Too Late for the Pebbles to Vote, Part 3

When we left off, we had examined the problem of self-organized criticality in social graphs, and were about to tackle the question of whether any more successful individual strategies exist. But before we dive into that, let’s talk about timing. And while we’re at it, let’s clarify something about scope.

c17dcf7ab8f9e1b0687a17fdc30f9f30

If you were wondering where the title came from, now you know.

The universalizing reflex is difficult to shake. Write about local effects and how they compound into regional ones, about the fact that we can only make decisions about our local behavior rather than deciding what will happen from the top down, and people will still ask you, “Yes, but what should we do at the top?” If you try to universalize local effects, you’ll find yourself trying to comb a sphere unsuccessfully. If frustration entertains you, then by all means enjoy yourself. Just know that you’ll never find a way to comb it flat.

I’m not writing about self-organized criticality in order to justify it. Like gravity, self-organized criticality admits neither justification nor blame. Anything that arises out of local interactions converges into an effect for which any individual actor can easily escape responsibility, and often they do. I’m not describing what I think should happen, merely what already happens. If that disturbs you, you’re not wrong! It disturbs the hell out of me too, especially when the state gets its hands on it. If you want different outcomes, though, you’re going to have to figure out how to get thousands if not millions of people to change their local strategies. This is, frankly, beyond me. The limits of my capacity are to tend my garden alongside others whose strategies are compatible with mine, and ignore the rest unless I have no other option. What I think should happen is only locally relevant. What I think could happen is only slightly less so.

There’s a military term, “operational tempo,” which refers to the overall duty cycle required of equipment and, most importantly, personnel. Maintaining a high operational tempo is a vital component of the sort of “shock and awe” tactics that wear opponents down reliably. Sociopaths know this well; the literature on sociopathy is rife with examples of adversaries setting the operational tempo for their targets. A sociopath who can shower a target with attacks from multiple directions has the opportunity to keep them off balance, in a responsive rather than proactive mode. Push hard enough from enough directions, and possibly the victim even becomes overwhelmed and stops functioning — a distributed denial of service.

However, this begs the question: What if they had a war and only a couple of people showed up?

You hire a lawyer for a legal battle. You hire a publicist for a PR war. But another important aspect of battlefield tactics is terrain. As much as the “digital rights” world — or the tech world in general — can feel like the entire scope of reality from time to time, given how immersive it can be once you’re in it, the rest of the world is quite a bit larger. DailyDot and Buzzfeed were interested in this story because it’s in their wheelhouse as part of the tech press. To the Associated Press, however, this was a brushfire war. And, to be perfectly honest, it is: just one more example of a petty would-be tyrant ejected from his would-be domain, and not a domain the wider world has any meaningful familiarity with, at that. Civil unrest in the Central African Republic is vastly more important, to the average reader, than sociopathy in open-source software; that’s the AP’s take, and I’m inclined to share their perspective.

One of the strategies sociopaths use to keep information silos sturdy is to mislead people about the state of the world outside their domain of influence. The controlling parents, determined to keep their daughter under their thumb, convince her that the only thing men really want is to violate and abandon her. The politician stokes constituents’ antipathy toward the outgroup, whether that’s Muslims or white trash. The cult leader convinces their followers that outsiders simply can’t understand the ways of the enlightened, and that people who express negative sentiments about the group are out to destroy it. The rockstar activist plays on non-rockstars’ fears of organized state opposition to their activism, and convinces non-rockstars that any challenge to the rockstar’s status is evidence of an organized plot against the activist group.

When you’re inside the silo, in other words, the world is small. Not only that, it has externally imposed boundaries. If the whole of your social reality inhabits one strongly-connected cluster, with no weak ties connecting you to “outsider” groups, parrying the slings and arrows of outrageous sociopathy can be the difference between staying connected to the social graph at all and effective ostracism. To arguably-eusocial animals like humans, the threat of isolation is a primal and deep one. But once you’re outside the silo, the threat evaporates, and in its place comes a new superpower: the power of perspective. Once your own dignity is no longer commingled with that of your adversary, you get to write your own criteria for what to dignify with your attention. The perpetrators of sick systems rely on people’s better natures, like loyalty, forgiveness, and a strong work ethic, to keep them coming back after every disappointment. Honor and thoroughness are also on that list. A person who can’t leave an insult unanswered is a person who can be baited, and a person who can be baited is a person who isn’t in control of their own attention. As anyone who’s ever been involved with the raising of a puppy or a grade schooler can confirm, when positive attention isn’t an option, negative attention beats no attention at all, and if an adversary is guiding the direction of your attention, you might as well be back in the silo.

Taking that control back for yourself has an even more important effect, though: it puts the operational tempo back in your hands, too. In the age of hot takes, it’s easy to believe that speed is the most important factor in responding to a reputational assault. However, trying to put this belief into practice is a recipe for burnout. The thing is, it’s easy to believe for the simple reason that so many other people already do. When it seems like everybody’s arguing about you, your instincts tell you to put up a robust defense. Your instincts, as it turns out, are full of shit. Once an avalanche has begun, your voice is no louder than that of any other pebble, and your exit is precarious until the ground settles. Focus your attention on more rewarding priorities, and act when you are ready — and no sooner.

This is actually just one instantiation of a general, lower-coordination-cost sociopath-resistance strategy that is a viable replacement for turtling: setting explicit boundaries and maintaining them. A person who respects a boundary will not cross that boundary. A person who also wants to signal their intent to respect a boundary will also keep a healthy distance back from it. Sociologist Ari Flynn, also a keen observer of abnormal psychology, points out that how a person responds to discovering they’ve crossed a boundary yields considerable information about their attitude toward boundaries in general: an honest person will try to find out how to make it right, while a bad actor will try to make it all about them.

Bad actors also keep trying. To a bad actor, a clearly defined boundary is like the battle of wits Scott Alexander describes as a result of “trying to control AIs through goals plus injunctions” — “Here’s something you want, and here are some rules telling you that you can’t get it. Can you find a loophole in the rules?” If one approach doesn’t work, a clever sociopath will keep coming up with new ones. A mediocre one will try the same approach on someone else, and an incompetent one will try it on people s/he has already tried it on. (I’ve encountered all three kinds.)

Recognizing this in the wild, however, can be hard. In Blind to Betrayal: Why We Fool Ourselves We Aren’t Being Fooled, research psychologist Jennifer Freyd explores the human tendency to systematically ignore mistreatment and treachery — a strategy for short-term self-protection that sets a person up for long-term harm. As Freyd explains:

The core idea [of betrayal trauma theory] is that forgetting and unawareness help the abuse victim survive. The theory draws on two facts about our nature as social beings and our dependence and reliance on others. First, we are extremely vulnerable in infancy, which gives rise to a powerful attachment system. Second, we have a constant need to make “social contracts” with other people in order to get our needs met. This has led to the development of a powerful cheater-detector system. These two aspects of our humanity serve us well, but when the person we are dependent on is also the person betraying us, our two standard responses to trouble conflict with each other.

Freyd focuses on trauma, but this tension also explains why people often write off minor boundary violations. When your cheater-detector system fires, only you know. You then have to decide whether you’re going to do anything about it. Options include confronting the cheater and alerting others about it. Doing something proactive might result in a redrafting of the social contracts that involve you, which is a potential threat to the attachments you rely on. This is especially true for people with an insecure attachment style. A person who has few or no secure attachments thus has an internal disincentive toward acting on their cheater-detector’s signals. For many people, the thought of losing a valued but insecurely-attached relationship is far more daunting than the notion of leaving a boundary violation unaddressed; taking action is scarier than staying still.

Relationships are iterated games, though — and they’re evolutionary. People’s strategies adapt as they learn about how the other players will react. When Mallory the sociopath observes that Alice grins and bears it when Mallory violates her boundaries, Mallory learns that Alice won’t make things difficult for him (or her). Alice also learns from this encounter: she trains herself not to respond when someone defects on her. Thus numbed, the next time Alice and Mallory interact, Mallory can betray her just a little harder, and if Alice sucks it up again, the cycle is poised to continue. Over time, as long as Alice cooperates, Mallory can shift Alice’s Overton window of tolerable behavior to ignore all kinds of abuses.

Given this, it’s tempting to attempt to define a rigid, comprehensive system of standards and defend them against all comers. This is most of what Honeywell proposes in her set of solutions for preventing “rock star” narcissists from taking up all the oxygen in a community. Her recommendations sound like good ideas on the surface. However, any mildly talented sociopath will have no problem end-running around all of them, usually by co-opting or distracting the organization’s leadership. As I’ve said before, sociopath strategies are battle-hardened, and some of them are effective counters to several of Honeywell’s suggestions at once. I’ve condensed these into the table below.

Defense Counterattack
Have explicit rules for conduct and enforce them for everyone

Assume that harassment reports are true and investigate them thoroughly

Watch for smaller signs of boundary pushing and react strongly

Call people out for monopolizing attention and credit

Enforce strict policies around sexual or romantic relationships within power structures

The sociopath “befriends” people with decision-making authority and/or social power. Those people make exceptions for the sociopath: rules turn out not to apply to him/her after all; investigations of the sociopath’s behavior are completely half-assed; people who Matter don’t react to boundary-pushing or spotlight-hogging and thus others conclude they won’t receive social support if they call it out; everyone studiously ignores that the sociopath and X are romantically involved; &c.
Make it easy for victims to find and coordinate with each other Sociopath gets a “friend” to join the affinity group and report back with information, sow misinformation and distrust, or both.
Build a “deep bench” of talent at every level of your organization

Build in checks for “failing up”

Distribute the “keys to the kingdom”

Sociopath interferes with HR / hiring / administration, making sure that “random” crises keep them so busy that no one has time to make sure these things are getting done. Sociopath becomes the irreplaceable person.
Flatten the organizational hierarchy as much as possible Tor’s organizational hierarchy was already flat, but this didn’t help them until Shari Steele came on board. Jake had co-opted leadership so thoroughly that they retaliated against Karen Reilly for reporting his behavior.
Avoid organizations becoming too central to people’s lives Sociopath slowly inculcates an atmosphere of paranoia: those outside the organization can’t be trusted. Often involves crisis-manufacturing. This one is really easy to pull off when everyone is on Slack or IRC.
Don’t create environments that make boundary violations more likely Sociopaths can organize these kinds of activities perfectly well on their own.

When I read Honeywell’s piece, I see a valiant effort to help her social-justice activist communities transition from a communal, socialized-mind-oriented mode of organization to a systematic, self-authoring-mind-oriented one. It’s a pity it’s doomed. Making sure that everyone in a group publicly identifies as a feminist, an anti-racist, or any other kind of do-gooder — that everyone sends all the right signals — was never enough to keep sufficiently subtle defectors out. This is the critical failure of the communal mode once any organization gets large enough. It’s great that identity-politics-oriented groups are finally starting to wake up to this fact.

Unfortunately, since sociopathy grew up as part of humanity, that means it evolved right alongside the very same efforts to develop comprehensive social systems that are breaking down on us now. Today’s sociopathy is a sociopathy that has learned to use our systems against us. We can learn to recognize this happening, but in order to do that, we have to be able to step outside the systems that we cherish the most and think about them like an adversary.

For example, it’s easy to think “okay, our group doesn’t like sexual predators, so we’ll ban sexual behavior within the group, and while we’re at it, we’ll also ban alcohol, since drinking impairs people’s decision-making.” On the surface, this sounds likely to be effective: it’s a bright line, right? Remember, though: “Here’s something you want, and here are some rules telling you that you can’t get it. Can you find a loophole in the rules?” Puritanical adherence to an object-level system creates exploitation vectors for bad actors. In an environment where having some trait T is a sin, there’s a strong incentive to appear non-T-like. This gives bad actors a new handle for gaining social control: the threat of impropriety. If everyone in a group is a convincing rumor or a planted bottle away from being ostracized, anyone without a conscience suddenly has an incredibly powerful weapon for undermining or getting rid of people who might inconvenience them by, say, not letting them get their way. It becomes even more powerful in groups where many members have low emotional intelligence, like technical groups. For people who score highly on measures of Machiavellian tendencies, high emotional intelligence is a force multiplier, as they’re able to use their emotional intelligence instrumentally to further their manipulative goals. In a low-emotional-intelligence environment, this is like shooting fish in a barrel.

A sufficiently manipulative person can even convince people to act in ways that betray their own consciences, as happened with the Tor organizer I mentioned before. It’s great to have standards, except when nobody’s willing to act on them. Even when you can’t count on your community to uphold the standards it’s adopted, though — or its members to act on their individual principles — you can always uphold your own.

That’s the grim reality of a world in which we have to trust other people: sometimes they let us down. No matter how watertight an organization’s Code of Conduct, if leadership wimps out on enforcing it — or only enforces it selectively — the code is worth less than the paper it’s written on. As an individual, about the only thing you can do about that is endeavor to spend your time around people with backbones. For all the debate that goes into their wording, rules and laws are abstract things which cannot act on their own. No matter how comprehensive the rules are, people will ultimately do whatever the hell they think they can get away with. Like ants, we operate in concert, but each of us acts alone.

The problem we face, then, is: in the face of Conway’s law and a structure prone to self-organized criticality, can we construct a stigmergy that resists bad actors without the high cost of large avalanches?

Mark Manson has noticed this too, from another direction:

In the attention economy, people are rewarded for extremism. They are rewarded for indulging their worst biases and stoking other people’s worst fears. They are rewarded for portraying the world as a place that is burning to the ground, whether it’s because of gay marriage, or police violence, or Islamic terrorism, or low interest rates. The internet has generated a platform where apocalyptic beliefs are celebrated and spread, and moderation and reason is something that becomes too arduous and boring to stand.

And this constant awareness of every fault and flaw of our humanity, combined with an inundation of doomsayers and narcissistic nihilists commanding our attention space, is what is causing this constant feeling of a chaotic and insecure world that doesn’t actually exist.

He also gets that the criticality is self-organized:

It’s us. We are going crazy. Each one of us, individually, capsized in the flood of negativity, we are ready to burn down the very structures on which the most successful civilizations in human history have been built.

Indulging our worst biases results, predictably, in error due to bias: if we aim at the wrong targets, we will hit the wrong targets. But our models of the world can also suffer from another class of errors: error due to variance. Err too far on the side of variance, and you’ll overfit to the random noise in your training set instead of the signal. Although there is always a tradeoff between bias and variance, the two are partially independent. This means that a model can simultaneously overfit due to hypersensitivity, and underfit due to bad assumptions.

I wrote about this last year in terms of precision and recall, another pair of properties we use to evaluate models in machine learning. As that post describes, the Schroedinger’s Rapist model is a high-bias, high-variance model with no false negatives but a brutally large number of false positives. Its one big advantage, when it comes to the meme’s own evolutionary fitness, is that people who adopt it feel like they have made themselves safer by doing so. “Trust no one of unavoidably broad class X” is another one of those ideas that sounds feasible (if draconian) on the surface — but it’s underspecified. Trust, in practice, is ditransitive: you trust someone with something. When that theme, the thing you’re trusting them with, is underspecified, that’s where a bad actor can nudge you toward redefining your boundaries farther and farther backward. “Trust everyone who signals Y” is equally underspecified, but even worse, because in a world where social media makes long-range (in graph-distance terms, not physical distance) signaling nearly free, a willful liar can find a new sucker every second. “You really think someone would do that? Just go on the internet and tell lies?” Look, if you hadn’t decided Reddit had cooties, you would have incorporated that meme into your thinking a decade ago and we wouldn’t need to have this conversation.

Ever found yourself realizing that things have gone too far, but can’t quite piece together how they got to be so bad? Often that’s the result of not recognizing your own boundaries in the first place. If you haven’t defined them, or are willing to let people get away with infringing on them in the interest of not rocking the boat socially, bad actors are happy to step in and define them — for their benefit, not yours. Sometimes, however, it’s the result of not recognizing boundary-pushing behavior, or not having a model for what that looks like. Like deadlines, a lot of people only notice their own boundaries from the whistling sound they make as they fly by.

I’m not saying never to re-evaluate your boundaries. Rather, never dial them back under duress, or in any other kind of stressful situation, for that matter. Do your reassessing afterward. Boundary-pushing is a dominance game in which merely feeling safe is tantamount to pissing yourself to keep warm. If your goal is to be safe, rather than to feel like everything is fine right up until your house burns down, there are two skills you have to learn. The first is to recognize dominance games in progress, and the second is to either exit or flip the script as the situation and your personal capacities call for it.

“Apply a particular set of object-level boundaries” can’t solve the problem of “people are often bad at holding their personal ground, especially in the moment.” If your boundaries are all object-level, a bad actor has only to set up a forced-error situation by incentivizing you to defend one at the expense of another. If you value your friends, s/he can use them as human shields, involving them such that drawing attention to the sociopath’s behavior brings harm to your friend. If you value an ideology, s/he can use it as a shield, associating him/herself with it so publicly and strongly that people fear that speaking up about the sociopath will “damage the brand.” The foolish man builds his house upon the nouns, and the clever sociopath turns those nouns into the walls of a silo.

The wise man builds his house upon the verbs: the purpose of boundaries is to protect your freedom of action. Action potential, like attention, is a finite resource, and everybody wants yours. Giving it away for free to the outrage of the day leaves you impoverished not only when it comes to local conflicts, but when it comes to tending your own garden. If you’re going to be the change you want to see in the world, you have to pick your battles. If you want to actually see some change, you’re going to have to make it locally.

Scope insensitivity comes into play here, too. We say we’re willing to dedicate value (i.e., pay) to prevent harm, but our instincts for estimating how much harm should correspond to how much value are wildly off. When Desvouges et al asked subjects how much they would spend to prevent migratory birds from drowning in oil-polluted ponds, on average the subjects were willing to dedicate less money to rescuing 20,000 birds ($78) than they were to rescuing 2,000 birds ($80). On a more timely note, when Bloomberg polled 749 likely voters in the 2016 election about the extent to which various actions of Donald Trump’s bothered them — “botheredness” is effectively a proxy variable for gut-check estimate of harm — only 44% were “bothered a lot” by the fraudulent Trump University, and 26% “bothered not at all.” By contrast, 62% found Trump mocking a disabled reporter very bothersome, and only 15% didn’t care. Thousands of little, far-away, invisible people got scammed, yet our instincts tell us an insult to one person we’re able to see is a greater harm. Once again, instinct is full of shit.

There’s no happy ending here. Maybe a few people will read this series and hit upon some local changes they can make to improve the stability of their own environment, but the pessimist in me isn’t about to put money on it. The most local environment is the one inside your own head, and if you’re content to feel like you’re on the right side of history even as it ends around you, nothing I have to say can help you. Satisfaction is itself an attractor, and when we fail to find satisfaction outside ourselves, we retreat to look for it within, even when what we find is nothing more than a tasty ligand. All the while, the sand keeps pouring.

Lots of essays end with an exhortation that some choice is yours. This time it’s even true. You can’t choose universal properties, but you can always choose where to expend your attention and effort. This has always been true, despite all the external demands for your resources. If you want to build something better, look directly around yourself first, and start there.


One parting observation:

Nearly everything I’ve said here also applies to the defining egregores of a two-party system.

Pleasant dreams.


Works cited and recommended reading:

Freyd, Jennifer, and Pamela Birrell. Blind to Betrayal: Why We Fool Ourselves We Aren’t Being Fooled.

Hintjens, Pieter. The Psychopath Code.

Issendai. “Sick Systems: How to Keep Someone With You Forever” et seq.:

“On Whittling Yourself Away”

“Qualities That Keep You in a Sick System”

McGregor, Jane and Tim. “Empathic people are natural targets for sociopaths — protect yourself.”

McGregor, Jane and Tim. The Empathy Trap: Understanding Antisocial Personalities.

Simon, George. In Sheep’s Clothing: Understanding and Dealing with Manipulative People.

U.S. Department of Defense Standards of Conduct Office. Encyclopedia of Ethical Failure.

Wallisch, Pascal. “Psychopaths in our midst — what you should know.”

Posted in Uncategorized | 5 Comments

Too Late for the Pebbles to Vote, Part 2

Previously, we discussed how sociopaths embed themselves into formerly healthy systems. Now let’s talk about what happens when those systems undergo self-organized criticality.

Consider a pile of sand. Trickle more sand onto it from above, and eventually it will undergo a phase transition: an avalanche will cascade down the pile.

As the sand piles up, the slope at different points on the surface of the pile grows steeper, until it passes the critical point at which the phase transition takes place. The trickle of sand, whatever its source, is what causes the dynamical system to evolve, driving the slope ever back up toward the critical point. Thanks to that property, the critical point is also an attractor. However, crucially, the overall order evident in the pile arises entirely from local interactions among grains of sand. Criticality events are thus self-organized.

Wars are self-organized criticality events. So are bank runs, epidemics, lynchings, black markets, riots, flash mobs, neuronal avalanches in your own brain’s neocortex, and evolution, as long as the metaphorical sand keeps pouring. Sure, some of these phenomena are beneficial — evolution definitely has a lot going for it — but they’re all unpredictable. Since humans are arguably eusocial, it stands to reason that frequent unpredictability in the social graphs we rely on to be human is profoundly disturbing. We don’t have a deterministic way to model this unpredictability, but wrapping your head around how it happens does make it a little less unsettling, and can point to ways to route around it.

A cellular automaton model, due to Bak, Tang, and Wiesenfeld, is the classic example of self-organized criticality. The grid of a cellular automaton is (usually) a directed graph where every vertex has out-degree 4 — each cell has four neighbors — but the model generalizes just fine to arbitrary directed graphs. You know, like social graphs.

Online social ties are weaker than meatspace ones, but this has the interesting side effect of making the online world “smaller”: on average, fewer degrees separate two arbitrary people on Facebook or Twitter than two arbitrary people offline. On social media, users choose whether to share messages from one to another, so any larger patterns in message-passing activity are self-organized. One such pattern, notable enough to have its own name, is the internet mob. The social graph self-reorganizes in the wake of an internet mob. That reorganization is a phase transition, as the low become high and the high become low. But the mob’s target’s social status and ties are not the only things that change. Ties also form and break between users participating in, defending against, or even just observing a mob as people follow and unfollow one another.

Some mobs form around an explicit demand, realistic or not — the Colbert Report was never in any serious danger of being cancelled — while others identify no extrinsic goals, only effects on the social graph itself. Crucially, however, both forms restructure the graph in some way.

This structural shift always comes with attrition costs. Some information flows break and may never reform. The side effects of these local interactions are personal, and their costs arise from the idiosyncratic utility functions of the individuals involved. Often this means that the costs are incomparable. Social media also brings the cost of engagement way down; as Justine Sacco discovered, these days it’s trivial to accuse someone from halfway around the planet. But it’s worse than that; even after a mob has become self-sustaining, more people continue to pile on, especially when messages traverse weak ties between distant groups and kick off all-new avalanches in new regions of the graph.

strong-weak-ties

Members of the black group are strongly connected to other members of their group, and likewise for the dark gray and white groups. The groups are interconnected by weak, “long-distance” ties. Reproduced from The Science of Social 2 by Dr. Michael Wu.

Remember Conway’s law? All systems copy the communication structures that brought them into being. When those systems are made of humans, that communication structure is the social graph. This is where that low average degree of separation turns out to be a problem. By traversing weak ties, messages rapidly escape a user’s personal social sphere and propagate to ones that user will never intersect. Our intuitions prepare us for a social sphere of about a hundred and fifty people. Even if we’re intellectually aware that our actions online are potentially visible to millions of people, our reflex is still to act as if our messages only travel as far and wide as in the pre-social-media days.

This is a cognitive bias, and there’s a name for it: scope insensitivity. Like the rabbits in Watership Down, able to count “one, two, three, four, lots,” beyond a certain point we’re unable to appreciate orders of magnitude. Furthermore, weak long-distance ties don’t give us much visibility into the size of the strongly-tied subgraphs we’re tapping into. Tens of thousands of individual decisions to shame Justine Sacco ended in her being the #1 trending topic on Twitter — and what do you suppose her mentions looked like? Self-organized criticality, with Sacco at ground zero. Sure, #NotAllRageMobs reach the top of the trending list, but they don’t have to go that far to have significant psychological effect on their targets. (Sociologist Kenneth Westhues, who studies workplace mobbing, argues that “many insights from [the workplace mobbing] literature can be adapted mutatis mutandis to public mobbing in cyberspace,” and I agree.)

In the end, maybe the best we can hope for is user interfaces that encourage us to sensitize ourselves to the scope of our actions — that is to say, to understand just how large of a conversation we’re throwing our two cents into. Would people refrain from piling on to someone already being piled on if they knew just how big the pile already was? Well, maybe some would. Some might do it anyway, out of malice or out of virtue-signaling. As Robert Kegan and Lisa Laskow Lahey point out in Immunity to Change, for many people, their sense of self “coheres by its alignment with, and loyalty to, that with which it identifies.” Virtue signaling is one way people express that alignment and loyalty to groups they affiliate with, and these days it’s cheap to do that on social media. Put another way, the mobbings will continue until the perverse incentives improve. There’s not much any of us can individually do about that, apart from refraining from joining in on what appears to be a mob.

That’s a decision characteristic of what Kegan and Lahey call the “self-authoring mind,” contrasted with the above-mentioned “socialized mind,” shaped primarily “by the definitions and expectations of our personal environment.” Not to put too fine a point on it, over the last few years, my social media filter bubble has shifted considerably toward the space of people who independently came to a principled stance against participation in mobs. However, given that the functional programming community, normally a bastion of cool reason and good cheer, tore itself apart over a moral panic just a few months ago, it’s clear that no community is immune to flaming controversy. Self-organized criticality means that the call really is coming from inside the house.

Here’s the moral question that not everyone answers the same way I do, which has led to some restructuring in my region of the graph, a local phase transition: when is it right to throw a handful of sand on the pile?

Some people draw a bright line and say “never.” I respect that. It is a consistent system. It was, in fact, my position for quite some time, and I can easily see how that comes across as throwing down for Team Not Mobbing. But one of the implications of being a self-authoring system is that it’s possible to revisit positions at which one has previously arrived, and, if necessary, rewrite them.

So here’s the core of the conundrum. Suppose you know of some information that’s about to go public. Suppose you also expect, let’s say to 95% confidence, that this event will kick off a mob in your immediate social sphere. An avalanche is coming. Compared to it, you are a pebble. The ground underneath and around you will move whether you do anything or not. What do you do?

I am a preference consequentialist, and this is a consequentialist analysis. I won’t be surprised if how much a person agrees with it correlates with how much of a consequentialist they are. I present it mainly in the interest of braindumping the abstractions I use to model these kinds of situations, which is as much in the interest of information sharing as anything else. There will be mathematics.

I am what they call a “stubborn cuss” where I come from, and if my only choices are to jump or be pushed, my inclination is to jump. Tor fell down where organizational accountability was concerned, at first, and as Karen Reilly’s experience bears out, had been doing so for a while. So that’s the direction I jumped. To be perfectly honest, I still don’t have anything resembling a good sense of what the effects of my decision were versus those of anyone else who spoke up, for whatever reason, about the entire situation. Self-organized chaotic systems are confounding like that.

If you observe them for long enough, though, patterns emerge. Westhues has been doing this since the mid-1990s. He remarks that “one way to grasp what academic mobbing is is to study what it is not,” and lists a series of cases. “Ganged up on or not,” he concludes of a professor who had falsified her credentials and been the target of student protests about the quality of her teaching, “she deserved to lose her job.” Appelbaum had already resigned before the mob broke out. Even if the mob did have an extrinsic demand, his resignation couldn’t have been it, because that was already over and done with.

Okay, but what about the intrinsic outcomes, the radical restructuring of the graph that ensued as the avalanche settled? Lovecruft has argued that removing abusers from opportunities to revictimize people is a necessary step in a process that may eventually lead to reconciliation. This is by definition a change in the shape of the social graph. Others counter that this is ostracism, and, well, that’s even true: that’s what it looks like when a whole lot of people decide to adopt a degrees-of-separation heuristic, or to play Exit, all at once.

Still others argue that allegations of wrongdoing should go before a criminal court rather than the court of public opinion. In general I agree with this, but when it comes to longstanding patterns of just-this-side-of-legally-actionable harm, criminal courts are useless. A bad actor who’s clever about repeatedly pushing ever closer to that line, or who crosses it but takes care not to leave evidence that would convince a jury beyond a reasonable doubt, is one who knows exactly what s/he’s doing and is gaming the system. When a person’s response to an allegation boils down to “no court will ever convict me,” as Tor volunteer Franklin Bynum pointed out, that sends a game-theoretically meaningful signal.

Signaling games are all about inference and credibility. From what a person says, what can you predict about what actions they’ll take? If a person makes a particular threat, how likely is it that they’ll be able to make good on it? “No court will ever convict me” is actually pretty credible when it comes to a pattern of boundary-violating behavior that, in many cases, indeed falls short of prosecutability. (Particularly coming from someone who trades on their charisma.) Courts don’t try patterns of behavior; they try individual cases. But when a pattern of boundary-pushing behavior is the problem, responding to public statements about that pattern with “you’ll never prove it” is itself an instance of the pattern. As signals go, to quite a few people, it was about the loudest “I’m about to defect!” Appelbaum could have possibly sent in a game where the players have memory.

Courts don’t try patterns of behavior, but organizations do. TQ and I once had an incredibly bizarre consulting gig (a compilers consulting gig, which just goes to show you that things can go completely pear-shaped in bloody any domain) that ended with one of the client’s investors asking us to audit the client’s code and give our professional opinion on whether the client had faked a particular demonstration. Out of professional courtesy, we did not inquire whether the investor had previously observed or had suspicions about inauthenticity on the client’s part. Meanwhile, however, the client was simultaneously emailing conflicting information to us, our business operations partner, and the investor — with whom I’d already been close friends for nearly a decade — trying to play us all off each other, as if we didn’t all have histories of interaction to draw on in our decision-making. “It’s like he thinks we’re all playing classical Prisoner’s Dilemma, while the four of us have been playing an iterated Stag Hunt for years already,” TQ observed.

Long story short (too late), the demo fell shy of outright fraud, but the client’s promises misrepresented what the code actually did to the point where the investor pulled out. We got a decent kill fee out of it, too, and a hell of a story to tell over beers. When money is on the line, patterns of behavior matter, and I infer from the investor’s action that there was one going on there. Not every act of fraud — or force, for that matter — rises to the level of criminality, but a pattern of repeated sub-actionable force or fraud is a pattern worth paying attention to. A pattern of sub-actionable force or fraud coupled with intimidation of people who try to address that pattern is a pattern of sociopathy. If you let a bad actor get away with “minor” violations, like plagiarism, you’re giving them license to expand that pattern into other, more flagrant disregard of other people’s personhood. “But we didn’t think he’d go so far as to rape people!” Of course you didn’t, because you were doing your level best not to think about it at all.

Investors have obvious strong incentives to detect net extractors of value accurately and quickly. Another organization with similarly strong incentives, believe it or not, is the military. Training a soldier isn’t cheap, which is why the recruitment and basic training process aims to identify people who aren’t going to acquire the physical and mental traits that soldiering requires and turn them back before their tenure entitles them to benefits. As everyone who’s been through basic can tell you, one blue falcon drags down the whole platoon. Even after recruits have become soldiers, though, the military still has strong incentives to identify and do something about serial defectors. Unit cohesion is a real phenomenon, for all the disagreement on how to define it, and one or a few people preying on the weaker members of a unit damages the structure of the organization. The military knows this, which is the reason its Equal Opportunity program exists: a set of regulations outlining a complaint protocol, and a cadre trained and detailed to handle complaints of discriminatory or harassing behavior. No, it’s not perfect, by any stretch of the imagination. The implementation of any human-driven process is only as rigorous as the people implementing it, and as we’ve already discussed, subverting human-driven processes for their own benefit is a skill at which sociopaths excel. However, like any military process, it’s broken down into bite-sized pieces for every step of the hierarchy. Some of them are even useful for non-hierarchical structures.

Fun fact: National Guard units have EO officers too, and I was one. Again and again during the training for that position, they hammer on the importance of documentation. We were instructed to impress that not just on people who bring complaints, but on the entire unit before anyone has anything to bring a complaint about. Human resources departments will tell you this too: document, document, document. This can be a difficult thing to keep track of when you’re stuck inside a sick system, a vortex of crisis and chaos that pretty accurately describes the internal climate at Tor over the last few years. And, well, the documentation suffered, that’s clear. But now there’s some evidence, fragmentary as it may be, of a pattern of consistent and unrepentant boundary violation, intimidation, bridge-burning, and self-aggrandizement.

Even when the individual acts that make up a pattern are calculated to skirt the boundaries of actionable behavior, military commanders have explicit leeway to respond to the pattern with actions up to and including court-martial, courtesy of the general article of the Uniform Code of Military Justice:

Though not specifically mentioned in this chapter, all disorders and neglects to the prejudice of good order and discipline in the armed forces, all conduct of a nature to bring discredit upon the armed forces, and crimes and offenses not capital, of which persons subject to this chapter may be guilty, shall be taken cognizance of by a general, special, or summary court-martial, according to the nature and degree of the offense, and shall be punished at the discretion of that court.

It’s the catch-all clause that Kink.com installed a bunch of new rules in lieu of, an exception funnel that exists because sometimes people decide that having one is better than the alternative. Realistically, any form of at-will employment implicitly carries this clause too. If a person can be fired for no reason whatsoever, they can certainly be fired for a pattern of behavior. Companies have this option; organizations that don’t maintain contractual relationships with their constituents face paths that are not so clear-cut, for better or for worse.

But I take my cues about exception handling, as I do with a surprisingly large number of other life lessons, from the Zen of Python:

Errors should never pass silently.
Unless explicitly silenced.

When a person’s behavior leaves a pattern of damage in the social fabric, that is an exception going silently unhandled. The whisper network did not prevent the damage that has occurred. It remains to be seen what effect the mob-driven documentation will have. Will it achieve the effect of warning others about a recurring source of error (I suppose nominative determinism wins yet again), or will the damaging side effects of the phase transition prove too overwhelming for some clusters of the graph to bear? Even other consequentialists and I might part ways here, because of that incomparability problem I mentioned earlier. I don’t really have a good answer to that, or to deontologists or virtue ethicists either. At the end of the day, I spoke up because of two things: 1) I knew that several of the allegations were true, and 2) if I jumped in front of the shitstorm and got my points out of the way, it would be far harder to dismiss as some nefarious SJW plot. Sometimes cross-partisanship actually matters.

I don’t expect to change anyone’s mind here, because people don’t develop their ethical principles in a vacuum. That said, however, situations like these are the ones that prompt people to re-examine their premises. Once you’re at the point of post-hoc analysis, you’re picking apart the problem of “how did this happen?” I’m more interested in “how do we keep this from continuing to happen, on a much broader scale?” The threat of mobs clearly isn’t enough. Nor would I expect it to be, because in the arms race between sociopaths and the organizations they prey on, sociopath strategies evolve to avoid unambiguous identification and thereby avoid angry eyes. “That guy fucked up, but I won’t be so sloppy,” observes the sociopath who’s just seen a mob take another sociopath down. Like any arms race, it is destined to end in mutually assured destruction. But as long as bad actors continue to drive the sick systems they create toward their critical points, there will be avalanches. Whether you call it spontaneous order or revolutionary spontaneity, self-organized criticality is a property of the system itself.

The only thing that can counteract self-organized aggregate behavior is different individual behavior that aggregates into a different emergent behavior. A sick system self-perpetuates until its constituents decide to stop constituting it, but just stopping a behavior doesn’t really help you if doing so leaves you vulnerable. As lousy of a defense as “hunker down and hope it all goes away soon” is over the long term, it’s a strategy, which for many people beats no strategy at all. It’s a strategy that increases the costs of coordination, which is a net negative to honest actors in the system. But turtling is a highly self-protective strategy, which poses a challenge: any proposed replacement strategy that lowers the cost of coordination among honest actors also must not be significantly less self-protective, for idiosyncratic, context-sensitive, and highly variable values of “significantly.”

I have some thoughts about this too. But they’ll have to wait till our final installment.

Posted in Uncategorized | 12 Comments

Too Late for the Pebbles to Vote, Part 1

There’s a pattern most observers of human interaction have noticed, common enough to have earned its own aphorism: “nice guys finish last.” Or, refactored, “bad actors are unusually good at winning.” The phenomenon shows up in business, in politics, in war, in activism, in religion, in parenting, in nearly every collaborative form of human undertaking. If some cooperative effort generates a valuable resource, tangible or intangible, some people will try to subvert the effort in order to divert more of that resource to themselves. Money, admiration, votes, information, regulatory capacity, credibility, influence, authority: all of these and more are vulnerable to capture.

Social engineering, as a field, thus far has focused primarily on hit-and-run tactics: get in, get information (and/or leave a device behind), get out. Adversaries who adaptively capture value from the organizations with which they involve themselves are subtler and more complex. Noticing them, and responding effectively, requires a different set of skills than realizing that’s not the IT guy on the phone or that a particular email is a phish. Most importantly, it requires learning to identify patterns of behavior over time.

Having recently been adjacent to the sudden publicity of one such pattern of behavior, I have a lot to discuss about the general mechanisms that give rise to both these patterns and the criticality events — the social media jargon is “shitstorms” — they occasionally generate, and also about this specific incident. We’re going to talk about narcissism and its side effects, and how bad actors can damage good organizations. We’re going to talk about how bad things happen to good people, how all kinds of people make bad decisions, and also how organizations live and die. We’re going to talk about self-organized criticality. There will be game theory, and management theory, and inside baseball, and multiple levels from which to view things, and even a diagram or two, so if diagrams aren’t your thing, you might as well bail out now. There will also be some practical advice, toward the end.

But first, let’s talk about David Chapman’s 2015 essay, “Geeks, MOPs, and sociopaths in subculture evolution.”


In Chapman’s analysis, a subculture’s growth passes through three phases. First come the geeks, the creators and their True Fans whose interest in a niche topic gets a scene moving. Then come the MOPs, short for “Members Of Public,” looking for entertainment, new experiences, and something cool to be part of. Finally, along come the sociopaths, net extractors of value whose long-term aim is to siphon cultural, social, and liquid capital from the social graph of geeks and MOPs. Sociopaths don’t just take, unless they’re not very good at what they do. Many sociopaths contribute just enough to gain a reputation for being prosocial, and keep their more predatory tendencies hidden until they’ve achieved enough social centrality to be difficult to kick out. It’s a survival strategy with a long pedigree; viruses that burn through their host species reservoir too quickly die off.

Corporations, of course, have their own subcultures, and it’s easy to see this pattern in the origin stories of Silicon Valley success stories like Google — and also those of every failed startup that goes under because somebody embezzled and got away with it. Ditto for nonprofits, activist movements, social networking platforms, and really anything that’s focused on growth. Which is a lot of things, these days.

Organizations have a strong incentive to remove net extractors of value. Would-be net extractors of value, then, have an even stronger incentive to keep themselves connected to the social graph. The plasticity of the human brain being what it is, this sometimes leads to some interesting cognitive innovations.

Narcissism, for example, when it rises to the level of a pathology, is a personality disorder. This is not sufficient, in and of itself, to qualify someone as a sociopath in Chapman’s model. A narcissist who knows what kind of behavior s/he is capable of, keeps capital-siphoning behaviors (like claiming credit for others’ work) in check, and remains a net contributor of value even when that contribution isn’t aligned with his/her personal incentives, is by definition not a sociopath. However, a large social graph can be a tempting source of narcissistic supply, the interpersonal support that feeds a narcissist’s fragile and hungry ego. A narcissist who coerces or cons others into maintaining the “superman” narrative that papers over that damaged ego is a narcissistic sociopath. Other personality disorders can develop in similar ways, such as with borderline sociopaths, who coerce or con others into holding up the black-and-white, good-versus-evil lens through which the borderline sees the world. A mere personal dysfunction, once weaponized, becomes something much larger and more dangerous.

If you’ve ever seen an apparently-thriving group suddenly implode, its members divided over their opinions about one particular person, chances are you’ve seen the end of a sociopath’s run. Last December, progressive PR firm FitzGibbon PR collapsed when it came out that founder Trevor FitzGibbon had a pattern of sexually assaulting and harassing his employees and even some of his firm’s clients. However, the progressivism is what elevates the FitzGibbon story to “man bites dog” levels of notoriety. Everyone loves to watch a hypocrite twist in the wind. Usually one hears about sociopath-driven organizational meltdowns through the grapevine, though, not the media. Fearing repercussions or bad publicity, firms often equivocate about the reasons behind a sudden departure or reorganization. This tendency is understandable from a self-preservation perspective, but it also covers a sociopath’s tracks. Ejected from one firm, a serial net extractor of value can pick right back up at another one. (Indeed, FitzGibbon had been disciplined for harassment at his previous firm.)

Which brings us to the Tor Project.

Tor is an anonymous routing network. Journalists, dissidents, law enforcement, queer people, drug dealers, abuse victims, and many other kinds of people who need privacy send and receive their Internet traffic through Tor’s encryption and routing scheme in order to keep site operators from knowing who and where they are. It’s an intricate system with a lot of moving parts, supported by a foundation that pays its developers through the grant funding it brings in. And about two months ago, Tor’s most visible employee, Jacob “ioerror” Appelbaum, abruptly resigned.

Before coming to Tor, Appelbaum already had a history of value-extracting behavior only occasionally noticeable enough to merit discipline. His 2008 Chaos Communication Congress talk presented, without credit, research that he had wheedled out of Len Sassaman, Dan Kaminsky, and me the previous year. Other researchers, like Travis Goodspeed and Joe Grand, learned the hard way that to work “with” Appelbaum meant to have him put in no effort, but take credit for theirs. As Violet Blue points out, his ragequit from San Francisco porn producer Kink.com followed a flotilla of employee rulebook updates he’d personally inspired.

There’s never a convenient time for a scandal involving a decade-plus of sexual and professional misconduct, and organizational cover-ups thereof, to break. It’s easy to think “oh, I have a lot on my plate right now,” or “oh, it’s not really my problem,” and keep your head down until the chaos subsides. I could have exercised either of those options, or any one of half a dozen others, when Tor announced Appelbaum’s resignation in a one-sentence blog post a week and change before my wedding. But there’s never a good time for a pattern of narcissistic sociopathy to be exposed; there is only too late, or even later. So I got vocal. So did some other folks. And Tor confirmed that Appelbaum had resigned over sexual misconduct. Nick Farr went public about how Appelbaum had stalked and intimidated him at a conference in Hamburg in December 2013. Appelbaum vowed he’d done nothing criminal and threatened legal action, and the media circus was on.

It turns out that when seven pseudonymous people, and a small handful of named ones, speak up in a situation like this one, reporters really, really want to talk to the people with real names attached. At the time, I was in Houston, taking care of final preparations for my June 11th wedding on Orcas Island. I also spent a lot of that time fielding journalists’ questions about things I’d learned from members of the community about Tor’s little “open secret,” about Appelbaum’s plagiarism, and about observing Appelbaum manhandling a woman in a bar from my vantage point about twenty feet away. Then I got on a plane, flew to Seattle, got on a ferry, and didn’t open my laptop until I returned to work the following Monday.

During that period, Appelbaum’s publicist apparently tracked down and released a statement from the woman involved in the bar incident, Jill Bähring. Bähring avers that her interactions with Appelbaum were entirely consensual, which I am relieved and pleased to hear. I’m not sure why anyone would expect any other reaction out of me, seeing as how I’ve sung the praises of making mistakes and owning them in public for so long that I’ve given invited talks on it. The interaction I observed took place within my line of sight but out of my earshot, and if I misinterpreted it, then I genuinely am sorry about that. Ultimately, Bähring makes her own decisions about what she consents to or doesn’t. If I was mistaken, well, good.

But Leigh Honeywell also makes her own decisions about what she consents to or doesn’t, and Karen Reilly likewise. Attacking me over a misinterpretation may be enough to distract some people from the full scope of a situation, but nothing about my error invalidates Honeywell and Reilly’s accounts about their own experiences. Isn’t it interesting that someone whose first public response to allegations of wrongdoing was “I apologize to people I’ve hurt or wronged!” hasn’t had a single word to say to either one in two months? Or to Alison Macrina, or to Isis Lovecruft? It’s as if the allegorical defendant against murder, arson, and jaywalking had no response to the murder or arson counts, but wanted to make damned sure the whole world knew he wasn’t a jaywalker.

*slow clap*

How low-rent of a publicist do you have to hire for them not to be able to keep a story that simple consistent? All Appelbaum had to do was swallow his pride and ask his publicist to bang out an apology of the “I’m sorry you feel that way” variety, and he could have maintained the semblance of high ground he tried to stake out in his initial statement. But the need to be adored — the narcissist’s defining quality, and the sociopath’s first rule of survival — is simply too alluring, the opportunity to gloat over seeing one’s prey stumble too difficult to resist.

Attention is a scarce commodity. What a person expends it on reveals information about that person’s priorities.

Isn’t it interesting when people show you what their preferences really are?


But that’s more than enough narcissistic supply for that particular attention junkie. Let’s talk about preference falsification spirals.

Honeywell correctly observes that whisper networks do not transmit information reliably. In her follow-on post, she advocates that communities “encourage and support private affinity groups for marginalized groups.” If this worked, it would be great, but Honeywell conveniently neglects to mention that this solution has its own critical failure mode: what happens to members of marginalized groups whom the existing affinity group considers unpersons? I can tell you, since it happened here: we had to organize on our own. Honeywell’s report came as a surprise to both me and Tor developer Andrea Shepard, because we weren’t part of that whisper network. Nor would we expect to be, given how Honeywell threw Andrea under the bus when Andrea tried to reach out to her for support in the past. If your affinity group refuses to warn or help Certain People who should otherwise definitionally fall under its auspices, then what you’re really saying is “make sure the sociopath rapes an unperson.”

Thanks, but no thanks. Nobody should have to suck up to an incumbent clique in order to learn where the missing stairs are. The truly marginalized are those with no affinity group, no sangha. Who’s supposed to help them?

It’s a tough question, because assessing other people’s preferences from their behavior can be difficult even when they notionally like you. I was surprised, after the news of the extent of Appelbaum’s behavior broke, to learn that several acquaintances whom I had written off as either intentionally or unintentionally enabling him (in the end, it doesn’t really matter which) had actually been warning other people about him for longer than I had. How did I make this mistake?

Well, social cartography is hard. Suppose you’re at an event full of people you kinda-sorta know, and one person who you know is a sociopath. Supposing you decide to stick around, how do you tell who you can trust? Naïvely, anyone who’s obviously buddy-buddy with the sociopath is right out. But what about people who interact with the sociopath’s friends? During my brief and uneventful stint as an international fugitive, both friends and friends-of-friends of the pusbag who was funneling information about me to the prosecution were happy to dump information into that funnel. I learned the hard way that maintaining a cordon sanitaire around a bad actor requires at least two degrees of separation and possibly more. Paranoid? Maybe. But the information leakage stopped. (If I suddenly stopped talking to you sometime in 2009, consider whether you have a friend who is a narc.)

Consider, however, how this plays out in tightly connected groups, where the maximum degree of separation between any two people is, let’s say, three. Suppose that Mallory is a sociopath who has independently harmed both Alice and Bob. Suppose further that Alice and Bob are three degrees of separation from one another, and each has an acquaintance who is friends with Mallory. Let’s call those acquaintances Charlie and Diane, respectively.

social-graph

Mallory, the sociopath, is one degree of separation away from each of Charlie and Diane, who are also one degree away from each other. Charlie’s friend Alice and Diane’s friend Bob are three degrees apart.

If Alice sees Diane and Mallory interacting, and then sees Diane and Bob interacting, the two-degrees-of-separation heuristic discourages Alice from interacting with Bob, since Bob appears to be a friend of a friend of Mallory. Likewise for Bob and Charlie in the equivalent scenario. How can Alice and Bob each find out that the other is also one of Mallory’s victims, and that they could help each other?

In business management, this kind of problem is known as an information silo, and it is a sociopath’s best friend. Lovecruft describes several of Appelbaum’s siloing techniques, such as threatening to smear anyone who spoke out against him as a closet fed. As recently leaked chat logs show, affiliation with intelligence agencies is a genuine hazard for some Tor contributors, which means that “fedjacketing” someone, or convincing others that they’re actually a fed, is an attack which can drive someone out of the community. (Side note: how the hell does urbandictionary.com not have an entry for fedjacketing yet? They have snitch jacket, with which Appelbaum has also threatened people.) But you don’t have to be someone for whom a fedjacketing would be career death for a sociopath to put you in an information silo. Fedjacketing is merely the infosec reductio ad absurdum of the reputational-damage siloing technique. If someone has made it clear to you that they’ll ruin your reputation — or any other part of your life — if you so much as breathe about how they treated you, that’s siloing. Pedophiles do it to children (“don’t tell your mom and dad, or they’ll put us both in jail”), cult leaders do it to their followers — anyone a sociopath can emotionally blackmail, s/he can isolate.

Discussing this with one acquaintance I had misread as an enabler, I asked: what should Alice and Bob do? “When in doubt, it might be a good idea to ask,” they suggested. But this presupposes that either Alice or Bob is insufficiently siloed as to make asking a viable option. My acquaintance also allowed that they had misread still other people for years simply due to not knowing that those people had cut ties with Appelbaum. I didn’t know my acquaintance’s preferences, nor they mine. My acquaintance didn’t know the other people’s preferences, nor vice versa. Because none of us expressed our preferences freely, we all falsified our preferences to one another without trying to, which I’m sure Appelbaum appreciated. People believed they had to play by the standard social rules, and that civility gave him room to maneuver. Once a sociopath achieves social centrality, concealed mistrust creates more information silos than the sociopath could ever create alone.

What else creates information silos? In some cases, the very people who are supposed to be in a position to break them down. Sociopaths don’t only target victims. They also target people in positions of authority, in order to groom them into enablers. This happened at Tor. The “open secret” was so open that, when Appelbaum didn’t show up to a biannual meeting in February and people put up a poster for others to write messages to him, someone wrote, “Thanks for a sexual-assault-free Tor meeting!” This infuriated one of the organizers, who had to be talked down from collecting handwriting samples to identify the writer. At an anonymity project, no less. Talk about not knowing your demographic.

This abandonment of the community’s core values speaks to just how far someone can be groomed away from their own core values. In part, this may have been due to Appelbaum’s dedication to conflating business and personal matters — playing off people’s unwillingness to overlap the two. When this tactic succeeds against a person in a position of organizational power, it incentivizes them to protect their so-called friend, to the overall detriment of the organization.

Friendship is all well and good, right up to the point when it becomes an excuse to abdicate a duty of care. You know, like the one a meeting organizer takes on with respect to every other attendee when they accept the responsibility of organization. If the organizer knew that his “friend” had serious boundary issues, why the hell didn’t he act to protect or at least warn people at the meetings Appelbaum did attend? As enablers, people in positions of authority are a force multiplier for sociopaths. Sociopaths love to recruit them as supporters, much the same as the way a middle-school Queen Bee puts on her most adorable face for the vice-principal. Why put in the effort to threaten victims when your pet authority figure will gladly do it for you? Co-opted authority figures turn preference falsification cascades into full-on waterfalls.

Scott Alexander muses:

I wonder if a good definition for “social cancer” might be any group that breaks the rules of cooperative behavior that bind society together in order to spread more quickly than it could legitimately achieve, and eventually take over the whole social body.

One cell in the body politic mutates, and starts to recruit others. Those recruited cells continue to perform the same functions they always have — building structures, transmitting signals, defending nearby cells — but now they do it in service of that mutant cell line. The tumor is the silo. If you find yourself breaking the rules — or, worse, your rules, your personal ethics — for someone on a regular basis, consider whether that charming friend of yours is inviting you to be part of their tumor.

How do you bust out of a sociopath’s information silo? Personally, I take my cues from Captain James Tiberius Kirk: when the rules are arrayed against you, break them. When a sociopath tries to leave you no “legitimate” maneuvers, Kobayashi Maru that shit as hard as you possibly can.

I also take cues from my husband. TQ has interacted with Appelbaum exactly twice. The first time, Appelbaum physically shoved him out of the way at my late husband Len’s wake in order to stage a dramatic fauxpology for plagiarizing me, Len, and Dan in 2008, begging to “put our differences aside.” (Protip: when someone later tries to shut you up about something they did to you because “we reconciled!”, it wasn’t a real apology in the first place.) The second time, Appelbaum walked up and sat on him.

Appelbaum behaves as if TQ were an object. Operationalizing people — understanding them as a function of what they can do, rather than who they are — is one thing. We’re autistic; we do it all the time. Operationalizing people without concern for their preferences or their bodily integrity is another thing entirely. Since then, with no concern whatsoever for social niceties, any time anyone has brought Appelbaum up in TQ’s presence, he asks, “Why are you giving the time of day to a sociopath?” It isn’t polite, but it sure does break the ice quickly. 

Similarly, a few years ago, Appelbaum applied to speak at a conference TQ and I regularly attend. This conference is neither streamed nor recorded, and speakers are encouraged to present works in progress. The organizer contacted us, unsure how to handle the situation. TQ replied, “I would no more invite a plagiarist to an unfinished-work conference than I would a pedophile to a playground.” The organizer rejected Appelbaum, and the conference went on theft-free. That’s more than a lot of other conference organizers, some of whom knew better, can say.

The one thing that protects sociopaths the most is their victims’ unwillingness to speak up, because the one thing that can hurt a sociopath is having their extraction racket exposed for the fraud it really is. People fear social repercussions for standing up to the Rock Star or the Queen Bee, but consider: if someone is stupid, venal, or corrupt enough to be a sociopath’s enabler, why would you even want to give them any of your social capital in the first place? You might feel like you have to, for the sake of social harmony, or because the subcultural niche that the sociopath has invaded is important to you, or because it’s your workplace and you really need the job. Even sociopaths themselves can experience this pressure. On Quora, diagnosed sociopath Thomas Pierson explains:

Why do [sociopaths] lie and manipulate? Because people punish you when you tell them the truth.

Giving in to fear, to the detriment of those around you, is how you become the bad guy. Lies don’t really protect anyone. They only kick the can down the road, and the reckoning will only be worse when it eventually comes. Suppressing the truth out of fear of being punished is the same as paying the Danegeld out of fear of being overpowered. It’s a form of the sunk cost fallacy, and Kipling had the right of it:

And that is called paying the Dane-geld;
But we’ve proved it again and again,
That if once you have paid him the Dane-geld
You never get rid of the Dane.

Social capital isn’t some magical thing that some people have and others don’t. Like any other form of currency, the locus of its power is in its exchange. (Yeah, we really are all Keynesians now.) In the case of social exchanges, those currencies are information, attention, and affective empathy. Sociopaths try to keep their victims from having relationships the sociopath isn’t involved in, because those are the relationships the sociopath can’t control or collect rent on in the form of secrets or adulation. Building up those relationships — from finding other victims, all the way up to entire parallel social circles where known sociopaths are unwelcome and their enablers receive little to no interaction — incrementally debases the sociopath’s social currency, faster and faster as the graph expands.

Internalizing this grants you a superpower: the power of giving exactly zero fucks. It’s the same power of giving zero fucks that Paulette Perlhach writes about in The Story of a Fuck-Off Fund, only denominated in graph connectivity rather than dollars. It takes the same kind of effort, but it pays off in the same kind of reward. When you give no fucks and tell the truth about a sociopath, two things happen. First, people who have been hurt and haven’t found their superpower yet will come find you. Second, the sociopath starts flailing. (One benefit of being right is that the facts line up on your side.) As accounts of the sociopath’s misdeeds come out, the sociopath’s narrative has to become more and more convoluted in order to keep the fanboys believing. “They’re all feds!” he shrieks. “Every last one of them!”

Uh-huh. Sure. Because the feds always assign multiple agents not only to target one guy who can’t even keep his dick in his pants, but to become his coworkers, don’t they? This is not exactly an inexpensive proposition. Reality check: if the feds had wanted to pull a honeytrap (which there’d be no reason to do, given his mascot-only status at Tor), everything would have been a lot more cut-and-dried.

Threats that work well in a silo don’t necessarily work so well at scale.

Of course, an actual programmer would know that scaling is hard.

Tomorrow, we’ll explore why that is.

Posted in Uncategorized | 14 Comments

Defcon Is Problematic

Defcon is hosted in a desert. This is exclusionary to people from cold regions, who cannot handle the heat.

Defcon is in Las Vegas. This is exclusionary to people who have gambling problems.

Defcon provides attendees with open bars. This is exclusionary to people who don’t drink.

Defcon wifi will get you hacked. This is exclusionary to technically-ignorant people who don’t understand the risks.

Defcon attendees will try to hack your phone. This is exclusionary to people who can’t afford burner phones.

Defcon ATMs are probably being skimmed. This is exclusionary to people who don’t have credit cards and must use cash to pay for things.

Defcon attendees have poor hygiene. This is exclusionary to people who are sensitive to scents.

Defcon goons walk around with sticks of deodorant to give to people who have poor hygiene. This is exclusionary to people who are sensitive to scents.

Many Defcon events are put on by, and marketed to, specific groups. This is exclusionary to people attending alone.

People attending alone can get in to events through social engineering. This is exclusionary to people with poor social skills.

Defcon is attended by 15,000 people. This is exclusionary to people overwhelmed by crowds.

Many employers pay for people to attend Defcon. This is exclusionary to people who work at startups with limited budgets.

Defcon is in North America. This is exclusionary to people who live in South America, Europe, Asia, Africa, Australia, and Antarctica.

The food in Las Vegas is fairly generic. This is exclusionary to people with dietary concerns such as vegetarianism or gluten free.

People accuse Defcon of being hostile to newcomers, and demand changes to make it more welcoming. This is exclusionary to the old-timers who made it what it is, and just want to have a good time with their good friends.

At Defcon, white men try to tell women what to think about the events that are put on. This is sexist and exclusionary to women.

Posted in Uncategorized | 2 Comments

How to Reformat Reality

We hear it every day: the future is here. If you didn’t think so last month — if you somehow thought we had more time before the Wired became real — take a look outside. Somewhere between 10 and 50 million humans have casually submerged themselves in augmented reality. We blinked and missed it. The individual pieces of tech had only just emerged when suddenly, they fused in a blinding instant.  It has happened before. It will happen again. It is only a matter of time before the next Big Shift blurs the boundary of the real even further. Meatspace and bitspace are converging, but you don’t need me to tell you that. 

You need me to tell you that you have been deceived.

History and technology have an uneasy relationship. Each one likes to change their partner, seemingly on a whim. Better minds than I have noticed this phenomenon:

Major technological possibilities, once uncovered, are invariably exploited in ways that maximally unleash their potential.

With major technologies, it becomes clear early on that the global impact is going to be of a certain magnitude and cause a corresponding amount of disruptive societal change.

In the broadest sense, tech enables history to change faster, or forces change to happen. But don’t take my word for it: just ask Martin Luther. The printing press ignited the Protestant Reformation and laid the foundations for the Renaissance by disrupting the Catholic Church’s stranglehold on culture and power. Ask the anonymous leaker of the Panama Papers, or Ed████ Sno████. The world will feel the ripples of their legacy forever. After all, the new PR stunt in Silicon Valley is to make efforts to eliminate unnecessary data collection on people who use your apps. (Unless, of course, you’re Big Zuck. But then again, does opium need PR? And do Facebook users count as people?)

Define the status quo of any time period as (a) the organization of power structures and (b) the dominant culture of that period, i.e. the zeitgeist. The Catholic Church, in medieval Europe, served both purposes. Today, the American government also plays both roles: starting with the world’s largest military, it co-opted public schools and media to attain near-complete dominance of both hearts and minds.

Organizations of this kind all but dictate their contemporary history. However, as we learned throughout the 20th century, stability is an equilibrium: if either the power structure or the zeitgeist of a nation experiences radical change, the other must change in turn, or be replaced — reformatted — by a superior equilibrium of power and culture.

You may ask: in the current year, isn’t it true we have more tech than ever? We should expect governments to adopt and enhance tech to efficiently maintain their power. Doesn’t that mean we would see fewer disruptions? Good question. After all, in Luther’s time, the power-to-culture ratio was weak. The Church could not send a cloud of drones to the Middle East. The Pope did not have eyes on every street, nor ears in every pocket. The priests had to rely on charisma and group cohesion to maintain the Pontifex. Small wonder Martin only needed a newspaper to unseat the old coot. The Panama leak, on the other hand, challenged a far more powerful tessellation of organizations, as did Mr. S██████. How did these two nail their ninety-five theses?

The answer: scalable tech is unimaginably powerful — yes, even more so than any modern government. You can set fire to a book and erase the work of a lifetime. You can duplicate a .pdf scan of a textbook in less than a minute. You can torrent an ebook in less than a second. The internet is the next best thing to limitless information, and by proxy, limitless power — or, more precisely, limitless culture. Today, the internet is the door on the community church. Everyone attends, and everyone sees the signs nailed thereupon. Yet if a billion signs upon a billion doors cannot disrupt today’s ruling power, today’s ruling power must be powerful indeed. Something is out of balance.

But where do you come into this?

You are a programmer. (If you’re not, you can be.) You pilot the machines that control the most complex system ever constructed by humankind. If you are employed, you use this power to do what a company tells you for around $30 an hour. This is a pretty safe bet. It pays the bills, and you sleep in your own bed. No one can blame you for following this path.

But if you think this is the only path, you have been deceived.

I am not from the Bay Area, although I have witnessed it. “Change the world,” the Bay choir sings, “by increasing advertising impressions.” Where I come from, that is what’s known as false advertising. Don’t wait up for Yelp to build factories that build factories.

If you want to change the world, study those who actually have.

Impactor Catalyst Technology
Martin Luther 95 Theses printing press
John D. Rockefeller Standard Oil industrial capitalism
Gavrilo Princip Assassinate Franz Ferdinand premodern geopolitics
Bill Gates, Steve Jobs Microsoft & Apple personal computing
Satoshi Nakamoto Bitcoin cryptocurrency * internet
Peter Thiel, Elon Musk Paypal, Founders Fund, Tesla, SpaceX, et al venture capital * internet
Mark Zuckerberg Facebook social media * internet

Each of these impactors catalyzed change with scalable technology. Martin Luther spread free thinking with the printing press, and killed the authority of the Church. Rockefeller saw the industrial Matrix, and simply bent the spoon. Princip believed he was fighting for his country, when in fact he and his friends just removed the a card at the base of the house. Gates and Jobs brought computers into every home and hand. Satoshi made money 100x faster and available everywhere. Thiel and Musk are geniuses at creating and capitalizing on massive opportunities. Zucky struck lucky, and made big friends.

What else did these humans have in common? Each of them was only one person. Somehow, the efforts of a single human brain were able to fundamentally restructure a huge part of the world. It follows, then, that at any given time, the world must have one or more human-brain-sized points of extreme sensitivity. For consistency’s sake, let’s call these points nodes. A node can be a city councilmember, a small business, or a community of artists. If it guides people to act in some way — that is, if it has the power to influence individuals — it is a node. If a node is sensitive enough to be affected by the actions of a single human, then it is “human-sized”. Each node can connect to any other node, and the sum of all nodes is the complete set of power forces that act on humankind. Sketching a comprehensive diagram is left as an exercise for the reader.

Certain nodes, by their nature, are exquisitely well-connected. Other nodes are overlooked by everyone around them, simply because of how hard they are to find. The most important nodes are both. Did you know that you can distribute 30,000 copies of whatever you want, wherever you want, with 10 lines of shell script? I didn’t. At least, not before somebody found and poked the node labeled <printer network>. Suddenly, the power flowed from <printers>, to <offices>, and downstream to <the media>, and finally to the minds of millions. Weev’s message notwithstanding, one cannot but acknowledge his efficiency. The <printer network> sat there for years before anyone poked it as hard as he did. The DAO’s access point was a little harder to find, but its bug bounty – a cool $40 million – was certainly worth the wade through Solidity. And <augmented reality> only needed an intellectual property from Nintendo to blast off in a matter of days.

Even the most well-guarded node structures that form the Power half of the status quo can be accessed and influenced in this way. The nodes labeled <state secrets> and <surveillance tyranny> happened to fall within reach of Ed████ Sno████. He didn’t plan to have access to these things, but because he did, he could act upon them. The Agency betrayed the trust of Americans, so Ed simply returned the favor. The resulting revelations shook the world’s foundations.

Hell, <Franz Ferdinand> parked right in front of Gavrilo Princip in an open-roof carriage. Gavrilo thought the assassination would avenge his people, but the fallout he precipitated would slaughter two generations of European children and bring the continent to its knees, while America rose to prominence on the utility bills. In terms of preventing downstream tragedies with a time machine, I would stop Princip before I stopped Hitler.

My point is, you don’t have to be a genius to reformat reality. (It helps, and if you don’t quite know what you’re doing there will be unintended consequences, but it’s not strictly necessary.) You simply need to apply well-directed pressure at a well-connected node. Take a look around. Determine which nodes are important. Determine the best action. Act.

The actors in the table above did not exactly “get away clean”. There are always unintended consequences to reformatting reality. However, it is impossible to blame (or credit) an impactor for the eventual consequences of an immediate action. The node-web is fractal and infinite, and the cascade of power is nigh unpredictable. A small twitch here could spiral out of control in any of a million directions. As such, one cannot fault an insurrectionist for having a poor model of the world. After all, if their model were more accurate, they wouldn’t get to have all the fun that a rioting revolutionary gets to have with other people’s property — they would have to settle for a more “standard” occupation, like sitting at a computer terminal and commanding the most powerful network in human history. You can’t make any money doing boring stuff like reading code. Code monkeys don’t get laid.

It is difficult to get a man to understand something when his salary depends upon his not understanding it.

For all of his ideological preoccupations, Sinclair knew what it was like to swim upstream. And although Michael Huemer misses the mark when he advocates passivity (let’s be precise: the world is undeniably better when most people take his advice, but you and I are different), he got the nature of the matter dead-on:

When one lacks a precise and detailed understanding of a complex system, any attempt to radically improve that system is more likely to disrupt the things that are working well than it is to repair the system’s imperfections.

However, there is a silver lining to this unknowable cloud of complexity. The world is shaped every day by advancements in technology. As tech increases in complexity and scale, the importance of tech grows, and so does its influence on the world. When technology defines reality, who understands technology understands reality.

What is a programmer if not an expert with a precise and detailed understanding of the software that runs our world, with its countless pulsing nodes? If we can see the Matrix, nothing can stop us from bending the spoon.

And if we don’t reformat reality, we are responsible for what happens to it.

Posted in Uncategorized | 10 Comments

You Come At The King, You’d Best Not Miss

Lambdaconf‘s got me thinking a lot about what the purpose of a conference is. I think people have a variety of reasons for attending. Some go to network, find jobs or employees. Some go to socialize; conferences are good Schelling points for meeting other nerds. Lots of people go to learn. At least, that’s what they tell their manager when filing the expense report. More than a fair share go just to have an excuse to travel. And, as I’ve recently discovered, some go to parasitize (symbiotically, of course) and run their own conference-in-the-conference.

Conference organizers have the unenviable task of balancing these competing priorities and trying to accommodate all of them. This is hard. I would say it’s impossible, but Mammon provides. The only cost is, er, cost; we could accommodate everyone if we had infinite money.

But we don’t. And so organizers prioritize. In my experience, they primarily prioritize learning. It’s the easiest way to get the deeper pockets to pay for it, after all. And I think this is part of why political drama has been showing up more and more. Well-meaning individuals think that other things should be prioritized, and advocate for this in every way they can.

This hit me pretty explicitly while reading through some of the conference info. Thanks to the generosity of all of you fine folks, the organizers have had lots and lots of resources with which to accommodate different priorities. They advertise child care, ample diversity scholarships, all of those things that justice-minded folks ask for. I had the contrarian thought: why should we have to do this?

To many people, even anti-SJW folks, the answer is obvious: some people have valuable contributions to make, but can’t because the conference is inaccessible to them. But this does not suggest that diversity scholarships are the answer. If the purpose of the conference is to facilitate learning and creation, then giving out scholarships on the basis of need, instead of merit, is counterproductive. Why should we do it?

I think that the conflict between the SJWs and the rest of us is a conflict of worldviews. For many, the priority is just to do their jobs, build cool things, and exercise their brains. For SJWs, this is not the priority. It’s the secondary benefit. The priority is, well, it’s in the name: Social Justice. Excluding people for any reason, but especially for their socioeconomic class, is unjust. Justice, being their priority, drives them to push for this change. Not against people who oppose justice. But against people who have other priorities. This might sound cold and heartless, but as Scott laid out, if you don’t accept some ethical behaviours as supererogatory, you’ll be paralyzed by the sheer mass of injustice present in a population of 7 billion people.

In their advocacy of justice priorities over others, they pressure conference organizers to make substantial accommodations in order for the underprivileged people to have a seat at the table. To them, this is reasonable, because seat-at-the-table is the goal. To the organizers, this is frustrating and nonsensical, because the goal is learning. They don’t like having to deal with something irrelevant to their priorities.

All of the above is a build-up to a dumb joke. SJ techies want leaders and organizers to go out of their way, spending time and money, in order to bring a group of outsiders into a community dedicated to learning. They want us to do all the work. That’s strange, though, because I thought it’s not my job to educate you. 

You come at someone, you’d best not miss; enemies can pick up dropped weapons.


A long time ago, Scott wrote a blog post about superweapons, and the dangers associated with them. He focuses on the culture war aspect. I think there is a more general principle here, and one that could use a highlight. Since social justice activism is a touchy subject, I’m going to go for a more neutral subject.

As a Canadian, seeing the 2016 election from the outside, people’s reactions to Trump confuse me. Especially as someone who appreciates well-designed systems, I can’t believe people’s gross ignorance of their own nation. People are so afraid of the terrible things Trump will do that protests like this happen. And yet, the vast majority of things people are afraid of are things he can’t do. Was I the only person who paid attention in civics class?

The US was founded as a nation as a response to an uprising against an autocrat. Its founders were horrified at the potential for another such autocrat to arise, and they designed their government accordingly. There was to be a strict separation of powers, with mutually opposed groups keeping each other in check. Most importantly, the office of the executive was intentionally crippled. The president was supposed to have very little power. The founders thought that mitigating potential bad leaders was more important than empowering potential good leaders.

So if Trump can’t do these bad things, what’s the problem? Well, the theory that the country was based on is solid. But you know what they say: In theory, there’s no difference between theory and practice. In practice, there is. Perfect, beautiful ideas never survive implementation. In this case, there are no backwards arcs in the state machine.

On paper, Trump can’t do anything too bad. In practice, he can, because previous presidents have set the precedent. People like to make fun of small-c conservatives who want government out of their lives. Libertarians are a favourite scapegoat online, for similar reasons. Every time a president said “we need the power to do X”, a libertarian said “no, we can’t let you do that; your powers are restricted for a reason.” In the case of, say, Obamacare, we looked at the libertarians and said “why do you hate poor people? Why do you want them to die? Can you be so heartless? Can’t you make an exception this one time?” You should have listened to them, in detail. Once a proof of concept is committed to master, it is the new feature. “One” time never is.

Over time, various factions have engaged in special pleading. “We need this superweapon, just this one time. Can’t you see the challenge we’re facing? Are you really going to demand principles when people are suffering?” The same argument turned Rome into a dictatorship, millennia ago. When you shoot your superweapon at the king, you’d best not miss. He can pick it up from your fallen comrades.


When we at Status 451 talk about libertarianism and anarchy, this is why. We recognize that the long-tail risk of a bad leader is a much, much greater cost than the benefit of a good leader, and seek to design the system to be robust in the face of such a threat. It’s an attitude that falls out pretty easily from systems-thinking backgrounds like engineering and software.

But when we talk about this, when anyone talks about this, there’s a subtle distinction to be made. There are two parallel systems being designed. One is the formal system, the law. The rules we draft regarding what people can and cannot do. This is the theory.

The other is the informal system, the social norms. These are the implicit rules. These are not what we permit or deny. These are what we tolerate or reject. This is the practice.

In many arguments, people will play one against the other, identifying discrepancies and inconsistencies in people’s positions on these subjects. Weaponized equivocation can be very useful, and hypocrisy is a damning accusation. I try very hard to be consistent. Above is a legal, formal, theoretical example. Below is the practice.

Recently, the New York Times revealed that Peter Thiel is Batman. It turns out, this Silicon Valley billionaire has been pursuing retribution against Gawker, ever since they targetted him. Since this dropped, many people have been highly critical. This is billionaires blatantly leveraging our legal system for personal vendettas, they say. We can’t let these plutocrats corrupt our society like this.

They only say this because they dropped the superweapon when they started to retreat.

The superweapon, counterintuitively, is not third party litigation financing. No, the superweapon is social in nature. The superweapon is the one that Gawker drew first. It is the superweapon of fighting in the culture wars By Any Means Necessary. It’s the superweapon of people willing to fight dirty against their identified enemies.

Gawker even admitted as much. They claimed, publicly, that they did this because they felt Thiel to be homophobic (so much for ‘oppressed people can’t be X-ist’), and that he deserved to be attacked. They chose to fight dirty. They took a private, intimate detail of someone’s life, and used it for their own agenda against his consent.

They chose to burn civility and common decency to score some points. Now, that floodgate is open. They’ve dropped that weapon, and now they’re in its sights.

The alternative is demilitarization, dismantling of the existing superweapons and an embargo on building more. This seems an unpopular position. People, so sure of their own rightness, stock the armory and prepare to go to war. They load, aim, fire…. and miss. And then it’s too late; the weapon is out there.

You all should have listened to us when you had the chance.

Posted in Uncategorized | 12 Comments

Miller’s Law in the Archipelago of Weird

How big is your filter bubble? bubbleWhat’s in it? What’s outside it?

Okay, next question: how can you tell?

In 2011, Eli Pariser defined a filter bubble as “that personal ecosystem of information that’s been catered by algorithms.” Unfortunately, he never defined a measure on filter bubbles. Like its name implies, a measure is a way to tell how big something is. Without one, we can talk qualitatively about filter bubbles, but if we want to get quantitative, we’re going to need to do some more work. We’ll start with a literature search.

Let’s assume that the continuum of human experience is a continuum in the mathematical sense. (Whether this is actually true has some interesting philosophical implications. If it is instead discrete, the analysis below is overkill, but still works.)

In the last few years, researchers have studied the frequency with which Facebook and web users, in aggregate, encounter news and opinions from a different ideological perspective. This is interesting, but doesn’t tell us anything about the size or interior of an individual user’s bubble. Dillahunt et al. 2015 defines a distance metric between search engine users based on the overlap between their personalized search results, but a metric only tells you how far apart two things are. If you want to know how much space a bubble takes up, you need a measure.

Nguyen et al. 2014 creates a similarity metric for movies from the Euclidean distance between vectors of descriptive tags, one vector per movie. The authors use this metric to evaluate a system that recommends movies based on titles a user has already rated. They take the top 15 recommendations for a user, compute their mean pairwise distance, and call that the content diversity of those recommendations. This still only tells us something analogous to the diameter of a filter bubble, though, and then only if you tilt your head and squint. However, because the set of all content that has ever been or ever will be created (and, thus, the set of topics that content has ever been or ever will be about) is finite (if unbounded), trivially a filter bubble in the real world is also a measure space. It’s not immediately clear to me whether there are any measures that tell us anything interesting about a filter bubble’s volume, but the Dirac measure δx(A) answers the question “is x inside or outside A’s filter bubble?” with 1 for “inside” and 0 for “outside.”

Facebook’s study found less of a difference between which news liberals see and which news conservatives see than Pariser expected. However, the researchers only looked at the liberal/conservative valence of articles, which they determined based on how the people who shared an article self-identified politically. They did not examine or identify the topics of articles. There’s been some work on topic identification in group filter bubbles, though, which still makes sense with our notion of filter bubbles as measure spaces. Here’s how.

Let T be all the topics a group discusses. The power set of T is all the possible subsets of T, so elements in the power set are also sets of topics. If each member of a group is only interested in some subset of the topics the overall group talks about, that means we can map each member of the group onto subsets of T. The resulting collection of subsets is a  σ-algebra, and along with T, it defines a measurable space: the topics in each member’s individual filter bubble constitute a measurable set which is an element of the power set of the overall set of topics. With our Dirac measure, the measurable space is also a measure space. (Mathematics nomenclature: not fucking helping since sometime before the 19th century.)

However, I have yet to see anyone attempt to empirically characterize topic filter bubbles outside of small political niches. What does it look like when, instead of along some increasingly meaningless two-party axis, you characterize groups by the topics they cluster around?

Well, it probably looks a little something like a continent and a bunch of islands:

Culture, in the part of the world in which I’ve been, and, for all I know, in other parts as well to which I cannot speak, has two rough parts: the Mainland and the Isles.

The Mainland is what calls itself the “mainstream” or “normal” culture.

You know… Mundania.

The Isles are everything else. Everything that’s not “mainstream” is an island.

Nobody knows how many Isles there are. They are wholly and utterly unmapped. Each one is its own subculture.

Some Isles are closer to the Mainland, and some further.

Some Isles are closer to others. Some are big. Some are small.

We — meaning I and a very large percentage of my readership — live in a collection of close Isles which form up an Archipelago. The SCA. Fandom. NERO. Etc.

This is the Archipelago of Weird.

No points for guessing I’m an Islander, obviously. I work on the Mainland, and I have acquaintances there, but I do my research and pretty much everything else that matters in the Isles. The Mainland can be a nice place to visit, but I have to speak Mainlander there, because they don’t understand my jargon. This has its annoying aspects, but also a tradeoff that’s sometimes useful: Mainlanders know nothing whatsoever about Island customs, history, or aspirations, nor do they care to. (“I work in computers.” “Oh! That must be interesting.”) The Isles can be fractious places, and from time to time it’s nice to evacuate away from the latest hostilities. Intra-Isle conflicts are the most exhausting, because they split a population into two or more groups, each with its own native shibboleths and taboos. Keeping track of norm proliferation takes a lot of attention.

Mainlanders, generally unaware that the Isles even exist, know nothing of Island norms either. And Mainlanders get really weirded out when Islanders try to explain them. Mainlanders dont want to understand Islander inside baseball. Hell, most Islanders don’t want to understand inside baseball from other islands, unless they’re like me and have a summer home on the Isle of Inside Baseball Dissectors. A Mainlander and someone from the Isle of Fen might both listen politely to an explanation of why the Burr-Feinstein encryption bill is a terrible idea, but neither one has any reason to be particularly interested, so if there’s something I need to get across to them, I’d better be entertaining first and informative second. Burr-Feinstein is inside baseball for someone from the Isle of Infosec, and technically so is the Wassenaar Arrangement, but international trade agreements are Isle of Policy inside baseball. δWassenaar(A) is almost certainly 0 for any Mainlander A, but it could be 1 or 0 even on my native island.

Being able to estimate δx(A) for many different topics (x) and people (A) is an incredibly useful skill for anyone who does much Isle-hopping, or anyone who travels between the Isles and the Mainland with any frequency. Being bad at estimating δx(A) is a recipe for awkward interactions at best, and interpersonal conflict at worst. Mainlanders who can’t estimate Dirac measures for Islanders can still usually get by if they find themselves on an Isle by mistake, depending on how amenable the local Islanders are to humoring foreigners. (Do not expect to be humored on the Isle of Outlaw Bikers.) Islanders who can’t estimate Dirac measures for Mainlanders are kind of screwed on the Mainland, especially if they don’t realize how far away the Mainland filter bubble is from their own. Double especially if they expect the Mainland filter bubble to overlap one-to-one, or even significantly, with their own.

This is something I have to keep firmly in mind every single time an allistic person comes to me with well-meaning questions about autism.

In my first post on this blog, I criticized a piece Sumana Harihareswara wrote about eliminating “inessential weirdnesses” in technology because of its inherently ableist stance on inclusion. Proclaiming command line interfaces and non-face-to-face interaction to be of a universally “unwelcoming” nature erases the subset of people for whom those affordances make a space more welcoming, not less, and many of those people are on the autism spectrum. Harihareswara reached out to me and linked me to a revision of the piece that she gave as a talk at LibrePlanet, hoping to “do better in discussion of ableism and competing access needs.” Encouraged by the lure of dialogue, I opened the transcript and started reading …

… only to discover that Inessential Weirdness Number One, LibrePlanet Edition was still “the command line.” The new Number Two was “not using small talk.”

robot-headdesk.gif

From the back, off the floor, through the hoop, nothing but headdesk.

In the OSCON version of the same talk, the “inessential weirdnesses” she digs into are contempt (which I can get behind, though frankly, contempt is a communication antipattern in all communities; tech isn’t special here), lack of consideration for religious observances (ditto) … and the command line. There’s also a laundry list of Inessential Weirdnesses Not Appearing In This Talk, and small talk is still one of them.

Look, I’ll be the first to admit that the Archipelago of the Spectrum is a place most people just don’t visit. Particularly people from the allistic Mainland, unless they work in special education or have a relative who lives there. But when someone claims a position of moral authority on how to be inclusive of people with wildly varying levels of ability — and telling people how they should or shouldn’t behave is inherently claiming a position of moral authority — it is ever so slightly incumbent on that person to do at least the equivalent of visiting Wikitravel beforehand. Here’s what you get when you Google “autism ‘small talk’.” With 138,000 results focused on how much of a challenge small talk presents for autistic people, it’s reasonable to assume there’s at least some there there.

When a dialogue reaches the point of “how could you possibly not know this,” I find it useful to apply Miller’s law:

In order to understand what another person is saying, you must assume that it is true and try to find out what it could be true of.

Harihareswara states her beliefs that not using small talk is “across-the-board inhospitable” and “off-putting to working-class people and women,” and that “some people get socialized” to “perform solo, freestanding utterances” while others don’t. I’m willing to assume that she believes these things. But what would have to be true in order for someone to form such beliefs?

Here is an incomplete list of possibilities, from least to most charitable:

  • She believes that the preferences of allistic people are more important than those of autistic people, and that autistic people should subordinate their needs to the preferences of the allistic majority.
  • She’s never met anyone for whom an expectation of small talk is itself inhospitable.
  • She’s never met anyone for whom freestanding explanations are the most natural way to communicate.
  • No one she’s met has been willing to admit having such uncanny-valley traits to her.
  • The possibility that there are actual people for whom her beliefs about human behavior do not hold did not occur to her.

I have to reject the first hypothesis, because a genuine allistic-supremacist wouldn’t have asked for feedback in the first place. Number four gets a bonus to its likelihood estimation because of the prevalence of the attitude that “indistinguishable from neurotypical is the same as cured” (protip: it isn’t), the stigma that people who are open about their mental health issues face, and how severely underdiagnosed autism is in women. (Here’s a partial taxonomy: white girls are highly sensitive people, white boys are autistic, and black boys have oppositional defiant disorder.)

I don’t have any further data to use in evaluating the likelihood of the other possibilities. One way or another, though, Miller’s Law leads me to conclude that δautism(Sumana) = 0. Whether it’s because she hasn’t gone looking for information or because no one has told her about it, the details of autistic lived experience are outside her filter bubble.

Thus, I went to the Isles for counsel. There’s a little place I like to hang out where all the regulars are autistic women, and all of them except me are feminists. I linked them the transcript of the LibrePlanet talk, and here’s what one of them — who also has some physical disabilities, and from whom I hope to have a guest post soon — had to say.

Look, the tech community is already not very inclusive of me.

It’s not very welcoming to women.

It’s not very welcoming to physically disabled people.

It’s not very welcoming to people with dyscalculia that affects their perception of time.

It’s not very welcoming to people who work best by focusing on one thing to completion or exhaustion instead of interleaving several tasks that must be mentally unpacked, expanded, reabsorbed in both breadth and depth of detail, then switched away from and packed back up again. The type of thinking that gives me wide-ranging expertise — jack of all trades, master of none, better than master of one — makes me nearly unemployable despite my skills and extensive crosstraining.

It’s not very welcoming to people who prefer both breadth and depth of detail instead of only one or the other, in general, come to that.

It’s not very welcoming to people who have suffered short-term memory loss and require, require, a single, assigned, constantly-messy space in which to work by viewing frequent reminders of work-in-progress as it shifts to long-term.

IT IS STILL MORE WELCOMING TO ME THAN MOST PROFESSIONS.

And there are areas where these things are close-to-least true of all the fields I am suited for: I do not have the emotional-labor aptitude for most traditionally-feminine work, I do not have the physical capacity to walk from office to office or lift 30 pounds or sit in the chair assigned to me for hours or perform repetitive arm/hand movements reliably, I do not have the ability to clock in or out on time, I do not have the ability to switch tasks flexibly during the day without excessive mental fatigue and loss of productivity, and I do not have the ability to leave a question unanswered without emotional fatigue and a need for frequent breaks, stimming, or other self-soothing behaviors.

There are areas of tech where these are serious barriers to employment.

There are areas of tech where these things matter less than whether I can pick up a problem, research it in detail, keep it all in my head, keep myself comfortable, and produce a result in shorter order than most people could.

If we’re going to talk about making tech more inclusive, we could talk about physical disability inclusion. We could talk about chronic illness inclusion. We could talk about cognitive disability inclusion. We could talk about neurodiversity inclusion in general.

We could even talk about the self-satisfied but false proclamation that tech is already a meritocracy where none of these things matter so long as I produce a result in shorter order (or of better quality, or both) than most people could, a proclamation that makes any of the other problems more difficult to solve by insisting that they cannot possibly exist, because we have always been a meritocracy (and at war with Eastasia) that would not tolerate such barriers and any such complaints therefore must come from special-snowflake syndrome harbored by incompetents.

We could also talk about the stuff the tech industry does/is/accommodates that still leaves it the closest thing I can have to self-sufficiency and a feeling of personal accomplishment and general utility, and the people who have been not just willing but eager to steer me into the small tech-harbors I’m most compatible with because they get it.

But hey, no, sure, allistic person, let’s talk about this one place you feel locked out of and how we can make it even better for the majority, who already run so many other industries to the exclusion of people like me, first. Let’s make sure the already-privileged majority is comfortable in all places, at all times, before appreciating small pockets of minority safety and accommodation, and asking what they used to do right before they, too, were colonized by the tyranny of the narrowly-defined “default” human being in need of additional comfort while I try to survive. THAT FEELS FUCKING INCLUSIVE TO ME, HELL YEAH.

The tech community is a place where many spectrum-dwellers feel welcome because people don’t expect us to engage in small talk or other social rituals that privilege the desire for a perception of harmony of experience over information sharing. Not everyone whose behavior supports anti-small-talk norms is autistic. In fact, I expect most people whose behavior upholds those norms are not merely not autistic, but have no idea that they’re making the community more welcoming for us. (Thanks for that, by the way, to everyone who does it, whether you knew that was what you were doing or not.)

In other words, anti-small-talk norms are a Chesterton’s fence. Expecting autistic people to get better at small talk in order to make allistics feel more welcome is like expecting people in wheelchairs to get better at walking in order to make physically abled people feel more welcome. It puts all the onus for change on the marginalized population rather than the marginalizing one, and it drives away people for whom the expectation is too much to ask. The fence is there because it protects marginalized people whose presence the community values, and people seeking to tear down the fence have not taken the time to follow Wikipedia’s editorial recommendations for handling that urge:

If you’re considering nominating something for deletion because it doesn’t appear to have any use or purpose, research its history first. You may find out why it was created, and perhaps understand that it still serves a purpose. Or if you do feel the issue it addressed is no longer valid, frame your argument for deletion in such a way that acknowledges that.

Deleting the norm of “small talk isn’t necessary here” and replacing it with a norm of “you must engage in small talk in order to make newcomers feel welcome” excludes those whom the original norm exists to include. For them, the issue the old norm addresses will remain valid no matter how many allistic women get tech jobs. Wouldn’t the truly inclusive solution be to give allistic newcomers more insight about autistic modes of interaction compared to allistic ones, explain how the tech community has evolved over time to include autistic communication styles, and encourage them to set aside their bias against not being small-talked to?

I’d think it would, at least. But this is a lot harder when people don’t even realize they have biases they need to examine, and worse still when the same people consider themselves to be the rightful arbiters of which biases are even worth examining.

As I mentioned earlier, Pariser defined a filter bubble as a personal ecosystem of information catered by algorithms. What this definition obscures, however, is that algorithm is nothing more than a fancy term for process, derived from the name of a 9th-century Persian mathematician. In every single one of those handwringing articles you see about “Are Algorithms Running Our Lives?”, you can safely replace “algorithm” with “process.” Do processes run our lives? Consider how many processes you ran through today on your way to taking out your phone or settling in at your computer, and you tell me. Taking a shower is a process. Making coffee is a process. Riding the bus and driving a car are processes. For that matter, so are the interactions you have with other people, whether you recognize those interactions as processes or not. Other people curate the information that they present to you just as you curate the information you present to them. The only novel purpose that “algorithms” in the handwringing-article sense serve is to remove the constraint of physical distance from the problem of who can curate information for whom. Whether online or in meatspace, there is still some process that filters what information you receive. The only salient difference is the extent to which you can control that process.

“But don’t Facebook’s and Twitter’s algorithms limit what information I see?” Yes, and so do the choices you make in friends. The fact that your friends cater the information that ends up in your filter bubble means that your choices in who to listen to determine whether you’re a Mainlander or an Archipelagian, which inside baseball means something to you and which doesn’t. If your filter bubble contains no outliers, where do you expect to learn that outliers exist, much less what their lives are like? If your goal is to make existing spaces more welcoming to the mainstream, what effect do you think that has on outliers? Especially when only the mainstream gets a say? If your goal is, instead, inclusivity of both mainstream and outlier populations, what actions do you think you could take to learn more about outliers and the Chesterton’s fences they rely on?

I’m leaving those questions open-ended rather than answering them because, frankly, by the norms that Harihareswara claims to support, it’s not my job to answer them. They are, however, absolutely not rhetorical. Mainstream feminism has some serious catching-up to do when it comes to learning about the lives of people who aren’t nice normal middle- to upper-class ladies, not to mention a lot of earned distrust. When you tell people that a skill to which they are inherently maladapted is a new requirement for participating in some culture, you are telling those people that they are no longer welcome in that culture. Bluntly, that is not your decision to make, and people are right not to trust the motivations of anyone who behaves as if they think it is. Too many of us have been burned too many times by people who told us “we want to make this a great place for everyone!”, only to find out that in practice, “everyone” actually means “all the allistics.” If you really do intend to be inclusive, then it’s on you to find ways to discover what marginalizations people experience — including the ones they’ve been socialized into believing it’s shameful to tell you about — and what accommodations they need, because, as the saying goes, intent isn’t magic.

Nor is it lost on me that I am sitting here patiently spergsplaining theory of mind to people who supposedly have it when I supposedly don’t. Allistics can get away with developing a theory of one mind — their own — because they can expect most of the people they interact with to have knowledge, perspectives, and a sensorium not all that different from theirs. Autists don’t get that option. Reaching adulthood, for us, means first learning how to function through a distorted sensorium, then learning to develop a theory of minds, plural, starting with ones different from our own. (Even when it comes to other autists. As the saying goes, if you’ve met one autist, you’ve met one autist.) Developing good Dirac measures for other people’s filter bubbles is an autistic survival skill that allistics can go their entire lives without realizing other people have to acquire.

Except you, dear allistic reader. δpro-small-talk norms marginalize autists(you) = 1, now. What will you do with your newly expanded filter bubble? Think carefully, because your credibility now hinges on it. Granted, that’s credibility among autists and among people who expect others to behave in a manner consistent with the values those others claim to hold, which I’m starting to think may be the exact same set. If you want to prove me wrong, you can start by taking up Miller’s Law yourself: what are the things I’ve explained true of?

Finding an answer to that question is going to require a lot of traveling to unfamiliar islands where the natives may be hostile specifically to people with your neurotype. Understanding why they’re hostile, and why what’s already going right for them is going right, is going to require updating your theory of minds. If you’re not prepared to do that, it won’t be a pleasant trip. If you are, however, then please enjoy your stay in the Archipelago.

Posted in Uncategorized | 21 Comments