There’s a pattern most observers of human interaction have noticed, common enough to have earned its own aphorism: “nice guys finish last.” Or, refactored, “bad actors are unusually good at winning.” The phenomenon shows up in business, in politics, in war, in activism, in religion, in parenting, in nearly every collaborative form of human undertaking. If some cooperative effort generates a valuable resource, tangible or intangible, some people will try to subvert the effort in order to divert more of that resource to themselves. Money, admiration, votes, information, regulatory capacity, credibility, influence, authority: all of these and more are vulnerable to capture.

Social engineering, as a field, thus far has focused primarily on hit-and-run tactics: get in, get information (and/or leave a device behind), get out. Adversaries who adaptively capture value from the organizations with which they involve themselves are subtler and more complex. Noticing them, and responding effectively, requires a different set of skills than realizing that’s not the IT guy on the phone or that a particular email is a phish. Most importantly, it requires learning to identify patterns of behavior over time.

Having recently been adjacent to the sudden publicity of one such pattern of behavior, I have a lot to discuss about the general mechanisms that give rise to both these patterns and the criticality events — the social media jargon is “shitstorms” — they occasionally generate, and also about this specific incident. We’re going to talk about narcissism and its side effects, and how bad actors can damage good organizations. We’re going to talk about how bad things happen to good people, how all kinds of people make bad decisions, and also how organizations live and die. We’re going to talk about self-organized criticality. There will be game theory, and management theory, and inside baseball, and multiple levels from which to view things, and even a diagram or two, so if diagrams aren’t your thing, you might as well bail out now. There will also be some practical advice, toward the end.

But first, let’s talk about David Chapman’s 2015 essay, “Geeks, MOPs, and sociopaths in subculture evolution.”

---

In Chapman’s analysis, a subculture’s growth passes through three phases. First come the geeks, the creators and their True Fans whose interest in a niche topic gets a scene moving. Then come the MOPs, short for “Members Of Public,” looking for entertainment, new experiences, and something cool to be part of. Finally, along come the sociopaths, net extractors of value whose long-term aim is to siphon cultural, social, and liquid capital from the social graph of geeks and MOPs. Sociopaths don’t just take, unless they’re not very good at what they do. Many sociopaths contribute just enough to gain a reputation for being prosocial, and keep their more predatory tendencies hidden until they’ve achieved enough social centrality to be difficult to kick out. It’s a survival strategy with a long pedigree; viruses that burn through their host species reservoir too quickly die off.

Corporations, of course, have their own subcultures, and it’s easy to see this pattern in the origin stories of Silicon Valley success stories like Google — and also those of every failed startup that goes under because somebody embezzled and got away with it. Ditto for nonprofits, activist movements, social networking platforms, and really anything that’s focused on growth. Which is a lot of things, these days.

Organizations have a strong incentive to remove net extractors of value. Would-be net extractors of value, then, have an even stronger incentive to keep themselves connected to the social graph. The plasticity of the human brain being what it is, this sometimes leads to some interesting cognitive innovations.

Narcissism, for example, when it rises to the level of a pathology, is a personality disorder. This is not sufficient, in and of itself, to qualify someone as a sociopath in Chapman’s model. A narcissist who knows what kind of behavior s/he is capable of, keeps capital-siphoning behaviors (like claiming credit for others’ work) in check, and remains a net contributor of value even when that contribution isn’t aligned with his/her personal incentives, is by definition not a sociopath. However, a large social graph can be a tempting source of narcissistic supply, the interpersonal support that feeds a narcissist’s fragile and hungry ego. A narcissist who coerces or cons others into maintaining the “superman” narrative that papers over that damaged ego is a narcissistic sociopath. Other personality disorders can develop in similar ways, such as with borderline sociopaths, who coerce or con others into holding up the black-and-white, good-versus-evil lens through which the borderline sees the world. A mere personal dysfunction, once weaponized, becomes something much larger and more dangerous.

If you’ve ever seen an apparently-thriving group suddenly implode, its members divided over their opinions about one particular person, chances are you’ve seen the end of a sociopath’s run. Last December, progressive PR firm FitzGibbon PR collapsed when it came out that founder Trevor FitzGibbon had a pattern of sexually assaulting and harassing his employees and even some of his firm’s clients. However, the progressivism is what elevates the FitzGibbon story to “man bites dog” levels of notoriety. Everyone loves to watch a hypocrite twist in the wind. Usually one hears about sociopath-driven organizational meltdowns through the grapevine, though, not the media. Fearing repercussions or bad publicity, firms often equivocate about the reasons behind a sudden departure or reorganization. This tendency is understandable from a self-preservation perspective, but it also covers a sociopath’s tracks. Ejected from one firm, a serial net extractor of value can pick right back up at another one. (Indeed, FitzGibbon had been disciplined for harassment at his previous firm.)

Which brings us to the Tor Project.

Tor is an anonymous routing network. Journalists, dissidents, law enforcement, queer people, drug dealers, abuse victims, and many other kinds of people who need privacy send and receive their Internet traffic through Tor’s encryption and routing scheme in order to keep site operators from knowing who and where they are. It’s an intricate system with a lot of moving parts, supported by a foundation that pays its developers through the grant funding it brings in. And about two months ago, Tor’s most visible employee, Jacob “ioerror” Appelbaum, abruptly resigned.

Before coming to Tor, Appelbaum already had a history of value-extracting behavior only occasionally noticeable enough to merit discipline. His 2008 Chaos Communication Congress talk presented, without credit, research that he had wheedled out of Len Sassaman, Dan Kaminsky, and me the previous year. Other researchers, like Travis Goodspeed and Joe Grand, learned the hard way that to work “with” Appelbaum meant to have him put in no effort, but take credit for theirs. As Violet Blue points out, his ragequit from San Francisco porn producer Kink.com followed a flotilla of employee rulebook updates he’d personally inspired.

There’s never a convenient time for a scandal involving a decade-plus of sexual and professional misconduct, and organizational cover-ups thereof, to break. It’s easy to think “oh, I have a lot on my plate right now,” or “oh, it’s not really my problem,” and keep your head down until the chaos subsides. I could have exercised either of those options, or any one of half a dozen others, when Tor announced Appelbaum’s resignation in a one-sentence blog post a week and change before my wedding. But there’s never a good time for a pattern of narcissistic sociopathy to be exposed; there is only too late, or even later. So I got vocal. So did some other folks. And Tor confirmed that Appelbaum had resigned over sexual misconduct. Nick Farr went public about how Appelbaum had stalked and intimidated him at a conference in Hamburg in December 2013. Appelbaum vowed he’d done nothing criminal and threatened legal action, and the media circus was on.

It turns out that when seven pseudonymous people, and a small handful of named ones, speak up in a situation like this one, reporters really, really want to talk to the people with real names attached. At the time, I was in Houston, taking care of final preparations for my June 11th wedding on Orcas Island. I also spent a lot of that time fielding journalists’ questions about things I’d learned from members of the community about Tor’s little “open secret,” about Appelbaum’s plagiarism, and about observing Appelbaum manhandling a woman in a bar from my vantage point about twenty feet away. Then I got on a plane, flew to Seattle, got on a ferry, and didn’t open my laptop until I returned to work the following Monday.

During that period, Appelbaum’s publicist apparently tracked down and released a statement from the woman involved in the bar incident, Jill Bähring. Bähring avers that her interactions with Appelbaum were entirely consensual, which I am relieved and pleased to hear. I’m not sure why anyone would expect any other reaction out of me, seeing as how I’ve sung the praises of making mistakes and owning them in public for so long that I’ve given invited talks on it. The interaction I observed took place within my line of sight but out of my earshot, and if I misinterpreted it, then I genuinely am sorry about that. Ultimately, Bähring makes her own decisions about what she consents to or doesn’t. If I was mistaken, well, good.

But Leigh Honeywell also makes her own decisions about what she consents to or doesn’t, and Karen Reilly likewise. Attacking me over a misinterpretation may be enough to distract some people from the full scope of a situation, but nothing about my error invalidates Honeywell and Reilly’s accounts about their own experiences. Isn’t it interesting that someone whose first public response to allegations of wrongdoing was “I apologize to people I’ve hurt or wronged!” hasn’t had a single word to say to either one in two months? Or to Alison Macrina, or to Isis Lovecruft? It’s as if the allegorical defendant against murder, arson, and jaywalking had no response to the murder or arson counts, but wanted to make damned sure the whole world knew he wasn’t a jaywalker.

*slow clap*

How low-rent of a publicist do you have to hire for them not to be able to keep a story that simple consistent? All Appelbaum had to do was swallow his pride and ask his publicist to bang out an apology of the “I’m sorry you feel that way” variety, and he could have maintained the semblance of high ground he tried to stake out in his initial statement. But the need to be adored — the narcissist’s defining quality, and the sociopath’s first rule of survival — is simply too alluring, the opportunity to gloat over seeing one’s prey stumble too difficult to resist.

Attention is a scarce commodity. What a person expends it on reveals information about that person’s priorities.

Isn’t it interesting when people show you what their preferences really are?

---

But that’s more than enough narcissistic supply for that particular attention junkie. Let’s talk about preference falsification spirals.

Honeywell correctly observes that whisper networks do not transmit information reliably. In her follow-on post, she advocates that communities “encourage and support private affinity groups for marginalized groups.” If this worked, it would be great, but Honeywell conveniently neglects to mention that this solution has its own critical failure mode: what happens to members of marginalized groups whom the existing affinity group considers unpersons? I can tell you, since it happened here: we had to organize on our own. Honeywell’s report came as a surprise to both me and Tor developer Andrea Shepard, because we weren’t part of that whisper network. Nor would we expect to be, given how Honeywell threw Andrea under the bus when Andrea tried to reach out to her for support in the past. If your affinity group refuses to warn or help Certain People who should otherwise definitionally fall under its auspices, then what you’re really saying is “make sure the sociopath rapes an unperson.”

Thanks, but no thanks. Nobody should have to suck up to an incumbent clique in order to learn where the missing stairs are. The truly marginalized are those with no affinity group, no sangha. Who’s supposed to help them?

It’s a tough question, because assessing other people’s preferences from their behavior can be difficult even when they notionally like you. I was surprised, after the news of the extent of Appelbaum’s behavior broke, to learn that several acquaintances whom I had written off as either intentionally or unintentionally enabling him (in the end, it doesn’t really matter which) had actually been warning other people about him for longer than I had. How did I make this mistake?

Well, social cartography is hard. Suppose you’re at an event full of people you kinda-sorta know, and one person who you know is a sociopath. Supposing you decide to stick around, how do you tell who you can trust? Naïvely, anyone who’s obviously buddy-buddy with the sociopath is right out. But what about people who interact with the sociopath’s friends? During my brief and uneventful stint as an international fugitive, both friends and friends-of-friends of the pusbag who was funneling information about me to the prosecution were happy to dump information into that funnel. I learned the hard way that maintaining a cordon sanitaire around a bad actor requires at least two degrees of separation and possibly more. Paranoid? Maybe. But the information leakage stopped. (If I suddenly stopped talking to you sometime in 2009, consider whether you have a friend who is a narc.)

Consider, however, how this plays out in tightly connected groups, where the maximum degree of separation between any two people is, let’s say, three. Suppose that Mallory is a sociopath who has independently harmed both Alice and Bob. Suppose further that Alice and Bob are three degrees of separation from one another, and each has an acquaintance who is friends with Mallory. Let’s call those acquaintances Charlie and Diane, respectively.

If Alice sees Diane and Mallory interacting, and then sees Diane and Bob interacting, the two-degrees-of-separation heuristic discourages Alice from interacting with Bob, since Bob appears to be a friend of a friend of Mallory. Likewise for Bob and Charlie in the equivalent scenario. How can Alice and Bob each find out that the other is also one of Mallory’s victims, and that they could help each other?

In business management, this kind of problem is known as an information silo, and it is a sociopath’s best friend. Lovecruft describes several of Appelbaum’s siloing techniques, such as threatening to smear anyone who spoke out against him as a closet fed. As recently leaked chat logs show, affiliation with intelligence agencies is a genuine hazard for some Tor contributors, which means that “fedjacketing” someone, or convincing others that they’re actually a fed, is an attack which can drive someone out of the community. (Side note: how the hell does urbandictionary.com not have an entry for fedjacketing yet? They have snitch jacket, with which Appelbaum has also threatened people.) But you don’t have to be someone for whom a fedjacketing would be career death for a sociopath to put you in an information silo. Fedjacketing is merely the infosec reductio ad absurdum of the reputational-damage siloing technique. If someone has made it clear to you that they’ll ruin your reputation — or any other part of your life — if you so much as breathe about how they treated you, that’s siloing. Pedophiles do it to children (“don’t tell your mom and dad, or they’ll put us both in jail”), cult leaders do it to their followers — anyone a sociopath can emotionally blackmail, s/he can isolate.

Discussing this with one acquaintance I had misread as an enabler, I asked: what should Alice and Bob do? “When in doubt, it might be a good idea to ask,” they suggested. But this presupposes that either Alice or Bob is insufficiently siloed as to make asking a viable option. My acquaintance also allowed that they had misread still other people for years simply due to not knowing that those people had cut ties with Appelbaum. I didn’t know my acquaintance’s preferences, nor they mine. My acquaintance didn’t know the other people’s preferences, nor vice versa. Because none of us expressed our preferences freely, we all falsified our preferences to one another without trying to, which I’m sure Appelbaum appreciated. People believed they had to play by the standard social rules, and that civility gave him room to maneuver. Once a sociopath achieves social centrality, concealed mistrust creates more information silos than the sociopath could ever create alone.

What else creates information silos? In some cases, the very people who are supposed to be in a position to break them down. Sociopaths don’t only target victims. They also target people in positions of authority, in order to groom them into enablers. This happened at Tor. The “open secret” was so open that, when Appelbaum didn’t show up to a biannual meeting in February and people put up a poster for others to write messages to him, someone wrote, “Thanks for a sexual-assault-free Tor meeting!” This infuriated one of the organizers, who had to be talked down from collecting handwriting samples to identify the writer. At an anonymity project, no less. Talk about not knowing your demographic.

This abandonment of the community’s core values speaks to just how far someone can be groomed away from their own core values. In part, this may have been due to Appelbaum’s dedication to conflating business and personal matters — playing off people’s unwillingness to overlap the two. When this tactic succeeds against a person in a position of organizational power, it incentivizes them to protect their so-called friend, to the overall detriment of the organization.

Friendship is all well and good, right up to the point when it becomes an excuse to abdicate a duty of care. You know, like the one a meeting organizer takes on with respect to every other attendee when they accept the responsibility of organization. If the organizer knew that his “friend” had serious boundary issues, why the hell didn’t he act to protect or at least warn people at the meetings Appelbaum did attend? As enablers, people in positions of authority are a force multiplier for sociopaths. Sociopaths love to recruit them as supporters, much the same as the way a middle-school Queen Bee puts on her most adorable face for the vice-principal. Why put in the effort to threaten victims when your pet authority figure will gladly do it for you? Co-opted authority figures turn preference falsification cascades into full-on waterfalls.

Scott Alexander muses:

I wonder if a good definition for “social cancer” might be any group that breaks the rules of cooperative behavior that bind society together in order to spread more quickly than it could legitimately achieve, and eventually take over the whole social body.

One cell in the body politic mutates, and starts to recruit others. Those recruited cells continue to perform the same functions they always have — building structures, transmitting signals, defending nearby cells — but now they do it in service of that mutant cell line. The tumor is the silo. If you find yourself breaking the rules — or, worse, your rules, your personal ethics — for someone on a regular basis, consider whether that charming friend of yours is inviting you to be part of their tumor.

How do you bust out of a sociopath’s information silo? Personally, I take my cues from Captain James Tiberius Kirk: when the rules are arrayed against you, break them. When a sociopath tries to leave you no “legitimate” maneuvers, Kobayashi Maru that shit as hard as you possibly can.

I also take cues from my husband. TQ has interacted with Appelbaum exactly twice. The first time, Appelbaum physically shoved him out of the way at my late husband Len’s wake in order to stage a dramatic fauxpology for plagiarizing me, Len, and Dan in 2008, begging to “put our differences aside.” (Protip: when someone later tries to shut you up about something they did to you because “we reconciled!”, it wasn’t a real apology in the first place.) The second time, Appelbaum walked up and sat on him.

Appelbaum behaves as if TQ were an object. Operationalizing people — understanding them as a function of what they can do, rather than who they are — is one thing. We’re autistic; we do it all the time. Operationalizing people without concern for their preferences or their bodily integrity is another thing entirely. Since then, with no concern whatsoever for social niceties, any time anyone has brought Appelbaum up in TQ’s presence, he asks, “Why are you giving the time of day to a sociopath?” It isn’t polite, but it sure does break the ice quickly. 

Similarly, a few years ago, Appelbaum applied to speak at a conference TQ and I regularly attend. This conference is neither streamed nor recorded, and speakers are encouraged to present works in progress. The organizer contacted us, unsure how to handle the situation. TQ replied, “I would no more invite a plagiarist to an unfinished-work conference than I would a pedophile to a playground.” The organizer rejected Appelbaum, and the conference went on theft-free. That’s more than a lot of other conference organizers, some of whom knew better, can say.

The one thing that protects sociopaths the most is their victims’ unwillingness to speak up, because the one thing that can hurt a sociopath is having their extraction racket exposed for the fraud it really is. People fear social repercussions for standing up to the Rock Star or the Queen Bee, but consider: if someone is stupid, venal, or corrupt enough to be a sociopath’s enabler, why would you even want to give them any of your social capital in the first place? You might feel like you have to, for the sake of social harmony, or because the subcultural niche that the sociopath has invaded is important to you, or because it’s your workplace and you really need the job. Even sociopaths themselves can experience this pressure. On Quora, diagnosed sociopath Thomas Pierson explains:

Why do [sociopaths] lie and manipulate? Because people punish you when you tell them the truth.

Giving in to fear, to the detriment of those around you, is how you become the bad guy. Lies don’t really protect anyone. They only kick the can down the road, and the reckoning will only be worse when it eventually comes. Suppressing the truth out of fear of being punished is the same as paying the Danegeld out of fear of being overpowered. It’s a form of the sunk cost fallacy, and Kipling had the right of it:

And that is called paying the Dane-geld;
But we’ve proved it again and again,
That if once you have paid him the Dane-geld
You never get rid of the Dane.

Social capital isn’t some magical thing that some people have and others don’t. Like any other form of currency, the locus of its power is in its exchange. (Yeah, we really are all Keynesians now.) In the case of social exchanges, those currencies are information, attention, and affective empathy. Sociopaths try to keep their victims from having relationships the sociopath isn’t involved in, because those are the relationships the sociopath can’t control or collect rent on in the form of secrets or adulation. Building up those relationships — from finding other victims, all the way up to entire parallel social circles where known sociopaths are unwelcome and their enablers receive little to no interaction — incrementally debases the sociopath’s social currency, faster and faster as the graph expands.

Internalizing this grants you a superpower: the power of giving exactly zero fucks. It’s the same power of giving zero fucks that Paulette Perlhach writes about in The Story of a Fuck-Off Fund, only denominated in graph connectivity rather than dollars. It takes the same kind of effort, but it pays off in the same kind of reward. When you give no fucks and tell the truth about a sociopath, two things happen. First, people who have been hurt and haven’t found their superpower yet will come find you. Second, the sociopath starts flailing. (One benefit of being right is that the facts line up on your side.) As accounts of the sociopath’s misdeeds come out, the sociopath’s narrative has to become more and more convoluted in order to keep the fanboys believing. “They’re all feds!” he shrieks. “Every last one of them!”

Uh-huh. Sure. Because the feds always assign multiple agents not only to target one guy who can’t even keep his dick in his pants, but to become his coworkers, don’t they? This is not exactly an inexpensive proposition. Reality check: if the feds had wanted to pull a honeytrap (which there’d be no reason to do, given his mascot-only status at Tor), everything would have been a lot more cut-and-dried.

Threats that work well in a silo don’t necessarily work so well at scale.

Of course, an actual programmer would know that scaling is hard.

Tomorrow, we’ll explore why that is.