When we left off, we had examined the problem of self-organized criticality in social graphs, and were about to tackle the question of whether any more successful individual strategies exist. But before we dive into that, let’s talk about timing. And while we’re at it, let’s clarify something about scope.

The universalizing reflex is difficult to shake. Write about local effects and how they compound into regional ones, about the fact that we can only make decisions about our local behavior rather than deciding what will happen from the top down, and people will still ask you, “Yes, but what should we do at the top?” If you try to universalize local effects, you’ll find yourself trying to comb a sphere unsuccessfully. If frustration entertains you, then by all means enjoy yourself. Just know that you’ll never find a way to comb it flat.

I’m not writing about self-organized criticality in order to justify it. Like gravity, self-organized criticality admits neither justification nor blame. Anything that arises out of local interactions converges into an effect for which any individual actor can easily escape responsibility, and often they do. I’m not describing what I think should happen, merely what already happens. If that disturbs you, you’re not wrong! It disturbs the hell out of me too, especially when the state gets its hands on it. If you want different outcomes, though, you’re going to have to figure out how to get thousands if not millions of people to change their local strategies. This is, frankly, beyond me. The limits of my capacity are to tend my garden alongside others whose strategies are compatible with mine, and ignore the rest unless I have no other option. What I think should happen is only locally relevant. What I think could happen is only slightly less so.

There’s a military term, “operational tempo,” which refers to the overall duty cycle required of equipment and, most importantly, personnel. Maintaining a high operational tempo is a vital component of the sort of “shock and awe” tactics that wear opponents down reliably. Sociopaths know this well; the literature on sociopathy is rife with examples of adversaries setting the operational tempo for their targets. A sociopath who can shower a target with attacks from multiple directions has the opportunity to keep them off balance, in a responsive rather than proactive mode. Push hard enough from enough directions, and possibly the victim even becomes overwhelmed and stops functioning — a distributed denial of service.

However, this begs the question: What if they had a war and only a couple of people showed up?

You hire a lawyer for a legal battle. You hire a publicist for a PR war. But another important aspect of battlefield tactics is terrain. As much as the “digital rights” world — or the tech world in general — can feel like the entire scope of reality from time to time, given how immersive it can be once you’re in it, the rest of the world is quite a bit larger. DailyDot and Buzzfeed were interested in this story because it’s in their wheelhouse as part of the tech press. To the Associated Press, however, this was a brushfire war. And, to be perfectly honest, it is: just one more example of a petty would-be tyrant ejected from his would-be domain, and not a domain the wider world has any meaningful familiarity with, at that. Civil unrest in the Central African Republic is vastly more important, to the average reader, than sociopathy in open-source software; that’s the AP’s take, and I’m inclined to share their perspective.

One of the strategies sociopaths use to keep information silos sturdy is to mislead people about the state of the world outside their domain of influence. The controlling parents, determined to keep their daughter under their thumb, convince her that the only thing men really want is to violate and abandon her. The politician stokes constituents’ antipathy toward the outgroup, whether that’s Muslims or white trash. The cult leader convinces their followers that outsiders simply can’t understand the ways of the enlightened, and that people who express negative sentiments about the group are out to destroy it. The rockstar activist plays on non-rockstars’ fears of organized state opposition to their activism, and convinces non-rockstars that any challenge to the rockstar’s status is evidence of an organized plot against the activist group.

When you’re inside the silo, in other words, the world is small. Not only that, it has externally imposed boundaries. If the whole of your social reality inhabits one strongly-connected cluster, with no weak ties connecting you to “outsider” groups, parrying the slings and arrows of outrageous sociopathy can be the difference between staying connected to the social graph at all and effective ostracism. To arguably-eusocial animals like humans, the threat of isolation is a primal and deep one. But once you’re outside the silo, the threat evaporates, and in its place comes a new superpower: the power of perspective. Once your own dignity is no longer commingled with that of your adversary, you get to write your own criteria for what to dignify with your attention. The perpetrators of sick systems rely on people’s better natures, like loyalty, forgiveness, and a strong work ethic, to keep them coming back after every disappointment. Honor and thoroughness are also on that list. A person who can’t leave an insult unanswered is a person who can be baited, and a person who can be baited is a person who isn’t in control of their own attention. As anyone who’s ever been involved with the raising of a puppy or a grade schooler can confirm, when positive attention isn’t an option, negative attention beats no attention at all, and if an adversary is guiding the direction of your attention, you might as well be back in the silo.

Taking that control back for yourself has an even more important effect, though: it puts the operational tempo back in your hands, too. In the age of hot takes, it’s easy to believe that speed is the most important factor in responding to a reputational assault. However, trying to put this belief into practice is a recipe for burnout. The thing is, it’s easy to believe for the simple reason that so many other people already do. When it seems like everybody’s arguing about you, your instincts tell you to put up a robust defense. Your instincts, as it turns out, are full of shit. Once an avalanche has begun, your voice is no louder than that of any other pebble, and your exit is precarious until the ground settles. Focus your attention on more rewarding priorities, and act when you are ready — and no sooner.

This is actually just one instantiation of a general, lower-coordination-cost sociopath-resistance strategy that is a viable replacement for turtling: setting explicit boundaries and maintaining them. A person who respects a boundary will not cross that boundary. A person who also wants to signal their intent to respect a boundary will also keep a healthy distance back from it. Sociologist Ari Flynn, also a keen observer of abnormal psychology, points out that how a person responds to discovering they’ve crossed a boundary yields considerable information about their attitude toward boundaries in general: an honest person will try to find out how to make it right, while a bad actor will try to make it all about them.

Bad actors also keep trying. To a bad actor, a clearly defined boundary is like the battle of wits Scott Alexander describes as a result of “trying to control AIs through goals plus injunctions” — “Here’s something you want, and here are some rules telling you that you can’t get it. Can you find a loophole in the rules?” If one approach doesn’t work, a clever sociopath will keep coming up with new ones. A mediocre one will try the same approach on someone else, and an incompetent one will try it on people s/he has already tried it on. (I’ve encountered all three kinds.)

Recognizing this in the wild, however, can be hard. In Blind to Betrayal: Why We Fool Ourselves We Aren’t Being Fooled, research psychologist Jennifer Freyd explores the human tendency to systematically ignore mistreatment and treachery — a strategy for short-term self-protection that sets a person up for long-term harm. As Freyd explains:

The core idea [of betrayal trauma theory] is that forgetting and unawareness help the abuse victim survive. The theory draws on two facts about our nature as social beings and our dependence and reliance on others. First, we are extremely vulnerable in infancy, which gives rise to a powerful attachment system. Second, we have a constant need to make “social contracts” with other people in order to get our needs met. This has led to the development of a powerful cheater-detector system. These two aspects of our humanity serve us well, but when the person we are dependent on is also the person betraying us, our two standard responses to trouble conflict with each other.

Freyd focuses on trauma, but this tension also explains why people often write off minor boundary violations. When your cheater-detector system fires, only you know. You then have to decide whether you’re going to do anything about it. Options include confronting the cheater and alerting others about it. Doing something proactive might result in a redrafting of the social contracts that involve you, which is a potential threat to the attachments you rely on. This is especially true for people with an insecure attachment style. A person who has few or no secure attachments thus has an internal disincentive toward acting on their cheater-detector’s signals. For many people, the thought of losing a valued but insecurely-attached relationship is far more daunting than the notion of leaving a boundary violation unaddressed; taking action is scarier than staying still.

Relationships are iterated games, though — and they’re evolutionary. People’s strategies adapt as they learn about how the other players will react. When Mallory the sociopath observes that Alice grins and bears it when Mallory violates her boundaries, Mallory learns that Alice won’t make things difficult for him (or her). Alice also learns from this encounter: she trains herself not to respond when someone defects on her. Thus numbed, the next time Alice and Mallory interact, Mallory can betray her just a little harder, and if Alice sucks it up again, the cycle is poised to continue. Over time, as long as Alice cooperates, Mallory can shift Alice’s Overton window of tolerable behavior to ignore all kinds of abuses.

Given this, it’s tempting to attempt to define a rigid, comprehensive system of standards and defend them against all comers. This is most of what Honeywell proposes in her set of solutions for preventing “rock star” narcissists from taking up all the oxygen in a community. Her recommendations sound like good ideas on the surface. However, any mildly talented sociopath will have no problem end-running around all of them, usually by co-opting or distracting the organization’s leadership. As I’ve said before, sociopath strategies are battle-hardened, and some of them are effective counters to several of Honeywell’s suggestions at once. I’ve condensed these into the table below.

Defense	Counterattack
Have explicit rules for conduct and enforce them for everyone Assume that harassment reports are true and investigate them thoroughly Watch for smaller signs of boundary pushing and react strongly Call people out for monopolizing attention and credit Enforce strict policies around sexual or romantic relationships within power structures	The sociopath “befriends” people with decision-making authority and/or social power. Those people make exceptions for the sociopath: rules turn out not to apply to him/her after all; investigations of the sociopath’s behavior are completely half-assed; people who Matter don’t react to boundary-pushing or spotlight-hogging and thus others conclude they won’t receive social support if they call it out; everyone studiously ignores that the sociopath and X are romantically involved; &c.
Make it easy for victims to find and coordinate with each other	Sociopath gets a “friend” to join the affinity group and report back with information, sow misinformation and distrust, or both.
Build a “deep bench” of talent at every level of your organization Build in checks for “failing up” Distribute the “keys to the kingdom”	Sociopath interferes with HR / hiring / administration, making sure that “random” crises keep them so busy that no one has time to make sure these things are getting done. Sociopath becomes the irreplaceable person.
Flatten the organizational hierarchy as much as possible	Tor’s organizational hierarchy was already flat, but this didn’t help them until Shari Steele came on board. Jake had co-opted leadership so thoroughly that they retaliated against Karen Reilly for reporting his behavior.
Avoid organizations becoming too central to people’s lives	Sociopath slowly inculcates an atmosphere of paranoia: those outside the organization can’t be trusted. Often involves crisis-manufacturing. This one is really easy to pull off when everyone is on Slack or IRC.
Don’t create environments that make boundary violations more likely	Sociopaths can organize these kinds of activities perfectly well on their own.

When I read Honeywell’s piece, I see a valiant effort to help her social-justice activist communities transition from a communal, socialized-mind-oriented mode of organization to a systematic, self-authoring-mind-oriented one. It’s a pity it’s doomed. Making sure that everyone in a group publicly identifies as a feminist, an anti-racist, or any other kind of do-gooder — that everyone sends all the right signals — was never enough to keep sufficiently subtle defectors out. This is the critical failure of the communal mode once any organization gets large enough. It’s great that identity-politics-oriented groups are finally starting to wake up to this fact.

Unfortunately, since sociopathy grew up as part of humanity, that means it evolved right alongside the very same efforts to develop comprehensive social systems that are breaking down on us now. Today’s sociopathy is a sociopathy that has learned to use our systems against us. We can learn to recognize this happening, but in order to do that, we have to be able to step outside the systems that we cherish the most and think about them like an adversary.

For example, it’s easy to think “okay, our group doesn’t like sexual predators, so we’ll ban sexual behavior within the group, and while we’re at it, we’ll also ban alcohol, since drinking impairs people’s decision-making.” On the surface, this sounds likely to be effective: it’s a bright line, right? Remember, though: “Here’s something you want, and here are some rules telling you that you can’t get it. Can you find a loophole in the rules?” Puritanical adherence to an object-level system creates exploitation vectors for bad actors. In an environment where having some trait T is a sin, there’s a strong incentive to appear non-T-like. This gives bad actors a new handle for gaining social control: the threat of impropriety. If everyone in a group is a convincing rumor or a planted bottle away from being ostracized, anyone without a conscience suddenly has an incredibly powerful weapon for undermining or getting rid of people who might inconvenience them by, say, not letting them get their way. It becomes even more powerful in groups where many members have low emotional intelligence, like technical groups. For people who score highly on measures of Machiavellian tendencies, high emotional intelligence is a force multiplier, as they’re able to use their emotional intelligence instrumentally to further their manipulative goals. In a low-emotional-intelligence environment, this is like shooting fish in a barrel.

A sufficiently manipulative person can even convince people to act in ways that betray their own consciences, as happened with the Tor organizer I mentioned before. It’s great to have standards, except when nobody’s willing to act on them. Even when you can’t count on your community to uphold the standards it’s adopted, though — or its members to act on their individual principles — you can always uphold your own.

That’s the grim reality of a world in which we have to trust other people: sometimes they let us down. No matter how watertight an organization’s Code of Conduct, if leadership wimps out on enforcing it — or only enforces it selectively — the code is worth less than the paper it’s written on. As an individual, about the only thing you can do about that is endeavor to spend your time around people with backbones. For all the debate that goes into their wording, rules and laws are abstract things which cannot act on their own. No matter how comprehensive the rules are, people will ultimately do whatever the hell they think they can get away with. Like ants, we operate in concert, but each of us acts alone.

The problem we face, then, is: in the face of Conway’s law and a structure prone to self-organized criticality, can we construct a stigmergy that resists bad actors without the high cost of large avalanches?

Mark Manson has noticed this too, from another direction:

In the attention economy, people are rewarded for extremism. They are rewarded for indulging their worst biases and stoking other people’s worst fears. They are rewarded for portraying the world as a place that is burning to the ground, whether it’s because of gay marriage, or police violence, or Islamic terrorism, or low interest rates. The internet has generated a platform where apocalyptic beliefs are celebrated and spread, and moderation and reason is something that becomes too arduous and boring to stand.

And this constant awareness of every fault and flaw of our humanity, combined with an inundation of doomsayers and narcissistic nihilists commanding our attention space, is what is causing this constant feeling of a chaotic and insecure world that doesn’t actually exist.

He also gets that the criticality is self-organized:

It’s us. We are going crazy. Each one of us, individually, capsized in the flood of negativity, we are ready to burn down the very structures on which the most successful civilizations in human history have been built.

Indulging our worst biases results, predictably, in error due to bias: if we aim at the wrong targets, we will hit the wrong targets. But our models of the world can also suffer from another class of errors: error due to variance. Err too far on the side of variance, and you’ll overfit to the random noise in your training set instead of the signal. Although there is always a tradeoff between bias and variance, the two are partially independent. This means that a model can simultaneously overfit due to hypersensitivity, and underfit due to bad assumptions.

I wrote about this last year in terms of precision and recall, another pair of properties we use to evaluate models in machine learning. As that post describes, the Schroedinger’s Rapist model is a high-bias, high-variance model with no false negatives but a brutally large number of false positives. Its one big advantage, when it comes to the meme’s own evolutionary fitness, is that people who adopt it feel like they have made themselves safer by doing so. “Trust no one of unavoidably broad class X” is another one of those ideas that sounds feasible (if draconian) on the surface — but it’s underspecified. Trust, in practice, is ditransitive: you trust someone with something. When that theme, the thing you’re trusting them with, is underspecified, that’s where a bad actor can nudge you toward redefining your boundaries farther and farther backward. “Trust everyone who signals Y” is equally underspecified, but even worse, because in a world where social media makes long-range (in graph-distance terms, not physical distance) signaling nearly free, a willful liar can find a new sucker every second. “You really think someone would do that? Just go on the internet and tell lies?” Look, if you hadn’t decided Reddit had cooties, you would have incorporated that meme into your thinking a decade ago and we wouldn’t need to have this conversation.

Ever found yourself realizing that things have gone too far, but can’t quite piece together how they got to be so bad? Often that’s the result of not recognizing your own boundaries in the first place. If you haven’t defined them, or are willing to let people get away with infringing on them in the interest of not rocking the boat socially, bad actors are happy to step in and define them — for their benefit, not yours. Sometimes, however, it’s the result of not recognizing boundary-pushing behavior, or not having a model for what that looks like. Like deadlines, a lot of people only notice their own boundaries from the whistling sound they make as they fly by.

I’m not saying never to re-evaluate your boundaries. Rather, never dial them back under duress, or in any other kind of stressful situation, for that matter. Do your reassessing afterward. Boundary-pushing is a dominance game in which merely feeling safe is tantamount to pissing yourself to keep warm. If your goal is to be safe, rather than to feel like everything is fine right up until your house burns down, there are two skills you have to learn. The first is to recognize dominance games in progress, and the second is to either exit or flip the script as the situation and your personal capacities call for it.

“Apply a particular set of object-level boundaries” can’t solve the problem of “people are often bad at holding their personal ground, especially in the moment.” If your boundaries are all object-level, a bad actor has only to set up a forced-error situation by incentivizing you to defend one at the expense of another. If you value your friends, s/he can use them as human shields, involving them such that drawing attention to the sociopath’s behavior brings harm to your friend. If you value an ideology, s/he can use it as a shield, associating him/herself with it so publicly and strongly that people fear that speaking up about the sociopath will “damage the brand.” The foolish man builds his house upon the nouns, and the clever sociopath turns those nouns into the walls of a silo.

The wise man builds his house upon the verbs: the purpose of boundaries is to protect your freedom of action. Action potential, like attention, is a finite resource, and everybody wants yours. Giving it away for free to the outrage of the day leaves you impoverished not only when it comes to local conflicts, but when it comes to tending your own garden. If you’re going to be the change you want to see in the world, you have to pick your battles. If you want to actually see some change, you’re going to have to make it locally.

Scope insensitivity comes into play here, too. We say we’re willing to dedicate value (i.e., pay) to prevent harm, but our instincts for estimating how much harm should correspond to how much value are wildly off. When Desvouges et al asked subjects how much they would spend to prevent migratory birds from drowning in oil-polluted ponds, on average the subjects were willing to dedicate less money to rescuing 20,000 birds ($78) than they were to rescuing 2,000 birds ($80). On a more timely note, when Bloomberg polled 749 likely voters in the 2016 election about the extent to which various actions of Donald Trump’s bothered them — “botheredness” is effectively a proxy variable for gut-check estimate of harm — only 44% were “bothered a lot” by the fraudulent Trump University, and 26% “bothered not at all.” By contrast, 62% found Trump mocking a disabled reporter very bothersome, and only 15% didn’t care. Thousands of little, far-away, invisible people got scammed, yet our instincts tell us an insult to one person we’re able to see is a greater harm. Once again, instinct is full of shit.

There’s no happy ending here. Maybe a few people will read this series and hit upon some local changes they can make to improve the stability of their own environment, but the pessimist in me isn’t about to put money on it. The most local environment is the one inside your own head, and if you’re content to feel like you’re on the right side of history even as it ends around you, nothing I have to say can help you. Satisfaction is itself an attractor, and when we fail to find satisfaction outside ourselves, we retreat to look for it within, even when what we find is nothing more than a tasty ligand. All the while, the sand keeps pouring.

Lots of essays end with an exhortation that some choice is yours. This time it’s even true. You can’t choose universal properties, but you can always choose where to expend your attention and effort. This has always been true, despite all the external demands for your resources. If you want to build something better, look directly around yourself first, and start there.

---

One parting observation:

Nearly everything I’ve said here also applies to the defining egregores of a two-party system.

Pleasant dreams.

---

Works cited and recommended reading:

Freyd, Jennifer, and Pamela Birrell. Blind to Betrayal: Why We Fool Ourselves We Aren’t Being Fooled.

Hintjens, Pieter. The Psychopath Code.

Issendai. “Sick Systems: How to Keep Someone With You Forever” et seq.:

“On Whittling Yourself Away”

“Qualities That Keep You in a Sick System”

McGregor, Jane and Tim. “Empathic people are natural targets for sociopaths — protect yourself.”

McGregor, Jane and Tim. The Empathy Trap: Understanding Antisocial Personalities.

Simon, George. In Sheep’s Clothing: Understanding and Dealing with Manipulative People.

U.S. Department of Defense Standards of Conduct Office. Encyclopedia of Ethical Failure.

Wallisch, Pascal. “Psychopaths in our midst — what you should know.”