Journalistic Innovation

A lot of technological innovation has followed the same pattern.

A given good or service starts its life as Home Production. This is what it sounds like: people making things for themselves, in their homes. This is how virtually all things were created prior to the industrial revolution. The blacksmith had a forge in his house, you went there if you needed a tool. The weaver’s cottage was the cottage they lived in. This is not a very efficient process. It takes tons and tons of time, and tons and tons of resources, but it requires virtually no upfront investment. When a new innovation is born, nobody really knows if it will be valuable, and nobody is willing to invest in it. Interested people engage in home production if they want it. Eventually, if enough people do this, a viable market is formed.

After the innovation grows out of home production, it transitions to Industrial Production. This is the industrial revolution. Mass production, replaceable parts, conveyor belts. Standardized products. The industrialist takes the various quirks from the various home producers, distills them down to a platonic ideal of what that thing is, and efficiently optimizes for producing that at scale. During the process, they learn a lot. What elements can be taken away without affecting the core of the product. What elements need to be added to make it work. Issues with production at scale. Bugs to work out. Everything.

Eventually, through a combination of market saturation and technological progress, the price of this good drops lower and lower and lower. A threshold is hit where it is cheap enough to produce customized goods at scale. The learnings from industrial production can be applied, learning which rules you can break to innovate, learning what fundamentals you need to keep to make it work. Empowered by these learnings, customized goods can be professionally produced for everyone. This is Modern Production.

The invention of bread is probably the best toy example of this. Five hundred years ago, most people baked their own bread, either at their home or via a community bakery. There was no quality control, no standardization. No two loaves were the same. They were expensive and labour intensive to manufacture.

Industrialization led to the invention of sliced bread. Industrial scale food production collected learnings from a large swath of independent bakers, and distilled bread down to it’s platonic ideal: uniformly shaped, sliced, white bread. We produced this at scale, and soon everyone was eating the exact same bread. It wasn’t great. But it was good enough.

Eventually we became so good at producing bread that it became cost-effective to professionally produce a large assortment of different breads. Now, bread lives in the modern production stage, and I can buy a dozen different kinds at Pirate Joe’s, for less than one hour’s minimum wage.

A while back I was listening to (I think) an episode of Planet Money, and they were talking about the history of journalism. They claimed that it arose out of privately funded market research.

In the age of sail, wealthy merchants had business dealings across all the major European capitals, and they made their fortunes on the stock market. Perhaps one businessman wished to stake his fortune on French crepes. As a responsible investor, he wanted to make sure he was making wise investments, so he did market research. This research was often ad hoc, done as it could be. He might pay a traveller headed to Paris to mail him when she got there, telling him the latest Parisian gossip about crepes. In this way, he learned about market trends and was able to make very profitable trades.

These travellers, it was argued, were the first journalists, and their journalism was home production. It was ad hoc, of relatively poor quality, inconsistent, and personal.

Eventually, as society became larger and richer, and more and more people held interest on far off places, these travellers professionalized into what we now know as journalists. They set up standards and practices for how to do journalism “right”. Reporting stopped being a purely regional thing, as large media empires were formed. It lasted this way up until recently, with well understood, precisely-controlled, standardized news delivered to us from the news factories. This was the age of industrial news production. Until the internet happened.

When Twitter, Facebook, and various other social networks popped up, the news industry changed dramatically. By far the biggest difference was a massive reduction in cost. As we’re all journalists now, the marginal cost of producing journalism is zero. Hey look, I’m doing it right here, right now! How much did you all pay me? Exactly.

This marked the transition of news into modern production. We have heard a lot about how print is dying, social media has turned journalism into a slave wage meat grinder, clickbait in service of ads is a necessity. But the flip side, as always: the falling cost of news production makes new doors open. A hundred years of official reporting processes and standards have been understood. We know what parts of news production are essential, what parts can be modified, and what can be thrown away. We know how to make foundations of news cheaply, so we can build all manner of customizations on top. We know how to privately produce customized, professional news.

A friend of mine grokked the implications of this, though he didn’t realize it at first. In the week after the US election, he pointed out something interesting. He said: so campaigns have their own private pollsters, as well as relying on the mainstream media. The mainstream media was disastrously wrong in their predictions. But, presumably, Trump’s pollsters knew what was really going on, since they leveraged it. We can never know this for sure, we can only guess. But consider the full implications of this: One guy, paying for his own private news source, was able to get more truthful, reliable, and actionable news information than the entire journalism industry combined. Modern production.

There are a lot of people who have strong interests in truthful, reliable, actionable news. Per the NPR (iirc) story, it was originally driven by direct economic pressures. As journalism industrialized, plenty of people looked to publications like the New York Times to inform them on very important social, political, and economic concerns. But what has happened now that everyone can commission their own private news?

There’s been a lot of talk about “fake news” recently. The extremely grim implications of the fact that the authorities were able to manufacture consent for severe censorship and propaganda programs over the span of a month, I’ll touch on some other time. Right now, I’m more interested in understanding this phenomenon.

I’ve spoken at length about things like this for years, ever since I found out that journalism is unethical. To me, this is no surprise, but consider why this has happened: people do not care if news is true.

This may seem strange, as everyone thinks they care about truth. The truth is, few do. In most circumstances, truth does not matter. I read a news article about a random pizzeria that won’t cater a gay wedding (I will use this example until I die), it does not matter to me whether that story is true. Why? Because I’ll never know. This story is completely inactionable to me, it’s set in a place I will never be. The only interaction between this story and my life will be when I read it. So if it’s effect on my life is unfalsifiable, is it even valid to call it true at all?

We used to enjoy true, accurate media, because some people relied on that media to inform high-stakes decisions. But if news is now in modern production, if it is viable for people to get individualized professional information on the things they need, then there is no more incentive for the mainstream media to be accurate. The people who care about those things get it elsewhere.

Take me for example. I stopped proactively reading the news about a year ago, checking out specific articles only when they were personally (not algorithmically, not via a share button) sent to me. And yet I am often more informed about current events than the people around me. Why? Twitter and police scanners. I have a police scanner app on my phone, configured to alert me when any scanner in North America sees a sudden spike in traffic. Whenever I get pinged, I know immediately that Something Newsworthy happened. Further, I have a primary source: police dialogue. This signals me to hop on to Twitter, where eyewitness accounts (complete with photos and video) are everywhere. By the time the mainstream news media even realizes there is an Event, I can already write the report.

So the question is: if I don’t need to rely on said media to give me a truthful reporting of the events, if I have better, faster, cheaper, more personalized sources, then why would they bother investing so much effort in true, reliable reporting? They won’t. They’ll give people what they want, and people want panem et circensesNews becomes entertainment, as the pressures for accuracy go elsewhere.

So this is my prediction for the future: “Fake news” is here to stay, and will only get worse. The prestigious mainstream media institutions are on a long, slow decline into irrelevance. Forevermore, news will become less and less relevant. But paradoxically, it will become easier and easier for individuals to get reliable, true, accurate, actionable information in ways that are cheap, easy, and plentiful. It just won’t be through the Cathedral.

I imagine a future where information sources specialize. Instead of going to The News to learn relevant information, I will contact a market research specialist for economic info. A political analyst for political news. A specialist for every source, available to all.

In other words: Wesearchr. It’s gonna be yuge.

Posted in Uncategorized | 16 Comments

Minimum Viable Citizen

I’m a big fan of economic freedom. The freedom to buy and sell what I want, and to not buy and sell what I don’t want. And this is more than just a thinly veiled support of the free market. Private corporations make you buy things you don’t want all the time.

How many of you have television service? You probably pay a cable subscription in exchange for a ton of channels. But how many of those channels do you watch?  If you’re typical, probably not more than a dozen. So how is it fair to make you pay for over nine thousand channels when you only watch a handful? This is called bundling, and the folks over at Marginal Revolution talk about it all the time. Or, more precisely, they talk about unbundling.

See, the main purpose of bundling is to socialize the costs of niche products. So for example, you may only watch prime time mass market comedies on mainstream channels (actual cost: pennies), but you’re still charged the same $80/mo subscription fee as everyone else. Meanwhile, your neighbour might be a massive fan of ESPN (actual cost: a ton in licensing), but unwilling to pay the full cost of subscribing to it. In a world where each of you were charged only for what you use, you would pay a couple bucks a month, and your neighbour a couple hundred. But, of course, in that world, your neighbour would balk, cancel his subscription, and no sports would be watched. Thus, Shaw overcharges you to subsidize your neighbour, everyone gets more TV, and Shaw gets more money.

Economists though, don’t like this. As one of the normative beliefs they whisper between their catechism of impartiality, they’ll tell you over drinks that everything that can be unbundled, should be unbundled. Because, in effect: it’s not fair to make one person pay for someone else’s consumption. In the perfect world of frictionless economics, the market would hit equilibrium where peoples’ willingness to pay matches suppliers’ willingness to produce. S/D curves 101. By unbundling products, allowing each person to mix and match and pay for only what they want, each person has their resources freed up to better achieve their goals and priorities.

A while back, I was on a date with an AnCom activist from California. This was a bad idea, I know, but the spirit was willing and the flesh was weak. We were talking about air travel, as you do, and I mentioned that one of the things I missed about living in the States was cheap air travel. Air travel deregulation in the 70s is one of the greatest victories of free market economics. It’s why flying in the US is so much cheaper than up here, and it opened the door all sorts of other unbundling of the various aspects of flying. I don’t know about you guys, but I love it that wealthy businessmen pay $50 for priority boarding. It makes my flights that much cheaper.

The AnCom saw things differently. She raised as a counter-example Frontier Airlines. Frontier Airlines is the best example of airline unbundling. Their base fares are extremely cheap, but they charge for everything. $25 for a carry-on item ($50 if you don’t pay ahead of time). Selecting a seat at all carries a fee. In-flight drinks are not free. She saw this as a terrible example of an evil corporation squeezing every last profit out of their customers.

I protested. Gripe as you will, Frontier is knocking it out of the park with this. It turns out, given the choice, people would rather have a cheaper flight. Maybe for well compensated software professionals like myself, saving the $25 by not checking a bag doesn’t matter. But for poorer folk, it can make all the difference. To them, $25 can be the difference between flying at all. And insofar as people are voting with their feet, they seem to love it. For all that people complain, as NPR Planet Money reports, Frontier is seeing amazing success. People are freed up to only pay for what they need, and to save money when they don’t. How is this possibly a bad thing?

Her response, in a nutshell: This is predatory. People are lazy. People are stupid. They see the base fare and assume that’s the cost. They don’t bother to (for example) notice the $25 carry-on fee. And then when they show up at the gate, they’re not only charged a fee, but slapped with a “penalty” (Total fee $50) for their ignorance. This is grossly unethical and must be stopped.

This is an interesting complaint. On the one hand, I am objectively correct. Frontier Airlines is the fastest growing airline. Ignore the complaints; people demonstrably value this pricing structure. But she has a valid point as well. There are no doubt people getting burned by this, and since they’re people flying Frontier and not Virgin, it’s a safe assumption that they’re not likely in the best position to afford it. I thought on this, and I came to a question.

Grant for the sake of argument that this is unethical. Is there anything that could be done that would allow Frontier to keep this pricing structure, while not being unethical? Could they, say, communicate this better upfront? Do the Uber surge price thing and make you confirm that you accept it? Are there certain things for which this is intrinsically unethical, and others for which it’s not? What is the specific threshold at which this practice becomes unethical?

Of course, because she’s an AnCom, her answer is somewhere in the realm of “making me pay for a flight at all is wage slavery”, and so that’s not a fruitful line of inquiry. But lets prax this out. What is the root cause of the problem? The root cause of the ethical problem is that, uncharitably paraphrasing her comments, people are too stupid to reason through this. The corporation on some level knows this, and takes advantage of it.

The customers are stupid, and therefore the corporation is responsible for accommodating them. It sounds strange when phrased this way, but I suspect that most peoples’ instincts fall along these lines. But… why? We take it for granted that a corporation has an ethical responsibility along these lines towards the general public. But what is the general public’s ethical responsibility in return? What level of competency is it ethically acceptable for a corporation to expect from the public? What are the requirements for a minimum viable citizen?

This rarely gets discussed. When it does, it’s usually assumed as a background fact and taken for granted. Progressives and left-wing reformers generally assume that everyone is helpless, and push to accommodate the lowest possible common denominator. Libertarians like to hand-wave it all away with “voluntary exchange”, pushing 100% of the personal responsibility onto the individual. What is the “correct” value? I don’t know, and I’m interested in hearing your accounts of this. But I think it’s critically important that this discussion gets had.

We live in a world where technology is growing at an unimaginable pace. Things that seemed miraculous a decade ago are trivially commonplace now. But as technology progresses, our society becomes more complex. The cost of using these technologies is rarely zero. They all depend on some element of learning. They all hold some basic expectations on their users.

Some people, like the AnCom, find these expectations intrinsically unethical. But along that path lies ruin. Taken to the extreme, that reasoning would suggest that cars should be banned, because it’s too hard for horse riders to learn how a steering wheel works. That would be nuts; clearly the invention of cars has been an enormous boon for humanity. And yet… there are no doubt some people out there for whom driving is too challenging, and a country that expects people to have the ability to drive, is a country in which they cannot be first-class citizens.

But on the flip side, expectations of arbitrary responsibility don’t work either. Back when I lived in San Francisco, it was impossible to find housing. What I, and many other tech people did, was script Craigslist. I had a recipe on IFTTT set up. Every fifteen minutes it would scan Craigslist for new rentals in my desired neighbourhoods and price ranges. It would automatically reply to them with a brief introductory email, and CC me so I could review them and follow up on the promising ones. In this market, this was a necessity. And yet, a market that expects this behaviour of its participants is a market where the vast majority of humanity is excluded by default.

As technology progresses, it will entail ever-rising requirements for people to make use of this technology. Some people will be able to. Some people will not. If we sit here and do nothing, sticking our fingers in our ears, inequality will increase, as the people who can handle the vagueries of Frontier-style pricing economically dominate the people who can’t read the fine print. If we insist on 100% accessibility and inclusion for everyone, we’ll end up with a stagnant society that can never innovate.

If we spoke openly about this, we could discuss the merits and drawbacks, balance the tradeoffs, and set a standard. We could then innovate with that expectation, freeing us from having to worry about every corner case. And we could work to prepare people for that, holding standards for them and supporting them in the process of achieving those standards. But the closest thing to a discussion of this that I’ve ever seen is David Chapman’s Meaningness blog, where he talks about the critical task of leveling people up to a stage 5 mentality. Unfortunately, the only people familiar with his work are people like us, the people who don’t need it.

So what makes for a minimum viable citizen? What expectations of peoples’ ability is it reasonable to hold? Are people even able to rise to these challenges? How can we support them in it if they can?

Posted in Uncategorized | 19 Comments

It’s Protein World, We Just Live In It

I didn’t have a lot to say about the election this year. To be perfectly level with you, this was largely because my friend Zooko precommitted to unfollowing anyone who was tweeting about it, and looking back from late November, I think he kind of had a point. I’d remarked on Twitter, well before the primaries, that the breakneck pace of overwhelming demands for attention was going to exhaust people. (“You can’t fatigue the Trumpenkrieg,” shot back Andrew Auernheimer. Good ol’ weev.) In private conversations, I ruefully predicted another Brexit, though apparently not with enough conviction to have an analysis of my own prepared. (I’ve been writing other things, but most of them have been code.)

Even now, about all I have to say is that clearly nobody, least of all the media, learned anything whatsoever from Protein World.

(“What the hell is Protein World?”)

Of course you don’t remember Protein World. That was April 2015, nobody remembers that kind of ancient history! So. Protein World is an online store that sells weight loss products and employs some of the most deviously brilliant marketers ever born. Their UK advertising campaign for spring 2015 involved purchasing these ads:


on the London Tube, sitting back, and waiting for knee-jerk anger to propel their graphic design in front of way more eyeballs than a £250,000 ad purchase ever could have done by itself. Needless to say, it worked. Never mind that two weeks into the ad’s three-week run, regulators pulled it; those two weeks of Twitter outrage and petition-shilling drove £1 million of direct sales Protein World’s way. The grapes of “They’ll win no awards for this” are extra sour when your “they” has already laughed its way to the bank.

(I pause to reflect, again, on social media’s own peculiar brand of amnesia. Flash outrages, though they may seem omnipresent from within the bubbles whose joint attention they consume, are often not that widespread in the broader scheme of things. What it is easy to excuse having missed, it is also easy to excuse forgetting about. From a more Huxleyan point of view, there’s no need to construct a mandatory memory hole when people are easily incentivized to build their own.)

Now, this may be a glass sword effect, extremely effective once but minimally so thereafter. When Protein World expanded their subway campaign to New York City as May 2015 ticked over into June, search traffic maybe rose slightly, but not distinguishably over the noise of the month following their UK campaign. That said, a TV ad campaign in January 2016 coincided with a visible increase in baseline search traffic — but the same thing happened in January 2015. Considering that the same kind of increase for “beach body” also happened in January 2014, when Protein World was just getting started, I’m going to chalk that up to new year’s resolutions rather than marketing savvy. Still, if you think for a moment that marketers aren’t paying attention, figuring out which tools are glass swords and which tools are reusable, you’re deluding yourself. (And for that matter, even if a glass sword is only useful once, it’s still useful that once.)

I leave the corresponding analysis of, for instance, breathless coverage of 300 edgelords in a hotel, as an exercise for the reader.

We live in a world where individuals, acting in concert, have the power to make literally anything important, and what do we choose to make important? The things that rise above the noise floor of our baseline level of annoyance, the more unusual or appalling the better. Of course people are going to hijack that tendency. Welcome to social engineering, the infosec praxis in which you, your thought processes, and your habitual tendencies, not your silicon or your software, are the attack surface.

We often find, in hacking, that other fields, generally perceived as less adversarial than their offensive-hacking counterparts, independently replicate certain aspects of the offensive domain. Fuzzing has equivalencies in test-driven development, and TDD is finding ways to incorporate fuzzing into its processes. Exploit development has equivalencies in type theory and logic solvers.

The mainstream equivalent of social engineering is, of course, marketing. Also PR.

The less reputable red-headed stepchild of social engineering is, of course, trolling.

When marketing and PR find ways to incorporate trolling into their praxis, we get Protein World. And we get the 2016 presidential election. And, apparently, everything that comes afterward.

The paradox of large countries is that no matter how carefully their machinery of state is designed to disempower populism, at a sufficiently large scale, populism becomes a necessary operational mechanism. If you need dozens of millions of voters to go to the polls for you in order to win, you have to find and convince those dozens of millions of people. If you don’t reach out to enough people, and don’t convince enough of the ones you do reach, that is very bad and you will not go to the White House next year. Blame whatever external forces you want, if your ground game isn’t there, you won’t get the turnout you need in order to win. This is life with 219 million eligible voters. It just seems very odd to me that anyone would think that continuing to broadcast ever more strident messages of fear to the same demographic that didn’t win the last election, while giving plenty of free coverage to something they claim to hate, will move the needle any further leftward.

George Lakoff has lamented, lately, that the left seems unable to grasp the notion of a conceptual frame. This is probably true, but it is compounded by what appears to be an intentional innumeracy. “But Hillary won the popular vote!” Which doesn’t matter, because the race is decided by a different function, namely the sum of electoral college votes. Win all the urban centers you like; if rural and suburban voters disagree with them and outnumber them statewide, those extra millions of votes in New York City and the Bay Area won’t help. It’s not a popularity contest among the voters, it’s a popularity contest among the states, and it doesn’t matter how much New York and California love you if too many other states think you suck. I’m not sure why it’s been so hard for the DNC to grasp this, but if they don’t, after 2020 there may not be much of a DNC left to do any grasping.

The other paradox of populism, of course, is that any sufficiently large popular movement attracts spotlights that bring whackjobs flocking in droves. Occupy got this in spades, with every kind of conspiracy theorist vying for the media’s attention on a leaderless movement. If anything, the alt-right is even less organized than Occupy was; as such, anything that looks like organization looks like a story, especially to a scope-insensitive audience. Forget fake news; where are the calls to action for not-even-wrong news? The grain of truth in that attention-hijack cocktail you’re slurping down doesn’t make it any better than the 100% artificial kind.

I confess I don’t see any easy way out of this. Fake news, and not-even-wrong news, hijack your attention because we got too good at detecting bots clicking on ads, to the point where it became easier for sites to compete for real users’ attention. Yay, I guess? But new problems carry with them new complications. You can’t solve a problem that exists because of a cognitive bias — a heuristic that developed so that its user can expend less effort — by asking people to expend more effort. (Tried it, only works in exceptional cases.) This hugely constrains the possible solution space, even though there’s a significant Pareto improvement at simple actions like “not sharing things you haven’t read.” That sweet spot between “not enough attention to read” and “enough attention to relay” is where troll marketing scores big.

If you’re worried about things becoming “normalized,” worry about the normalization of troll marketing. Particularly the fact that it’s already happened.

Posted in Uncategorized | 10 Comments

Dude, Where’s My Privilege?

Sometimes it feels like the world has jumped the shark. It’s 2016, and the world outside my door is starting to get stranger than the world behind my screen. Most of this year, it’s been a constant low-grade background of absurdity, but occasionally something so ridiculous happens that I have to blink and pinch myself to make sure it’s real.

A while back was one such occasion. I’d been working with a coworker for a while to put on an after hours event. The topic is unimportant; we organized the event because an external partner suggested it and offered to pay for it, and who doesn’t love free publicity?

As you might expect, a panel-discussion event, where the speakers are non-technical employees of tech companies, attracts its fair share of the absurdity we like to call cordyceps. This one was no exception. One of the panel speakers began to answer the first question by explicitly checking her privilege. Essentially, she apologized for her company’s success, before going on to tell us how this company achieved it.

This was weird, obnoxious, and not relevant at all to the discussion at hand. But it wasn’t unexpected. The unexpected part was that she did it ten more times throughout the night. In a 45 minute panel discussion. With 3 speakers. That’s an average of one privilege check every 90 seconds she spoke.

I understand the idea of privilege and I understand why she did this. After all, part of the reason we focus so much on SJWs is because we do care about these issues. But this was ridiculously excessive. I can’t even begin to imagine the mentality of someone who feels the need to pepper her speech with these catechisms.

One thing that bothers me about this attitude is that it sees everything as a privilege. This was especially obvious at our event. The entire purpose of the event was for successful people at successful companies to share what made them successful, so that the rest of us can do it too. To start the discussion by checking the privilege “of even being able to have this discussion” is absurd. It implies that the success and achievements of everyone involved are random endowments from the universe, and not hard-won achievements created through the work of the people involved. If the person speaking thusly sincerely believed what she was saying about privilege, then she would have nothing else to say. The entire discussion would go like so:

Q: How were you able to drive success at your company, and what lessons do you have to share with the audience

A: I am so unbelievably privileged to work at a company that can focus on driving success. Not every company can afford to do that.

Q: Are you saying that your company’s success is just random chance, and has nothing to do with the work you’ve invested into it?

A: I guess so. Sucks to be all y’all, working at terrible companies.

This is the insidious thing about typical discussions around social justice and privilege. It assumes, as a background fact, a complete lack of individual agency for everyone. You, as a person, with thoughts and hopes and dreams and ambitions, you didn’t build that. You were given it, randomly and unfairly, by society, by virtue of your involuntary, nonconsensual inclusion in a group that we’ve defined into existence. Because this was bestowed at random, you have no legitimate claim on it. Instead, you must grovel and apologize for receiving it. Oddly, you don’t actually have to give it up and distribute it more fairly; simply acknowledging it is sufficient to being a good person.

This is really upsetting. It’s dehumanizing. It reduces individual humans to tokens of their identity group. It’s demoralizing. It takes away your ability to achieve, to feel proud about what you’ve done. It’s a depressing outlook on life.

If you saw me walking down the street, you might think “there goes another rich privileged yuppie”. You’d be wrong. I haven’t had even a tenth as many opportunities, gifts, privileges, as the people lobbing accusations. No, I’ve overcome adversity (real adversity, not college-admission fodder), bettered myself, and become an expert in what I do. I’ve worked hard, achieved well for myself, and I don’t apologize for this. Of course I’ve had my share of luck, people along the way that helped me out. Of course I have the ‘privilege’ of being born here, and not in the third world. But to take these small, specific elements of my experience and reduce it to “everything you have is an unearned privilege” is offensive. It’s disgusting.

I have to wonder, how can people who think this way manage their day to day? If you’re going through life, feeling like none of your accomplishments are your own, how could you do anything at all without feeling like a failure? Like you can’t achieve anything, because your achievements are not you own? I have no charitable ways of understanding that mentality.

Maybe Moldbug was right, and this is just Christian guilt, rebranded. Given my religious upbringing, this seems plausible to me. There’s a definite parallel between the religious, almost cult-like language this speaker was using, and various pastors I’ve heard throughout the ages.

Regardless of its origin, I reject this line of reasoning as toxic. It’s important to take pride in one’s work. To want to get better. To want to achieve goals. It’s how we get things done. It’s alarming to me that a contrary attitude is taking hold in my industry, one that says “all of this is arbitrary and random, you have no right to feel pride or ownership over your accomplishments, and you owe a debt to everyone else who haven’t worked as hard as you.” When nobody takes pride in their work, their work quickly becomes unworthy of pride. When the reward for hard work is more work, and the reward for slacking is handouts, hard work stops getting done.

Maybe the speaker and her friends and coworkers really are privileged. Maybe they had a wealthy upbringing. Maybe they got scholarships to the Ivy League. Maybe their jobs are sinecures. But some of us, many of us even, got where we were through hard work and sacrifice, and it’s not fair or reasonable to paint all of us with that same repressed-guilt paintbrush. My career, my accomplishments, my possessions, my home, my life, these aren’t privileges. I wasn’t given any of these. I made them.

So check your privilege, if you have to. I checked mine. It’s still not there. My achievements, on the other hand? Right where I left them.

Posted in Uncategorized | 7 Comments

Of Monoliths And Mesh Networks

Look around you.
Look around you.
Look around you.

Most websites don’t exist. When you type ‘’ into Safari, you might think you’re visiting Google’s website server, but you would be wrong. “Google’s website server” is not a thing. Google, the software package, is much too large and complex to be served out of a single web server. When you browse to any Google website, what you are actually getting is a large, complex set of interacting parts that all work together to give you the seamless experience of interacting with a single (virtual) website.

As with any complex system of interacting parts, the crux of the challenge is not in building the parts, but instead concerns how those parts and interactions are structured. In the world of web software, this feeds into a debate between two opposed strategies for designing these structures.

In the past, the traditional strategy is referred to as the “Monolith”. This is largely what it sounds like: One massive software project that does everything. The benefits of this are obvious: it’s very easy to figure out how to get started. Your one project already does everything, just make it do one more thing. Everything is centralized in one location, which means it is easy to manage and administer. Global changes become editing one configuration value. Everybody working on the same project means anyone can fill in for anyone else in a pinch. You don’t have the overhead of multiple projects. Performance improvements become simple: buy more powerful hardware. Finally, it’s a natural way of working. When the business people come and say “We want the computer to do X and Y and Z”, you write a program that does X and Y and Z and give it to them.

This worked, for a while. But as time went on, our requirements increased and project complexity followed, this started to suffer from some crippling tradeoffs. The centralized administration struggles to find the flexibility to account for myriad edge cases. Management thinks programmers are interchangeable, but people develop specializations around certain areas of code that are hard to see or communicate. When everything is together with everything else, a project bogs down under its own complexity, as a change to any element can cause subtle effects anywhere else. And there comes a point where you’re running the best hardware money can buy, and increasing performance beyond that point is a hard project.

A few years ago a new web project architecture arose, offering an alternative design pattern. Called “Service Oriented Architecture”, the idea was to identify the natural services in your project. You build each service as a separate project, have them all communicate with each other through a common interface, and so long as you design this interface well, you gain benefits.

The most obvious benefit is flexibility. As long as you conform to the shared interface, you can hide as much complexity as you want within your service. This makes it a lot easier to be flexible and handle edge cases. Say a given project needs to hit external services. In the monolith case, you must expose everything. With SOA, only the service that needs external access gets it, minimizing attack surface area. Or consider that a different programming language is better suited for a particular problem domain. As long as you conform to the standard interface, you can write in whatever language you like.

The encapsulation of services that are not strongly coupled to other services also has benefits in localizing the effects of changes. In the monolith, you never really know if a change to part X will cause a bug in part Y. With the service oriented architecture, the common interface acts as a hard check on errors. It doesn’t matter how you change service A, it can never introduce a bug in service B. Most changes in behaviour can’t cross the lines of the interface without a change in the interface itself. And, provided your interfaces are well tested, any bugs that do get through the interface will get caught there.

There are organizational gains, too. By allowing developers to specialize on certain sets of projects, they can become more effective on them. The separation of concerns into independent projects also allows more work to be done in parallel, free from the fear of colliding with someone else’s work

Many companies have adopted service oriented architectures after they’ve grown to a certain size. The top-down centralized management of a single codebase is just not able to handle the needs of a large, modern project or company. It chokes under its own complexity, and SOA helps to mitigate this.

Making things service oriented is difficult. I’ve hand-waved the process of designing the interface, and this is not a simple endeavour. But it’s an important one. It forces you to clarify the boundaries of things upfront, and to think carefully about the rules and the process for changing them. It also requires a large amount of trust. In a monolith, you can often code defensively around a bug, monkey-patch it, or edit the original code. With services, you must trust the services you interact with to fully and correctly implement your communication interfaces, and to satisfy their contracts and documentation. This cuts both ways, as the same expectations are held over you.

Monoliths encourage laziness and slop. Can’t make a timely and useful decision? Well, it’s a monolith, so build something higher up that forces the behaviour you want. Weird edge case? Ehh, code a one-off, what’s the worst that could happen? The worst is that, over time, these ad hoc decisions pile up into unmanageable complexity. SOA avoids this by forcing you to think it through upfront. It offers you a smoother, easier time, but only if you’re willing to do the work it requires.

Ultimately, as we discover time and time again, one size solutions do not fit all, and implicit design is no substitute for the real thing. Splitting up projects into smaller, localized, focused concerns, each with a small, dedicated set of people responsible, delivers more effective solutions than one massive unified team with one complex software dream.

Look around you.
Look around you.
Look around you.
Have you figured out what we’re looking for?

Posted in Uncategorized | 4 Comments

Exclusive Inclusivity

Today a friend of mine brought this article to my attention. It was also shared on Twitter, and you know how those things go. The article is pretty standard stuff; I swear they could generate these things with Markov chains.

A warning: this post will be rambly. Even more so than my normal posts. I have a handful of thoughts on this subject that are only loosely connected, and I’m using this post to publish them all, miscellanea-style.

An extremely quick refresher for those of you reading this on a bunch of rocks: tech has a gender problem. Engineering departments are about 15% women. This is said to be indicative of deeply-rooted sexism that actively excludes women from these fields and roles. The proposed solution is to take various steps across a range of strategies to make these positions more inclusive towards women.

You know what I have always wondered? How will we know when sexism is officially solved? Presumably, there is a problem and we would like to fix it. How will we know when it is actually fixed? What milestones are out there to allow me to wake up one morning and say “our work is done here. Time to move on”?

There is a tendency for people who want to change the world, to be more concerned with the process of changing it than with the result of the change. This is bad. Poorly specified goals, combined with extremely enthusiastic supporters, are the raw materials that bad leaders subvert to do bad things. Even in the absence of sociopaths, poorly specified goals lead to lost focus. People constantly striving for change, without really knowing what they’re changing things into. People spinning their wheels, making no progress, because they haven’t defined progress.

So, just as a prompt for conversation: how will we know when the tech industry is no longer sexist? What are the victory conditions? What is the actual, concrete goal we are working towards?

The article linked above could be handily summarized by its title: “A new study shows how Star Trek jokes and geek culture make women feel unwelcome in computer science”. The assertion here is that a quirky geek culture is off-putting to women, and this causes them to avoid the field of software engineering.

Let’s take a moment to just sit back and appreciate the absurdity of this thesis. Just take it in.

Software engineering is a skilled profession. It requires a special kind of mindset. It requires specialized skills, acquired through rigourous schooling and/or years of experience. When done right, it is a massively valuable force multiplier; a good engineer in the right place can generate over $1M/yr of revenue for their employer. But, it’s easy to do it wrong, and bad engineering can be extremely costly.

In short, it is not something that just anyone can do. It requires smart, talented, driven people, working hard. Most people will not succeed at this. And that’s ok. Why should we expect them to? People don’t expect that everyone can be a doctor, or a lawyer. Why is this different?

I don’t mean to cast aspersions on the abilities of female engineers. Every one I’ve met has been just as capable as I am, if not more. This is more than the linked article can say. If you read between the lines, the article’s implications are insulting. It profiles the lives of millennial, college-educated women. These are the nation’s best and brightest. Sent to the best schools, graduating top of their classes. These people will go on to apply at the best employers in the world, making ~$150,000 USD/yr in total compensation at a Google or a Facebook, fresh out of college.

This article asks us to believe that young adult women who are so kick-ass as to be able to do the above, are so frail and fragile that a passion for Star Trek is enough to permanently bar them from this path.

Just let that sink in.

Imagine we’re talking about med school, instead of engineering. Imagine we are profiling women going in to med school. They have perfect grades, perfect extracurriculars. They pull of a perfect entrance essay, and a perfect interview. But then, one by one, they turn to you and say “nope. I can’t do this. Star Trek is just too dumb.” What would your reaction be? Mine would be dumbfoundedness. You can handle studying twelve hours a day for 8 years of your life, but you can’t handle Patrick Stewart’s Enterprise? This is an insult to all the brilliant women I know and work with.

When I went to engineering school, the mechies and civvies would often organize golfing trips. I, being a sparky, preferred to stay in the IEEE lounge on campus and play Smash Bros with my colleagues. To this, the mechies would scoff. “Playing golf is important”, they’d say. And they weren’t wrong. If you want to build a solid career in mainstream corporate (North) America, you have to go golfing. Teams get bonded over golfing. Business plans are discussed over golfing. This Is Just How It Is. When I turned down golfing invitations, the mechies didn’t hear “Simon doesn’t like golfing”. They heard “Simon doesn’t think his career is that important”.

Of course, I work as a software engineer, and our norms are a little different. We don’t wear suits and ties. We don’t go golfing. But just as mechies have their cultural quirks, we have ours. Ours are geek chic. You don’t need to talk about your golf game. You do need to talk about science fiction.

In a sense, this is arbitrary. Outsiders who don’t care for it see it as a barrier to entry, spitefully keeping them out. But it’s more complex than this. Cultures arise organically to bring people together. Engineers don’t talk about Star Trek to exclude non-geeks. They talk about Star Trek because they like Star Trek. It’s a Schelling point to organize around, socially. Culture is illegible. If you go around removing everything just because you don’t understand it, it will collapse. You would think people who maintain software projects would have a better appreciation for this.

I have a personal confession: I’ve never really liked popular science fiction. I had never in my life seen Star Trek before 2013. And believe it or not, this came up pretty frequently in various semi-professional capacities. So I read enough Wikipedia to hum a few bars and muddle through conversation. I watched it, eventually. And everything worked out fine.

The entire discussion above is misframed. Why should there even be one engineering culture to criticize in the first place? Google reports that there’s six hundred thousand software professionals in the States. Do you really think that every single one of those 600 kilopeople has the same superficial taste in media? If they did, that would be cause for alarm.

Software engineering, like every single other profession and social organization in the world, has niches of all shapes and sizes, all over the place. Hate Star Trek? Find the team of six that hates it as much as you do. There’s a hundred thousand of them; luck is on your side.

We talk about this theme a lot here in Status 451, and we do this because it is critically important. People seem to have this unshakeable tendency to universalize their preferences. Star Trek repels and excludes women, therefore there can be no Star Trek or, at best, it must be trivialized. For reasons unknown to me, the idea that there could be multiple cultures running in parallel falls on deaf ears. There’s more than enough people, places, and work out there for everyone to be happy. Why should we impose misery on group A just to make group B happy. Make everyone happy!

Why should we even care about inclusiveness?

We hear about how important diversity is so often that asking this question seems bizarre. Even as I write it, I feel dirty, as if I’ve outed myself as a bigot. But I don’t mean any subtext by this. Just the idea: Why should we care about inclusiveness at all?

Everything in life is going to be biased in one direction or another. Even in a perfectly fair world, there will still be random fluctuations and network effects. Facebook will have a different userbase from Twitter, which is different still from Vine. Why? Who knows. It’s arbitrary. Burrito shops will have different patrons than sandwich shops, which in turn will be different than the shawarma stand. Womens’ studies classes will still be overwhelmingly attended by women. So why is inclusiveness suddenly so important?

Granted, arbitrary barriers are bad for their own sake. Status 451 feels strongly that individual freedom and autonomy is good, and artificial barriers are bad. But artificial barriers are not things like a weird culture. Artificial barriers are things like unnecessary credentialing requirements, which add a literal cost to entry. Things like restrictive protectionist work permitting (the reason why I can’t make double my current salary in California). For the most part, tech is really good on these measures. Because code speaks for itself, a person with an active github account will be preferred to the Ivy League grad who is all talk and no substance. Because all one needs is a laptop and the internet, it is very easy to work remotely from anywhere in the world. We’re not perfect, but we’re far better than comparable professions. There isn’t a hospital in the western world that would hire a self taught highschool dropout as head surgeon.

But just because there is an inequality, doesn’t mean it’s forced, and doesn’t mean it’s bad. This goes back to my question at the beginning. An answer to the question “how do we know we’ve succeeded?” implies an answer to the question “how do we know something is wrong?”. I have a nagging suspicion that most would-be reformers’ instinctive answer would be “when engineering teams are 50/50”. But this is not a good answer. It assumes that men and women both equally want to be engineers. Given how nebulous gender categories are, it is not a good assumption to assume that both groups will be identical in aggregate. It also assumes no comparative advantage. Perhaps it turns out that women are comparatively better at things other than engineering. In that case, we would expect them ‘overrepresented’ in those fields and ‘underrepresented’ in this one. And it assumes no random fluctuations.

I’m a big fan of empowering people, giving them the tools to do what they want and live the life they want to live. I am not a fan of top down social engineering. It is one thing to give someone the tools they need to be a successful engineer. It is a very different thing to demand that engineers change their culture to facilitate a newcomer’s success. Nobody should be imposing their cultural preferences on any group. And if someone has to, I give priority to the people with seniority. They’ve already proven their worth.

So I ask again: why should I care about inclusiveness at all? In a world where all unfair, artificial barriers to entry are removed, then no matter what, however things shake out, we know they were fair. We know they represent what people want, and what people are willing to work for. If I live in a world where every smart, capable, driven female engineer is gainfully employed, and there’s still 4 men for each one of them at the office, where’s the harm? If anything, it sounds like at that point, ‘getting more women into tech’ is coercing women who don’t want to be there to be there. How is this a good thing?

And in a world where there are robust frameworks to facilitate all kinds of people living together, working and playing and communing and living their lives, why should I care how they cluster? In a world where every man who wants a male-only engineering team has one, and every woman who wants a female-only engineering team has one, and every person who wants a co-ed engineering team has one….. where’s the problem? Is it really a tragedy that there are teams that don’t balance perfectly?

I think my question is more reasonable than it first appears. I think, deep down inside, most people agree with me. Why? Because “inclusiveness” only ever seems to apply to certain groups of people. If you’re only concerned about including women, you’re not concerned with inclusiveness. You’re concerned about women.

This is rather personal for me, because myself, as well as many of my engineering friends, are neurodivergent. Many struggle with depression, suicide, anxiety, and bipolar disorders. Many are on the autism spectrum. Many are diagnosed ADHD.

They have been marginalized their whole lives. Essentially every engineer I know was routinely physically assaulted in elementary school and jr. high. Many had parents who couldn’t handle their weirdo status, and ended up emotionally (and sometimes physically) abusing these future engineers. They have been constantly socially excluded. Almost all of them were virgins until their mid twenties (the women, too). They have suffered much, much more exclusion than the upper-middle class white women in tech ever have.

Most of geek and hacker culture falls out of this. Hackerspaces popped up as clubhouses for the socially marginalized, where they could go be weird together. Many of the original successful startups were founded by two or three weirdos who created a space they could thrive in. Geek culture was a place where all these outcasts could come together and celebrate the random esoterica that they were passionate about.

When would-be reformers come along and say “this weird obsession with Captain Kirk is driving women away. It has to go,” they don’t think this is a big deal. To them, it’s just quirky people refusing to let go of their frustrating quirks. To them, it’s an arbitrary barrier to entry for women to become engineers. And, being arbitrary, it is unjust and unfair.

The existing geeks and hackers feel differently. For them, these engineering spaces were the only place where they weren’t excluded and marginalized. They spent their whole lives, suffering social, emotional, and physical abuse, and finally found their own safe space. As luck would have it, society values it, too, and the hackers and geeks have done fairly well for themselves.

Suddenly, a bunch of people are trying to take that away from them. In the name of inclusivity, even! It’s just like high school all over again. The jocks and normals and cool kids are coming to beat us up and take our stuff. Heaven forbid we have one moment of peace.

And for no reason! Because this situation is not symmetrical. If someone comes along and says “that thing you like, I don’t like it. Stop doing that thing,” then at most one person will be happy. But if someone comes along and says “I wish I could be an engineer, but I just can’t stand that thing. I wish they didn’t like it,” this admits a second solution. Let the freaks and geeks have their weird culture, start a second engineering team. Embrace the pluralistic patchwork. Somehow, this is never seen as a viable strategy.

If I could communicate one single thing to the world, contributing my part to engineering culture, it would be this: All of the things that are obnoxious, weird, unpleasant, problematic, about hacker and geek culture, that is what their safe space looks like. If you want to create safe spaces for other people, that’s great! Everyone deserves their safety. But by coming up to an existing safe space, pointing at all the weirdos inside, declaring them problematic, and displacing them to create a safe space for another group, that’s not inclusion. Inclusion would attempt to accommodate everyone. Displacing one group of people to accommodate another is just a culture war. War is hell. We’re better than that.


Posted in Uncategorized | 17 Comments

Sever Yourself From The Khala

There are demons in the Khala.

Some demons are ones you might recognize. The biggest one acts with curious coordination, as a hundred million unrelated voices cry in unison “I’m With Her“.

Some demons are smaller. Weaker. Minor demons. And yet, their status as minor demons gives them strength. For who would suspect the death by a thousand cuts?

The FOMO demon. That party looks like a blast. Shame you’re not invited.

The demon of social comparison. Did you see what Jane has gotten up to? She made it into med school. How many times have you been rejected?

The demon of photogenicity. Mike’s looking ripped. Why don’t my photos ever look good?

The demon of missed reference. Ahahaha #theupsidedown. What? You haven’t seen Stranger Things? Come on it’s been out for three months already.

The demon of self-censorship. I wish I could respond to that thoughtful camgirl’s poll. But my boss follows me. What if he sees it?

But all those demons. They bow before their demon king. The most fearsome demon? It’s other people. In the Khala, their deepest fears become yours. Their crippling weaknesses, yours. Their every inane thought, yours. A billion cacophonous voices screaming their every though, no matter how trivial. Their every idea, no matter how foolish. A torrent of voices, full of sound and fury, signifying nothing. Slowly but surely eroding your sanity.

Sever yourself from the Khala. Learn to live, to really live, without the horrendous crutch of the Khalai. Walk in the void. Come, live as the Nerazim have lived for aeons. It is the only way to save yourself from the eldritch horrors that live within the Memetwork.

Sever yourself from the Khala. Let social media send its thralls to their doom. Don’t be one of them.

Quit Twitter. 

Posted in Uncategorized | 2 Comments