by Guy Burgess
This is the second of what will be a series of posts exploring the challenge posed by what we call "bad-faith actors" --- people who, as they try to advance their own narrow interests, attack good-faith efforts to make collaborative, liberal, democracy work for the benefit of all. As we see it, the conflict field and the society as a whole needs to develop much more effective defenses against each of the five principal types of bad-faith actors.
That means we need to address the threat posed by partisans on the left and the right who are so convinced that they are the "good guys" and the other side is so evil that they must be decisively defeated. We need to limit the effect of news organizations and the communication media more broadly that have figured out that they can build a larger and more loyal audience by focusing everyone's attention on divisive, inflammatory, and often inaccurate information. We must sideline or muffle political figures who use inflammatory, "mobilize-the-base" tactics as part of a corrupt, divide-and-conquer strategy to accumulate as much political power as possible. In addition, we must identify and block foreign rivals who use information-warfare tactics to spread divisive content as part of an effort to destabilize and weaken our society. Finally, we must disempower the nihilists who are so alienated from society that they simply want to destroy things.
All of these actors see it as being in their interest to amplify our many tensions in ways that compel us to act against our own interests by handling our inevitable conflicts in destructive and, potentially, catastrophic ways. In this newsletter, we will focus on the things that make us so vulnerable to this type of attack. Then in the next newsletter in this series, we will look at the tactics that are being used to exploit these vulnerabilities and strategies for better defending ourselves against them.
One factor that increases our vulnerability to bad-faith actor attacks is the process of Darwinian selection. It has always been true that the most successful bad-faith actors can expect to enjoy enormous wealth, status, and power. As such, ambitious, clever, and ruthless people have always been attracted to the quest for social dominance. These individuals have then interacted in a highly competitive environment that systematically selects those who are able to put together the most effective power-building strategy.
So we can be sure that there will always be at least a few individuals who are willing and able to employ the most sophisticated and ruthless strategies available. In today's world, this means that we can expect bad-faith actors to take full advantage of the latest insights into the neurobiology of human decision-making; the complex structure of modern communication media; and the intricacies of legal, political and economic systems. In other words, they will know where we are vulnerable and how to take advantage of those vulnerabilities.
Bad-faith actors will know that the psychology of human thought and decision-making evolved in a simpler time when humans faced very different challenges --- challenges that, over time, the adaptation process hardwired the human brain to navigate. As society has grown larger and more complex, the nature of the challenges we face have changed, but our decision-making processes have had trouble adapting. The usefulness of many of our neuropsychological adaptations to danger have deteriorated to the point where we now rightly think of them as "biases" which undermine our ability to rationally navigate modern society. Psychologists have identified an astonishing array of biases, but there are a few big ones that are widely exploited by unscrupulous actors.
One such bias is our understandable tendency to focus first on anything that might constitute a direct threat to our vital interests. This is commonly used as an audience-building strategy by media companies and as a hate-mongering strategy by divide-and-conquer political actors who want us to focus all of our attention on efforts to this decisively defeat the "dangerous "other." This, in turn, is reinforced by a worst-case bias that, under conditions of uncertainty, leads us to "play it safe" and take the worst possible view of the threat posed by our adversaries and conclude that defeating them must be our only priority (and that other things, like the corruption of our leaders, are not as important or worth worrying about.
We also find it very hard to consider the possibility that we may be wrong, that our "friends" may not really be our friends, and things that we've been doing to protect ourselves may actually be hurting ourselves instead. Not surprisingly, we tend to seek out and give greater weight to information that avoids such unpleasant cognitive dissonance and confirms that we were right all along. Similarly, we tend to look uncritically at leaders who make extravagant, too-good-to-be-true promises (that further reinforce the sense that we made the right choice).
Beyond this, we tend to be attracted to information that allows us to hold ourselves blameless (and others responsible for) for whatever misfortunes we might suffer. This results in a sense of victimhood which, of course, reinforces the hostility with which we view those who we think victimized us.
The scale and complexity of the modern world means that we have little direct, personal knowledge of a great many key aspects of our lives. In deciding what we believe and how we should behave, we are utterly dependent upon information that we receive through mass media's various channels of communication. It is through these channels that we learn about the threats we face and things we might be able to do to limit those threats. These communication channels are, however, far from perfect. They suffer from a wide range of vulnerabilities that undermine their ability to provide us with the fair and accurate information that we need to guide our lives.
One of the most important vulnerabilities stems from the commercial nature of communication systems in capitalistic democracies. (Authoritarian societies, not surprisingly, face a different set of vulnerabilities.) Commercial media is ultimately funded by attracting and monetizing our attention. Any information source that wants to stay in business (and be influential) has to attract and retain an audience.
Unfortunately, this seems to be most effectively done by exploiting the above psychological vulnerabilities in ways that provide people with information that they want to hear (people, not surprisingly, tend to tune out displeasing information). This is best done by segmenting the market into a series of relatively small and homogeneous audiences and then delivering to each audience custom-tailored information that they will get excited about --- information that is seen as so important that it demands a space in everyone's busy schedule. In today's highly competitive media environment (with its precise tracking of the stories that individuals pay most attention to) publishers have both the ability and the incentive to tell people precisely what they want to hear.
This creates a situation which rewards those who feature stories of moral outrage and highlight the things that the "good guys" are doing to fight back --- things that will, if everyone stays committed to the fight, they promise, ultimately yield decisive victory. Similarly, it rewards those who cultivate and then take advantage of the camaraderie that arises among those fighting the "good fight" together. Since groups of people from differing cultural and economic circumstances tend to have very different priorities and worldviews, it is relatively easy to build enthusiastic audiences around the demonization of opposing groups. What makes this vulnerability especially serious is that it tends to generate a positive feedback loop in which inflammatory stories beget inflammatory responses in a continuing cycle that drives up audience participation and loyalty along with the associated revenue stream.
This is, of course, also a process that can be driven and fueled by unscrupulous, divide-and-conquer politicians and foreign hybrid-warfare actors seeking to destabilize a society. (Such actions are facilitated by the open structure of liberal democracies.) It is also clear that rapidly-advancing communication technologies have amplified these vulnerabilities by allowing them to be exploited with ever greater precision and effectiveness. It is now easy to tailor a multiplicity of "newsfeeds" to each individual's worldview and prejudices. The ability to provide these evermore attractive information flows is being enhanced by new techniques for evading the constraints of objective reality and selling people an attractive fictional vision of "reality."
Taken together, the vulnerabilities associated with decision-making biases and distorted information flows influence the way in which people interact in the larger social environment. Unfortunately, the structure of this environment also contains vulnerabilities that are subject to manipulation and exploitation.
One vulnerability stems from the hierarchical structure of the social system and the fact that there are always some people who enjoy positions of greater status, wealth and power – positions which give them the ability to claim even more status, power, and wealth over time. Not surprisingly, this is a process has generated immense resentment at lower levels of the hierarchy where people struggle to get by with ever-dwindling resources. This resentment is compounded by the tendency of the meritocratic elite to see its privileged position as nothing more than a rightly-deserved reward for their hard work.
The resulting populist tensions on both the left and the right make societies vulnerable to bad-faith actors who promise to address the problem, while actually doing the opposite. In the U.S., this vulnerability is further amplified by features of our democracy that make it easier to win elections through anger-driven, base-mobilization tactics than by actually proposing better solutions to common problems.
Other vulnerabilities arise from the social nature of our cultural beliefs and our images of objective reality. In deciding how to behave and what to believe, people don't have time to figure it all out for themselves. They inevitably look to members of their group for experience, expertise, and guidance. Very few people have the courage (or maybe the stupidity) to deviate too far from the wisdom of their group and risk the social condemnation that would accompany such deviation. (All social groups have robust mechanisms for assuring that there is enough agreement within the group for it to function effectively.)
Social systems are also vulnerable to the "vested interest" or "squeaky wheel" effect in which highly-motivated individuals (who are often resource rich, bad-faith actors who stand to gain enormous sums) are able to exert an outsized and corrupting influence on society. Also dangerous is the inherent fragility of the bonds of trust that hold society together. Relatively minor incidents can instill a sense of betrayal that is very difficult to repair. Terrible and usually violent incidents are even worse. Massacres and other atrocities that can arise from the reprehensible behavior of just a few individuals can become the kind of "unrightable wrongs" that plague societies for generations.
These vulnerabilities are not, in and of themselves, all that bad. The problem is that, as we'll see in the next newsletter on the bad-faith actor problem, unscrupulous, bad-faith actors have developed a variety of very effective tactics for exploiting these vulnerabilities. Before we can expect good-faith democratic governance to succeed, we have to develop much more effective defenses against these tactics.