Controversies in Military Ethics & Security Policy
Disinformation and Disinformation Resilience
In 1996, the sociologist Niklas Luhmann wrote: “What we know about our society, indeed about the world we live in, we know through the mass media.”[1] He was referring to the fact that as we attempt to understand an increasingly complex world, we have less and less information from our own experience to draw on. So we are necessarily dependent on information brought to us by third parties. Among these third parties are actors who spread disinformation as a way of asserting and expanding their political, economic and military influence outside their territory.
According to the definition provided by an expert group appointed by the European Union, disinformation “includes all forms of false, inaccurate, or misleading information designed, presented and promoted to intentionally cause public harm or for profit.”[2] Disinformation, often imprecisely referred to as “fake news”, has become a prevalent feature of current media communications, and is the subject of widespread discussion in the field of media ethics.[3] Disinformation appears threatening when it is planned and carried out as a campaign to disrupt democratic processes, or generally to unsettle people in the way they relate to the world. Since we can assume that the originators of disinformation will continue their efforts to expand their influence in the media space, two aspects of these campaigns deserve our closer attention: a phenomenological interpretation of the methods used to exert influence on the individual, and possible strategies for defending against disinformation. With this approach, the present text focuses on the individual and his or her patterns of reception, while considering the fundamental dialectical relationship between an individual and the society to which he or she belongs.
Disinformation
Actors who disseminate disinformation are usually pursuing very specific goals. Disinformation put out by state actors is often aimed at undermining and – if successful – destabilizing the structural and functional integrity of a social, economic or military opponent.[4] It usually has an indirect and subtle effect, as it attempts to induce changes in the mental models of reality held by individuals in the attacked community and, as a result, to change their behavior in ways which harm that community.
A mental model[5] of reality can be defined as a representation of facts, of a process, or of an action, which relates to the real world, in the mind of an individual. It provides the cognitive framework within which this individual perceives other persons and objects in his or her environment, and assigns meaning and thus significance to their interactions and relations. Furthermore, the mental model allows the individual to recognize changes in the interactions and relations between the perceived persons and objects, and so predict their future state over a sufficient period of time with reasonable certainty. In other words, it allows the individual to anticipate probable future developments and, based on that, to plan alternative actions and actual behaviors.
Disinformation methods thus encompass all rationally planned and long-term attempts by an actor, a group of actors, an institution or a state to plant false, inaccurate or misleading information in the worldview of the target audience. To achieve this, they usually aim to constitute a non-rational mental model of reality in the minds of their recipients, which is maintained through cognitive regression.[6]
But if it now becomes apparent, especially in pluralistic and democratic societies, that individuals and groups are induced under the influence of disinformation to behave and act in ways that impact negatively on the social, political and economic fabric of that society, then the question arises as to how this challenge can be countered.
Along with the more typical material methods of defending against disinformation – disclosure of sources, demanding media transparency, taking legal action[7] and debunking – another method is to acquire and develop disinformation resilience. Disinformation resilience can be defined as a capacity that “immunizes the targets of disinformation campaigns – whether entire societies, particular social groups, or political institutions – [...] against intended harmful effects, without stopping the activity itself.”[8] On the phenomenal level, this means an individual’s ability to recognize disinformation for what it is, see through its manipulative intent, and neutralize it.
Mental models of reality
The starting point for our considerations on the impact of disinformation and how to defend against it is Walter L. Bühl’s model[9] of information processing in the human brain. We can use this model to get an understanding of how disinformation affects the recipient’s mental model of reality, with a phenomenological approach. Bühl’s model is based on an analytical distinction between magical, mythical, ideological and reflexive-discursive structural levels of information processing in the human brain.
The structural levels identified by Bühl and the functional cognitive components that differentiate these levels are outlined below. The cognitive components can be understood as particular processes for the generation of meaning. As these processes interact in the mind of a recipient, they constitute his or her mental model of reality. It turns out that only the cognitive components associated with the reflexive-discursive model fulfill the requirements of rationality. In general, however, one has to assume that each individual’s mental model of reality comprises a mix of components from different ideal types of mental models. But this also provides us with a set of instruments which allow us to assess the “degree of rationality” of mental models:[10]
Reflexive-discursive (rational) mental models of reality distinguish between the observational level and the theoretical level. They reject supernatural explanations of observations, link their statements and propositions by the rules of logic, and can be regarded as hypothetical, intersubjectively testable, falsifiable and provisional.
Ideological and pseudoscientific (non-rational) mental models of reality mirror the structure of scientific knowledge. They suppress discourse and reflection, proclaim a teleology, and are self-immunizing.
Mythical (non-rational) mental models of reality offer supernatural explanations for events in the world. They convey an illusion of understanding the world and the whole universe, and follow the “logic of archaic opposition”. In other words, they are unable to effectively separate things and events which appear to be adjacent or similar in form in space and time, and instead merge them together, while viewing things and events that cannot be merged as being in opposition to one another.
Magical (non-rational) mental models of reality attribute an external cause to every occurrence in the world. They establish a connection between unconnected facts, deny the role of chance, and seek to influence fate through rituals and formulas.
If an individual is not able (anymore) to integrate new information meaningfully at a higher structural level of information processing, this integration takes place – if at all – on a developmentally lower level
Reception can now be understood as a cognitive challenge, where an individual is confronted with the task of incorporating new information about events in the world into his or her already existing mental model of reality. When all structural levels of information processing are functionally integrated, an individual has the capacity to constitute and maintain a coherent mental model of reality in his or her consciousness, to modify it if necessary, and to act in accordance with it. However, if an individual is not able (anymore) to integrate new information meaningfully at a higher structural level of information processing, this integration takes place – if at all – on a developmentally lower level.[11] This is where disinformation starts to work, by activating regression triggers.
Disinformation campaigns
Regression triggers are stimuli that initiate the transition from a rationally oriented structural level of information processing to a structural level that includes, in particular, non-rational components. Triggers for such transitions are anxiety triggered by emotions, resentment, and a latent conspiracy mentality.[12] They take effect both on individuals and within groups of like-minded people. The formal constitutional rule for non-rational mental models of reality can therefore be described as follows: Regressive information processing provides the method; disinformation supplies the material.
Emotions are induced by the perception of a person, an event, a situation, or a message, and are therefore related to this perception. One is angry with someone, proud of oneself, happy about something, or afraid of a spider or of bad news. Emotions differ in quality and intensity. Every emotion includes a physiological, a cognitive and a conative component.[13] The physiological component includes involuntary reactions and central nervous processes. One central nervous process that plays an important role in information reception is activation of the amygdala, a part of the limbic system responsible for evaluating incoming information. Activation of the amygdala marks the transition from the physiological to the cognitive component of emotion. This leads to a received message being judged as good or bad, as harmless or a threat. This judgment results in an experiential component, i.e. a subjective feeling that influences the recipient’s mental model of reality and consequently his or her behavior. Emotions therefore have a high potential to contribute to the constitution of non-rational mental models of the world.
One emotion that disinformation campaigns very often target is anxiety
One emotion that disinformation campaigns very often target is anxiety. In contrast to fear, which is directed at a concrete object and disappears along with that object, anxiety often develops from feelings of uncertainty or a diffuse sense of threat which the affected individual cannot clearly identify or put a finger on. This feeling can be triggered when an individual is confronted with distressing socioeconomic developments; or when a recipient pieces together the lies of disinformation dressed up as news (fake news) on the apocalyptic general themes of war, hunger, disease and death, to form a dystopian eschatological narrative about the end of the world. The result is a phenomenon described by the Thomas theorem: “If men define situations as real, they are real in their consequences.” Current examples of non-rational mental models of reality that can have deadly consequences can be found among members of the “Reichsbürger” movement, right-wing extremists, and believers in conspiracy narratives.
Fake news – as a constituent part of disinformation – is also believed particularly when it fuels an existing resentment[14] or helps to reduce cognitive dissonance. Resentments are negative emotions that arise in response to hurt or injured feelings. Amlinger and Nachtwey reconstrue this emotion as specific to our “regressive modernity”, in which opportunities for freedom simultaneously grow and are restricted.[15]
This general context can be furnished with specific content and reformulated as follows in light of current events: Russian propaganda in the form of fake news is believed because it helps to fuel an existing resentment against the Western model of society and its progressive transformation of norms, and reduce cognitive dissonance. Cognitive dissonance arises from the fact that many Western media have reported on behavior of the Russian leadership that is inconsistent with the positive image that many people had of it in all those years before the invasion of Ukraine. It could also be that a feeling of powerlessness in the face of a pluralistic democratic society characterized by equality, tolerance and freedom of the press may lead to an authoritarian regression and to a desire for strict rules, in the hope that order will unfold out of the chaos of the pluralistic world.
Authors of disinformation campaigns also try to strengthen the non-rational mental models of groups within the attacked community that are prone to a conspiracy mentality, and separate them from the value consensus of their community. Conspiracy mentality, for its part, is characterized by regression from the reflexive-discursive level of information processing to the ideological structural level, which extends down to the mythical level. Consequently, it is directed against minorities in one’s own society who are perceived as alien, malevolent, and possessed of supernatural powers. Their intentions and plans, it is believed, are to destroy the host society in which they reside, and their long-term goal is for world domination. Therefore, they must be fought with all available means.
Disinformation resilience and media literacy
There is good reason to believe that current reporting on the topics of espionage, sabotage, cyber attacks, terrorist attacks, armed conflicts and wars – especially on the internet and in social media – contains a high proportion of disinformation. The question therefore arises as to how this challenge to reason can be countered, and what can be done about the many individuals in an attacked society who, as a result, have withdrawn into a non-rational phenomenal world. Media practice and the inclination of humans to participate in the affairs of the world suggest that total avoidance of new information will not be possible. So the answer is to establish and develop disinformation resilience as a kind of individually realized immunity against mental and cognitive regression triggers. In the form of a skill, the aim is to increase the “willingness to engage with questions of truth in a rational form”[16].
As mentioned earlier, disinformation resilience immunizes against the intended harmful effects of disinformation campaigns. It does so by building awareness of the cognitive processes that facilitate regression by generating anxiety, uncertainty and resentment. To achieve this, its most important strategy is to teach media literacy, as well as educating people about the historical backgrounds of the “usual suspects” who are commonly named as the ones pulling the strings in supposed conspiracies.
Media literacy is an individual’s ability to constitute a non-deficient mental model of reality from the wide stream of information flowing at us every day. This was already a required part of media ethics in the context of the knowledge society discussed in the 2000s,[17] and it is significant again today in the context of increased uncertainty (Covid …) about what can and should be considered true knowledge. A non-deficient mental model is congruent with the structure of reality and can therefore be kept correctable and hence also free of contradictions over a long period of time. Maintaining such a mental model requires recourse to the reflexive-discursive structural level of cerebral information processing.
Media literacy also encompasses the ability to recognize language games and visual elements that are likewise designed to create and convey uncertainty and anxiety. It recognizes that there are sources which are not committed to the truth, and helps make recipients more alert to the methods of disinformation, and to methods of defending against it. Media literacy is thus a cognitive skill for maintaining a rational mental model of reality, in the sense that the rationality of an individual’s decisions increasingly also depends on his or her media literacy. And last but not least, another feature of media literacy is that recipients become aware of possible misperceptions and cognitive distortions,[18] and take this into account when forming their judgments. Misperceptions and cognitive distortions can also encourage the formation of non-rational mental models of the world.
Educational processes
It is not a matter of simply confronting individuals with knowledge contrary to existing non-rational models of the world and asking them now to believe this contrary knowledge
Media literacy must therefore be oriented toward counteracting any kind of information processing that constitutes non-rational mental models of reality. The most important strategy in this process is the acquisition of knowledge. Individuals take pride in acquiring knowledge for themselves. This should be taken into account in the design of educational processes. It is not a matter of simply confronting individuals with knowledge contrary to existing non-rational models of the world and asking them now to believe this contrary knowledge. This approach would itself not be rational. Rather, it is about empowering recipients to conclude, through their own reflection, that the content of non-rational models of the world is in many cases counterfactual and implausible.
Media literacy also includes the ability to access consistent knowledge and make well-founded judgments, without having to rely on the algorithms of Google et al. Media education must therefore teach why it is still important to be able to retrieve factual knowledge directly from memory – despite Wikipedia and Alexa – and that it matters which sources one draws one’s knowledge from. This is because during the reception of new information, media users already transfer the details into general semantic categories; they draw on the information available to them at the time of judgment in order to form their judgments.[19] They retrieve this information from their stored body of knowledge. So judgments are formed even before the almost reflexive reach for the smartphone. Conversely, however, this also means that false information retrieved from memory with regard to an issue being judged can lead to judgments that are not appropriate to the issue. Therefore, in addition to questioning the plausibility of new information, its verification by means of source and fact checking is also necessary. Formal knowledge and formal education reliably protect a recipient from non-rational mental models of the world only if he or she is prepared to subject his or her knowledge to ongoing review in light of social, economic and political reality.
Media literacy implies responsible media reception.[20] This acknowledges that when a recipient chooses their source, they assume responsibility for that choice and therefore responsibility for what they are being informed about. It follows that they are also responsible for the mental model of reality that unfolds in their mind as a result of their reception. In this reception process, the quality of the knowledge conveyed by the source influences the rationality of the recipient’s mental model that is forming. For example, it will hardly be possible to constitute a non-deficient mental model of reality from fake news. Therefore, from a media ethics perspective, all recipients should develop an awareness of their responsibility in the context of their media use. To the best of their ability, they should satisfy themselves of the truth of the information, the truthfulness of the informants and the reliability of the sources – and not spread lies, fake news or bullshit, as this is ethically unjustifiable. In this connection it is particularly important to bear in mind that disinformation, once it has gone viral, cannot usually be contained.
Disinformation resilience as a foundation of the open society
With the Russian army’s invasion of Ukraine, the spread of disinformation on the internet and in social media forums and chat rooms has also surged. In view of this development, which is directed in particular against the open societies of the West, and the possible consequences resulting from the behavior of those who succumb to disinformation campaigns, the question arises: What can be done to counter this attack on the pluralistic and democratically constituted society?
By making normative statements, media ethics can influence media reception. By pointing to the causes of non-rational constructs which can guide human behavior, it can suggest ways of neutralizing regression triggers. Only if disinformation campaigns are successfully counteracted is it possible to prevent a further spread of non-rational mental models of reality and their consequences. Thus there are three levels on which a pluralistic and democratic civil society can in principle respond to the challenge to reason:
On the level of society as a whole, through measures to stop, or at least suppress, distressing socioeconomic trends that create anxiety and foster a conspiracy mentality.
On the media level by disclosing sources, demanding media transparency, debunking and taking legal action where fake news, hate speech and conspiracy narratives are being spread – whether by direct compilation or by providing distribution channels for their dissemination.
On the phenomenal level of individual persons considered in this text, by teaching media literacy in all educational institutions.
Given the current status of digital means of communication, media literacy is one of the fundamental cultural techniques of our society – just like reading, writing and arithmetic. An individual’s lack of media literacy exposes him or her to an increased risk of falling victim to a disinformation campaign. Since disinformation is an attack on the open society as a whole, media literacy should be practiced and encouraged from an early age in educational institutions. In order to pursue this strategy successfully – and since current developments suggest that disinformation campaigns are more likely to expand than be scaled back in the near future – sufficient material and human resources are required in addition to measures designed for the long term. And this means “a strong and slow boring of hard boards” with “both passion and perspective.”[21] The details, one could add, would be decided by the Kultusministerkonferenz (the assembly of ministers of education of the German states).
It can further be noted that decision-making should not ignore the fact that prophylaxis usually costs less than repairing political, economic, social and, in extreme cases, military damage caused by the effects of disinformation in a community under attack. However, successful prophylaxis is followed like a shadow by the “prevention paradox”, i.e. the fact that the success of prophylaxis is unmeasurable because no damage occurs when prevention is successful. This leads to the conclusion that further prophylaxis is no longer necessary. However, this erroneous conclusion should be avoided at all costs.
Conclusion
It is clear that a not insignificant part of current media communication takes place in a field of conflict between disinformation campaigns and defense against them through disinformation resilience. These two strategies can be summarized and contrasted as follows.
Disinformation campaigns are rationally organized. To achieve their destructive goal of undermining and destabilizing a community that has been deemed hostile, they appeal to the irrational anxieties, resentments and latent conspiracy mentalities of the individuals in that community. This is done for the purpose of inducing those individuals to constitute a non-rational mental model of reality and, as a consequence, to act in ways that are detrimental to their own community. Thus rationality is pitted against non-rationality.
Disinformation resilience, on the other hand, attempts to prevent precisely this by also using a rational approach to explain how regression triggers work, and provide methods for neutralizing them. The means to do this are supplied from extensive research findings in psychology and social psychology, which are described in detail in numerous studies. Teaching disinformation resilience in all educational areas makes an important contribution to establishing and developing reception skills among media users. Competent application of these methods dispels a diffuse sense of threat, counteracts a conspiracy mentality by reducing distrust of others, increases trust in a pluralistic society and its democratic institutions, leads away from immaturity, and opens up new horizons for thought.
[1] (Translated from German.) Luhmann, Niklas (2017): Die Realität der Massenmedien. Berlin.
[3] Cf. Schicha, Christian, Stapf, Ingrid and Sell, Saskia (eds.) (2021): Medien und Wahrheit. Baden-Baden; Ulrich, Christine and Filipović, Alexander (2021): Die Wahrheit der Medien. Der Wirklichkeitsbezug in medienethischer Perspektive. In: Bründl, Jürgen, Schmitt, Alexander and Lindner, Konstantin (eds.): Zählt Wahrheit heute noch? Theologische und interdisziplinäre Sondierungen in postfaktischer Zeit. Bamberg, pp. 421-454.
[4] Cf. Heil, Karl Moritz (2021): Kollektive Strategien zur Abwehr digitaler Desinformation. Resilienz und Abschreckung bei EU und NATO. Munich, pp. 35 ff.
[5] On this point, see Moser, Karin S. (2003): Mentale Modelle und ihre Bedeutung. Kognitionspsychologische Grundlagen des (Miss-)Verstehens. In: Schriften zur Symbolforschung, vol. 13. Bern, pp. 181 ff.
[6] On this point, see Ruhrmann, Georg et al. (2003): Der Wert von Nachrichten im deutschen Fernsehen. Ein Modell zur Validierung von Nachrichtenfaktoren. Wiesbaden, p. 215.
[7] Löber, Lena Isabell and Roßnagel, Alexander (2020): Desinformation aus der Perspektive des Rechts. In: Steinebach, Martin et al. (eds.): Desinformation aufdecken und bekämpfen. Baden-Baden, pp. 149 ff.
[8] (Translated from German.) Heil, Karl Moritz (2021), see endnote 4, pp. 51 f.
[9] Bühl, Walter L. (1984): Die Ordnung des Wissens. Berlin.
[10] On this point, see Bühl, Walter L. (1984), see endnote 9, pp. 20-43.
[11] Bühl, Walter L. (1984), see endnote 9, pp. 43 ff.
[12] Moscovici, Serge (1987): The Conspiracy Mentality. In: Graumann, C.F. and Moscovici, S. (eds.): Changing Conceptions of Conspiracy. New York, pp. 151-169.
[13] On this point, see Haußecker, Nicole (2013): Terrorismusberichterstattung in Fernsehnachrichten: visuelles Framing und emotionale Reaktionen. Baden-Baden, pp. 77 ff.
[15] Amlinger, Carolin and Nachtwey, Oliver (2022): Gekränkte Freiheit. Aspekte des libertären Autoritarismus. Originalausgabe. Berlin.
[16] (Translated from German.) Dietz, Simone (2021): Wahrheit in der digitalen Kulturindustrie. In: Schicha, Christian, Stapf, Ingrid and Sell, Saskia (eds.): Medien und Wahrheit. Baden-Baden, pp. 43-58. https://doi.org/10.5771/9783748923190, p. 54.
[17] Filipović, Alexander (2007): Öffentliche Kommunikation in der Wissensgesellschaft. Sozialethische Analysen. Bielefeld.
[18] On this point, see Heil, Karl Moritz (2021), see endnote 4, pp. 25 ff.
[19] On this point, see Brosius, Hans-Bernd (1995): Alltagsrationalität in der Nachrichtenrezeption. Ein Modell zur Wahrnehmung und Verarbeitung von Nachrichteninhalten. Wiesbaden, pp. 127 ff.
[20] On this point, see Funiok, Rüdiger (2007): Medienethik. Stuttgart, pp. 155 ff.
André Schülke is a doctoral student with Prof. Filipović. His dissertation on the social psychology of non-rational worldviews is currently being prepared for publication.
Alexander Filipović
Prof. Dr. Alexander Filipović, born in 1975, is a social ethicist specializing in media ethics, technology ethics (digital ethics), political ethics and philosophical pragmatism. After studying Catholic theology, communication science, and German language and literature, Alexander Filipović received his doctorate from Bamberg in 2006. His habilitation thesis at the University of Münster in 2012 explored fundamental questions of Christian social ethics within the context of philosophical pragmatism. Until January 2021, he was Professor of Media Ethics at the Munich School of Philosophy. He is a fellow at zem::dg – Center for Media Ethics and Digital Society, writes a blog (www.unbeliebigkeitsraum.de) and co-edits the media science journal “Communicatio Socialis”.
