Dartmouth engineering researchers have developed a new approach for detecting a speaker’s intent to mislead.
The approach’s framework, which could be developed to extract opinion from “fake news,” among other uses, was recently published as part of a paper in Journal of Experimental & Theoretical Artificial Intelligence.
Although previous studies have examined deception, this is possibly the first study to look at a speaker’s intent.
The researchers posit that while a true story can be manipulated into various deceiving forms, the intent, rather than the content of the communication, determines whether the communication is deceptive or not.
For example, the speaker could be misinformed or make a wrong assumption, meaning the speaker made an unintentional error but did not attempt to deceive.
“Deceptive intent to mislead listeners on purpose poses a much larger threat than unintentional mistakes,” said Eugene Santos Jr., co-author and professor of engineering at Thayer School of Engineering at Dartmouth.
“To the best of our knowledge, our algorithm is the only method that detects deception and at the same time discriminates malicious acts from benign acts.”
The researchers developed a unique approach and resulting algorithm that can tell deception apart from all benign communications by retrieving the universal features of deceptive reasoning.
However, the framework is currently limited by the amount of data needed to measure a speaker’s deviation from their past arguments; the study used data from a 2009 survey of 100 participants on their opinions on controversial topics, as well as a 2011 dataset of 800 real and 400 fictitious reviews of the same 20 hotels.
Santos believes the framework could be further developed to help readers distinguish and closely examine the intent of “fake news,” allowing the reader to determine if a reasonable, logical argument is used or if opinion plays a strong role. In further studies, Santos hopes to examine the ripple effect of misinformation, including its impacts.
In the study, the researchers use the popular 2001 film Ocean’s Eleven to illustrate how the framework can be used to examine a deceiver’s arguments, which in reality may go against his true beliefs, resulting in a falsified final expectation.
For example, in the movie, a group of thieves break into a bank vault while simultaneously revealing to the owner that he is being robbed in order to negotiate.
The thieves supply the owner with false information, namely that they will only take half the money if the owner doesn’t call police.
However, the thieves expect the owner to call police, which he does, so the thieves then disguise themselves as police to steal the entirety of the vault contents.
Because Ocean’s Eleven is a scripted film, viewers can be sure of the thieves’ intent – to steal all of the money – and how it conflicts with what they tell the owner—that they will only take half.
This illustrates how the thieves were able to deceive the owner and anticipate his actions due to the fact that the thieves and owner had different information and therefore perceived the scene differently.
“People expect things to work in a certain way,” said Santos, “just like the thieves knew that the owner would call police when he found out he was being robbed.
So, in this scenario, the thieves used that knowledge to convince the owner to come to a certain conclusion and follow the standard path of expectations. They forced their deception intent so the owner would reach the conclusions the thieves desired.”
In popular culture, verbal and non-verbal behaviors such as facial expressions are often used to determine if someone is lying, but the co-authors note that those cues are not always reliable.
“We have found that models based on reasoning intent are more reliable than verbal changes and personal differences, and thus are better at distinguishing intentional lies from other types of information distortion,” said co-author Deqing Li, who worked on the paper as part of her Ph.D. thesis at Thayer.
Most philosophers think that at least some experiences have representational content: they represent the world as being a certain way.1 Representational content dictates accuracy conditions, namely, what would need to be the case in order for the experience to be accurate.
Inner speech, that “interior monologue” or familiar voice inside your head, is something that we experience, and that experience of inner speech seems to have representational content: it seems to “tell” the subject that something is going on in the world.
Our central question is: What is it that the experience of inner speech is telling the subject is going on in the world, and could it, in some circumstances, be telling the subject something inaccurate? In other words: When, if ever, does the experience of inner speech mislead?
This may seem like a strange question to ask, and its importance may not be immediately obvious, but answering it has a number of significant implications.
To start with, the question about whether the experience of inner speech can mislead requires us to answer a more basic question first: what sorts of things enter into the representational content of an experience of inner speech?
This question is of tremendous importance since it tells us what the epistemic weight of an experience of inner speech is, namely, the content that it carries. In particular, if we view the experience of inner speech as important to self-knowledge, the content of the experience will tell us more precisely what the route to that self-knowledge is.
A more specific implication of an answer to this question is that there are unusual experiences (often in the context of psychiatric diagnoses), such as auditory verbal hallucinations (AVHs), which are taken by a number of theorists to involve inner speech (Frith 1992, Seal et al. 2004, Jones & Fernyhough 2007).
If we think of AVHs as experiences of inner speech, we can usefully ask ourselves: is this experience of inner speech telling the subject something inaccurate? And if it is, what aspects of the world aren’t, and which are, the way they are represented as being?
At this point it is important to clarify two things. First, there is the question of what exactly we mean by an “experience of inner speech”.
Some might want to say that inner speech simply is an experience. Others might want to say that inner speech is something that we do, and which we have an experience of. At this stage we remain neutral between these two, but it will become clear later on that our position is more in line with the latter. Second, it is important to clarify that we are talking about the experiential content of an experience of inner speech and not its linguistic content.
We are not talking about utterances of inner speech linguistically expressing inaccuracies. Thus to draw an analogy with outer speech experience, if someone says “Madrid is the Capital of France”, although they have said something inaccurate, my experience is accurate to the extent that it has accurately represented various features of the utterance, for example, the speech sounds produced, and perhaps more besides (a central part of this chapter is the controversy surrounding this). Now, the extent to which this analogy with outer speech holds is itself up for dispute and will depend upon how we think of inner speech.
We proceed as follows. We start by presenting an intuitively appealing view according to which an episode of inner speech is an imaginative episode, and therefore cannot mislead (at least not in the relevant sense).
We criticize this view and reject it in favour of the view that inner speech is actually a kind of speech, rather than merely imagined speech. We then present a view about the representational content of speech experience generally, and then apply it to inner speech in particular. We end, in light of this, by presenting the different ways in which inner speech could potentially mislead.
Content without Commitment: Inner Speech as Imagination
It is important to distinguish representational content from psychological force. Perceiving and believing have representational content, but they also have a certain psychological force: they don’t merely represent a content, they represent that content as accurate. In other words, they involve, by their very nature, a certain commitment to the world being the way represented.2
Other psychological states or events (such as suppositions or certain imaginings etc.) on the other hand may represent something but lack that kind of commitment to what is going on in the world. If you voluntarily imagine a pink unicorn, it cannot be regarded as an inaccurate experience just because there is no such thing in front of you (or no such thing in existence at all).3
The experience is not even in the running for accuracy. That said, the experience has representational content: it is of (or represents) a pink unicorn. What you have in this case of imagining is content (something is represented, it is about something) without commitment to accuracy. Another way of thinking about this lack of commitment to accuracy is that the imaginative episode is not presenting an aspect of the world over and above the experience itself (and so, trivially, it cannot do so inaccurately).
Some might think that inner speech is like that. Inner speech, on this view, is like imagining yourself speaking (and hearing yourself speak).
The experience does not inform you about something going on in the world, and, as such, it cannot be wrong since the world cannot act as a benchmark against which the experience can fall short.
There simply is the experience, presenting itself pure and simple. At most, if this is true, an inner speech experience tells you the immediate and infallible fact that you are having that very experience.
On such a view, inner speech may represent certain things, which would be reflected in the phenomenology of inner speech, much in the same way as imagining a pink unicorn represents certain things (like pinkness and unicorns), and this too is reflected in the phenomenology of the experience, but neither experience purports to tell you anything about the world beyond the experience itself. On such a view, inner speech, as a variety of imagination, cannot be inaccurate: it just is what it is.4
But is inner speech an instance of imagination? We think that the answer is no. A crucial step to seeing why this is the case involves an appreciation of the distinction between imagination and imagery. Imagination is a whole psychological event in its own right. People are engaged in acts of imagination.
These acts of imagination enable them to appreciate, in potentially many different ways, non-actual scenarios, and, when they are engaged in such acts, they may be motivated to do so by a number of different things. They may be trying to judge whether they could have jumped over that river, reason about a social situation, or simply engage in imagination for the pleasure of it. Furthermore it is in the nature of imagination to have content without commitment (which is not to say that it cannot serve, and fail to serve, a given function).
These acts of imagination often will recruit or make use of imagery in many modalities, but there will also be aspects to the imaginative experience that aren’t purely imagistic. Imagery, in contrast, is not in itself a complete psychological event.
It features as a component of such events. Whereas people imagine things, people don’t “imagize” or “do imagery”. When people imagine things, imagery may be involved, but it is not all that is involved. And, crucially, imagery is also involved in many psychological events that aren’t imaginings. For example, imagery may be involved in episodic recollections. It may even be involved in certain judgements (see, e.g. Langland-Hassan 2015).
In other words it may be involved in psychological events that, unlike imagining (in the sense that we are using the term), have an inbuilt commitment to how things are (or were, in the case of memory) in the world.
In light of this, it is too quick to move from the (accurate) observation that inner speech involves imagery to the conclusion that an episode of inner speech is a case of imagination. And if it is not a case of imagination then it seems, at least in principle, that, as an experience, it can be committed to telling you something about the world.5
Inner Speech as Speech
If inner speech is not imagination, then what is it? In line with a number of other theorists (Vygotsky 1987/1934, Fernyhough 1996, Martinez-Manrique & Vicente 2010) our answer is: it is speech. It is speech in two important senses. First, it is a productive rather than recreative activity. Second, its primal use is in making speech acts: asserting, questioning, insulting etc. We take these points in turn.
Inner speech as productive rather than re-creative
To see the productive rather than re-creative nature of inner speech we need to ask ourselves not just, “What is inner speech?” or “What does it look like once developed?” but also: “How and why did it develop?” One attractive theory (which originates in Vygotsky 1987/1934), which carries both evolutionary and developmental plausibility, states that inner speech starts off as speech (namely, outer or “overt” speech). That is to say, whatever function inner speech plays, once it has developed, is played by outer speech in children who have not yet developed the capacity to engage in inner speech. This capacity to engage in inner speech is usually seen as partly constituted by the capacity to inhibit the overt production of speech.
According to this story, inner speech is the end product of a developmental trajectory that begins with private speech. “Private speech” refers to outer speech that is not produced for the benefit of anyone other than the speaker. Young children will first, under the guidance of a caregiver, learn to reason verbally, but out loud, for the benefit of guiding their thinking and attention. Over time, they learn to “internalize” this speech, to inhibit its overt production. However, as with many cases of motoric inhibition, vestiges of the motor processes remain. Evidence of motoric involvement in inner speech has been empirically supported by several electromyographical (EMG) studies, measuring muscular activity during inner speech, some of which date as far back as the early 1930s (e.g. Jacobsen 1931). In short, these discovered that, when you engage in inner speech, muscles in the face and throat, associated with speaking, are activated (see also Rapin et al. 2013).
There have been brain-imaging studies (fMRI) presenting results that are very much in keeping with the distinction between a productive phenomenon, namely, inner speech proper, and a re-creative imaginative phenomenon, imagined speech. In particular, Tian & Poeppel (2012) and Tian, Zarate, and Poeppel (2016) have shown that there are two very different ways of generating auditory-verbal imagery, namely, of activating relevant areas of auditory sensory cortices in the absence of external sensory stimulation. One, which corresponds to inner speech (which they call “articulation imagery”) is induced through “motor simulation”, i.e., is initiated “top-down” by activation in areas of prefrontal and motor cortex associated with speaking. The other, which corresponds to inner hearing/imagined speech, is induced, in line with more standard accounts of imagery (including in other modalities, such as vision), via a memory-based mechanism (e.g. Kosslyn 1994), i.e., by the re-creation of a sensory event (derived, to some extent, from past sensory events). While the former mechanism involves trying to produce something directly (and its inhibition results in imagery being activated as part of the sensory predictions of the completed action), the latter involves trying to re-create the sensory effects of a past or constructed scenario. There is a sense in which imagining hearing something entirely new (i.e., not previously experienced) is “producing something”, but not in the same sense that inner speaking is productive. Unlike the latter, it involves the recreation of the sensory effect of an event, in this case an event that has never happened.
This distinction between a productive and re-creative phenomenon may map onto a phenomenological distinction between two different forms that auditory-verbal imagery can take. Using descriptive experience sampling (DES), Hurlburt and colleagues (Hurlburt, Heavey, and Kelsey 2013) isolated two differently reported phenomena: “inner speaking” on the one hand, and “inner hearing” on the other. The former may correspond to the top-down mechanism of generating imagery that Tian and colleagues isolated; the latter, to the more bottom-up mechanism. Nevertheless, equating Hurlburt’s “inner speaking” with “inner speech” does not suffice to show that “inner speech” is not a case of imagination. The reason for this is that it seems plausible that inner speaking can take part in imaginative episodes as well as in more authentic or ecologically valid instances of inner speech. If you imagine yourself going up to someone and speaking to them, nothing prevents this from engaging the sort of top-down imagery that Tian and colleagues isolate, or in having phenomenological features more akin to inner speaking than to inner hearing. What we actually need is three-way distinction among the phenomena that make use of auditory-verbal imagery: (i) a genuinely productive phenomenon (which we are about to introduce, and which constitutes ecologically valid inner speech); (ii) a re-creative productive phenomenon (like the case of imagining yourself speak to someone, which involves inner speaking); and (iii) a re-creative sensory phenomenon (inner hearing). Whereas (ii) involves the same (or much of the same) apparatus as (i), it is used in a different context and for a different purpose (i.e., for the recreation of a counterfactual scenario). On the other hand, (iii) recruits sensory imagery for a similar re-creative activity as (ii). The genuinely productive phenomenon, namely, (i), is what we examine now.
Inner speech acts as the main form of inner speech
Following Roessler (2016) we can distinguish between a “mere act of inner speech” and an “inner speech act”, in a way that perfectly mirrors the distinction between a “mere act of speech” and a “speech act”. Although there are different accounts of speech acts (see Austin 1962, Searle 1969, Bach & Harnish 1979 for some classic formulations) everyone agrees that speech acts are closely tied to the speaker’s mental state in a way that mere acts of speech are not. If you change the mental state in relevant ways, then you change the speech act in relevant ways. Indeed, if you remove the mental state, then you thereby remove the speech act altogether. Examples will make things clearer. Reciting a poem, or repeating an address so as to remember it, is an act of speech, but it is not a speech act. This is, in part, because the speaker, in reciting, or repeating, does not mean what is being said, and any potential variations in the subject’s mental states are compatible with the same act being performed (and variations in what is repeated or recited do not thereby signal similar variations in the subject’s mental states). In stark contrast, sincerely asserting, requesting, demanding, questioning are speech acts. These require the person performing them to be in certain states of mind. For example, an assertion (if sincere) requires the asserter to believe what they are asserting, a question (if sincere) requires the questioner to have the desire to know the answer to the question, and so on.
This fact adds further weight to the point that inner speech is not imagined speech, but rather is speech. Consider the following:
- Jane asserted that p
- Jane imagined asserting that p
- Jane asserted in inner speech that p
Whereas 3 implies 1, 2 does not. In fact, if anything, 2 implies that 1 is false: merely imagining asserting rules out actually asserting (just like imagining raising your right hand rules out you actually doing so). On the other hand, an assertion in inner speech is a perfectly good instance of assertion.6 And insofar as 1 and 3 are both assertions, they both, if sincere, require that Jane be in a certain mental state (i.e., believing that p). In a related manner, assertions that p are treated as evidence for the attribution of the mental states that they (if sincere) require (or express), in this case, believing that p. Thus if someone asserts, “Paris is the capital of France”, you will (other things being equal) think that they believe that Paris is the capital of France. The same applies to other kinds of speech acts, and other kinds of speech acts are intimately tied to other kinds of mental state. Orders and requests are tied to goals, questions are tied to desires to know, compliments are tied to positive evaluations, insults to negative evaluations, etc. And when people request, question, compliment, or insult, if we take them to be sincere, we thereby take them to be in those mental states.
Of course, there is one rather perplexing feature of inner speech, construed as an inner speech act, which is: why do we engage in it at all? Usually when we assert, question, or insult in outer speech, we have an addressee. We are speaking our minds to someone else. When we assert, question or insult in inner speech, who are we doing it for? Who are we speaking our minds to? The answer is: ourselves.
Organisms that live in groups, that cooperate and communicate, can do so very successfully without inner speech, and also without the need to directly introspect. They simply need to express themselves to their conspecifics. These communicative acts do not require the organism to have reflected on, or even have prior access to, its own mental state: the expression can be spontaneous and unreflective. However, once produced, these communicative acts can be perceived and interpreted by the agent who produced them. But of course, this cannot be regularly used as a way of accessing your mental states, since that would involve making your beliefs, desires, plans, and evaluations entirely public. That would often be, at best, socially unacceptable, and, at worst, downright dangerous. Inner speech can be understood in part as a solution to this problem of indiscretion: it is a way of expressing, and hence accessing and reflecting upon, your own state of mind without thereby having to risk giving that information away to others.7
There are many theorists who would be in general agreement with this picture (e.g. Jackendoff 1996, Clark 1996, Carruthers 2011). One interesting feature of positing this role for inner speech is that it suggests that we (at least sometimes, perhaps always) lack other more direct means of reflecting on our mental states. Our view is that inner speech certainly helps a great deal with reflection on our minds, but there are certainly ways of so reflecting that don’t make use of inner speech.
More information: Deqing Li et al, Discriminating deception from truth and misinformation: an intent-level approach, Journal of Experimental & Theoretical Artificial Intelligence (2019). DOI: 10.1080/0952813X.2019.1652354
