Skip to content

I suggest it was Professor Plum, in the library, with the arsenic: the unreliability of brain experience detection

A woman has been convicted in Mumbai for murder, based on a new brain-based experience detection technology. As can be predicted, many regard this as Orwellian while others hope technologies like this could transform the courtroom "as much as DNA evidence has". But there are big problems. The most obvious one is the question of whether the technology actually works, let alone works well enough to be suitable for convicting somebody. The analogy with DNA evidence points at the second big problem: the legal institutions need to learn how to use it well. There are very good reasons for experts in psychology, neuroscience and forensics are troubled by this case.

The technology used in the Mumbai case is called the Brain Electrical Oscillations Signature test (BEOS), and has been developed by Champadi Raman Mukundan. It appears to be based on placing EEG electrodes on the scalp of a person to record brain activity, and then having an investigator read the prosecution’s version of the details of the crime ("I bought arsenic") together with neutral statements ("The sky is blue"). The resulting brain activity is processed to detect whether there is any memory recall of past experiences or just plain cognition as a response to each question. In the Indian case the investigator claimed that the scans were proof of  "experiential knowledge" of having committed the murder.

Most neuroscientists are very critical of this kind of test. Polygraph "lie detectors" have always been suspect because they mainly measure anxiety/arousal. The cause of the signal could be guilt, mental effort or emotions, making the detector fairly useless (despite loud claims by their supporters). Similarly memory retrieval can occur for a wide variety of reasons: when hearing "I bought arsenic" you might involuntarily recall reading a mystery novel where arsenic was in the plot, or your latest store visit. Individual variations could also possibly add to this risk. If, for example, a
certain personality type or mindset led people to have vivid
recollections more often when under tension, they would
appear to be remembering a crime when they actually were just tense
from hearing a murder-related statement.

Another key issue is whether the test can distinguish between recollections and vivid imaginations. It has been demonstrated that the same or similar areas in the brain tend to become activated when we experience something and when we imagine it vividly (e.g. this paper, this review and in particular this paper, which shows that sentences high on imagery can trigger processing similar to reminiscence).

These problems can perhaps be overcome. There is no reason to think that recognition or recall cannot be detected with some degree of reliability – in principle, even if it may not be useful in legal practice. But if we are to get there clearly much testing and investigation needs to be done. Unfortfunately BEOS has not been subjected to outside scrutiny or peer-reviewed publication. While the system is presented (and presumably promoted) to law enforcement at various meetings, it has not to my knowledge been presented at neuroscientific meetings. What clearly is needed is documentation and independent verification showing that the system actually does distinguish between experienced and non-experienced situations, does not have a high rate of false positives and does not discriminate against certain persons. Before that, basing legal decisions on the technology should be regarded as about as valid as basing them on fortune-telling.

The second problem is more profound. I believe there are good reasons to think police investigators or even many forensic psychologists to be up to using the system without biasing it. Witness and interrogation psychology are subtle things, and we already know enough to be very sceptical about what people say, especially when there are elements of coercion or influence involved. We also know from countless psychological experiments that it is very hard for the experimenter not to influence a test subject even unconsciously in the direction of the desired outcome. Taken together, this suggests that the reading of details to the accused matter enormously. A slight shift in stress, a different style of sentence or content between the neutral and the non-neutral statements would completely bias the outcome. In psychology much effort goes into making word lists as similar to each other as possible, hoping to avoid bias. Do we have the confidence that investigators will go to the same length to make the statement of a crime and the neutral words have equal emotional valence, intonation and other subtle properties?

There are good reasons to expect that this sort of system would not be used neutrally. Police has a strong confirmation bias. Famously the Japanese legal system has a 95% confession rate followed by a 99.8% conviction rate due to various biases, and other legal systems also have bias. Even a relatively useless lie detector can still be used thanks to the bogus pipeline effect: by convincing the accused that all lies will be revealed, the chance of getting a confession (true or not) are increased. That alone would make many police forces very happy and not too eager to follow the instruction manual too closely.

DNA testing requires careful handling of samples, checking for contamination from officers at the scene, database management, handling integrity problems and having a policy for storing evidence that may in the future exonerate convicted people.  These  requirements  involve setting up routines and labs, educating people and taking care with the data. If they are not met DNA testing will be useless or worse. Similarly brain-scanning could probably become very useful, but it would need a highly trained organisation to work. As long as the technology is unreliable or its reliability not understood, it should not be used. And given the biases of police, it might be ethically better to only use it for exonerating people by showing that they at least do not recall a crime than proving that they did it.

Share on