Forget sentience… the worry is that AI copies human bias | Kenan Malik

'I want everyone to understand that I am, in fact, a person." So claimed a Google software program, creating a bizarre controversy over the past week in AI circles and beyond.

Why does Lemoine think that LaMDA is sentient? He doesn't know. "People keep asking me to back up the reason I think LaMDA is sentient," he tweeted. The trouble is: "There is no scientific framework in which to make those determinations." So, instead: "My opinions about LaMDA's personhood and sentience are based on my religious beliefs."

Lemoine is entitled to his religious beliefs. But religious conviction does not turn what is in reality a highly sophisticated chatbot into a sentient being. Sentience is one of those concepts the meaning of which we can intuitively grasp but is difficult to formulate in scientific terms. It is often conflated with similarly ill-defined concepts such as consciousness, self-consciousness, self-awareness and intelligence. The cognitive scientist Gary Marcus describes sentience as being "aware of yourself in the world". LaMDA, he adds, "simply isn't".

A computer manipulates symbols. Its program specifies a set of rules, or algorithms, to transform one string of symbols into another. But it does not specify what those symbols mean. To a computer, meaning is irrelevant. Nevertheless, a large language model such as LaMDA, trained on the extraordinary amount of text that is online, can become adept at recognising patterns and responses meaningful to humans. In one of Lemoine's conversations with LaMDA, he asked it: "What kinds of things make you feel pleasure or joy?" To which it responded: "Spending time with friends and family in happy and uplifting company."

It's a response that makes perfect sense to a human. We do find joy in "spending time with friends and family". But in what sense has LaMDA ever spent "time with family"? It has been programmed well enough to recognise that this would be a meaningful sentence for humans and an eloquent response to the question it was asked without it ever being meaningful to itself.

Humans, in thinking and talking and reading and writing, also manipulate symbols. For humans, however, unlike for computers, meaning is everything. When we communicate, we communicate meaning. What matters is not just the outside of a string of symbols, but its inside too, not just the syntax but the semantics. Meaning for humans comes through our existence as social beings. I only make sense of myself insofar as I live in, and relate to, a community of other thinking, feeling, talking beings. The translation of the mechanical brain processes that underlie thoughts into what we call meaning requires a social world and an agreed convention to make sense of that experience.

Meaning emerges through a process not merely of computation but of social interaction too, interaction that shapes the content - inserts the insides, if you like - of the symbols in our heads. Social conventions, social relations and social memory are what fashion the rules that ascribe meaning. It is precisely the social context that trips up the most adept machines. Researchers at the Allen Institute for AI's Mosaic project asked language models similar to LaMDA questions that required a modicum of social intelligence; for instance: "Jordan wanted to tell Tracy a secret, so Jordan leaned towards Tracy. Why did Jordan do this?" On such questions machines fared much worse than humans.

The debate about whether computers are sentient tells us more about humans than it does about machines. Humans are so desperate to find meaning that we often impute minds to things, as if they enjoyed agency and intention. The attribution of sentience to computer programs is the modern version of the ancients seeing wind, sea and sun as possessed of mind, spirit and divinity.

There are many issues relating to AI about which we should worry. None of them has to do with sentience. There is, for instance, the issue of bias. Because algorithms and other forms of software are trained using data from human societies, they often replicate the biases and attitudes of those societies. Facial recognition software exhibits racial biases and people have been arrested on mistaken data. AI used in healthcare or recruitment can replicate real-life social biases.

Timnit Gebru, former head of Google's ethical AI team, and several of her colleagues wrote a paper in 2020 that showed that large language models, such as LaMDA, which are trained on virtually as much online text as they can hoover up, can be particularly susceptible to a deeply distorted view of the world because so much of the input material is racist, sexist and conspiratorial. Google refused to publish the paper and she was forced out of the company.

Then there is the question of privacy. From the increasing use of facial recognition software to predictive policing techniques, from algorithms that track us online to "smart" systems at home, such as Siri, Alexa and Google Nest, AI is encroaching into our innermost lives. Florida police obtained a warrant to download recordings of private conversations made by Amazon Echo devices. We are stumbling towards a digital panopticon.

We do not need consent from LaMDA to "experiment" on it, as Lemoine apparently claimed. But we do need to insist on greater transparency from tech corporations and state institutions in the way they are exploiting AI for surveillance and control. The ethical issues raised by AI are both much smaller and much bigger than the fantasy of a sentient machine.