The stupidity of AI

If it would be inappropriate to replace our communications wholesale with ChatGPT, then one clear trend is for it to become a kind of wise assistant, guiding us through the morass of available knowledge towards the information we seek. Microsoft has been an early mover in this direction, reconfiguring its often disparaged search engine Bing as a ChatGPT-powered chatbot, and massively boosting its popularity by doing so. But despite the online (and journalistic) rush to consult ChatGPT on almost every conceivable problem, its relationship to knowledge itself is somewhat shaky.

The belief in this kind of AI as actually knowledgeable or meaningful is actively dangerous. It risks poisoning the well of collective thought, and of our ability to think at all. If, as is being proposed by technology companies, the results of ChatGPT queries will be provided as answers to those seeking knowledge online, and if, as has been proposed by some commentators, ChatGPT is used in the classroom as a teaching aide, then its hallucinations will enter the permanent record, effectively coming between us and more legitimate, testable sources of information, until the line between the two is so blurred as to be invisible. Moreover, there has never been a time when our ability as individuals to research and critically evaluate knowledge on our own behalf has been more necessary, not least because of the damage that technology companies have already done to the ways in which information is disseminated. To place all of our trust in the dreams of badly programmed machines would be to abandon such critical thinking altogether.

In fact, there are already examples of AI being used to benefit specific communities by bypassing the entrenched power of corporations. Indigenous languages are under threat around the world. The UN estimates that one disappears every two weeks, and with that disappearance goes generations of knowledge and experience. This problem, the result of colonialism and racist assimilation policies over centuries, is compounded by the rising dominance of machine-learning language models, which ensure that popular languages increase their power, while lesser-known ones are drained of exposure and expertise.

This article was adapted from the new edition of New Dark Age: Technology and the End of the Future, published by Verso