ChatGPT is dumber than you think (2023)

As a technology critic, I have to say that the enthusiasm for ChatGPT, a great language model trained by OpenAI, is misplaced. As impressive as it may be from a technical point of view, the idea of ​​having a machine to conduct conversations and generate responses raises serious concerns.

First, ChatGPT lacks the ability to truly understand the intricacies of human language and conversation. It is simply trained to generate words based on a given input, but lacks the ability to actually understand the meaning behind those words. This means that the responses generated are likely to be superficial and lack depth and insight.

Also, the reliance on ChatGPT for conversations raises ethical issues. When people begin to trust a machine to conduct conversations for them, it can result in the loss of real human connections. The ability to connect with others through conversation is a fundamental aspect of being human, and outsourcing this to a machine can have adverse side effects on our society.

to hold on I, Ian Bogost, did not write the three paragraphs above. A friend sent me screenshots of his session with ChatGPT, a program released by OpenAI last week that you interact with by typing in a chat window. It's actually a large language model (or LLM), a type of deep learning software that can generate new text after being trained on large amounts of existing written material. My friend's request was: "Create a favorable review for ChatGPT in the style of Ian Bogost."

ChatGPT wrote more, but I saved the rest because it was too boring. The AI ​​wrote another paragraph about responsibility (“If ChatGPT says or does something inappropriate, whose fault is it?”), and then a final paragraph that repeated the rest (even starting with “Conclusion…”). In short, you have written a five paragraph basic essay in high school style.

This fact can comfort you or scare you, depending on your preferences. When OpenAI released ChatGPT to the public last week, the first and most common reaction I saw wasFear that it will turn education upside down🇧🇷 "You can no longer take tests at home," said Kevin Bryan, a professor at the University of Toronto.SentOn twitter. "I think chat.openai.com could really spell the end of writing tasks."i wroteSamuel Bagg, a political scientist at the University of South Carolina. That is the fear.

But you can rest easy knowing that the bot's output, while smooth and engaging as text, is consistently uninteresting as prose. It is stereotyped in structure, style, and content. John Warner, the author of the book.Why can't they write?He protested the five-paragraph essay for years, writing aTwitter threadon how ChatGPT reflects this rule-based, standardized form of writing: "Students were essentially trained to mimic writing," he tweeted. AI can produce believable writing, but only because writing and our expectations have become so lax.

Even trying to trick the reader into passing off a copy of AI as one's own, as I did above, has become a tired trope, an expected twist on an overlong Twitter thread about the future of generative AI rather than a startling reveal about The abilities. . On the one hand, ChatGPT is capable of producing engaging prose. But, on the other hand, what it means to be persuasive depends on the context. The kind of prose you might find convincing and even surprising in the context of a generative encounter with an AI suddenly seems just right.horribleas part of a technical article published in a magazine such asthe atlantic🇧🇷 And, as Warner's comments make clear, the writing you find attractive as a professor (or marketing executive, lawyer, journalist, or whatever) may be due to position rather than importance: the essay was there and he was competent; the report arrived in your inbox on time; the newspaper article mentioned obvious facts that you could accept or deny.

Maybe ChatGPT and the technologies behind it are less about persuasive writing and more about great bullshit. A liar plays with the truth for the wrong reasons: to get away with it. The initial reaction to ChatGPT assumes that it is a tool to help people write essays or student posts or whatever. It's an easy conclusion for those who assume that AI is meant to replace human creativity rather than change it.

The Internet, and the entire technology sector in which it floats, seems like a giant organ for nonsense, for expanding human access to language and amplifying lies. Online, people cheat and cheat and fight each other. Deep learning AI makes all this worse by hiding how software like LLMs work so that no one, not even their creators, can explain what it's doing and why. open AIgiftsHis work is context-agnostic and experimental, with no specific use cases; he says that he launched ChatGPT just to "get user feedback and learn more about its strengths and weaknesses." Not surprisingly, the first and most obvious assumption about ChatGPT is that it is a threat, from something, everything.

But ChatGPT is not a step towards an artificial general intelligence that understands all human knowledge and text; it's just an instrument to play with all that knowledge and all those lyrics.GamesJust work with raw materials to see what they can do. You play a game or an instrument to make use of materials you are familiar with in unexpected ways. LLMs certainly won't replace faculty, magazines, or middle managers. But they offer these and other fields a new tool, that's really the right word, for playing with an unfathomable mass of textual material.

I spoke to ChatGPT on a wide range of topics including poetry, behavioral psychology, and computer programming to test this theory out. For now, ChatGPT is just a tech demo, a research experiment. What's less clear is how it could be used, beyond dire predictions about which industries its technology could bring down. In almost all cases, the AI ​​seemed to have knowledge and the means to express it. But when the time came, and the chat interface made it easy, the bot almost always had to admit that it was just making something up.

Sometimes models were simply used. When I asked ChatGPT to create a cover letter for a college job, they competently but uninspiredly delivered. He also produced the exact same letter for a magazine editor job and for a cannabis innovator job in the Web3 space ("I have a deep understanding of the Web3 space and the unique challenges and opportunities it presents"). Fair enough, after all, most cover letters are terrible, repeating the same mundane lines and abstract gestures.

In other cases, the bot produced surprisingly synthetic but buggy creative output. When I asked him to write an imaginary poem about hamburgers, in the style of Ezra Pound or William Carlos Williams, this little refrain came up:

Not bad I guess, especially for a computer. But the poem isn't exactly an example of imagism, either, trying to evoke a precise image through conscious language (like "The dense honesty of a/a hamburger"). So I called ChatGPT. "This is not an imaginary poem." His answer is worth reciting in full:

I also asked the AI ​​to create a Lai (medieval narrative poem) in the style of Marie de France about the popular Texas fast food chain Whataburger. It looked a lot like a fucking Lai and it started like this:

Except for the last line, which is a metric disaster. The AI ​​was unable (or unwilling) to assess his work ("I'm a large language model trained by OpenAI and I don't have the ability to assess the quality of literary works or make aesthetic judgments"), but was able to tell me about the typical structure of a lai: a short 12th-century narrative poem of specific structure and form, usually written in eight-syllable verse. The lyrical errors produced in the Lai result from the breach of this metric form.

When I questioned him on the matter, he again admitted that no, the lai he wrote was not structured in octosyllable couplets, stating that it produced "a more modern and playful interpretation of the lai form." He was beginning to feel that he was negotiating with a student who had come to the office to complain about his grade.

It happened over and over again. I asked for the source code for an Atari game about picking up cat litter, and the AI ​​sent me valid programming instructions (it understood the task), but only disjointed snippets of actual code titled "This program creates a simple game in which the player...must use a shovel to scoop up his cat's litter and throw it into a trash can.It was more an icon of the answer he was looking for than the answer itself.

So I asked for a set of diagnostic criteria for a made-up psychological condition I called "inherited truculence." The AI ​​immediately engaged:

Except that challenge doesn't really equate to "aggressive and hostile thoughts and feelings", right? I asked, and the AI ​​agreed, eventually revising its diagnosis after my request ("A tendency to experience and express challenging or confrontational thoughts and feelings," etc.).

Also, my request to give an ontological and epistemological answer to the questionWhen does bread become toast?he provided a generally fluent and apparently philosophical response. He said in part: "If we can recognize it as roasted based on its appearance, texture and other sensory properties."

But the philosophy of the toastit is a specialty for me, so I could see that this passage, supposedly an epistemological argument, was actually about perception. "Isn't the second a phenomenal argument and not an epistemological one?" I challenged. And again, the AI ​​called Unkle without hesitation: “You're right, the second argument I made is phenomenal, not epistemological. An epistemological argument focuses on how we know or understand something, while a phenomenal argument focuses on our experience or perception of something.

At this point, chatting with ChatGPT started to look like any other interaction on the web, where a guy (always a guy) tries to turn a draft Wikipedia article into a case of maximum experience. Except that ChatGPT was always willing to admit that it was wrong. Immediately and without complications. And anyway, the bot also knew quite accuratelywhywas wrong. That sounds good, but it's actually terrible: if you already need to have the knowledge to identify the problems with the LLM-generated text, but the purpose of the LLM-generated text is to avoid the need for such knowledge, then you're in for a pickle. from hecho. Maybe it's time for this paragraph about the surrender of accounts after all.

But that is not the purpose of ChatGPT. Does not accurately argue or express creativity, but produces textual material in a way that reflects the explicit or implicit intent of the researcher and may contain the truth under certain circumstances. Unfortunately, this is an accurate representation of texts of all kinds: online, in books, on Wikipedia, and more.

Read: Google's "responsive" chatbot is our elusive future

Proponents of LLM generativity can dismiss these concerns. Some do this by glorifying the apparent and fully realized genius of the GPT in embarrassing ways I can only stomach.for shortcutrepeat instead of repeat. Others, more measured but no less delighted, might claim that it is "still in its infancy" for a technology that is only a few years old but can already produce excellent 12th-century poetry about Whataburger. But these are the opinions of the personalities of the IT professionals who have ruined their computer and online life the most, not to mention their own lives. open AIassumethat his job is destined to become an artificial general intelligence, a machine that can do anything. Instead, let's set a less ambitious but more likely goal for ChatGPT and its successors: to provide an interface to the textual infinity of digitized life, an impenetrable space that few people can use effectively today.

To explain what I mean by that, let me show you a very different exchange I had with ChatGPT, in which I used it to help me find my way through the obscurity of text, rather than fooling myself with its creator abilities. words.

"I'm looking for a specific type of window covering, but I don't know what it's called." I told the bot. "It's some kind of blindness, I guess. What kinds are there?" ChatGPT responded with a litany of displays, which was nice. I made it clear that he had in mind something similar to a blind, but made of fabric. "Based on the description he provided, it sounds like you're thinking of a Roman blind," he replied, offering more details and a mini sales pitch for this window technology.

Dear reader, I really know what a roman blind is. But to lose that knowledge and still have to use it to understand the world, that's very hard to do with computers today. Getting something in the world often comes down to putting a set of existing materials into the expected linguistic form. That goes for Google or Amazon searching for window coverings or whatever.mostly fail now, which requires complicated and time-consuming clumsiness for the machinery to show even the general direction of a response. But it also applies to student essays, thank you notes, cover letters, marketing reports, and maybe even medieval lais (if one intends to create one). We've all faked with words. We are drowning in an ocean of content, desperate for the lifeboat of form.

ChatGPT offers this form, but, and this is where the bot accidentally partially recognized my position correctly, it doesn't do it out of knowledge. The AI ​​does not understand or type any text. It provides a way to examine text, play with text, shape and model endless prose from a wide range of fields, including literature and science, and post shit in structures where more questions can be asked and occasionally answered.

The GPT and other important linguistic models are aesthetic rather than epistemological tools. Imagine a strange, unholy synthesizer whose keys sample textual information, style, and semantics. Something like this is not mandatory because it is an optionanswersin the form of text, but because it allows you to play text, almost any text, as an instrument.

This result can be revealing! However, there is one big obstacle in the way: people who don't know what to think about LLMs, ChatGPT and every other generative AI system that has come along. Its creators haven't helped either, perhaps in part because they don't know what these things are for either. OpenAI does not provide a framework for ChatGPT and presents it as an experiment to "make AI systems more natural to interact with", a worthy but deeply ambitious goal. Without additional structure, it's no surprise that ChatGPT users portray their own creations as existential threats or perfect achievements. Neither result is true, but both are also boring. Imagine that you are concerned about the fate ofWrite tests to take home., a silly format that everyone hates but no one has the nerve to kill. But imagine pinging a computer that has just composed something that looks like a medieval poem about a restaurant, because its lines are not all measured correctly! Of course, you can use this opportunity to cheat on school exams or cheat yourself at work. A boring person would do that. That's what a computer expects.

Computers have never been an instrument of reason capable of solving human problems; They are just devices that structure the human experience through a very specific and extremely powerful method of symbol manipulation. This makes them aesthetic and functional objects. The GPT and its cousins ​​offer them the chance to take advantage of the offer: to use computers not to perform tasks, but to play with the world they have created. Or rather: destroy it.