How AI Works—in Plain English

 

 

Science fiction author Arthur C. Clarke once said, "Any sufficiently advanced technology is indistinguishable from magic," yet AI isn't magic, and you don't need to be a computer scientist to understand how ChatGPT works.

If it hasn't already, AI will soon be present in your home and business. Ideally, you should be ready.

Since ChatGPT was released by OpenAI 10 months ago, academics and journalists have been working to describe how AI algorithms work in clear, understandable language.

Three Reads. From beginner to advanced, here are three articles to  help you understand how generative AI operates.

  1. How transformers work (The Financial Times)
    -       In 2017, a group of Google researchers named their novel method for designing neural networks "Transformers," which sparked the current AI revolution. (And no, there is no connection to the movies.)

-       In their work, "Attention is all you need," the researchers presented a simplified method for creating AI language algorithms.

-       The Financial Times' visualization leads you through a few straightforward instances of transformer operation and explains why they enhanced the capabilities of AI chatbots.

-       Word count: 3000. Math required: minimal. 

2   Go deeper into large language models (LLM) (Ars Technica)

-       Every time you ask ChatGPT or any other LLM a question, it makes a sizable amount of calculations before typing you a response.

-       To forecast the following word in any sequence, the LLM has mapped mountains of words to enormous collections of numbers behind the cursor.

-       Journalists Timothy B. Lee and Sean Trott lay out how these "word vectors" operate—and how dozens of layers of machine-learning "neurons" pass clues along in the AI brain to zero in on a good answer.

-       This is the best explanation of artificial intelligence if you only read one.

-       Word count: 6000. Math required: modest. 

3.  The mathematician's perspective, for the rest of us (Stephen Wolfram)

-       Like all computer software, AI programs work by translating everything they touch into numbers. The most recent generative AI programs operate on an unfathomably enormous scale, requiring billions of words to train them and billions of "cycles," or processor operations, to produce a single response.

-       Wolfram's opus is an excellent read if you're searching for a better knowledge of how it all works, not just for language AI but also for image-making systems. It also covers subjects that most articles skip over, such as the enigmatic "temperature" variables that control how much randomness a system introduces into its results.

-       In between the clear lines of his explanations, you can also detect Wolfram's annoyance at how much of the development of today's AI is based on conjecture and common "lore" rather than actual research.

-       Word count: 18,000. Math required: considerable—but it's still understandable even if you never took calculus.

Bonus Read. "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?" provides a look into some of the most scathing critiques of the current AI boom.

-       The well-known and contentious paper from 2021 (authored by Emily Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell) anticipated some of the risks and issues associated with advancing generative AI into widespread public use before we have fully understood and mitigated the risks, including bias, misinformation, privacy violations, and transparency failures.

-       Compared to the other articles that were suggested, it is more scholarly yet still accessible to lay readers.

The Bottom Line. Each of these links represents a creative attempt to reduce a difficult abstraction to a concept that anybody can understand. And none of these could have been created by ChatGPT or any other AI in existence today.

 

Subscribe to Technology This Week