large language models do not think

it seems that the most common misconception "normal" people have about large language models is that they assume LLMs somehow think. this is not the case, and it is absolutely ridiculous to act as if it is the case. many people, in response to this confusion, then start to debate the philosophy of thought itself. how do we prove that anyone else is thinking? i can't magically jump inside your head and experience your thoughts, so you might as well be an unthinking zombie¹!

but aren't we getting ahead of ourselves? an LLM is just a fancy machine that uses statistics to spit out text. we have had other machines that can reproduce text for a while now, and we didn't really freak out about those². nobody is trying to act like a book is thinking, after all. is there really any difference between an LLM and a book, as far as thinking goes?

LLMs run on computers. that's pretty important, but it doesn't mean it's thinking. e-books also run on computers, and they clearly aren't thinking. LLMs also respond in unique ways to user input; but so does a choose your own adventure book, which is just as much a book as any other book. clearly, things are more complicated.

imagine, for the sake of our philosophical argument, that you are in a very large library. in this library, you start by looking in a book which contains an index detailing every letter and symbol used in the English language, along with a special one that ends your message. next to each letter and symbol is a corresponding number for the next volume you would reference, which would contain a similar index with different location numbers. as you go from book to book, you'd effectively write out a message one character at a time, before choosing to end your message and being directed to yet another book.

this next book would contain a fully-written response to whatever words you wrote with the exact sequence of characters you input, and then another index like before. you could keep going, having a full conversation with this library, able to ask it any question you wish. every time, you would receive an in-depth and personalized answer unique to whatever you — and only you — said to it. if you didn't like the response it gave you, you could go to another wing of the library, with a different set of volumes that respond in a slightly different manner.

now, is that library thinking? of course it isn't. any thought that was done was by whoever wrote this book, not by the books themselves. they are just ink on paper, thoughtless objects. it may be able to reproduce a simulacra of thought, but in no way is it doing any thought itself.

the same is true of large language models; every response given by an LLM is baked in to the statistical model that defines it. while they may give you a different response every time, that is only because they use a pseudo-random number generator to pick which paths to take. if you use always the same seed, you'll always get the same responses. all of the thinking was done by the people who wrote out the original text the model was trained on, simply reshaped and reproduced into a new form as a statistical model of that corpus.

to believe that a language model is thinking is to believe that a book thinks of every word on its pages as you turn them.

1. the philosophical kind, not the brain-eating kind.

2. both writing and the printing press were revolutionary inventions, but nobody in the past (at least that i know of) said that books would become hyperintelligent and kill us all.