AI Has a significantly Larger Working Memory Than The Human Brain: What It Really Means | newssparx

The Meaning of "Working Memory" for an AI

Working memory in AI is typically defined by what is referred to as a context window, which is the aggregate amount of text that a model can handle while creating output at one time. It has your prompt, the files you uploaded, previous messages of the conversation itself and its own earlier answers. This means that everything within that window is available to the model right now and anything outside that window simply isn't there. Contextual windows have seen remarkable growth within a just few years. The early large language models could contain a couple of thousand tokens, basically just a few pages worth of text. Current state of the art models work with context windows in the hundreds of thousands of tokens (some systems claim they do millions). For context, a 200k token window comprises around 150,000 English words or two novels’ worth of words.

The Significance of Working Memory Neuroimaging in Humans

The difference lies in how human working memory works. It maintains the brain's workspace for storing and manipulating information for a matter of seconds while you're doing something, remembering a phone number long enough to dial it, or keeping track of the start of a sentence while you finish reading it. For decades, the standard had been what has come to be known as "the magic number seven" that people can deal roughly with seven items simultaneously. Recent cognitive science research has actually revised that number down, pretty far down, indicating the true limit is probably closer to somewhere nearer about four meaningful chunks of information for most people and most tasks. That's still a pretty tiny number compared to the raw capacity of AI systems, but working memory isn't just about item count, it's also influenced by how rich each "chunk" is, which way your attention leans, and what you happen to be thinking or paying attention to right now.

It is also the reason why working memory often is labeled as one of the cornerstones of common cognition. The one that allows you to track a conversation, perform mental math and figure out navigation directions in your mind rather than writing it down on paper. However, in contrast to an AI's context window, which retains all information equally for the duration of the conversation, human working memory deteriorates in a matter of seconds and is extremely prone to interruptions; a single distraction can erase what you were just thinking about.

Bigger Isn't Always Better

Reading a long contract in seconds or having dozens of documents at once the capacity advantage for AI is real and Incredibly powerful with large context windows. But mere size doesn't mean a model actually makes effective use of all of it. Researchers have observed a "lost in the middle" effect, where models remember things at the beginning and end of a long context better than information hidden in between these two ranges. AI is capable of so much more than all the human stuff, but it doesn't always use that space evenly.

The Distinction Between Working and Long Term Memory

To be clear, a big context window is not the same as long term memory. Unless the system provides a distinct memory layer for saving data across sessions, whatever that was in that window usually disappears as a chat finishes. Similar to how your own working memory clears away unless something is stored into longer term memory first, a model with a million token window can perform exceptionally well in one lengthy conversation but yet recalling nothing about you in a subsequent one.

Why It's Significant

A big context window allows AI to process more information at once than a human could comprehend for lengthy papers or codebases. However, don't think that a large window ensures flawless recall, crucial information can still be missed, particularly during lengthy inputs. Additionally, a dedicated memory feature rather than merely a large context window is necessary for anything you need to remember over the course of days or weeks.