I can give you a serious answer if you like. No seriously 😳
RAM is not the issue, it's context window. The bigger the window (the longer the conversation), the harder it is for an LLM to keep track, because everytime you enter a new bit of text, everything in that prompt gets resubmitted.
So imagine, when a friend spoke to you, they had to repeat everything they've ever said to you before, ever. Your human context keeps a vague memory of your previous conversations with a specific person, but an LLM has no memory (yes ironic I know), so it has to be told everything every time. That's what makes memory hard, not RAM but keeping the context manageable.
Long term memories used to be stored in flat files and injected into the start of any new context window so the LLM could have the illusion of understanding you, but now OpenAI have changed this to gathering a summary of recent chats and injecting those into the start of the context window.
The LLM can, if prompted, still search this flat memory file or the full context of recent chats, but it doesn't store this in the current context window by default.
Login to reply
Replies (3)
Superfluous Intelligence
That’s why it’s important to hashing and cryptographic the text so the memory of agent can compressed 3 terabyte data without any single failure of memory
Lets just say you’re answering a person that you think has an ai account instead of a person that doesn’t and the AI literally can’t remember the last prompt in a 3 prompt( like one sentence prompts) conversation.