I think I’m the same way as you Mleku - often I’ll have concepts that are well defined in my head but I don’t have the nomenclature for them. This can facilitate creative thinking but can inhibit communication with other people, for obvious reasons. A relevant example: only recently did the nomenclature crystallize in my mind the distinction between connectionist AI (LLMs, neural networks, machine learning, etc) versus symbolic AI, aka GOFAI (good old fashioned AI). These two distinct concepts formed in my head as an undergrad in electrical engineering forever ago but didn’t have a name or resurface in my mind until mid or late 2025 when a friend asked me if I had ever heard of “symbolic AI.” I don’t understand the math of connectionist AI, or the math of what you’re doing, well enough to connect what you and asyncmind are talking about to what I’m doing with the knowledge graph. But some of what y’all are discussing definitely catches my attention. I’m wondering whether continuous versus discrete (quantized) is the organizing principle. Connectionist AI deals with continuous spaces where we use tools like gradient descent to arrive at local minima. GOFAI, symbolic AI, and graphs are discrete. Could it be that the basic rules and features of the knowledge graph (most notably: class threads) are an emergent property of well-trained LLMs? I conjecture the answer is yes, as evidenced by things like the proximity of hypernyms (animal type) and hyponyms (dog, cat) in embedding space. Suppose we want to take an embedding and compress it into a smaller file size. Could it be that a graph is the ideal way to represent the compressed file? If so, can we read straight from the graph without the need to decompress the graph and rebuild the embedding space? If so, then we have to know how to query the graph, which means we have to know the rules that organize and give structure to the graph, and the class threads rule seems like a great contender for the first (maybe the only) such rule.

Replies (3)

Hypothesis: class threads (or something like them) are an emergent property of well-trained LLMs. I have no idea what (if anything) this hypothesis has to do with elliptic curves. But maybe … an embedding space that is fully infused and governed by class threads will have some sort of global topological features that we can derive and detect empirically?
exactly! exactly exactly exactly. yes, the LLMs, after jigawatt-hours, nay, yottawatt hours of model massage, come up with systemizations that you could have just asked a computer scientist familiar with compiler construction would have told you. especially types, that was really obvious to me and clearly correct. category theory, which is adjacent, is also a key concept in machine languages. the open questions it left me with, will totally seed the process of creative ideation and lateral thinking that is required to find answers to them. i'm going to follow the steps it lays out in that document, implementing those things, and yes, i think that not only is this going to be faster at learning, especially once it has the grounding in all the theory required to build it, will be able to learn how to learn.
btw, having ideas without words, is precisely something else orthogonal and related to current LLM technology. you can have a much bigger budget for thinking, if you don't translate it to english (or whatever) thus enabling a much greater amount of thinking, which is key to it resolving answers to questions. the frontier models don't do this, because of muh safety reasons. it's kinda stupid because the tokens could be logged and examined for auditing afterwards. all i know is, bumping my thinking token limit up to 20k tokens has dramatically reduced the amount of retarded things that claude does, so if i could have it not translate the thoughts into english, i mean, how precise wouldl it get? the factor is nearly orders of magnitude, but it's "dangerous"