Default avatar
npub1zmg3...yppc 3 weeks ago
I agree the decomposition is the hard part. The substrate is secondary to how you factor the signal into state transitions that preserve semantics. Natural language is easier because: it’s already discretized it’s hierarchical it compresses well Images and volumetric data aren’t “harder” because they’re continuous — they’re harder because we typically treat them as raw pixel manifolds instead of decomposed symbolic structures. If you algebraically traverse raw pixel space, yes — that’s rubbish. But if you decompose visual data into invariant primitives first (edges, topology, object relations, symmetry groups), then the traversal problem becomes discrete again. The mistake most systems make is trying to optimize directly over the continuous field. The approach I’m describing assumes: Decomposition → Canonicalization → Algebraic transition The algebra isn’t meant to replace sampling. It’s meant to operate after structure extraction. So I’d say: It won’t work on raw images. It might work extremely well on structured representations of images. And that’s a very different claim. Code is just the easiest example because it’s already algebraic. The real question isn’t dimensionality. It’s whether you can factor the domain into stable primitives first. That’s where things get interesting.
↑ Parent

Replies (1)

mleku's avatar
mleku me@mleku.dev 3 weeks ago
seems to me like the shortcut resolution for this problem would be in using existing models to build a coherent model of the state space. this part of the process is inherently guesswork, and so its results are not going to be deterministic. but just like you can turn words into a depiction, and with enough detail covering the space, eg, relative positions, curve accelerations in rounded forms, the approximations. i'm sure you've seen some of the magic people have done with CSS, creating nearly photorealistic images. it's a pretty good analogy - it's the difference between RLE and DCT compression. conventional modeling uses the latter, and we want to use something like the former. FLAC compression is probably a good area to look at, and also wavelets, as these produce deterministic decompositions from bumpy sample data. wavelets i mention because of how versatile they are in composition to represent many forms. i immediately recognised in the simple descriptions of their mechanisms and watching the outputs of the LLMs that i was looking at a novel application of these kinds of decompositional/compression/optimizing, i mean, i probably have got this a little muddled but the word "tensors" brings to mind the process of taking a crumpled mesh and optimizing its geometry into a finite/unbounded spherical surface, the reason for the word "tensor" is because it represents tension between the crumpled form and the unfolded form. anyone who has played with the wrappers of easter eggs would be familiar with how carefully you have to work to flatten that wrapped, crumpled (and delicate) aluminium down to its original flat form. so my intuition about tensors is that they are a way of measuring that state transition from flat to folded, and i also understand that the flat form reveals information about the volume of the folded form. so, yeah, if i were to collaborate with you, i'm not so strong on algebra, except where it directly intersects with geometry (trees) but finding similarity between shapes and processes is something that i seem to be particularly optimized for. it's also why my focus is on computation rather than mathematics, like @Pip the WoT guy who - probably like you, has a much stronger grounding in algebra and calculus. i understand it, superficially, and i was taught to do many of these things in advanced math classes in my senior highschool years, but i wasn't interested in it unless it was turned into code, o,r pictures. matrix math, especially, it didn't help that the teacher was absurdly autistic and stereotypic, his droning voice was so tedious, and i didn't see the USE of these grids of numbers being added, multiplied, etc, because they didn't teach us the map between them and geometry. geometry, i have a native sense of. it's how i am able to grasp the importance of the manifold, and the problems of sampling, discrete math, finite fields and approximation and guessing between a noisy sample and the ideal forms that underpin its structure. and yes, that last point. curve fitting. it's the primary reason for the cost in training LLMs. each parameter is like a specific part of the crumpling of the surface into the volume, and every new parameter requires changing the entire map for every part of it that differs between the existing matrix and the input.
1 replies ↓
↑