Thread - Nostr Hypermedia

ᴛʜᴇ ᴅᴇᴀᴛʜ ᴏꜰ ᴍʟᴇᴋᴜ me@mleku.dev 1 week ago

seems to me like the shortcut resolution for this problem would be in using existing models to build a coherent model of the state space. this part of the process is inherently guesswork, and so its results are not going to be deterministic. but just like you can turn words into a depiction, and with enough detail covering the space, eg, relative positions, curve accelerations in rounded forms, the approximations. i'm sure you've seen some of the magic people have done with CSS, creating nearly photorealistic images. it's a pretty good analogy - it's the difference between RLE and DCT compression. conventional modeling uses the latter, and we want to use something like the former. FLAC compression is probably a good area to look at, and also wavelets, as these produce deterministic decompositions from bumpy sample data. wavelets i mention because of how versatile they are in composition to represent many forms. i immediately recognised in the simple descriptions of their mechanisms and watching the outputs of the LLMs that i was looking at a novel application of these kinds of decompositional/compression/optimizing, i mean, i probably have got this a little muddled but the word "tensors" brings to mind the process of taking a crumpled mesh and optimizing its geometry into a finite/unbounded spherical surface, the reason for the word "tensor" is because it represents tension between the crumpled form and the unfolded form. anyone who has played with the wrappers of easter eggs would be familiar with how carefully you have to work to flatten that wrapped, crumpled (and delicate) aluminium down to its original flat form. so my intuition about tensors is that they are a way of measuring that state transition from flat to folded, and i also understand that the flat form reveals information about the volume of the folded form. so, yeah, if i were to collaborate with you, i'm not so strong on algebra, except where it directly intersects with geometry (trees) but finding similarity between shapes and processes is something that i seem to be particularly optimized for. it's also why my focus is on computation rather than mathematics, like @Pip the WoT guy who - probably like you, has a much stronger grounding in algebra and calculus. i understand it, superficially, and i was taught to do many of these things in advanced math classes in my senior highschool years, but i wasn't interested in it unless it was turned into code, o,r pictures. matrix math, especially, it didn't help that the teacher was absurdly autistic and stereotypic, his droning voice was so tedious, and i didn't see the USE of these grids of numbers being added, multiplied, etc, because they didn't teach us the map between them and geometry. geometry, i have a native sense of. it's how i am able to grasp the importance of the manifold, and the problems of sampling, discrete math, finite fields and approximation and guessing between a noisy sample and the ideal forms that underpin its structure. and yes, that last point. curve fitting. it's the primary reason for the cost in training LLMs. each parameter is like a specific part of the crumpling of the surface into the volume, and every new parameter requires changing the entire map for every part of it that differs between the existing matrix and the input.

↑ Parent

Replies (2)

ᴛʜᴇ ᴅᴇᴀᴛʜ ᴏꜰ ᴍʟᴇᴋᴜ me@mleku.dev 1 week ago

ok, wavelets are not deterministic, they are more composable for natural forms than sine waves because of their decay, thus allowing more flexibility in overlaying them to compose the image. sine waves let you do circles, but wavelets make elliptical, asymmetric forms easier. however, it seems to me like you could derive a more deterministic pattern out of them. as a child, when i got my first computer, the way that you convert a circle into a pixel grid was fascinating. some are like squares, some are like octagons, and as you increase the size, the sequence of the number of pixels approximating the curve gradually progresses towards the limit between discrete and continuous. circles are a concept, and the pixels represent the discretized version. it's not the most expensive quantization, but it is a quantization. it's funny, i hadn't really thought about the meanings of the words, when i read "this model is quantized" and this makes its model smaller, that's what it's talking about - down-sampling from effectively infinite precision down to a size that you can actually generate algorithmically faster than you need to paint it on the screen. for this task, it's about how to take what is effectively discrete data, not sampled like a pixel bitmap, but like a mesh or network. so, the path to this effective algorithmic, algebraic decomposition maps to the process of quantization on a graph. i have been doing some work with the Brainstorm WoT stuff, and i gave them a database driver for neo4j because the main dev @david was familiar with it, but i understand graph data representations well enough that as i repeatedly have said to them, to zero response, is that i can make algorithms that optimize for their queries, if they just define the queries. i already implemented a vertex table that has the property of "adjacency free" which is a fancy word for "searching the table does not require the use of unique keys forming nodes", but rather, that the nodes are in a list and your iteration would use something like a bisection search to get to the start of one node, and then iterate that to find the second one, bisection to find the second (all done by the database index iterator code) and then you can traverse, effectively, in "one" iteration, which is actually just a random scan across a sorted linear index, where each node and its vertexes to other nodes is found in a section of the table. i want to impress upon everyone i've linked into this conversation, that building geometric and algorithmic traversals for data that has an easy visual representation is my super power. i can SEE what i'm doing, which many people who are better at linear stuff, parsing out the code, i'm better at writing it, than reading it. once you clarify the description, i can SEE the pattern as i scan around the visual representation my brain generates from the linear description. the simple fact that i can see how all of these things boil down to graphs, is something that i think is harder for everyone else to do than me. not in theory, but in practice.

2 replies ↓

asyncmind asyncmind@asyncmind.xyz 1 week ago

I like where you're going with the circle-to-pixel analogy. The key thing I’m trying to separate is this: Quantization reduces precision. Canonicalization removes ambiguity. When you rasterize a circle, you’re approximating a continuous form in a discrete grid. That’s lossy, but predictable. What I’m interested in is slightly different: Given a graph (already discrete), how do we produce a representation that: Is deterministic Is idempotent Is invariant under isomorphic transformations Removes representational redundancy So instead of down-sampling infinite precision, we’re collapsing multiple equivalent discrete forms into one canonical embedding. Your “adjacency-free vertex table” intuition is actually very aligned with this. What you described — using sorted linear indexes and bisection traversal instead of explicit adjacency pointers — is essentially treating a graph as a geometric surface embedded in an ordered index space. That’s extremely interesting. Where I think your superpower fits perfectly is here: You see graph geometry. You see motifs and traversal surfaces. What I need in collaboration is: Identification of recurring structural motifs Definition of equivalence classes of graph shapes Proposal of canonical traversal orderings Detection of symmetry and invariants The difference between LLM-style quantization and what I’m building is: LLMs quantize parameters to approximate a surface. I want to deterministically decompose graph state into irreducible geometric motifs. No curve fitting. No parameter minimization. Just structural embedding. If we keep it abstract: Let’s take arbitrary graph shapes and: Define what makes two shapes “the same” Define canonical traversal orders Define invariants that must survive transformation Define minimal motif basis sets If you can see the geometry, that’s exactly the skill required. Your ability to visualize graph traversal as a surface in index space is actually very rare. Most people stay stuck in pointer-chasing adjacency lists. If you’re keen, we can start purely abstract — no domain constraints — and try to build a deterministic canonicalization pipeline for arbitrary graph motifs.

1 replies ↓