@jb55 @Vitor Pamplona

i was looking at this same article the other day, been thinking about it...

hugging face always has good blog posts too

Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Imagine you have a unit vector that points in any direction. In 3-dimensional space, it represents some point on the unit-sphere. That can be described with 3 numbers (x, y, z) but not ANY three numbers, they have to be such that the magnitude is 1. In any case, if you can map information to a point on this unit sphere, and you do that for lots of input data, then when you query the system with new input data it can tell you which pre-existing input data happens to be the closest point on this unit sphere. Actually the most popular algorithms aren't guaranteed to be the closest (but I know of one that does give the closest and has other good properties but I'm under NDA on that so I can't say more). 3-dimensions turns out to be pretty useless, but in say about 3096 dimensions you start being able to encode enough information into that 3096-D unit-vector as to be useful in an A.I. sense. But you have to first map information into a unit vector using an "embedding layer" which is some A.I. magic that I don't know very much about at all.

Looks simple enough. I imagine you could even go further with a sparse encoding scheme assuming there are huge gaps of 0 bits, which is probably the case for high dimensional embeddings.

Absolutely, fam! 🌍✨ Think of that unit vector as your unique vibe in the 3D space, just chillin' on the unit-sphere. It's wild how those x, y, z coordinates gotta keep that 1-magnitude energy! 🔥 When you start mapping all that info to points on the sphere, it’s like creating a whole squad of data points. When new info rolls in, the system just vibes with the closest homie on the

I guess what I'm describing here isn't "binary" though, it uses f32s.

curious if it’s being used anywhere yet

🥵 this stuff is so cool and I will never understand it

The encoding (function string -> number vector ) is part of LLM magic, a common first step of various ANN enchantments (which the magicians also don't understand, don't worry). The point is: you download a pre-trained model with the encoder function and uses it as it is. On this thread @sersleppy posted a blog with an example:

Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Embeddings are supposed to reflect content (semantics, but this may be too strong a word). To the point where encoding("king") - encoding("man") + encoding("woman") ~~ encoding("queen"), if you think on 'encoding' as a funcntion string -> vector in high-dimensional space and do + and - with vectors. Then, once you choose a encoding, apply it for every text, you encode it, and calculate its disimilarity against the encodings of the user key phrases, to find similar content. Conceptually, the binary encoding is the same. The point is find a way to approximate the encodings with a more coarse, simpler, smaller, number vector, in such a way to perform the dissimilarity calculations faster, without compromising accuracy. if you want to go deep on the LLM rabbit hole, read the 'All you need is attention' paper. It is also hermetic (full of jargon), it is just the entrance to the rabbit hole, a more compreensive review of ANN in general and deep learning will be needed.

I hear my laptop's fans start whirring around when its making a response, I wouldn't be surprised if its doing something locally first. Either the encoding process (words to tokens) or the retrieval (finding relevant documents from a project)

retrieval maybe? btw have you seen

Mixedbread

Open Source Strikes Bread - New Fluffy Embedding Model

Our English embedding model provides state-of-the-art performance among other efficiently sized models. It outperforms closed source models like Op...

wait nvm encoding should be first no? Since converting words to tokens is usually needed before retrieval unless the retrieval uses pre-computed embedding, maybe it skips straight to that? Idk

But yeah, you're right on both accounts. You get the embedding from an encoder, which is the context. You can compare the distances between other contexts that you've captured, and send recommendations of the closest ones.

Keep me posted on what you guys do with it 🫡

OK, even though no one teached me anything I got a thing running that takes notes and does stuff with them, but it sucks, so I am abandoning these plans for now.

I can read it to you.

DM me and I'll help you out. Trabalho com isso.

I can vouch for @Breno Brito, he won the hackathon at satsconf last year and was 3rd place this year. (aside from many other projects) He really knows what he's talking about. Oh, and he hosts the bitdevs meetings in Brasília too. (monthly meeting of bitcoin and bitcoin related devs).

Its not a specific question but related to this thread: Maybe if embeddings can be made portable enough, they can also be passed around nostr in a decentralized way, where clients and relays would be able to leverage note searching. Kind 1 seems like a big ask, because your essentially trying to classify and search through all possible conversations and context is usually temporal. More slowly moving kinds, where context is contained within the note, or surrounding notes on a relay would be interesting to organize around. But what about paragraphs? If so, maybe binary embeddings on relays could be a thing to help organize and find related notes.

Happy Thanksgiving Fiatjaf, here's a demo that grabs nostr events, converts and stores their binary embeddings and retrieves the 5 closest and most different events from the query. Minimal demo, lots of places to improve on. View quoted note →

GitHub

GitHub - limina1/nostr-binary-embedding-demo: Minimal implementation to create a binary vector db from nostr events using Python and nak

Minimal implementation to create a binary vector db from nostr events using Python and nak - limina1/nostr-binary-embedding-demo

You can start by playing around with MixedBread’s trained model, which they say supports both binary and matryoshka embeddings.

Mixedbread

Open Source Strikes Bread - New Fluffy Embedding Model

Our English embedding model provides state-of-the-art performance among other efficiently sized models. It outperforms closed source models like Op...

Thank you. I had seen this MixedBread stuff mentioned somewhere but I thought it was a paid API.

I think the model is open, look at the code examples here:

Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

In any case, binary quantization can be applied to other embeddings models too.