events need to store their ID, but pubkeys both author and p tags can be replaced with the pubkey index.
yes, this structure enables full graph traversals, currently it just speeds up those kinds of normal tag searches only, i can add this, and the necessary migration for existing databases to upgrade to it. i am going to add it because it seems to me it would save a lot of space in the database, (which is already compact as binary - including eliding all escaping as it's binary), and since the pubkey table will likely be quite small, probably will live in memory mostly and rarely substantially impact query performance.
i'm doing this now. once it's done, graph-native queries could become a thing, with that bidirectional table, with extremely fast iterations after an initial seek.
Login to reply
Replies (1)
claude is thoughtfully adding a proper LRU cache for the pubkey index as well so repeated lookups over a small cluster of events will stay in memory and not even call the database engine. i figured badger would probably do this to some extent automatically but probably an explicit LRU cache should be there for especially your use case but just to cut down the second iteration required for event fetches on at least the pubkeys/ptags
the ones for e tags will always require a second iteration of the event tables, which can actually mean iterating three separate tables, or maybe it's only two, as it has a "small event" table which inlines them into the key table avoiding a second (and usual) value table iteration to fetch the event data.
should be a good fit with adding WoT stuff to amethyst