yeah, you are kinda diving in the deep end there.
i had no idea how to design indexes until i was drowning in fiatjaf's badger index design. then i figured out a few more indexes i could make to improve search performance (and just recently added an inlining for events up to 1kb, as this allows skipping the value table iteration). been thinking also about adding specific graph-oriented indexes to orly specifically for follow list based social graph searches, stuff like "third degree follows with nip-05 domain example.com"
i'm currrently in the process of implementing a dgraph and neo4j database driver, that would allow you to bypass the relay and use the graphql endpoint or other interface as provided by the graph database engine (like dgraphs' DQLs, which are a scripting language for custom graph traversals).
i don't actually know what kind of searches are actually being used for these things. nostr:nprofile1qythwumn8ghj7un9d3shjtnswf5k6ctv9ehx2ap0qyt8wumn8ghj7ct4w35zumn0wd68yvfwvdhk6tcpz4mhxue69uhk5etvd3ukv6tndqhxcctwvshsz9thwden5te0wfjkccte9ejxzmt4wvhxjme0qythwumn8ghj7un9d3shjtnwdaehgu3wvfskuep0qqs0dqlgwq6l0t20gnstnr8mm9fhu9j9t2fv6wxwl3xtx8dh24l4aus0truwy maybe you can point me at info about what kind of graph traversals are needed for a WoT query? i assume the vertex codebase has it in there, i'll be able to figure it out. the intellectual challenge of designing a new type of search index would be amusing to me.
I don't actually use a graph database. In the past I tried neo4j and memgraph (similar but in memory), and both were just too slow. So I builder my own database, specifically designed for the operations I needed to make generating and updating random walks very fast. This allowed to lower the execution time of pagerank by a factor of 100x compared to traditional algorithms.
For the search, I use my other SQLite database to do a full text search for the profiles. Then I rank these profiles using Pagerank, and I create a composite rank which is a combination of search similarity, and reputation. The result is what is powering npub.world
