Traditionally when using databases you have to serialize data in and out. With nostrdb your in-memory data structures are the same as the disk representation. It uses virtual memory to map the entire database (up to 32gb). When accessing data the operating system only reads parts of this file into memory when accessing data. The end result is you get the speed of an in-memory data structure but without having to allocate 32gb of memory to read it all (bad and impossible) Since nostrdb uses lmdb, it arranges the data into an efficient search tree so that the most common accessed data is always hot in the cache. The end result is that the code is much simpler and faster. The next step is to replace the in memory note cache with nostrdb as well. This will put damus apps in a league of their own, thanks for your patience as I hook all this up πŸ™
npub1ahck...7v97
Profiles in nostrdb (#1520) This adds profiles to nostrdb and replaces our in-memory and ProfileDatabase cache with it - Remove in-memory Profiles caches, nostrdb is as fast as an in-memory cache - Remove ProfileDatabase and just use nostrdb directly ## TODO - [ ] cache lnurl properly - [ ] add database versions for migrations nostrdb-profiles, 5 commits https://github.com/damus-io/damus/pull/1520
View quoted note →

Replies (23)

I came out of a web developer bootcamp but have no professional experience yet, still a newbie. Could I use nostrdb in place of a traditional backend (with a SQL dB) when building a website? Like a cooking recipe site, for example (user provides title, ingredients list, directions list, author name, image which make up a single record in your recipe dB): could those all be NOSTR notes instead of rows in a SQL dB?
I did but it barely shows up in the profiler because its in the nanosecond range and doesn’t get picked up by the profiler πŸ˜†
This is close to the way that Redis manages its .rdb images (as distinct from their alternative AOF file format.
lmdb guarantees that when you have a pointer to a page it will remain there until the lifetime of the program. It has copy on write semantics when updating records. Notes are immutable anyways so this is not a problem.
Oh don't know yet. I built a recipe website as a project in my coding bootcamp several months back and while I was looking for a free option to host it (I did eventually find a free option for the backend--for a long time all the free options I saw were for frontend only, once Heroku shifted to paid only) I remember at one point I wondered if I could do it with NOSTR instead of a database and just build a frontend that simply collects notes that have been tagged as recipes. The workings of NOSTR are still totally foreign to me, though, so I quickly gave up that idea so I could get on with my project and finish the bootcamp. But it remains this niggling half-formed idea in the back of my mind.
So that was a lie. It only guarantees pointer validity during the transaction or until the next write, i guess lmdb moves pages around
↑