nos.lol db size 587 gb
pruning older, less important, low WoT ones
Login to reply
Replies (16)
Wot?
Web of Trust
Always actually wondered who ran those relays, at 587 gb that's one hell of a service you've provided.
thanks. i should do a better job. like automating this pruning 😃
The feds have a back up 👍
How long did it take to grow to that size?
This should be highly compressible, especially with a dictionary zstd
3+ years
i pruned a few times before
good idea @npub1yxpr...qud4
especially maybe old events ?
So one could easily run a relay on a 1TB phone
👋
Or put the database on ZFS. But. There is always a but.
ZFS is a cow filesystem, it may be slower, but it may actually be faster because of the compression. Everything depends on the underlying hardware
i wish there was deflate option. then the clients could be directly served from db
Zstd dictionanaries should be small, max 1MB, and often 128kb to keep zstd fast. If every klient pulled such dictionary once a few months, all event data could be kept and served compressed, very efficiently. This is probably what FB does in it's client?
Compression is underrated, we have very fast cpus now, very fast algos like zstd, proper setup will actually increase the client speed and decrease the network data being sent
yes compression looks doable. maybe even tell my clanker to do deflate because then there is no decompression, directly serving to clients over websockets.
Better bitcoin than AI. Honestly.