Thread - Nostr Hypermedia

another cool thing is if you're using ollama locally on your macbook on a plane, and you have no wifi connection, this will still work. Everything in notedeck is offline first. The question gets sent to the local model, the model then responds with a tool query ({query:cats, limit:3}), notedeck parses this query, understands it, queries the local notedeck database, formats a response and send it back to the local ai. the local ai then takes those formatted notes and replies with a summary. this all happens back and forth locally on your computer. how neat is that!?

jb55

still working on the response rendering, but was able to get dave working with my wireguard ollama instance (on a plane!) private, local, nostr ai assistant.

View quoted note →

Replies (11)

Ben Weeks ⚡ ben@bengweeks.github.io 10 months ago

Nice. Would regular users put the API end point address into settings?

jb55 _@jb55.com 10 months ago

yeah ux would be just select ollama provider since it is a standard endpoint

Ben Weeks ⚡ ben@bengweeks.github.io 10 months ago

What model are you running and what’s the performance like on the Mac?

jb55 _@jb55.com 10 months ago

Just this one i had on my ollama instance but i think there are better ones now View quoted note →

Ben Weeks ⚡ ben@bengweeks.github.io 10 months ago

I’m using deepseek-r1:14B at the moment but need to try more out. Currently running it concurrently against GPT-4o (which queries OpenAI API) in Open Web UI to compare output (which is a nice feature) I think I need a bigger GPU! Also been building a custom model (prompt) for Nini.

n 10 months ago

(・∀・)ｲｲﾈ!!

npub15pr6...5uyf 10 months ago

What AI model is integrated into Notedeck and what made you choose it?

jb55

still working on the response rendering, but was able to get dave working with my wireguard ollama instance (on a plane!) private, local, nostr ai assistant.

View quoted note →

1 replies ↓

jb55 _@jb55.com 10 months ago

you can use any model that has tools support

1 replies ↓

npub15pr6...5uyf 10 months ago

So is there a default one that comes with it, or do you have to manually import it in the settings?

1 replies ↓

jb55 _@jb55.com 10 months ago

still deciding on that. maybe would be 4o funded with zaps

1 replies ↓

npub15pr6...5uyf 10 months ago

Please no AI by default. Make that an option in the settings for easy import. While I do appreciate the feature. I am one of those type of people that AI still creeps me out a bit and I am not the only one. When I do use it. It is isolated inside of a virtual machine. Having it native on Notedeck by default and not able to be removed would be a bit of a non-starter situation for me. Purely from a security and privacy standpoint.