For real!! I was thinking I would need to daisy chain some Mac minis to do it!!

What’s the token rate? I tried once to run an ai model on a raspberry pi…needless to say it was unusable lol 😂

To be fair, I am running the distilled 14B model, not the full 700B,… I don’t have the hardware 😁 But I feel it is enough for what I need. I use an Obsidian AI chat plugin that talks to Ollama. I use it to give context to notes, reword sentences, give suggestions, simple text stuff 😁

Hmm not sure but it’s snappy enough for me. I don’t really use it in the terminal, I use it in Obsidian 😬