Thread

why that cpu ?

It’s workstation grade. Building a server. Other than that, no particular reason.

Interesting CPU choice. With that CPU and GPU are you planning on building a server to run local AI models?

Yes… oh very much yes.

And this is still in the plans, but selling access to the AI as well.

I have wanted to figure out how to run a local AI model for maximum censorship resistance. What model did you land on? Do you plan on doing any tutorials? Any links or advice would be very much appreciated.

Yeah, I plan on using Kobold.cpp for loading the AI, the model I’m using will be dolphin-mixtral (to start off), and possibly using Silly Tavern for the chat interface. I could do some tutorials that are blog based and put them on my website if that would help.

I am definitely interested in following along

Cool. I’ll do a write up then.