@Guy Swann what GPU are you running in your AI rig? I am trying to fine tune a model and the lack of adequate VRAM is really making this difficult.
Login to reply
Replies (4)
Might help. Idk shit about fuck tho 😶
What local LLM models are you using that actually seem to be useful? Please describe a successful use case you've had with it!
View quoted note →
I don't know shit about shit but went with a 4060ti with 16gb of ram for just dipping my toes in and dropping a fortune. Been running some of the lower parameter R1s and coder models with good speed.
I’m actually using a 3090 still and it pretty much kills it. Was planning to upgrade to a few 3090s or 4090s (2 or 4) to ramp it up a little but didn’t want to do it until my studio was done and my workflow was stable… I’m not wondering if either will ever occur 😅
But just a single 3090 is pretty damn useful for any and all Ai stuff.
Awesome. And yes I'm scraping by with a 1660. I actually just successfully tuned a small 3B model but I had to get real creative with the script. There is probably a easier way instead of reinventing the wheel but I wanted to use my own data set and model set. I really need to upgrade