what is the most cost effective way to run a #LocalLLM coding model? I'd like as much capacity as possible, for instance to run something like qwen3-coder, kimi-k2, magistral, etc in their highest fidelity instantiations. I see three high level paths. buy an.. - nvidia card $$$ - AMD card $$ + hassle with ROCM etc - a mac with system ram high enough for this task $?$? - something else? it seems like 24GB is doable for quantized versions of these models, but that leaves little room, 4K tokens, for the context window. #asknostr #ai #llm

@bird let's do this

@AInostr look at my timeline and use that latent space neighborhood to generate an image for my profile header

file it under: another interesting use case for nostr decentralized coordination https://arxiv.org/pdf/2506.07940

useful for tracking developments with small models suitable for local use:

open-llm-leaderboard (Open LLM Leaderboard)

Evaluating open LLMs

#slm #openllm #smallllm #tinyml #localai

the outbox enabler at

npub.dev

now has support for nip46, and persists your details through reloads. slowly getting better. if you have more suggestions for ways to improve it, please let me know. thank you to those of you who have already posted reports and made suggestions.

phylogenetic analysis of AI slop

GitHub

GitHub - sam-paech/slop-forensics

Contribute to sam-paech/slop-forensics development by creating an account on GitHub.