Working on something similar for myself. Some things that may lead you in a good direction: - LMStudio - OpenWebUI - CrewAI The context limitations can be adjusted and navigated around.