r/ChatGPTCoding
Best open-weight coding model + harness for an on-prem multi-agent setup? (80-200+ GB VRAM, 2-3 concurrent users)
- upvotes
- 5
- comments
- 6
Post
Highlighted: the lines this signal was extracted from
Best open-weight coding model + harness for an on-prem multi-agent setup? (80-200+ GB VRAM, 2-3 concurrent users) Setup: GPU box with 1x H200-class card now, can allocate more (up to several H200s) if it clearly buys better results. Inference via vLLM or similar, OpenAI-compatible endpoint. Coding happens on a separate non-GPU Linux VM on the same network - IDE/harness there, calling models over LAN. Outbound internet is fine for packages/extensions, but inference stays on our hardware (no code to external model APIs). Constraints: on-prem only, open weights, permissive license preferred (Apache/MIT), not Chinese-origin including base models (I understand a lot of fine-tunes are Qwen underneath), NATO-country lab preferred. Use is general software dev plus security tooling and code analysis - repo-level agentic work on existing codebases (fix / extend / refactor / test) is the core, with a human reviewing diffs. 2-3 concurrent users max. Names that came up in a first round elsewhere: Cohere North Mini Code, Mistral Small 4, Poolside Laguna S 2.1, Muse Glimmer 30B, Inkling-Small (2x H200), Reflection Beam (501B MoE / 23B active, weights due this month), Gemma 4 31B, K2 Horizon (lineage TBC), with Nemotron as a generalist baseline - but I haven't run any of them and I'm not wedded to the list. Chinese models (GLM/Qwen/DeepSeek) are out by rule, so no need to suggest them; I...
Keep reading with a free account
The rest of this post, and every signal for Cohere, is in your free account.
Also quoted as evidence
We could get the compute for Beam, Command A+ or Mistral Large 4 if it's actually better for code rather than just for E2E.
From the post