Run your coding agent on your own hardware

Updated August 10, 2026

Goal: run an agent in crystl against a model you host yourself, instead of the frontier API.

What you need

  • A model server you can reach: Ollama, LM Studio, vLLM, or llama.cpp, on your Mac or on a box on your network.
  • For Claude Code, an Anthropic-compatible server. Current LM Studio, vLLM, and llama.cpp releases provide a Messages endpoint directly; put a translation proxy such as LiteLLM in front of Ollama. Codex, aider, opencode, and goose can use the Ollama or OpenAI-compatible connection directly.
  • Configuring open-model connections, reading the reachability line, and running an endpoint by typing its agent name are free on every plan. Only crystl shard create from the CLI is a Guild control command.

Steps

  1. Start the server. For example, ollama serve on the machine with the GPU. For Claude Code, put a proxy in front of it:

    litellm --model ollama/qwen3-coder --port 4000

  2. Add an open-model connection. Open Settings → agents → providers & keys, scroll past the hosted keys to open models, and add one:

    Field Value
    Label closet-box
    Kind anthropic-compatible for the Claude Code proxy, Ollama or OpenAI-compatible for everything else
    Base URL the address as seen from your Mac, for example http://localhost:4000
    Key the proxy or server key if it has one, empty if it doesn’t
    Start command the CLI that calls it — claude, codex, aider. crystl fills this in for you

    The start command is what turns the connection into an agent you can type. With claude and a label of closet-box, you get claude@closet-box.

  3. Check that it answers before you spend a shard on it:

    crystl status

  4. Start the agent against the endpoint. A shard that was already open keeps the old values, so it has to be a new one either way.

    With an anthropic-compatible endpoint (the LiteLLM proxy above), Claude Code gets a name you can type in any new shard — claude@closet-box — while plain claude keeps running hosted Claude.

    Codex reads its endpoint from launch flags rather than the environment, so it is started with --local:

    crystl shard create --gem myapp --agent codex --local closet-box \
  --prompt "fix the lint errors in src/"

    --local is checked before any shard opens. A label that doesn’t exist, or is the wrong kind for that agent, gets you an error listing your endpoints instead of a worker that quietly ran somewhere else.

What you’ll see when it works

crystl status prints a reachability line next to the memory telemetry:

local model: closet-box http://100.x.y.z:11434 reachable (loaded: qwen3-coder · pulled: +5)

loaded means the model is in VRAM and will answer now. pulled means it’s on disk and cold, and the first request can take minutes with no output while it loads. In a Claude Code shard, echo $ANTHROPIC_BASE_URL returns your slot’s URL rather than nothing.

Two limits. A good local coder model is genuinely useful on mechanical work and will disappoint you on an architectural decision, so keep the hard calls with the strongest model you have. And a local worker under a hosted manager is not an air gap: the manager’s whole conversation still goes to a hosted API.

Go deeper