Open Models

Updated August 12, 2026

Open models can run on your own hardware or through a managed host such as z.ai. crystl gives both the same setup: save the connection once, check that it is reachable, and point a shard at it by name.

That makes open models useful for more than a local experiment. Route test runs, lint fixes, migrations, documentation passes, and other token-heavy work to open-model workers, and bring in a frontier model where it earns its rate, with orchestration the clearest case: a frontier lead planning and delegating to open-model workers. Every worker stays visible and steerable as a normal crystl shard.

The setup lives with your hosted provider keys: Settings → agents → providers & keys (this page used to be called api keys). Hosted keys are at the top, open-model connections below them. See API key storage for the hosted-key settings.

Tutorials

What a connection stores

A connection is a saved description of one model server. It has:

Field What it does
Label A name for the slot, so you can tell your laptop from the closet box
Kind Ollama, OpenAI-compatible (LM Studio, vLLM, llama.cpp), or anthropic-compatible
Base URL Where the endpoint lives, as seen from your Mac
As seen from remote Optional. The URL an agent running on another machine should use instead (see remote vantage)
Key Optional. A per-slot token, stored in your Keychain and shown masked
Env var aliases Which environment variables crystl exports for this slot. Editable, so an unusual tool can get the variable it actually reads
Enable toggle Off means crystl exports nothing for this slot

The defaults cover the common cases, so most people never touch the aliases:

Kind Typical base URL Exports
Ollama http://localhost:11434 OLLAMA_API_BASE, OLLAMA_HOST
OpenAI-compatible http://localhost:1234/v1 OPENAI_API_BASE, OPENAI_BASE_URL
anthropic-compatible http://localhost:4000 ANTHROPIC_BASE_URL, plus ANTHROPIC_AUTH_TOKEN / ANTHROPIC_API_KEY

vLLM and llama.cpp both speak the OpenAI API, so they use the OpenAI-compatible kind with their own port.

Claude Code on open models

Claude Code reads ANTHROPIC_BASE_URL, so an anthropic-compatible slot is all it takes. When a shard is served by one of these slots, crystl owns the auth variables for that shard too: it sends the slot’s own key, or a documented dummy value when the endpoint doesn’t need one.

That last part is the safety rule, and it is worth stating plainly. A real hosted Anthropic key is never sent to a configured endpoint. If you point Claude Code at somebody else’s server, your frontier key stays home.

There are two ways people use this, and both are first-class.

A local Anthropic-compatible server

Current LM Studio, vLLM, and llama.cpp servers expose the Anthropic Messages API directly. Save their base URL as an anthropic-compatible connection and Claude Code can use it without a proxy. LM Studio documents the complete path in its Claude Code guide.

Ollama does not expose the same Messages endpoint through crystl’s Ollama connection. Put a translation proxy such as LiteLLM in front of it when you want Claude Code rather than Codex or another OpenAI/Ollama-native agent.

# on the machine with the GPU
litellm --model ollama/qwen3-coder --port 4000

Then add a connection:

Field Value
Label proxy-box
Kind anthropic-compatible
Base URL http://localhost:4000
Key leave empty if the proxy is open, or paste the proxy’s master key

Open a new shard and Claude Code runs on your own hardware. Nothing about the prompt leaves the machine.

A hosted open model

You do not need your own GPU. Managed services such as z.ai host open models and provide an Anthropic-compatible URL for Claude Code. Get the service’s API key, save its base URL and key in a slot, and Claude Code runs against its models instead of the frontier API. Follow the z.ai tutorial for the full account-to-shard setup.

Field Value
Label z-ai
Kind anthropic-compatible
Base URL https://api.z.ai/api/anthropic
Key the key from that provider

Two things about this. These are third-party services with their own terms, their own retention policy, and their own view of your code, so read the terms the same way you would for any vendor. And quality varies by task: one of these can be great at mechanical work and weak on the reasoning-heavy change, and that ordering is not stable between providers or over time. Try it on your own work rather than trusting a benchmark. We don’t rank them here and we don’t quote prices, because both change faster than this page does.

Type its name to run it

An anthropic-compatible endpoint you save becomes an agent named after itself. An endpoint called closet-box that runs claude gives you claude@closet-box, and you type that in a shard the same way you type claude:

claude@closet-box

Plain claude in the same shard still runs hosted Claude. That is the point: the two sit side by side, so you can put one task on an open model without moving all of them. The agent is your harness either way — only the model behind it changes.

Add --size to pick which of the endpoint’s three models runs:

claude@closet-box --size small

Those three sizes are the same ones on Settings → agents → agents. One record, edited from either place, so they cannot disagree.

If you would rather every plain claude go to the endpoint, turn on also run plain claude here in its settings. Off is not “disabled” — the name always works.

Turning it on hands plain claude this endpoint’s address and its key, so those sessions can use only the models this endpoint serves, and they bill against its key rather than a signed-in subscription. On an endpoint running on your own machine that includes every hosted model name, so a session that asks for one fails rather than falling back to Anthropic. If you want the local model some of the time and hosted Claude the rest, leave this off and type claude@your-endpoint when you want it.

This is how the env-steered agents work: claude, and aider, opencode and goose, all read their endpoint out of the environment. Codex does not — see below.

Setting one up from the command line

Your agent can do this for you. It knows which API format a provider speaks and what its models are called, which is the part that takes the reading:

crystl endpoint set --label closet-box --kind anthropic \
  --url https://api.example.com/anthropic \
  --agent claude --standard some-model-id
crystl endpoint list

Keys never go on the command line, because a command line lands in your shell history. Pipe one in instead:

pbpaste | crystl endpoint set --label closet-box --key-stdin

Configuring endpoints this way is free on every plan.

Codex on open models

Codex is the exception to the section above. It reaches an open model through its own launch flags rather than the environment, so it is started with --local and does not get a typeable codex@endpoint name:

crystl shard create --gem myapp --agent codex --local closet-box \
  --prompt "fix the lint errors in src/"

That resolves to codex --oss --local-provider ollama for an Ollama slot, or --local-provider lmstudio for an OpenAI-compatible one. Verified against codex-cli 0.147.0.

If your Ollama isn’t on the default address, crystl also exports CODEX_OSS_BASE_URL. Treat that one as best effort: it comes from the provider table inside the Codex binary rather than from documented behavior, so it may change under you. On a default-port setup you need nothing at all.

The lightweight agents

aider, opencode, and goose still work the way they always did. They read the endpoint URL out of the environment, and --local validates that the slot exists before the shard opens.

crystl shard create --gem myapp --agent aider    --local closet-box --prompt "fix the lint errors in src/"
crystl shard create --gem myapp --agent opencode --local closet-box --prompt "run the test suite and report failures"
crystl shard create --gem myapp --agent goose    --local closet-box --prompt "rename the helper across the repo"

goose is launchable now too, steered by GOOSE_MODEL. It’s prompt-run form only, so give it a --prompt.

Model id shapes

The CLIs name models differently, and getting this wrong is the most common first-run stumble.

  • aider takes LiteLLM-style ids: provider/model. For Ollama, use ollama_chat/qwen3-coder rather than ollama/qwen3-coder. Both prefixes resolve, but ollama_chat uses the chat endpoint and gives noticeably better results.
  • opencode takes provider/model, for example ollama/qwen3-coder.
  • goose takes the bare model name in GOOSE_MODEL.

Spawning workers with –local

Typing the endpoint’s own name covers the everyday case. --local is for the other one: starting a shard from the CLI, usually a worker you are fanning out.

--local <label> names the endpoint you want, and crystl resolves it per agent:

Agent What --local does
codex Adds --oss and the matching --local-provider, plus CODEX_OSS_BASE_URL when the address is non-default
claude Checks that the shard’s environment really points at that endpoint. The env vars do the steering
aider, opencode, goose Validates the slot. The environment carries the URL, same as before

The check happens before any shard opens. Name a slot that doesn’t exist, or one that’s disabled, or one whose kind is wrong for the agent you asked for, and you get an error listing your configured slots instead of a shard that quietly ran against the wrong model. A worker that silently used the frontier API when you meant to use your GPU is an expensive surprise, so crystl would rather fail early.

Per-slot keys

The key field solves three problems:

  • Hosted open models need their provider’s key, and it should not be your Anthropic key.
  • vLLM started with --api-key rejects anything without a matching token.
  • OpenAI-SDK tools refuse to start with no OPENAI_API_KEY at all, even against a keyless local server. A keyless slot exports the dummy value local so those tools start.

Keys live in your macOS Keychain, not in a config file, and the settings field shows them masked.

Verify it with crystl status

You don’t have to guess whether the endpoint is up. crystl status prints a reachability line next to the memory telemetry:

$ crystl status
memory: app 412 MB · pressure normal
local model: closet-box http://100.x.y.z:11434 reachable (loaded: qwen3-coder · pulled: +5)

Three states:

  • reachable: something answered.
  • unreachable: nothing answered at that URL from your Mac’s vantage.
  • checking…: the first probe is still running. Read it again in a moment.

Loaded versus pulled

For Ollama slots the reachable line splits the models in two, and the difference matters for how long your first request takes.

  • loaded means the model is in VRAM right now. Ask it something and it answers.
  • pulled means the model is on disk but cold. The first request loads it, and on a big model that can take minutes with no output while it happens.

So loaded: qwen3-coder · pulled: +5 reads as: one model ready to go, five more you could use if you’re willing to wait for the load. If a worker looks hung on its first turn, this line is usually the answer.

Auth challenges count as reachable

anthropic-compatible and OpenAI-compatible slots probe /v1/models. A 401 or 403 back is treated as reachable, on purpose. Something is listening and it told you your credentials are the problem, which is a different fix from a dead address. Check the key, not the URL.

The probe is cached for about 20 seconds and times out after 1.5, so status never blocks or hangs on a sleeping server. The same information rides on GET /gems, so the API and the phone app see it too.

Remote vantage: when the agent is somewhere else

Here’s the case that trips people up. You have a home server with a GPU, and you reach it from your Mac over Tailscale at http://100.x.y.z:11434. That address is correct from your desk. It is the wrong address for an agent running on that server over SSH, because from there the endpoint is simply localhost.

That’s what as seen from remote is for. Fill in both:

Field Value
Base URL http://100.x.y.z:11434
As seen from remote http://localhost:11434

Local shards get the base URL. SSH shards get the remote-vantage URL. If you leave the remote field empty, SSH shards fall back to the base URL, which is the right answer whenever the address works from both sides.

How the URLs reach your agents

Every new shard gets the enabled slots’ URLs as environment variables at launch. Three rules worth knowing:

  • You always win. If your shell profile already exports OLLAMA_HOST, crystl leaves it alone. Anything you export yourself beats anything crystl would have injected, so a per-project override in .zshrc or a direnv file keeps working exactly as before.
  • Existing shards don’t change. Environment variables are set when a process starts, so a shard that was already running when you edited the slot keeps the old values. Open a new shard, or restart the one you’re in.
  • Disabled means absent. Turning a slot off doesn’t set an empty value, it just stops exporting.

Let crystl fill in the model names

You don’t have to go read the endpoint’s model list yourself. Ask for a suggestion:

crystl agent profile suggest --agent codex --slot closet-box

crystl reads the models the endpoint serves and proposes a small, standard, and large mapping for that agent. It shows you the mapping and waits for a yes. It never writes a profile unprompted. The agents settings tab does the same thing by click: probed model names fill the size fields when you pick them.

Set it by hand instead if you’d rather:

crystl agent profile set --agent aider \
  --small ollama_chat/<your-small-model> \
  --standard ollama_chat/qwen3-coder \
  --large ollama_chat/<your-largest-model> \
  --model-via '--model {model}'

After that, anything that asks for a size gets the right model.

Naming an endpoint in a model value

Anywhere crystl accepts a model you can write model@endpoint, naming one of your connections by its label: glm-4.6@glm. It works inside a command’s own flag too, so claude --model glm-4.6@glm is a valid size value.

An endpoint can be an agent

A connection’s edit form can bind one callable agent to it. Give the endpoint a start command and the pairing becomes an agent in its own right, spelled command@endpoint: claude@glm runs claude against your glm connection. It carries its own small, standard, and large, shows up in crystl agent list, and gets a row on the agents settings tab like any other agent.

That spelling names an agent, not a model. Typing it into a size field is caught with a hint showing the accepted forms.

Fanning out onto your own hardware

A local endpoint is one machine with one pool of memory, and that changes the arithmetic an orchestrator is used to. Hosted agents scale out; your GPU does not. Guidance for an orchestrating agent:

  • Check the status line before fanning out. The reachability line is a precondition, not a nice-to-have. Read it first, then decide.
  • Keep it to one or two local workers. Past that they queue behind each other on the same hardware and everything gets slower, including the one you were waiting on.
  • Unreachable means route around it, not debug it. If the endpoint isn’t answering, spawn a hosted agent for that task and move on. Chasing a sleeping GPU is not the orchestrator’s job.
  • Idle is not finished. Workers on hookless agents (aider, opencode, goose) report done or blocked automatically when their process exits, so trust that signal. Mid-run idle is a guess based on CPU, and a worker waiting on a cold model looks exactly like a worker that has stopped. Wait for the exit report.

A hosted open model doesn’t have the one-machine problem, so fan out on it the way you would on any hosted agent.

Caveats

Know what leaves the network. “Local model” and “private” are not the same claim, and the topology decides which one you get.

Topology What leaves your network
Agent on your Mac, model on your Mac or your LAN box Nothing. This is the private one
Local model worker, hosted agent orchestrating it The orchestrator’s whole conversation still goes to a hosted API. The worker being local doesn’t change that
Claude Code against a hosted open model Everything Claude Code sends goes to that provider instead of Anthropic. Different vendor, not fewer vendors

So a local worker under a hosted manager is not an air gap. Only the first row is.

Open models are not frontier models. A good open coder model is genuinely useful on mechanical work. Give it an architectural decision and you may be disappointed. Match the task to the model, and keep the hard calls with the strongest model you have.

Troubleshooting

The agent’s answers are garbled, or tool calls print as raw JSON:

The server’s context window is almost always the cause. Ollama defaults to 4096 tokens, and an agent harness like Claude Code or Codex carries a standing prompt bigger than that on its own, so the server silently truncates it and the model loses the plot. crystl warns about this two ways: crystl status shows a ctx warning next to reachable, and a --local spawn tells you in its create response. The fix lives on the server, not in crystl:

# systemd (Linux)
sudo systemctl edit ollama
# add under [Service]:
#   Environment="OLLAMA_CONTEXT_LENGTH=16384"
sudo systemctl restart ollama

On a Mac running Ollama directly, launchctl setenv OLLAMA_CONTEXT_LENGTH 16384 and restart the app. 16384 is a sane floor for agent work; more costs RAM.

Every turn takes minutes:

Check crystl status for a cpu-only tag. That means the loaded model reports no GPU memory in use, so inference runs at CPU speed, and every turn also re-processes the prompt at that speed. Your options, in order of effect: use a smaller model (a 7B is several times faster than a 14B on CPU), accept it for short tasks only, or move the endpoint to a machine with a GPU the server can use. A cpu-only endpoint is fine for one worker on small tasks. Do not fan out onto it.

unreachable in crystl status:

  1. Is the endpoint actually running? curl http://localhost:11434/api/tags for Ollama, or curl http://localhost:1234/v1/models for an OpenAI-compatible or anthropic-compatible one.
  2. Is the address right from your Mac’s vantage? crystl probes from your Mac, so localhost in the base URL means the Mac’s localhost, never the remote box’s.
  3. For an endpoint on another machine, check the firewall and the tunnel. Ollama binds to loopback by default, so a LAN or Tailscale peer can’t reach it until you set OLLAMA_HOST=0.0.0.0 on the server. On Tailscale, confirm both machines are up in tailscale status.

reachable but every request is rejected:

The probe counts a 401 or 403 as reachable on purpose, so this is the auth case. Put the provider’s key in the slot’s key field, and check you copied the whole thing.

Claude Code still hits the frontier API:

You probably typed claude rather than the endpoint’s own name. Plain claude stays on Anthropic unless you turn on also run plain claude here; claude@your-endpoint does not depend on that toggle. If you did use the endpoint’s name and it still went to Anthropic, the shard was open before you added the endpoint — environment is fixed when a process starts, so open a new one.

“claude.ai connectors are disabled because ANTHROPIC_API_KEY or another auth source is set”:

Expected on any Anthropic-compatible endpoint, and nothing is wrong. The auth source is the endpoint’s own key, which is what makes the session yours rather than Anthropic’s. Claude Code disables claude.ai connectors whenever a session authenticates with a key instead of your login, and says so once at startup. Do not unset the variable to silence it — that is the credential, and removing it sends the session back to Anthropic.

--local errored and no shard opened:

That’s the design. The message lists your configured slots, so compare the label you typed against that list, and check the slot is enabled and the right kind for the agent.

The agent starts but can’t find a model:

  • Check the model id shape. ollama_chat/qwen3-coder for aider, ollama/qwen3-coder for opencode.
  • Confirm the model is actually pulled. The reachable line lists what the endpoint is serving; if your model isn’t in that list, pull it first.
  • Tags are part of the name. A bare model name and the same name with a :tag suffix are two different models to the server, so copy the id from the reachable line exactly.

The first request takes forever, then works:

Cold model. Check whether the status line has it under pulled rather than loaded. Loading a large model into VRAM can take minutes, and there’s no progress output while it happens.

It worked in a new shard but not the one I was in:

That’s the expected behavior. Environment variables are fixed when a process starts. Open a new shard or restart the current one after any change to a slot.

A variable has the wrong value:

Something in your shell profile is exporting it, and your export wins on purpose. Run echo $OLLAMA_HOST in the shard to see what it resolved to, then either remove your export or set the slot’s base URL to match it.

Availability

Configuring open-model connections is free on every plan — in Settings, and from the CLI with crystl endpoint set. So is reading the reachability line in crystl status, and so is typing an endpoint’s agent name in a shard.

Spawning shards from the CLI with crystl shard create is a Guild control command, like the rest of the CLI’s mutations.