Connect a local model

Halbert ships no model and names none on any screen. It offers slots — a role it needs filled — and each slot points at a connection you configured. Connecting a local model is therefore two separate things: adding the connection, and pointing a slot at something that connection serves.

The first part takes about a minute. The second part is where this page spends its time, because “the model runs on this machine” is a claim the software checks in exactly one place, and most of what is on screen does not check it.

Before you start

Something has to be serving models on this machine already. Halbert does not install or manage a model runtime; it connects to one that is running. It probes three fixed loopback ports and nothing else:

Engine	Port
Ollama	11434
LM Studio	1234
The Apple Intelligence bridge	11435

The ports are fixed deliberately — no address you type reaches that probe — so discovery cannot be turned into a way of making the machine fetch a URL of someone else’s choosing. Anything else, including a runtime on another machine, is added by hand and is not discovered.

1. See what this machine is already serving

GET /api/llm/discover answers, and writes nothing: it reports what is running and deliberately does not register anything. On a machine with all three engines up:

$ curl -s -H "Authorization: Bearer $(cat ~/.local/state/halbert/api-token)" \
    http://localhost:8000/api/llm/discover \
  | python3 -c '
import json, sys
for name, engine in json.load(sys.stdin)["data"].items():
    print(name, engine["running"], engine["url"], len(engine["models"]), "models")'
ollama True http://localhost:11434 33 models
lm_studio True http://localhost:1234 34 models
apple_foundation True http://127.0.0.1:11435 1 models

The token is required. The dashboard does not answer callers it cannot identify, and being on loopback is not a credential — every other process running as you is on loopback too.

2. Add the connection

Open Settings → Intelligence → Models & Providers. The card is titled AI Models. Expand Providers at the bottom of it and the list ends in Add a provider, where the local choices are Ollama, LM Studio and Apple Intelligence. Each arrives with its loopback address pre-filled and needs no key. A provider already configured is not offered a second time, so the menu is shorter than that list as soon as you have used it.

Two things happen on their own, before you touch anything:

• On a Mac where the Apple Intelligence bridge is running, that connection is registered for you and assigned to the Secure (Local) slot. The card says so in a banner.
• On a machine with no connections saved at all, a running Ollama is registered for you.

Only one of those two runs on a given machine

The Apple Intelligence step goes first and leaves the saved list non-empty, and the Ollama step only fires when that list is empty. So on a Mac with Apple Intelligence available, Ollama is not registered for you even though it is running and the discovery probe can see it — observed on a first read of a fresh configuration here. Add it yourself; it takes one menu.

3. Point a slot at a model

The card lists four roles, in four columns — ROLE, ENDPOINT, MODEL, TEST:

Role	What it is for
Chat (Guide)	Carries the conversation. The only role that is not optional.
Specialist	Deep reasoning and multi-step plans.
Vision	Screenshots and sensor images. Inherits the chat model unless you set it.
Secure (Local)	Sensitive operations. The only role that refuses a connection that is not local.

Pick the connection in ENDPOINT, pick a model in MODEL. The write lands as soon as you choose; there is no save button. TEST sends a one-word prompt and reports what came back.

That is the whole procedure. What follows is how to know it worked.

4. Check that it is actually local

One function decides this, everywhere, for everything: is_local_model() in halbert_core/halbert_core/model/llm_config.py. Nothing else is a second opinion. The capability probe that reports whether this machine has a local model calls it; the probe behind the secure tier calls it; the store calls it before it will enable the Secure slot. From a clone, so can you:

python -c '
from halbert_core.model.llm_config import is_local_model, resolve
for slot in ("chat_model", "specialist_model", "vision_model", "secure_model"):
    m = resolve(slot)
    if m is None:
        print(slot, "- off")
    else:
        print(slot, "-", "local" if is_local_model(m.model, m.url, m.provider) else "NOT local")'

On the machine this page was written against, that prints:

MLX not available - install with: pip install mlx mlx-lm
chat_model - NOT local
specialist_model - NOT local
vision_model - off
secure_model - local

The first line is a warning from an optional runtime that is not installed, not an error. The four that follow are the answer: two slots on that machine reach off it, and their addresses are localhost.

The rule, exactly

Three clauses, all of which must hold.

1. No cloud tag. A model whose name ends in :cloud is a cloud model wherever it is served from. This is checked first and nothing overrides it.
2. A local provider. ollama, llamacpp, mlx and lm-studio run a model on this machine’s own runtime; mlx and apple-foundation have no network path at all. Outside that set the answer is no, whatever the address says.
3. A loopback address — unless the provider is on-device, where there is no address to judge.

The third clause is necessary and never sufficient, and that is the whole point of the rule. It replaces the inference the address is loopback, therefore local, which is wrong here for a concrete reason: a :cloud model is served through the local daemon and run by the vendor. The request goes to localhost:11434 and the prompt goes to the internet. Of the 33 models the Ollama daemon listed on this machine, 9 were of that kind.

Run against the one function, the cases come out like this:

Model name	Address	Provider	Local?
a-model	http://localhost:11434	ollama	yes
a-model:cloud	http://localhost:11434	ollama	no
a-model	http://192.168.1.50:11434	ollama	no
a-model	http://localhost:11434	openai	no
a-model	(none)	apple-foundation	yes
(empty)	http://localhost:11434	ollama	no

The last row is the design: an unnamed model cannot be proven local, because the name is where the tag would be. Unknown reads as not local everywhere, since a default that reads as local is the bug the whole rule exists to prevent.

Three things that look like a locality check and are not

Each of these was run on the real interface, against a real daemon, and behaved as described.

The Local badge belongs to the provider, not the model

Assign a :cloud-tagged model to the Chat slot on a loopback Ollama connection and the row reads:

Chat (Guide): (a :cloud-tagged model) on Ollama.   [ Local ]

The badge’s own screen-reader text is Local — runs on this machine. It is wrong, and it is wrong by construction rather than by accident: the picker decides a model’s locality from a table of providers, where ollama is marked local and every model it serves inherits that. The tag is never consulted on that path.

The badge answers a different question

Read the Local badge as this connection is a local runtime, which is true. Do not read it as this model runs here, which the badge does not check.

A passing Test means the model answered, not where from

TEST posts a one-word prompt and reports success if anything came back. Pressed twice on the same Chat row, against the same loopback Ollama connection, changing only which model the row named:

untagged model       Passed: Model responded successfully (load: 1.3s)
:cloud-tagged model  Passed: Model responded successfully

Same connection, same button, same green Passed. The only difference is an optional load figure, which the route appends whenever the engine reports one and omits when it does not — pressed a second time on the same already-resident local model it read (load: 0.0s), so it is a timing detail and not a signal about where anything ran. A passing test is evidence the model works. It is no evidence at all about where it works.

Quick setup picks the largest model, and a cloud entry looks like the largest

While no chat model is configured, the card offers one button: Use the largest model that fits my hardware. It chooses the largest model already installed on local Ollama that fits the memory it measured — and it has no idea what :cloud means.

A cloud-tagged entry is listed by the daemon with a download size of a few hundred bytes and a parameter count in the hundreds of billions. To a chooser that ranks by parameters and rejects only what does not fit, that entry is both enormous and free. It wins.

Pressed on this machine — 33 models listed, 24 of them genuinely local — the button assigned a :cloud-tagged model to the Chat slot, and the row then displayed the Local badge described above.

Do not use the button if what you want is a local model

Pick the connection and the model yourself in the two dropdowns. The same chooser runs the first-run auto-selection, so an instance that has opted into it with first_run: → auto_select_model: true can land the same way without anyone pressing anything.

The one slot that refuses

The Secure (Local) slot is the exception, and it is enforced in the store rather than in the interface. Assign it a :cloud model on a loopback connection and the assignment is accepted, recorded, and stored disabled:

$ TOKEN=$(cat ~/.local/state/halbert/api-token)
$ curl -s -X PUT http://localhost:8000/llm/config \
    -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
    -d '{"llm_config": {"secure_model": {"enabled": true,
         "endpoint_id": "ep_ollama", "model": "example-model:cloud"}}}' \
  | python3 -c 'import json,sys; print(json.dumps(json.load(sys.stdin)["data"]["llm_config"]["secure_model"]))'
{"enabled": false, "endpoint_id": "ep_ollama", "model": "example-model:cloud"}

The same call against the same connection, with an untagged name, is kept:

{"enabled": true, "endpoint_id": "ep_ollama", "model": "example-model"}

The endpoint id is minted per installation; GET /llm/config reports the one your connection has.

The server log says why, in one line naming the model, the address and the provider. Nothing on screen does: the row keeps showing the model you chose, and its Local badge simply disappears. It is disabled rather than erased on purpose — the interface is meant to be able to show what was configured and why it will not be used — but today the only place that reason is stated is the log.

So the secure slot cannot be used with a cloud model. It can look, on a quick glance, as though it has been configured with one and is fine.

What is not there yet

• No command reports locality. model-status prints the model, the provider and whether it is loaded; model-list-all lists cloud-tagged and local models together under the same heading. The one-liner in step 4 is the check.
• No screen states the refusal. A disabled secure slot is a missing badge, not a sentence.
• The picker offers three local connections — Ollama, LM Studio and Apple Intelligence. The locality rule also recognises llamacpp and mlx as local runtimes, but there is no way to add either from the interface.

Next

Model locality and the secure tier covers why the rule is shaped this way rather than how to satisfy it. Capabilities, not variants explains why connecting a local model turns a capability on with no flag edited in between — the probe behind that capability is the same function this page has been about.