Setup: bring your own model, bring your own cloud

Keel ships no model and no cloud account. It runs against whatever you already have, which keeps the licensing simple and keeps your documents on infrastructure you control. This page covers the four ways people point it at something, and what to do when a first run stalls.

The short version:

git clone https://github.com/BlakeR-M/keel.git
cd keel
python -m venv .venv

source .venv/bin/activate     # macOS and Linux
.venv\Scriptsctivate        # Windows

pip install -e ".[dev]"
keel up

keel up does the whole first run: finds a chat server you already have, loads the fixture corpus, starts the appliance and opens it. Each step is skipped once it has happened, so running it again just starts the server.

The fixture corpus is five invented documents, there so the question box has something to answer and so the access-control comparison has a restricted document to withhold. Installing Keel to answer from your own work instead? Run keel up --empty and arrive at a store holding nothing. Already ran it the other way? keel documents clear empties the store, and the fixtures stay on disk under fixtures/ for whenever you want them back.

The pieces separately, when you want to watch each one:

keel setup      # finds a chat server you already run and writes .env
keel doctor     # confirms the configuration, and names the fix for anything unready
keel serve      # http://127.0.0.1:8400

keel setup looks for a chat server answering on this machine. Ollama, LM Studio, llama.cpp and vLLM all expose the same OpenAI-compatible /v1/models, so whichever you have is found and its model list read back to you. Choosing one writes KEEL_LOCAL_LLM_BASE_URL and KEEL_LOCAL_LLM_MODEL into .env, then loads the fixture corpus so there is something to ask straight away.

What Keel downloads

One thing, once: the embedding and reranking models that retrieval uses, about 150 MB total (BAAI/bge-small-en-v1.5 and Xenova/ms-marco-MiniLM-L-6-v2, both ONNX on the CPU through fastembed). They land in the fastembed cache on first use and are read from disk after that.

Warm that cache while you have a network, and every later run works with the cable out. Air-gap mode sets HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1 for you, so a warm cache plus KEEL_AIRGAP=1 is a complete offline install. The Docker image bakes both models in at build time, so a container needs no download at all.

Everything else is yours to provide.

Option 1: a model on your own machine

The default, and the one that keeps every document and every question on your hardware.

Runtime	Start it	Endpoint `keel setup` looks for
Ollama	`ollama serve`, then `ollama pull llama3.1:8b`	`http://127.0.0.1:11434/v1`
LM Studio	Load a model, switch on the local server	`http://127.0.0.1:1234/v1`
llama.cpp	`llama-server -m model.gguf --port 8081 --jinja`	`http://127.0.0.1:8081/v1`
vLLM	`vllm serve <model>`	`http://127.0.0.1:8000/v1`

Then:

keel setup

To skip the search and name one directly:

keel setup --base-url http://127.0.0.1:11434/v1 --model llama3.1:8b

Any OpenAI-compatible server works, including ones not in that table. If it answers GET /v1/models and POST /v1/chat/completions, Keel can use it.

Model size and answer quality. Retrieval, the entitlement filter, refusals, screening, the ledger and the approval queue all behave the same whatever model you attach. The model affects the wording of answers. A 3B model is enough to see the system work; 7B and up reads noticeably better. keel eval scores whichever one you attach, so you can measure the difference rather than guess at it.

Option 2: a hosted OpenAI-compatible endpoint

OpenAI, OpenRouter, Groq, Together and most others speak the same API, so they work with the same two settings plus a key:

keel setup --base-url https://api.openai.com/v1 --model gpt-4o-mini --api-key sk-...

Read this before choosing it. A hosted model means the passages retrieved for each question leave your network and reach that provider. That is a reasonable trade for a small business with ordinary documents, and it is the wrong choice where the documents are the reason you wanted Keel. Air-gap mode refuses those hosts by design, so KEEL_AIRGAP=1 and a hosted model are mutually exclusive, and that is deliberate rather than an oversight.

The entitlement filter still runs before generation either way, so the provider only ever receives passages the person asking was entitled to read.

Anthropic's API uses a different request shape, so pointing --base-url at it directly falls over. Reach it through a proxy that presents an OpenAI-compatible surface, or open an issue if a first-class provider would be useful to you.

Option 3: your own Azure tenancy

The cloud profile runs inside your subscription on a user-assigned managed identity, with no key in the code and no key in the configuration. The Bicep template disables local key authentication on both Azure OpenAI and Azure AI Search, so a key is not merely unused, it is turned off.

If you already have the resources:

keel setup --profile azure \
  --azure-openai-endpoint https://<resource>.openai.azure.com \
  --azure-search-endpoint https://<search>.search.windows.net \
  --chat-deployment gpt-4o-mini \
  --embed-deployment text-embedding-3-small

If you want Keel to create them in your subscription:

az login
.\deploy\azure\deploy.ps1 -ResourceGroup keel-rg -Location australiaeast -WhatIf   # preview
.\deploy\azure\deploy.ps1 -ResourceGroup keel-rg -Location australiaeast           # deploy

The script checks the Azure CLI, previews with az deployment group what-if, creates the resource group, deploys the template, prints the endpoints and smoke-tests /health. Edit image in deploy/azure/main.bicepparam first so it points at your own container registry. Full detail lives in deploy/azure/README.md.

Install the cloud extra so the credential libraries are present:

pip install -e ".[azure]"

Option 4: Docker

Two stacks, and the first one is the short path. With a model runtime already on the machine, this needs no GGUF placed anywhere: Keel reaches your runtime through the host bridge and loads the fixture corpus on first start.

docker compose -f deploy/onprem/docker-compose.host-model.yml up -d --build
# then open http://127.0.0.1:8400

It defaults to Ollama on 11434. Point it elsewhere before up:

KEEL_HOST_MODEL_URL=http://host.docker.internal:1234/v1 KEEL_HOST_MODEL=your-model   docker compose -f deploy/onprem/docker-compose.host-model.yml up -d --build

The guard stays on in that stack, with the host bridge as the one destination it permits. Prove it from inside the container:

docker compose -f deploy/onprem/docker-compose.host-model.yml exec keel   python -m keel.web.airgap_probe data.attacker.example

To keep everything in containers instead, docker-compose.yml brings its own llama.cpp server:

# put a GGUF at deploy/onprem/models/model.gguf, or set KEEL_MODELS_DIR and KEEL_MODEL_FILE
docker compose -f deploy/onprem/docker-compose.yml up -d --build
docker compose -f deploy/onprem/docker-compose.yml exec keel keel ingest --manifest fixtures/corpus.yaml

Swap in docker-compose.gpu.yml for CUDA offload. docs/onprem.md covers the model swap, sizing, backups and the network-level air-gap proof with docker run --network none.

Adding your own documents

Two ways, and they run the same pipeline.

From the command line, covered in the tutorial:

keel ingest ./handbook.pdf --tags public
keel ingest --manifest your-corpus.yaml

Or from the admin page, under Documents. Choose a file, name the access tags, and it goes through section-aware chunking, the injection screen and a ledger row exactly as a command-line ingest does. Keel reads PDF, DOCX, Markdown, HTML and plain text, up to 25 MB.

Tags are the whole access-control model, so the form asks for them rather than guessing. A reader holding any one of a document's tags can retrieve it, an empty field means public, and the filter applies from the moment the upload finishes: a document tagged finance is invisible to everyone without that tag, in retrieval, in citations and in the source viewer alike.

The upload route lives behind the admin guard, so beyond loopback it needs KEEL_ADMIN_TOKEN like every other write. A deployment that sets KEEL_DEMO_READONLY=1 has no upload route at all: it is left unregistered rather than registered and refusing, which is how the hosted demo runs.

When a first run stalls

keel doctor

Every check names the fix rather than printing a stack trace:

profile: local
   ok  data directory  D:\keel\data is writable
   ok  air-gap         off, so outbound connections are unrestricted
check  model endpoint  http://127.0.0.1:8081/v1 is out of reach. ConnectTimeout: timed out
                       Something is answering elsewhere on this machine: Ollama at
                       http://127.0.0.1:11434/v1. Run `keel setup` to point at it.
   ok  corpus          5 document(s) in the store

The four that catch almost everything:

Symptom	What it usually is
Model endpoint out of reach	The server is on a different port. `keel doctor` probes the well-known ones and tells you where it found something.
Model name rejected, endpoint fine	`KEEL_LOCAL_LLM_MODEL` differs from what the server serves, often by a tag such as `:8b`. `keel doctor` lists the real names.
Every question refused	The store holds no documents. Run `keel ingest --manifest fixtures/corpus.yaml`, or ingest your own.
Air-gap on, every model call refused	The model is on a host outside the allow list. Add it with `KEEL_AIRGAP_ALLOW_HOSTS`, or move the model to loopback.

Refusals that say "the sources do not cover the question" are the entitlement filter and the relevance gate working. That answer costs zero model calls, so it appears instantly even with no model attached at all.

Settings worth knowing

keel setup writes only what it needs. Everything else has a working default and lives in .env.example.

Setting	Default	What it decides
`KEEL_PROFILE`	`local`	`local` for your own model server, `azure` for your own tenancy
`KEEL_LOCAL_LLM_BASE_URL`	`http://127.0.0.1:8081/v1`	The OpenAI-compatible endpoint
`KEEL_LOCAL_LLM_MODEL`	`qwen2.5-3b-instruct`	The model name that endpoint serves
`KEEL_LOCAL_LLM_TIMEOUT`	`120`	Seconds per model call. Raise it on a slow CPU
`KEEL_LOCAL_LLM_WAKE_SECONDS`	`0`	Seconds a call keeps retrying a model host that is still waking from sleep. Set it when the model server sleeps when idle; page loads then start the wake early
`KEEL_DATA_DIR`	`./data`	Where the SQLite store and the ledger live
`KEEL_AIRGAP`	`0`	`1` refuses every outbound connection outside the allow list
`KEEL_AIRGAP_ALLOW_HOSTS`	unset	Extra hosts the guard permits, comma separated
`KEEL_FRONT_PAGE`	`auto`	What `/` serves. `auto` gives the question box on an installed appliance and the overview on the hosted demo. `overview` and `chat` decide it outright, and `/about` keeps the overview either way
`KEEL_HOST`	`127.0.0.1`	Binding beyond loopback turns the admin guard on
`KEEL_ADMIN_TOKEN`	unset	Needed for `/admin` once bound beyond loopback
`KEEL_PROXY_TOKEN`	unset	Lets an authenticating reverse proxy assert identity
`KEEL_MIN_RELEVANCE`	`0.15`	Below this fused score Keel refuses rather than guesses

Tutorial walks the whole appliance, from ingest through the agent to the ledger.
Web app covers the routes and the identity model behind a reverse proxy.
On premise covers Compose, GPU offload and offline operation.
Architecture is the module-by-module walkthrough.