AI CV screening on a local LLM with Ollama

If you want AI CV screening on a local LLM, so that CV text is never sent to an outside AI service, you need two things: a model server you run yourself, and recruiting software that can talk to it. This guide shows how to do private AI CV screening with Vekt, our open-source recruiting platform, and Ollama: the exact settings, a diagram of where the CV text goes with each AI provider Vekt supports, what the identifier removal does and doesn’t catch, and why the score you get back is a reading-order hint rather than a measurement.

Every Vekt fact below was checked in the Vekt repository (README, .env.example, docker-compose.yml, lib/ai.ts, lib/redact.ts, lib/scoring.ts and docs/compliance/) on 10 November 2026. Facts about Ollama, Docker, OpenAI and OpenRouter link to their own documentation.

The settings

Vekt picks its AI provider from one environment variable. The README lists four values:

`AI_PROVIDER`	What scores the CV	Settings it needs
`mock` (default)	Nothing: a pseudo-score derived from a hash of the prompt	none
`openai`	OpenAI’s API (`OPENAI_MODEL`, default `gpt-4o`)	`OPENAI_API_KEY`
`openrouter`	A model reached through OpenRouter (default `openai/gpt-4o-mini`)	`OPENROUTER_API_KEY`, `OPENROUTER_MODEL`
`ollama`	A model on an Ollama server you run	`OLLAMA_BASE_URL`, `OLLAMA_MODEL`

For Ollama, these are the three lines in .env. The values are the ones .env.example ships with:

AI_PROVIDER="ollama"
OLLAMA_BASE_URL="http://localhost:11434"
OLLAMA_MODEL="llama3.2"

That works when you run Vekt from source on the same machine as Ollama. With Docker, localhost is the wrong address: inside the app container it means the container itself, not your server. There is a second trap too. The shipped docker-compose.yml passes OLLAMA_BASE_URL and OLLAMA_MODEL through as empty strings when you leave them unset, and Vekt only falls back to its defaults when the variables are missing, not when they are empty. So in Docker, always set both explicitly.

Running Ollama next to Vekt with Docker Compose

The simplest private setup puts Ollama in the same Compose project as Vekt, with no port published to the outside. Compose gives every service a DNS name on the project’s default network (Networking in Compose), so the app reaches Ollama at http://ollama:11434 and nothing else can.

If you followed our Docker Compose tutorial, add the ollama service and its volume to the same docker-compose.override.yml. If you are starting fresh, save this as docker-compose.override.yml next to Vekt’s docker-compose.yml:

services:
  ollama:
    image: ollama/ollama
    volumes:
      - ollama:/root/.ollama          # downloaded models
    environment:
      OLLAMA_CONTEXT_LENGTH: "8192"   # see "Context length" below
      OLLAMA_NO_CLOUD: "1"            # local models only
    restart: unless-stopped

volumes:
  ollama:

Then point Vekt at it in .env:

AI_PROVIDER="ollama"
OLLAMA_BASE_URL="http://ollama:11434"
OLLAMA_MODEL="llama3.2"

Start the stack, download the model once, and check that the app picked up the new settings:

docker compose -f docker-compose.yml -f docker-compose.override.yml up -d
docker compose exec ollama ollama pull llama3.2
docker compose logs -f app

Apply to a test job with a CV of your own. In the app log you should see AI: sending evaluation request with provider: "ollama" and endpoint: "http://ollama:11434/api/chat", followed by AI: evaluation completed with the score. If the model answers with something Vekt can’t use (not JSON, or JSON without a score, reasoning, pros and cons), Vekt logs the response and the evaluation step fails, so watch the log the first time you try a new model.

The container line above runs on CPU. Ollama’s Docker page gives the GPU variants: NVIDIA needs the NVIDIA Container Toolkit, and AMD uses the ollama/ollama:rocm image. The .env.example default, llama3.2, is a starting point, not a recommendation: try the models your hardware can run, and compare them on CVs you have already decided on (see below).

Two Ollama settings in that file are worth explaining.

Context length. Ollama’s default context window depends on the GPU memory it finds: 4k tokens below 24 GiB of VRAM, more above that (Context length). Vekt doesn’t set a context size per request, so the server default applies, and the prompt holds the system instructions, the job description, your optional weighting criteria and the whole CV. The docs don’t say what happens when a prompt is longer than the window, so don’t find out with a real candidate: set OLLAMA_CONTEXT_LENGTH, then run docker compose exec ollama ollama ps and check the CONTEXT column. A larger window uses more memory.

Cloud features. Ollama can also run cloud models, tagged :cloud, on Ollama’s own servers. Ollama’s FAQ says that when you run locally, “We don’t see your prompts or data”, and that OLLAMA_NO_CLOUD=1 turns the cloud features off. For CV screening that is the point of the exercise, so set it.

If Ollama runs on another machine, such as a GPU box on your network, set OLLAMA_BASE_URL to that machine. Ollama binds to 127.0.0.1 by default and is opened up with OLLAMA_HOST (FAQ). Vekt’s adapter sends no credentials with its requests, so protect that port at the network level, with a private network or VPN and a firewall, and remember the CV text now crosses that network.

Where the CV text goes, provider by provider

This is the path an application takes through Vekt, from app/api/candidates/route.ts through lib/queue.ts and lib/ai.ts:

 Candidate uploads a PDF CV
            │
            ▼
 ┌──────────────────────────────┐    PDF file    ──► volume uploads
 │ Vekt app (your server)       │    CV text     ──► SQLite, volume db-data
 │  1. unpdf extracts the text  │
 │  2. lib/redact.ts removes    │
 │     direct identifiers       │
 │  3. prompt = job title +     │
 │     description + criteria + │
 │     redacted CV text         │
 └──────────────┬───────────────┘
                │ AI_PROVIDER = ?
   ┌────────────┼──────────────┬────────────────────┐
   ▼            ▼              ▼                    ▼
 mock         ollama         openai              openrouter
 nothing      ollama:11434   api.openai.com      openrouter.ai
 is sent      container on   (OpenAI)            then the model
 (hash of     your server                        provider it
 the prompt)                                     routes to
   │            │              │                    │
   └────────────┴──────┬───────┴────────────────────┘
                       ▼
 Evaluation row in SQLite: score, reasoning, pros, cons,
 the prompt as sent (redacted CV text included), provider,
 model, prompt hash, evaluator version, redaction counts

 Candidate emails, whatever the provider ──► Resend

What that means in practice:

With ollama (or mock), scoring happens on hardware you run, so candidate data stays on your servers. The one exception is email: status emails to candidates go through Resend whichever provider you choose.
With openai, the redacted CV text goes to OpenAI. OpenAI’s data controls page says API data is not used for training unless you opt in, and that abuse-monitoring logs are kept for up to 30 days by default. OpenAI offers a European region at eu.api.openai.com for eligible customers, but Vekt’s adapter calls api.openai.com, so using it would need a code change.
With openrouter, the text goes to OpenRouter and on to whichever company hosts the model. OpenRouter’s logging page says each of those providers has its own retention policy, and lets you restrict requests to providers with zero data retention.
In every case, the prompt Vekt sent, including the redacted CV text, is stored with the evaluation in your database. It is deleted with the candidate when the retention period you set runs out.

Identifier removal: a mitigation, not a guarantee

Before the prompt is built, lib/redact.ts replaces direct identifiers in the CV text with placeholders and counts what it replaced. Here is exactly what the current code catches, and what it doesn’t:

Removed before the model sees the text	Not removed
Email addresses → `[EMAIL]`	Postal addresses
Web addresses, including `linkedin.com/` and `github.com/` links → `[URL]`	University and school names
Phone numbers with an international prefix, or runs of 7 to 15 digits → `[PHONE]`	Languages spoken
Lines labelled date of birth (in English, Dutch, Danish or Norwegian)	Military service, career gaps, graduation years
Lines labelled nationality, citizenship, marital status, gender or sex	Hobbies, memberships, writing style
Each part of the name the candidate typed in the form → `[CANDIDATE]`	A name spelled differently in the CV than in the form

Photos never reach the model, because only the extracted text is sent. The counts are saved with each evaluation (for example {"email":1,"phone":1,"name":3}), so you can see whether the removal ran.

Two side effects follow from how it works. Name matching is a whole-word, case-insensitive replacement, so a candidate called Rose Grant also loses the words “rose” and “grant” elsewhere in the CV. And the phone pattern is deliberately conservative, so that date ranges like “2019 - 2023” survive: some locally formatted numbers will get through.

Vekt’s own accuracy and limitations document is blunt about the rest: proxies such as university names, languages, military service, career gaps and writing style “carry signals of origin, age and gender that the model can and will use.” Running the model locally changes where the text goes, not what the model can infer from it.

Why the scores are uncalibrated

The model is asked for an integer from 0 to 100, two sentences of reasoning, and lists of pros and cons. Vekt rounds and clamps the number and compares it with the job’s threshold, 75 by default. At or above the threshold, the candidate is shortlisted. Below it, the candidate is marked needs review, not rejected, unless an admin has turned on AUTO_REJECT_BELOW_THRESHOLD.

That number is not a measurement. The limitations document says language models “are poor at calibrated numeric judgement”, that the same CV rescored “can drift by several points”, and that the threshold “therefore sits on a noisy signal”. Vekt has never been measured against real hiring outcomes and claims no accuracy figure; we wrote about why we publish that.

Switching to a local model adds one more consequence: scores from different models, providers or prompt versions are not comparable. Every evaluation records its provider, model, prompt hash and evaluator version, so after a switch you can tell old scores from new ones. Don’t rank them against each other, and if a role was open during the switch, re-read the needs review pile rather than trusting the old cut-off.

Before you rely on a local model, do what the limitations document asks of any deployment: score a set of past applications whose outcome you know, with your exact model, prompt and threshold, and check how often a candidate you hired would have landed below the threshold. Repeat it whenever you change the model.

What this setup gives you

Running the screening on Ollama gives you three things a hosted AI API doesn’t: the CV text stays on hardware you run, no second company’s retention policy applies to it, and the model doesn’t change underneath you unless you pull a new one. It doesn’t make the score more accurate, remove bias, or answer your organisation’s questions about lawful basis, privacy notices and candidate requests. Those stay your job.

More on the platform is on the Vekt home page, the code and the MIT licence are on our open-source page, and the full install is in the Docker Compose tutorial.

FAQ

Can I screen CVs with AI without sending them to OpenAI?

Yes. Set AI_PROVIDER="ollama" and point OLLAMA_BASE_URL at an Ollama server you run. Vekt then sends the redacted CV text only to that server. Candidate emails still go through Resend.

Which Ollama model should I use for CV screening?

Vekt’s example configuration uses llama3.2, but there is no tested recommendation: no model has been measured for this task in Vekt. Pick what your hardware runs, then compare candidates’ scores against decisions you have already made before you rely on it.

Why does Vekt fail to reach Ollama in Docker?

Usually because OLLAMA_BASE_URL is http://localhost:11434, which inside the app container points at the container itself. Run Ollama as a service in the same Compose project and use http://ollama:11434, or use an address the container can actually reach.

Does removing names and contact details make AI screening unbiased?

No. Vekt removes direct identifiers such as names, emails, phone numbers, links and labelled personal attributes, but not proxies like universities, languages or career gaps. It is a mitigation, which is why a person reviews every candidate below the threshold by default.

Are scores from a local model comparable with OpenAI scores?

No. Scores are not comparable across models, providers or prompt versions. Vekt stores the provider, model and evaluator version with each evaluation so you can keep them apart.