OllamaClient

Category: LLM Providers

Source: ollama_client.dart

Classes

OllamaClient

LLM client for Ollama local API with streaming.

Ollama uses NDJSON streaming (not SSE) and its own message format. Tool calling uses OpenAI-compatible tool schemas but returns arguments as parsed objects (not JSON strings).

num_ctx injection. Ollama silently defaults to num_ctx: 2048 for every request regardless of what the model was trained with — a notorious footgun that silently truncates agent loops. We resolve the model's real window once (exact catalog → daemon /api/show → catalog base-name) and always inject options.num_ctx = min(resolved, ollamaNumCtxCeiling), falling back to [ollamaDefaultNumCtx] (not 2048) when nothing resolves. The resolved real window is exposed via [contextWindow] for the context-occupancy gauge (null when only the default applied).

Constructor

OllamaClient({
    required this.model,
    required this.systemPrompt,
    String baseUrl = 'http://localhost:11434',
    int? contextWindow,
    int? contextWindowFallback,
    http.Client Function()? requestClientFactory,
  })

Properties

Property	Type	Description
model	String
systemPrompt	String
contextWindow	int? get
contextWindow	int? get

Methods

Stream<LlmChunk> stream(List<Message> messages, {List<Tool>? tools})

Constants

Name	Type	Description
ollamaNumCtxCeiling	int	Hard ceiling on the num_ctx override Glue will send. Keeps us from forwarding absurd context windows (some catalogue entries claim 1M+) that would blow past the user's RAM budget on mid-range GPUs. 128K is comfortably above every real agent conversation and matches what the upstream ecosystem (Continue, Cline, opencode) settled on. Exposed publicly so tests can assert it without magic-number copies.
ollamaDefaultNumCtx	int	Default num_ctx for Ollama models whose real context window cannot be resolved from the catalog or the daemon. Anything but Ollama's silent 2048 default; deliberately conservative so mid-range GPUs stay safe.