¶ (c) Technocore - All Rights Reserved.
NOTICE: All information contained herein is, and remains
the property of TechnoCore.
The intellectual and technical concepts contained
herein are proprietary to TechnoCore and dissemination of this information or reproduction of this material
is strictly forbidden unless prior written permission is obtained
from TechnoCore.
¶ Ollama MCP Provider
The ObjAiMcpOllama class provides an interface to a local Ollama server, allowing you to run open-source language models on your own hardware. It is part of the Multi-Cloud Provider (MCP) framework and inherits from the ObjAiMcpBase class.
¶ Initialization
The class is initialized with the following parameters:
def __init__(self, db=0, api_key: str = "", model: str = "mistral", max_turns: int = 50):
db: The database connection object.api_key: Not used for Ollama.model: The name of the Ollama model to use (e.g., "mistral", "llama2").max_turns: Maximum number of conversation turns sent to the model in eachchat()call. Oldest turns beyond this limit are excluded from the request (but remain in_contextfor reference). Defaults to 50.
¶ Methods
¶ prompt
Sends a one-shot prompt to the Ollama model and returns the response. Retries up to MAX_RETRIES times on transient failures with RETRY_DELAY_S seconds between attempts.
def prompt(self, role: str = "", prompt: str = "", image_base64: str = "", options: dict | None = None) -> str:
role: The system role for the AI (e.g., "You are a helpful assistant.").prompt: The user's prompt or question.image_base64: A base64-encoded image for multimodal (vision) prompts.options: Optional dict of Ollama generation options (e.g.,{"temperature": 0.7, "seed": 42}).
¶ embed
Generates an embedding vector for text using the local model. Use a dedicated embedding model such as nomic-embed-text or mxbai-embed-large for best results.
def embed(self, text: str = "") -> list:
¶ chat
Sends a message in a multi-turn conversation, accumulating history in self._context. Subsequent calls include the full conversation history automatically.
def chat(self, message: str, role: str = "", options: dict | None = None) -> str:
¶ reset_context
Clears the accumulated multi-turn conversation history.
def reset_context() -> None:
¶ stream_prompt
Streams a prompt response token-by-token. Yields text chunks as they arrive from the model — useful for TUI or CLI interfaces that display responses in real time.
def stream_prompt(self, role: str = "", prompt: str = "", options: dict | None = None):
for chunk in provider.stream_prompt("You are helpful.", "Tell me a story."):
print(chunk, end="", flush=True)
¶ get_model_registry
Returns the curated model list from resource.config/ollama-models.yaml. Each entry has name, use, tags, and default keys. Returns [] if the file is absent.
registry = ObjAiMcpOllama.get_model_registry()
for m in registry:
flag = "✅" if m["default"] else "➖"
print(f"{flag} {m['name']:25} {m['use']}")
¶ pull_model_set
Pulls models from the registry. When default_only=True (the default), only models marked default: true are pulled. Returns the list of model names that were successfully pulled. Failed pulls are logged via debug() and skipped.
# Pull mistral + nomic-embed-text + llava (the default set)
pulled = provider.pull_model_set()
# Pull the full registry including optional models
pulled = provider.pull_model_set(default_only=False)
¶ smoke_test
Sends a minimal prompt to verify the model actually responds. Returns True when a non-empty response is received.
¶ warm_model
Pre-loads the model into VRAM using ollama.generate(prompt="", keep_alive="5m"). Returns True on success.
¶ get_loaded_models
Returns the names of models currently loaded into VRAM via ollama.ps().
¶ get_capabilities
Inspects model metadata via ollama.show() and returns {"vision": bool, "context_length": int | None}.
¶ Usage
To use the Ollama provider, you must have an Ollama server running. No API key is required.
# One-shot prompt
ai_obj = ObjAI(db=0, model="mcp:ollama:mistral")
response = ai_obj.prompt("You are a helpful assistant.", "What is the speed of light?")
print(response)
# Multi-turn conversation
from ObjAiMcpOllama import ObjAiMcpOllama
provider = ObjAiMcpOllama(model="mistral")
provider.chat("Hello! What can you help me with?")
provider.chat("Tell me more about the first topic.")
provider.reset_context() # start fresh
# Vision (image analysis)
ai_obj = ObjAI(db=0, model="mcp:ollama:llava")
response = ai_obj.prompt("Describe what you see.", image_base64=base64_str)
# Embeddings
provider = ObjAiMcpOllama(model="nomic-embed-text")
vector = provider.embed("some text to embed")
¶ Reasoning models: think defaults to OFF
Added 2026-09-01. Reasoning models — qwen3, deepseek-r1, gpt-oss and
their kin — emit a hidden reasoning stream before the answer, and
ollama returns it in thinking, not message.content. Every entry
point here now takes think, and it defaults to off.
Two things went wrong with it on, both measured on qwen3:8b:
| call | content | done_reason |
|---|---|---|
think on, num_predict=16 |
'' |
length |
think off, same budget |
'OK' |
stop |
A bounded generation can spend its entire budget reasoning and
return nothing, and every unbounded call pays the reasoning tokens in
latency — pure cost for a one-line classification.
smoke_test() is the sharpest case: it asks for "Reply: OK" and judges
the model unhealthy when content comes back empty. With thinking on, a
perfectly healthy model fails it. smoke_test() and the cluster's node
calibration therefore pass think=False explicitly, whatever the
default is set to — the first asserts on content, the second is a
tokens-per-second benchmark that a reasoning preamble distorts.
The gate is the model's reported capability, not its name.
model_supports_thinking() asks ollama.show() and looks for
thinking in capabilities; a name list would be wrong the day a new
reasoning model ships. One round trip per model per process, cached in
THINKING_CAPABILITY. A model without the capability is left
byte-identical — no new kwarg reaches it, even if a caller passes
think=True.
provider = ObjAiMcpOllama(model="qwen3:8b")
provider.prompt("You are terse.", "Reply: OK") # no reasoning
provider.prompt("Solve this.", puzzle, think=True) # reasoning on
ObjAiMcpOllamaCluster imports apply_think from this module rather
than repeating the rule, so both the cluster path and the single-node
fallback behave the same and share one capability cache. A test asserts
they are literally the same function object — a second copy is how
these drift.