The ObjAiMcpGroq class routes prompts through
Groq, which serves open-weight models
on its own LPU hardware behind an OpenAI-compatible
endpoint. The free tier offers generous rate limits on
several large models and is several times faster than
local Ollama for the same workload. Inherits from
ObjAiMcpBase and reuses the official openai Python
client with a custom base_url.
def __init__(
self,
db: object = 0,
api_key: str = "",
model: str = "llama-3.3-70b-versatile",
) -> None
db: database connection.api_key: Groq API key (gsk_...).model: Groq model id (no vendor prefix).In config.yaml:
ai_mcp_groq:
api_key: gsk_XXXXXXXXXXXXXXXXXXXXXXXX
from ObjAI import ObjAI
ai = ObjAI(
db=0,
model="mcp:groq:llama-3.3-70b-versatile",
)
print(ai.prompt(
"You are a credit-risk analyst.",
"Summarise the outcome distribution.",
))
To use Groq for the sim email AI analysis, set
ai.sim_analysis_model in config.yaml:
ai:
sim_analysis_model: mcp:groq:llama-3.3-70b-versatile
(Subject to change — check the Groq model list.)
| Model | Notes |
|---|---|
llama-3.3-70b-versatile |
Strong general reasoning |
llama-3.1-8b-instant |
Fastest, smaller |
qwen/qwen3-32b |
Good code / structured output |
mixtral-8x7b-32768 |
Long context |
llama-3.2-90b-vision-preview |
Multimodal |
Vision input (image_base64) is only honoured by
vision-capable models — sending it to a text-only model
surfaces a 400 from the API.