LLMs and Models

LLMs and Models#

Sending conversations to LLM providers, and the model metadata gptme uses to pick defaults, count tokens, and check capabilities. See Providers and Models for the user guide, and Provider Integration Guide for adding a provider.

LLM#

gptme.llm.get_available_models(provider: Literal['openai', 'openai-subscription', 'anthropic', 'azure', 'openrouter', 'requesty', 'gptme', 'gemini', 'groq', 'xai', 'grok-subscription', 'deepseek', 'moonshot', 'nvidia', 'local', 'mock'] | CustomProvider) → list[ModelMeta]#

Get available models from a provider.

Parameters:

provider – The provider to get models from

Returns:

List of ModelMeta objects

Raises:

ValueError – If provider doesn’t support listing models
Exception – If API request fails

gptme.llm.get_model_from_api_key(api_key: str) → tuple[str, Literal['openai', 'openai-subscription', 'anthropic', 'azure', 'openrouter', 'requesty', 'gptme', 'gemini', 'groq', 'xai', 'grok-subscription', 'deepseek', 'moonshot', 'nvidia', 'local', 'mock'] | CustomProvider, str] | None#: Guess the model from the API key prefix.

gptme.llm.get_provider_from_model(model: str) → Literal['openai', 'openai-subscription', 'anthropic', 'azure', 'openrouter', 'requesty', 'gptme', 'gemini', 'groq', 'xai', 'grok-subscription', 'deepseek', 'moonshot', 'nvidia', 'local', 'mock'] | CustomProvider#

Extract provider from fully qualified model name.

Returns the provider (built-in BuiltinProvider or CustomProvider).

gptme.llm.guess_provider_from_config() → Literal['openai', 'openai-subscription', 'anthropic', 'azure', 'openrouter', 'requesty', 'gptme', 'gemini', 'groq', 'xai', 'grok-subscription', 'deepseek', 'moonshot', 'nvidia', 'local', 'mock'] | CustomProvider | None#: Guess the provider to use from the configuration.

gptme.llm.init_llm(provider: Literal['openai', 'openai-subscription', 'anthropic', 'azure', 'openrouter', 'requesty', 'gptme', 'gemini', 'groq', 'xai', 'grok-subscription', 'deepseek', 'moonshot', 'nvidia', 'local', 'mock'] | CustomProvider)#

Initialize LLM client for a given provider if not already initialized.

Parameters:: provider – Provider name (built-in or custom)

gptme.llm.is_provider_error(e: BaseException) → bool#

Whether an exception is a recoverable LLM provider/transport failure.

Requires the _gptme_from_llm_reply tag so untagged openai/anthropic/httpx/requests errors from tools or hooks still propagate. See gptme/gptme#3668. requests covers openai-subscription (and similar HTTP backends) which do not raise SDK types.

gptme.llm.list_available_providers() → list[tuple[Literal['openai', 'openai-subscription', 'anthropic', 'azure', 'openrouter', 'requesty', 'gptme', 'gemini', 'groq', 'xai', 'grok-subscription', 'deepseek', 'moonshot', 'nvidia', 'local', 'mock'] | CustomProvider, str]]#

List all available providers based on configured API keys or OAuth tokens.

Returns:: List of tuples (provider, auth_source) for configured providers. auth_source is an env var name for API key providers, or “oauth” for OAuth-based providers like openai-subscription.

gptme.llm.mark_llm_reply_origin(exc: BaseException) → None#

Mark an exception as raised from the provider call inside reply().

Applied after GENERATION_PRE hooks so a hook/tool failure is not treated as a recoverable LLM outage.

gptme.llm.summarize(msg: str | Message | list[Message]) → Message#: Uses a cheap LLM to summarize long outputs.

Models#

Model metadata, resolution, and listing.

Split from the original monolithic models.py into sub-modules:

types: Provider types, ModelMeta, constants
data: Static MODELS dict with per-provider model metadata
resolution: Model lookup, alias resolution, default model management
listing: Model listing, filtering, and display formatting

class gptme.llm.models.CustomProvider#

Represents a custom provider configured by the user.

Subclasses str so it can be used anywhere a provider string is expected, but is distinguishable from plain strings and built-in Provider literals.

class gptme.llm.models.ModelMeta#

ModelMeta(provider: Union[Literal[‘openai’, ‘openai-subscription’, ‘anthropic’, ‘azure’, ‘openrouter’, ‘requesty’, ‘gptme’, ‘gemini’, ‘groq’, ‘xai’, ‘grok-subscription’, ‘deepseek’, ‘moonshot’, ‘nvidia’, ‘local’, ‘mock’], gptme.llm.models.types.CustomProvider, Literal[‘unknown’]], model: str, context: int, max_output: int | None = None, supports_streaming: bool = True, supports_vision: bool = False, supports_reasoning: bool = False, supports_responses_api: bool = False, supports_parallel_tool_calls: bool = False, supports_strict_tools: bool = False, supports_mid_system: bool = True, price_input: float = 0, price_output: float = 0, knowledge_cutoff: datetime.datetime | None = None, deprecated: bool = False, default_tool_format: ‘ToolFormat | None’ = None, preferred_edit_format: Optional[Literal[‘diff’, ‘whole’]] = None, pricing_type: Literal[‘per_token’, ‘subscription’] = ‘per_token’)

__init__(provider: Literal['openai', 'openai-subscription', 'anthropic', 'azure', 'openrouter', 'requesty', 'gptme', 'gemini', 'groq', 'xai', 'grok-subscription', 'deepseek', 'moonshot', 'nvidia', 'local', 'mock'] | CustomProvider | Literal['unknown'], model: str, context: int, max_output: int | None = None, supports_streaming: bool = True, supports_vision: bool = False, supports_reasoning: bool = False, supports_responses_api: bool = False, supports_parallel_tool_calls: bool = False, supports_strict_tools: bool = False, supports_mid_system: bool = True, price_input: float = 0, price_output: float = 0, knowledge_cutoff: datetime | None = None, deprecated: bool = False, default_tool_format: ToolFormat | None = None, preferred_edit_format: Literal['diff', 'whole'] | None = None, pricing_type: Literal['per_token', 'subscription'] = 'per_token') → None#

property provider_key: str#: Return the provider identifier used for availability filtering.

class gptme.llm.models.ProviderPlugin#

A third-party LLM provider registered via the gptme.providers entry point group.

Install a provider plugin with:

pip install gptme-provider-minimax

The plugin package declares the entry point in its pyproject.toml:

[project.entry-points."gptme.providers"]
minimax = "gptme_provider_minimax:provider"

Where provider is a ProviderPlugin instance exported from the package.

Example (inside the plugin package):

from gptme.llm.models import ModelMeta, ProviderPlugin

provider = ProviderPlugin(
    name="minimax",
    api_key_env="MINIMAX_API_KEY",
    base_url="https://api.minimax.chat/v1",
    models=[
        ModelMeta(
            provider="unknown",
            model="minimax/MiniMax-M3",
            context=1_000_000,
            price_input=0.6,
            price_output=2.4,
            supports_vision=True,
            supports_reasoning=True,
        ),
        ModelMeta(
            provider="unknown",
            model="minimax/MiniMax-M2.7",
            context=204_800,
            price_input=0.3,
            price_output=1.2,
            supports_reasoning=True,
        ),
    ],
)

__init__(name: str, api_key_env: str, base_url: str, models: list[ModelMeta] = <factory>, init: Callable[[Config], None] | None = None) → None#

api_key_env: str#: Name of the environment variable that holds the API key, e.g. "MINIMAX_API_KEY".

base_url: str#: Base URL for the OpenAI-compatible API endpoint, e.g. "https://api.minimax.chat/v1".

init: Callable[[Config], None] | None = None#

Optional custom initialisation function.

Called once before the first request is made. Custom init functions must register an OpenAI-compatible client for this provider before returning (for example by calling gptme.llm.llm_openai.init(provider, config)), because plugin traffic is routed through the OpenAI client path. If None, the provider is auto-initialised as an OpenAI-compatible client using base_url and the key from api_key_env.

models: list[ModelMeta]#

List of ModelMeta objects describing the available models.

The provider field of each ModelMeta should be "unknown" and the model field should be the fully-qualified name ("<provider>/<model>").

name: str#: Provider name, e.g. "minimax". Must be unique across all installed providers.

gptme.llm.models.format_recommended_models(fmt: Literal['table', 'rst', 'markdown', 'json'] = 'table') → str#: Render the recommended-model table in the given format.

gptme.llm.models.get_default_model_summary() → ModelMeta | None#

Get the summary model for the default provider.

Returns the cheaper summary model if available for the provider, otherwise returns the default model itself (for local providers, etc.).

gptme.llm.models.get_model_list(provider_filter: str | None = None, vision_only: bool = False, reasoning_only: bool = False, include_deprecated: bool = False, dynamic_fetch: bool = True) → list[ModelMeta]#

Get list of available models with optional filtering.

This is the underlying function used by list_models() and command completers. Results are cached for 5 minutes when dynamic_fetch=True to avoid repeated API calls.

Parameters:

provider_filter – Only include models from this provider
vision_only – Only include models with vision support
reasoning_only – Only include models with reasoning support
include_deprecated – Include deprecated/sunset models (default: False)
dynamic_fetch – Fetch dynamic models from APIs where available

Returns:

List of ModelMeta objects

gptme.llm.models.get_recommended_model(provider: Literal['openai', 'openai-subscription', 'anthropic', 'azure', 'openrouter', 'requesty', 'gptme', 'gemini', 'groq', 'xai', 'grok-subscription', 'deepseek', 'moonshot', 'nvidia', 'local', 'mock'] | CustomProvider) → str#: Return the recommended model name (without provider prefix).

gptme.llm.models.get_summary_model(provider: Literal['openai', 'openai-subscription', 'anthropic', 'azure', 'openrouter', 'requesty', 'gptme', 'gemini', 'groq', 'xai', 'grok-subscription', 'deepseek', 'moonshot', 'nvidia', 'local', 'mock'] | CustomProvider) → str | None#: Return a cheaper/faster summary model, or None to reuse the main model.

gptme.llm.models.infer_supports_mid_system(*names: str) → bool#

Whether these model identifiers accept non-leading system messages.

Qwen3.5’s stock chat template raises System message must be at the beginning. Match qwen3.5 / qwen3_5 only — not the earlier Qwen3 family. Returns False if any name looks like Qwen3.5.

gptme.llm.models.is_custom_provider(provider: str) → bool#: Check if the provider is a custom provider configured by the user.

gptme.llm.models.list_models(provider_filter: str | None = None, show_pricing: bool = False, vision_only: bool = False, reasoning_only: bool = False, include_deprecated: bool = False, simple_format: bool = False, dynamic_fetch: bool = True, available_only: bool = False, json_output: bool = False) → None#

List available models with optional filtering.

Parameters:

provider_filter – Only show models from this provider
show_pricing – Include pricing information
vision_only – Only show models with vision support
reasoning_only – Only show models with reasoning support
include_deprecated – Include deprecated/sunset models
simple_format – Output one model per line as provider/model
dynamic_fetch – Fetch dynamic models from APIs where available
available_only – Only show models from configured providers
json_output – Output as JSON

gptme.llm.models.model_to_dict(model: ModelMeta) → dict[str, Any]#: Convert a ModelMeta to a JSON-serializable dict.

gptme.llm.models.recommended_models_rows() → list[dict[str, str]]#: Rows of (provider, recommended, summary) in PROVIDERS order.

LLMs and Models

Contents

LLMs and Models#

LLM#

Models#