LiteLLMProvider - ICRL

Constructor

LiteLLMProvider(
    model: str = "gpt-4o-mini",
    temperature: float = 0.7,
    max_tokens: int | None = None,
    system_prompt: str | None = None,
    **kwargs: Any,
)

Methods

await provider.complete(messages)
provider.complete_sync(messages)
provider.get_token_profile()
provider.get_last_call_profile()

Behavior Notes

Adds optional system prompt automatically.
Applies defensive truncation before completion calls.
Includes token-budget safety helpers and retry paths for token/output-limit errors.
Uses max_tokens safety logic (does not set both max_tokens and max_completion_tokens).