Skip to main content

​
Constructor

LiteLLMProvider(
    model: str = "gpt-4o-mini",
    temperature: float = 0.7,
    max_tokens: int | None = None,
    system_prompt: str | None = None,
    **kwargs: Any,
)

​
Methods

await provider.complete(messages)
provider.complete_sync(messages)
provider.get_token_profile()
provider.get_last_call_profile()

​
Behavior Notes

  • Adds optional system prompt automatically.
  • Applies defensive truncation before completion calls.
  • Includes token-budget safety helpers and retry paths for token/output-limit errors.
  • Uses max_tokens safety logic (does not set both max_tokens and max_completion_tokens).