Skip to main content

v1.67.4-stable

Krrish Dholakia
Ishaan Jaffer

Key Highlights​

Let's dive in.

New Models / Updated Models​

  • OpenAI
    1. Added gpt-image-1 cost tracking Get Started
    2. Bug fix: added cost tracking for gpt-image-1 when quality is unspecified PR
  • Azure
    1. Fixed timestamp granularities passing to whisper in Azure Get Started
    2. Added azure/gpt-image-1 pricing Get Started, PR
    3. Added cost tracking for azure/computer-use-preview, azure/gpt-4o-audio-preview-2024-12-17, azure/gpt-4o-mini-audio-preview-2024-12-17 PR
  • Bedrock
    1. Added support for all compatible Bedrock parameters when model="arn:.." (Bedrock application inference profile models) Get started, PR
    2. Fixed wrong system prompt transformation PR
  • VertexAI / Google AI Studio
    1. Allow setting budget_tokens=0 for gemini-2.5-flash Get Started,PR
    2. Ensure returned usage includes thinking token usage PR
    3. Added cost tracking for gemini-2.5-pro-preview-03-25 PR
  • Cohere
    1. Added support for cohere command-a-03-2025 Get Started, PR
  • SageMaker
    1. Added support for max_completion_tokens parameter Get Started, PR
  • Responses API
    1. Added support for GET and DELETE operations - /v1/responses/{response_id} Get Started
    2. Added session management support for non-OpenAI models PR
    3. Added routing affinity to maintain model consistency within sessions Get Started, PR

Spend Tracking Improvements​

  • Bug Fix: Fixed spend tracking bug, ensuring default litellm params aren't modified in memory PR
  • Deprecation Dates: Added deprecation dates for Azure, VertexAI models PR

Management Endpoints / UI​

Users​

  • User Info Panel: Added a new user information pane PR
  • Global Sorting/Filtering:
    • Added global filtering to Users tab PR
    • Enabled global sorting to find users with highest spend PR
    • Support for filtering by user ID PR

Teams​

  • Team Filtering:
    • Added team-based filtering to the models page PR
    • Support for filtering by team ID and team name PR
  • Team Model Selector: Bug fix for team model selection PR

Keys​

  • Key Management:
    • Support for cross-filtering and filtering by key hash PR
    • Fixed key alias reset when resetting filters PR
    • Fixed table rendering on key creation PR

Logs​

  • Session Logs: Added UI Session Logs documentation PR

Authentication & Security​

  • Required Authentication: Authentication now required for all dashboard pages PR
  • SSO Fixes: Fixed SSO user login invalid token error PR
  • Encrypted Tokens: Moved UI to encrypted token usage PR
  • Token Expiry: Added token expiry logic to user dashboard PR

UI Refinements​

  • Fixed UI Flicker: Addressed UI flickering issues in Dashboard PR
  • Improved Terminology: Better loading and no-data states on Keys and Tools pages PR
  • Azure Model Support: Fixed editing Azure public model names and changing model names after creation PR

Logging / Guardrail Integrations​

  • Datadog:
    1. Fixed Datadog LLM observability logging Get Started, PR
  • Prometheus / Grafana:
    1. Enable datasource selection on LiteLLM Grafana Template Get Started, PR
  • AgentOps:
    1. Added AgentOps Integration Get Started, PR
  • Arize:
    1. Added missing attributes for Arize & Phoenix Integration Get Started, PR

General Proxy Improvements​

  • Caching: Fixed caching to account for thinking or reasoning_effort config PR
  • Model Groups: Fixed handling for cases where user sets model_group inside model_info PR
  • Passthrough Endpoints: Ensured PassthroughStandardLoggingPayload is logged with method, URL, request/response body PR
  • Fix SQL Injection: Fixed potential SQL injection vulnerability in spend_management_endpoints.py PR

Helm​

  • Fixed serviceAccountName on migration job PR

Full Changelog​

The complete list of changes can be found in the GitHub release notes.