LiteLLM

Overview

LiteLLM can be deployed as an internal proxy in front of MixRoute. Applications call LiteLLM, and LiteLLM forwards requests to MixRoute while providing unified authentication, model discovery, Dashboard management, virtual keys, and usage records. LiteLLM official Docker Compose deployment guide: https://docs.litellm.ai/docs/proxy/deploy

Docker Compose deployment

Prepare three files:

.
├── .env
├── config.yaml
└── docker-compose.yml

.env:

MIXROUTE_API_KEY=<mixroute-api-key>
LITELLM_MASTER_KEY=<litellm-master-key>
LITELLM_SALT_KEY=<stable-random-salt-key>
POSTGRES_PASSWORD=<postgres-password>

Variable	Description
`MIXROUTE_API_KEY`	MixRoute API key used by LiteLLM when forwarding requests to MixRoute
`LITELLM_MASTER_KEY`	LiteLLM administrator key, used for API access and Dashboard login
`LITELLM_SALT_KEY`	Stable random value used by LiteLLM to encrypt stored keys
`POSTGRES_PASSWORD`	Password for the PostgreSQL service inside Docker Compose

docker-compose.yml:

services:
  postgres:
    image: postgres:17
    container_name: litellm-postgres
    restart: unless-stopped
    environment:
      POSTGRES_DB: litellm
      POSTGRES_USER: litellm
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U litellm -d litellm"]
      interval: 5s
      timeout: 5s
      retries: 20
    volumes:
      - litellm-postgres-data:/var/lib/postgresql/data

  litellm:
    image: ghcr.io/berriai/litellm-database:main-stable
    container_name: litellm
    restart: unless-stopped
    depends_on:
      postgres:
        condition: service_healthy
    env_file:
      - .env
    environment:
      DATABASE_URL: postgresql://litellm:${POSTGRES_PASSWORD}@postgres:5432/litellm
      USE_PRISMA_MIGRATE: "True"
      STORE_MODEL_IN_DB: "True"
    command: ["--config", "/app/config.yaml", "--host", "0.0.0.0", "--port", "4000"]
    ports:
      - "4000:4000"
    volumes:
      - ./config.yaml:/app/config.yaml:ro

volumes:
  litellm-postgres-data:

Variable	Description
`DATABASE_URL`	PostgreSQL connection string for Dashboard, virtual keys, usage records, and persisted model configuration
`USE_PRISMA_MIGRATE`	Runs database migrations on startup when set to `True`
`STORE_MODEL_IN_DB`	Persists model changes made in Dashboard/API to PostgreSQL when set to `True`

Start LiteLLM:

docker compose pull
docker compose up -d

Connect MixRoute

MixRoute Base URL:

https://api.mixroute.ai/v1

Configure LiteLLM in config.yaml:

model_list:
  - model_name: o3-mini
    litellm_params:
      model: openai/o3-mini-2025-01-31
      api_base: https://api.mixroute.ai/v1
      api_key: os.environ/MIXROUTE_API_KEY
    model_info:
      mode: chat

  - model_name: openai/*
    litellm_params:
      model: openai/*
      api_base: https://api.mixroute.ai/v1
      api_key: os.environ/MIXROUTE_API_KEY

  - model_name: anthropic/*
    litellm_params:
      model: anthropic/*
      api_base: https://api.mixroute.ai
      api_key: os.environ/MIXROUTE_API_KEY

  - model_name: gemini/*
    litellm_params:
      model: gemini/*
      api_base: https://api.mixroute.ai/v1
      api_key: os.environ/MIXROUTE_API_KEY

litellm_settings:
  drop_params: true
  request_timeout: 600
  telemetry: false
  check_provider_endpoint: true

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL
  store_model_in_db: true

Configuration	Purpose
`openai/*`	Routes OpenAI-compatible requests to MixRoute `/v1`; suitable for text, embeddings, images, and video APIs
`anthropic/*`	Routes Anthropic-compatible requests through MixRoute; LiteLLM handles the `/v1/messages` path
`gemini/*`	Routes Gemini-compatible requests through MixRoute
`check_provider_endpoint: true`	Enables LiteLLM model discovery through MixRoute `/v1/models`
`drop_params: true`	Drops unsupported upstream parameters to improve cross-provider compatibility
`store_model_in_db: true`	Stores model changes made from the Dashboard/API in PostgreSQL
`o3-mini`	Example alias that maps a short model name to the MixRoute model ID

openai/ is the LiteLLM OpenAI-compatible provider prefix. It does not mean that the upstream model must come from OpenAI.

Client usage

OpenAI-compatible Base URL:

<LITELLM_OPENAI_BASE_URL>

Request header:

Authorization: Bearer <LITELLM_API_KEY>

List models:

curl -sS <LITELLM_OPENAI_BASE_URL>/models \
  -H "Authorization: Bearer <LITELLM_API_KEY>"

Use the returned model IDs in requests, for example openai/gpt-5.5, anthropic/claude-sonnet-5, gemini/gemini-2.5-flash, or the alias o3-mini.

Verification

curl -sS <LITELLM_BASE_URL>/health/liveliness

curl -sS <LITELLM_OPENAI_BASE_URL>/chat/completions \
  -H "Authorization: Bearer <LITELLM_API_KEY>" \
  -H "Content-Type: application/json" \
  --data '{
    "model": "openai/gpt-5.5",
    "messages": [{"role": "user", "content": "Return exactly OK"}],
    "max_tokens": 8
  }'

curl -sS <LITELLM_BASE_URL>/v1beta/models/gemini/gemini-2.5-flash:generateContent \
  -H "Authorization: Bearer <LITELLM_API_KEY>" \
  -H "Content-Type: application/json" \
  --data '{
    "contents": [{"role": "user", "parts": [{"text": "Return exactly OK"}]}],
    "generationConfig": {"maxOutputTokens": 8}
  }'

Gemini generateContent uses the LiteLLM frontend path /v1beta/models/gemini/<model-id>:generateContent; the upstream MixRoute base remains https://api.mixroute.ai/v1.

Conversational AI

Programming

Engineering

Translation

Overview

Docker Compose deployment

Connect MixRoute

Client usage

Verification

​Overview

​Docker Compose deployment

​Connect MixRoute

​Client usage

​Verification

Overview

Docker Compose deployment

Connect MixRoute

Client usage

Verification