> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mixroute.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# LiteLLM

> Deploy LiteLLM with Docker Compose and route requests to MixRoute

## Overview

LiteLLM can be deployed as an internal proxy in front of MixRoute. Applications call LiteLLM, and LiteLLM forwards requests to MixRoute while providing unified authentication, model discovery, Dashboard management, virtual keys, and usage records.

LiteLLM official Docker Compose deployment guide: [https://docs.litellm.ai/docs/proxy/deploy](https://docs.litellm.ai/docs/proxy/deploy)

## Docker Compose deployment

Prepare three files:

```text theme={null}
.
├── .env
├── config.yaml
└── docker-compose.yml
```

`.env`:

```env theme={null}
MIXROUTE_API_KEY=<mixroute-api-key>
LITELLM_MASTER_KEY=<litellm-master-key>
LITELLM_SALT_KEY=<stable-random-salt-key>
POSTGRES_PASSWORD=<postgres-password>
```

| Variable             | Description                                                           |
| -------------------- | --------------------------------------------------------------------- |
| `MIXROUTE_API_KEY`   | MixRoute API key used by LiteLLM when forwarding requests to MixRoute |
| `LITELLM_MASTER_KEY` | LiteLLM administrator key, used for API access and Dashboard login    |
| `LITELLM_SALT_KEY`   | Stable random value used by LiteLLM to encrypt stored keys            |
| `POSTGRES_PASSWORD`  | Password for the PostgreSQL service inside Docker Compose             |

`docker-compose.yml`:

```yaml theme={null}
services:
  postgres:
    image: postgres:17
    container_name: litellm-postgres
    restart: unless-stopped
    environment:
      POSTGRES_DB: litellm
      POSTGRES_USER: litellm
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U litellm -d litellm"]
      interval: 5s
      timeout: 5s
      retries: 20
    volumes:
      - litellm-postgres-data:/var/lib/postgresql/data

  litellm:
    image: ghcr.io/berriai/litellm-database:main-stable
    container_name: litellm
    restart: unless-stopped
    depends_on:
      postgres:
        condition: service_healthy
    env_file:
      - .env
    environment:
      DATABASE_URL: postgresql://litellm:${POSTGRES_PASSWORD}@postgres:5432/litellm
      USE_PRISMA_MIGRATE: "True"
      STORE_MODEL_IN_DB: "True"
    command: ["--config", "/app/config.yaml", "--host", "0.0.0.0", "--port", "4000"]
    ports:
      - "4000:4000"
    volumes:
      - ./config.yaml:/app/config.yaml:ro

volumes:
  litellm-postgres-data:
```

| Variable             | Description                                                                                                |
| -------------------- | ---------------------------------------------------------------------------------------------------------- |
| `DATABASE_URL`       | PostgreSQL connection string for Dashboard, virtual keys, usage records, and persisted model configuration |
| `USE_PRISMA_MIGRATE` | Runs database migrations on startup when set to `True`                                                     |
| `STORE_MODEL_IN_DB`  | Persists model changes made in Dashboard/API to PostgreSQL when set to `True`                              |

Start LiteLLM:

```bash theme={null}
docker compose pull
docker compose up -d
```

## Connect MixRoute

MixRoute Base URL:

```text theme={null}
https://api.mixroute.ai/v1
```

Configure LiteLLM in `config.yaml`:

```yaml theme={null}
model_list:
  - model_name: o3-mini
    litellm_params:
      model: openai/o3-mini-2025-01-31
      api_base: https://api.mixroute.ai/v1
      api_key: os.environ/MIXROUTE_API_KEY
    model_info:
      mode: chat

  - model_name: openai/*
    litellm_params:
      model: openai/*
      api_base: https://api.mixroute.ai/v1
      api_key: os.environ/MIXROUTE_API_KEY

  - model_name: anthropic/*
    litellm_params:
      model: anthropic/*
      api_base: https://api.mixroute.ai
      api_key: os.environ/MIXROUTE_API_KEY

  - model_name: gemini/*
    litellm_params:
      model: gemini/*
      api_base: https://api.mixroute.ai/v1
      api_key: os.environ/MIXROUTE_API_KEY

litellm_settings:
  drop_params: true
  request_timeout: 600
  telemetry: false
  check_provider_endpoint: true

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL
  store_model_in_db: true
```

| Configuration                   | Purpose                                                                                                    |
| ------------------------------- | ---------------------------------------------------------------------------------------------------------- |
| `openai/*`                      | Routes OpenAI-compatible requests to MixRoute `/v1`; suitable for text, embeddings, images, and video APIs |
| `anthropic/*`                   | Routes Anthropic-compatible requests through MixRoute; LiteLLM handles the `/v1/messages` path             |
| `gemini/*`                      | Routes Gemini-compatible requests through MixRoute                                                         |
| `check_provider_endpoint: true` | Enables LiteLLM model discovery through MixRoute `/v1/models`                                              |
| `drop_params: true`             | Drops unsupported upstream parameters to improve cross-provider compatibility                              |
| `store_model_in_db: true`       | Stores model changes made from the Dashboard/API in PostgreSQL                                             |
| `o3-mini`                       | Example alias that maps a short model name to the MixRoute model ID                                        |

`openai/` is the LiteLLM OpenAI-compatible provider prefix. It does not mean that the upstream model must come from OpenAI.

## Client usage

OpenAI-compatible Base URL:

```text theme={null}
<LITELLM_OPENAI_BASE_URL>
```

Request header:

```http theme={null}
Authorization: Bearer <LITELLM_API_KEY>
```

List models:

```bash theme={null}
curl -sS <LITELLM_OPENAI_BASE_URL>/models \
  -H "Authorization: Bearer <LITELLM_API_KEY>"
```

Use the returned model IDs in requests, for example `openai/gpt-5.5`, `anthropic/claude-sonnet-5`, `gemini/gemini-2.5-flash`, or the alias `o3-mini`.

## Verification

```bash theme={null}
curl -sS <LITELLM_BASE_URL>/health/liveliness
```

```bash theme={null}
curl -sS <LITELLM_OPENAI_BASE_URL>/chat/completions \
  -H "Authorization: Bearer <LITELLM_API_KEY>" \
  -H "Content-Type: application/json" \
  --data '{
    "model": "openai/gpt-5.5",
    "messages": [{"role": "user", "content": "Return exactly OK"}],
    "max_tokens": 8
  }'
```

```bash theme={null}
curl -sS <LITELLM_BASE_URL>/v1beta/models/gemini/gemini-2.5-flash:generateContent \
  -H "Authorization: Bearer <LITELLM_API_KEY>" \
  -H "Content-Type: application/json" \
  --data '{
    "contents": [{"role": "user", "parts": [{"text": "Return exactly OK"}]}],
    "generationConfig": {"maxOutputTokens": 8}
  }'
```

Gemini `generateContent` uses the LiteLLM frontend path `/v1beta/models/gemini/<model-id>:generateContent`; the upstream MixRoute base remains `https://api.mixroute.ai/v1`.