Self-hosted VPN & networking projects

\n\n- [Setup IPsec VPN](https://github.com/hwdsl2/setup-ipsec-vpn)\n- [IPsec VPN on Docker](https://github.com/hwdsl2/docker-ipsec-vpn-server)\n- [WireGuard](https://github.com/hwdsl2/docker-wireguard)\n- [OpenVPN](https://github.com/hwdsl2/docker-openvpn)\n- [Headscale](https://github.com/hwdsl2/docker-headscale)\n\n

\n\n## Requirements\n\n- A Linux server (local or cloud) with Docker installed\n- At least one LLM provider API key (OpenAI, Anthropic, Groq, etc.) **or** a locally running Ollama server, such as [InferCrate](https://github.com/hwdsl2/infercrate)\n- TCP port 4000 (or your configured port) open and accessible\n\nNo LLM provider keys are required to start the proxy — the server starts successfully with an empty model list. Add models at any time using `litellm_manage`.\n\nFor internet-facing deployments, see [Using a reverse proxy](#using-a-reverse-proxy) to add HTTPS.\n\n## Download\n\nGet the trusted build from the [Docker Hub registry](https://hub.docker.com/r/hwdsl2/litellm-server/):\n\n```bash\ndocker pull hwdsl2/litellm-server\n```\n\nAlternatively, you may download from [Quay.io](https://quay.io/repository/hwdsl2/litellm-server):\n\n```bash\ndocker pull quay.io/hwdsl2/litellm-server\ndocker image tag quay.io/hwdsl2/litellm-server hwdsl2/litellm-server\n```\n\nSupported platforms: `linux/amd64` and `linux/arm64`.\n\n## Environment variables\n\nAll variables are optional. The master API key is auto-generated on first start if `LITELLM_MASTER_KEY` is not set.\n\nThis Docker image uses the following variables, that can be declared in an `env` file (see [example](https://github.com/hwdsl2/gatewaycrate/blob/main/litellm.env.example)):\n\n| Variable | Description | Default |\n|---|---|---|\n| `LITELLM_MASTER_KEY` | Master API key for the proxy | Auto-generated |\n| `LITELLM_PORT` | TCP port for the proxy (1–65535) | `4000` |\n| `LITELLM_HOST` | Hostname or IP shown in startup info and `--showkey` output | Auto-detected |\n| `LITELLM_LOG_LEVEL` | Log level: `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL` | `INFO` |\n| `LITELLM_OPENAI_API_KEY` | OpenAI API key — auto-adds `gpt-4o`, `gpt-4o-mini` | *(not set)* |\n| `LITELLM_ANTHROPIC_API_KEY` | Anthropic API key — auto-adds `claude-3-6-sonnet` (latest) | *(not set)* |\n| `LITELLM_GROQ_API_KEY` | Groq API key — auto-adds `llama-3.3-70b` | *(not set)* |\n| `LITELLM_GEMINI_API_KEY` | Google Gemini API key — auto-adds `gemini-2.0-flash` | *(not set)* |\n| `LITELLM_OLLAMA_BASE_URL` | Ollama base URL — ensures `ollama/llama3.2:3b` and `ollama-chat/llama3.2:3b` | *(not set)* |\n| `LITELLM_OLLAMA_API_KEY` | API key for the Ollama endpoint (auto-read from shared volume in [self-hosted-ai-stack](https://github.com/hwdsl2/self-hosted-ai-stack)) | *(not set)* |\n| `LITELLM_DATABASE_URL` | PostgreSQL URL — enables virtual key management | *(not set)* |\n| `LITELLM_POSTGRES_PASSWORD_FILE` | File containing the Compose Postgres password; used only when `LITELLM_DATABASE_URL` is not set | *(not set)* |\n| `LITELLM_MCP_URL` | Compatible MCP HTTP endpoint URL (for example, UplinkCrate), registered on every start | *(not set)* |\n| `LITELLM_MCP_API_KEY` | Bearer token for the MCP endpoint (required for authenticated endpoints such as UplinkCrate) | *(not set)* |\n| `LITELLM_DISABLE_USAGE_COUNTS` | Set to `1` to disable anonymous aggregate usage counts. | *(not set)* |\n\n**Note:** In your `env` file, you may enclose values in single quotes, e.g. `VAR='value'`. Do not add spaces around `=`. If you change `LITELLM_PORT`, update the `-p` flag in the `docker run` command accordingly.\n\nExample using an `env` file:\n\n```bash\ncp litellm.env.example litellm.env\n# Edit litellm.env and set your API keys, then:\ndocker run \\\n --name litellm \\\n --restart=always \\\n -v litellm-data:/etc/litellm \\\n -v ./litellm.env:/litellm.env:ro \\\n -p 4000:4000/tcp \\\n -d hwdsl2/litellm-server\n```\n\nThe env file is bind-mounted into the container, so changes are picked up on every restart without recreating the container.\n\n## Model management\n\nUse `docker exec` to manage models with the `litellm_manage` helper script. Models are stored in `config.yaml` inside the Docker volume and persist across container restarts.\n\n**Note:** `--addmodel` and `--removemodel` write to `config.yaml` and automatically restart the proxy to apply the change.\n\nWhen `LITELLM_OLLAMA_BASE_URL` is set, the container keeps both default Ollama aliases in `config.yaml`: `ollama/llama3.2:3b` for backward compatibility and `ollama-chat/llama3.2:3b` for chat-native Ollama behavior. Use the `ollama-chat/...` alias for streaming tool calls.\n\n**List configured models:**\n\n```bash\ndocker exec litellm litellm_manage --listmodels\n```\n\n**Add a model with an API key:**\n\n```bash\n# OpenAI\ndocker exec litellm litellm_manage --addmodel openai/gpt-4o --key sk-...\n\n# Anthropic\ndocker exec litellm litellm_manage --addmodel anthropic/claude-3-6-sonnet-latest --key sk-ant-...\n\n# Groq\ndocker exec litellm litellm_manage --addmodel groq/llama-3.3-70b-versatile --key gsk_...\n\n# Add with a custom display name (alias)\ndocker exec litellm litellm_manage --addmodel openai/gpt-4o --key sk-... --alias my-gpt4\n\n# Mark a model as function-calling capable\ndocker exec litellm litellm_manage \\\n --addmodel ollama_chat/llama3.2:3b \\\n --alias ollama-chat/llama3.2:3b \\\n --base-url http://host.docker.internal:11434 \\\n --supports-function-calling\n```\n\n**Add a local Ollama model:**\n\n```bash\n# Connect to Ollama running on the Docker host\ndocker exec litellm litellm_manage \\\n --addmodel ollama/llama3.2:3b \\\n --base-url http://host.docker.internal:11434\n```\n\nFor Ollama models that need native chat behavior or streaming tool calls, use the `ollama_chat/...` provider model with a user-facing alias such as `ollama-chat/llama3.2:3b`.\n\n**Remove a model** (use the `id` field from `--listmodels`):\n\n```bash\ndocker exec litellm litellm_manage --removemodel \n```\n\n**Show the master key** (if you need to look it up):\n\n```bash\n# Full output with endpoint info\ndocker exec litellm litellm_manage --showkey\n\n# Key only (for scripting — no IP or endpoint info displayed)\ndocker exec litellm litellm_manage --getkey\n```\n\n## UplinkCrate integration\n\nSet `LITELLM_MCP_URL` in your `litellm.env` file to connect GatewayCrate to a compatible MCP HTTP endpoint, such as UplinkCrate. Set `LITELLM_MCP_API_KEY` if the endpoint requires Bearer authentication. AI clients can then call MCP tools through the LiteLLM proxy.\n\nWhen `LITELLM_MCP_URL` is set, an `mcp_servers:` block is injected into `config.yaml` on every container start — no manual YAML editing required.\n\n**Example: connect to UplinkCrate:**\n\n```bash\n# In litellm.env:\nLITELLM_MCP_URL=http://mcp:3000/mcp\nLITELLM_MCP_API_KEY=mcp-xxxx... # get with: docker exec mcp mcp_manage --showkey\n```\n\nAfter setting these values, restart the container:\n\n```bash\ndocker compose restart litellm\n# or: docker restart litellm\n```\n\n**Manage MCP servers with `litellm_manage`:**\n\n```bash\n# List configured MCP servers\ndocker exec litellm litellm_manage --listmcp\n\n# Add an MCP server manually\ndocker exec litellm litellm_manage --addmcp my-gateway http://mcp:3000/mcp --key mcp-xxxx\n\n# Remove an MCP server\ndocker exec litellm litellm_manage --removemcp my-gateway\n```\n\n**Note:** `--addmcp` and `--removemcp` write to `config.yaml` and automatically restart the proxy. MCP servers added via `LITELLM_MCP_URL` are named `docker_mcp_gateway` in the config and can be managed with `--removemcp docker_mcp_gateway`.\n\n## Virtual key management\n\nVirtual keys are scoped API keys you can issue to users or applications. Each key can optionally restrict which models it may access, set a maximum spend budget, and have an expiry. Virtual keys require a PostgreSQL database, which is included in the default `docker-compose.yml`.\n\n**Create a virtual key:**\n\n```bash\n# Basic key (no restrictions)\ndocker exec litellm litellm_manage --createkey\n\n# Key with alias, model restrictions, budget, and expiry\ndocker exec litellm litellm_manage --createkey \\\n --alias dev-key \\\n --models gpt-4o,claude-3-6-sonnet \\\n --budget 20.0 \\\n --expires 30d\n```\n\n**List all virtual keys:**\n\n```bash\ndocker exec litellm litellm_manage --listkeys\n```\n\n**Delete a virtual key:**\n\n```bash\ndocker exec litellm litellm_manage --deletekey sk-...\n```\n\n## Using the proxy with OpenAI SDK\n\nPoint apps that use the OpenAI SDK at your proxy by setting two environment variables:\n\n```bash\nexport OPENAI_API_KEY=\"\"\nexport OPENAI_BASE_URL=\"http://:4000\"\n```\n\nFor Python:\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(\n api_key=\"\",\n base_url=\"http://:4000\",\n)\n\nresponse = client.chat.completions.create(\n model=\"gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n)\nprint(response.choices[0].message.content)\n```\n\nThe proxy is compatible with any tool or library that supports the OpenAI API format.\n\n## Persistent data\n\nAll proxy data is stored in the Docker volume (`/etc/litellm` inside the container):\n\n```\n/etc/litellm/\n├── config.yaml # Proxy configuration and model list (created once, preserved across restarts)\n├── .master_key # Master API key (auto-generated, or synced from LITELLM_MASTER_KEY)\n├── .initialized # First-run marker\n└── .db_configured # Present when LITELLM_DATABASE_URL is set (used by litellm_manage)\n```\n\nBack up the Docker volume to preserve your master key and configured models.\n\n## Using docker-compose\n\n```bash\ncp litellm.env.example litellm.env\n# Edit litellm.env and set your API keys, then:\ndocker compose up -d\ndocker logs litellm\n```\n\nFresh Compose installs generate a random PostgreSQL password automatically and store it in the `litellm-secrets` volume. Existing default installs continue to use the legacy `litellm` database password for compatibility. If you previously customized the database password, set `LITELLM_POSTGRES_PASSWORD` in your shell environment to that password before running `docker compose up -d`, or keep an explicit `LITELLM_DATABASE_URL` override in `litellm.env`.\n\nWhen upgrading an existing checkout, run `docker compose pull` before `docker compose up -d` so the LiteLLM image supports `LITELLM_POSTGRES_PASSWORD_FILE`.\n\nExample `docker-compose.yml` (already included):\n\n```yaml\nservices:\n litellm-init:\n image: alpine:3.24\n container_name: litellm-init\n restart: \"no\"\n environment:\n - LITELLM_POSTGRES_PASSWORD=${LITELLM_POSTGRES_PASSWORD:-}\n volumes:\n - litellm-db:/var/lib/postgresql:ro\n - litellm-secrets:/var/lib/litellm-secrets\n - ./scripts/litellm-init.sh:/usr/local/bin/litellm-init.sh:ro\n entrypoint: [\"/bin/sh\", \"/usr/local/bin/litellm-init.sh\"]\n\n db:\n image: postgres:18\n container_name: litellm-db\n restart: always\n environment:\n POSTGRES_USER: litellm\n POSTGRES_PASSWORD_FILE: /var/lib/litellm-secrets/postgres_password\n POSTGRES_DB: litellm\n volumes:\n - litellm-db:/var/lib/postgresql\n - litellm-secrets:/var/lib/litellm-secrets:ro\n depends_on:\n litellm-init:\n condition: service_completed_successfully\n healthcheck:\n test: [\"CMD-SHELL\", \"pg_isready -U litellm\"]\n interval: 15s\n timeout: 5s\n retries: 5\n\n litellm:\n image: hwdsl2/litellm-server\n container_name: litellm\n restart: always\n ports:\n - \"4000:4000/tcp\" # For a host-based reverse proxy, change to \"127.0.0.1:4000:4000/tcp\"\n environment:\n - LITELLM_POSTGRES_PASSWORD_FILE=/var/lib/litellm-secrets/postgres_password\n volumes:\n - litellm-data:/etc/litellm\n - litellm-secrets:/var/lib/litellm-secrets:ro\n - ./litellm.env:/litellm.env:ro\n depends_on:\n db:\n condition: service_healthy\n\nvolumes:\n litellm-data:\n name: litellm-data\n litellm-secrets:\n name: litellm-secrets\n litellm-db:\n name: litellm-db\n```\n\n**Note:** For internet-facing deployments, use a [reverse proxy](#using-a-reverse-proxy) to add HTTPS. Also change `\"4000:4000/tcp\"` to `\"127.0.0.1:4000:4000/tcp\"` in `docker-compose.yml`, to prevent direct access to the unencrypted port.\n\n## Using a reverse proxy\n\nFor internet-facing deployments, place a reverse proxy in front of LiteLLM to handle HTTPS termination. The server works without HTTPS on a local or trusted network, but HTTPS is recommended when the API endpoint is exposed to the internet.\n\nUse one of the following addresses to reach the LiteLLM container from your reverse proxy:\n\n- **`litellm:4000`** — if your reverse proxy runs as a container in the **same Docker network** as LiteLLM (e.g. defined in the same `docker-compose.yml`).\n- **`127.0.0.1:4000`** — if your reverse proxy runs **on the host** and port `4000` is published (the default `docker-compose.yml` publishes it).\n\n**Example with [Caddy](https://caddyserver.com/docs/) ([Docker image](https://hub.docker.com/_/caddy))** (automatic TLS via Let's Encrypt, reverse proxy in the same Docker network):\n\n`Caddyfile`:\n```\nlitellm.example.com {\n reverse_proxy litellm:4000\n}\n```\n\n**Example with nginx** (reverse proxy on the host):\n\n```nginx\nserver {\n listen 443 ssl;\n server_name litellm.example.com;\n\n ssl_certificate /path/to/cert.pem;\n ssl_certificate_key /path/to/key.pem;\n\n location / {\n proxy_pass http://127.0.0.1:4000;\n proxy_set_header Host $host;\n proxy_set_header X-Real-IP $remote_addr;\n proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;\n proxy_set_header X-Forwarded-Proto $scheme;\n proxy_read_timeout 300s;\n proxy_buffering off;\n }\n}\n```\n\nAfter setting up a reverse proxy, set `LITELLM_HOST=litellm.example.com` in your `env` file so that the correct endpoint URL is shown in the startup logs and `litellm_manage --showkey` output.\n\nThe auto-generated master API key is required for all API requests. Keep it secure when the server is accessible from the public internet.\n\n## Update Docker image\n\nTo update the Docker image and container, first [download](#download) the latest version:\n\n```bash\ndocker pull hwdsl2/litellm-server\n```\n\nIf the Docker image is already up to date, you should see:\n\n```\nStatus: Image is up to date for hwdsl2/litellm-server:latest\n```\n\nOtherwise, it will download the latest version. Remove and re-create the container:\n\n```bash\ndocker rm -f litellm\n# Then re-run the docker run command from Quick start with the same volume and port.\n```\n\nYour data is preserved in the `litellm-data` volume.\n\n## Using with other AI services\n\nLiteLLM can be used as the AI gateway in a broader self-hosted AI setup.\n\nFor full and lightweight Docker Compose stacks, manual `docker run` examples, and voice/RAG/MCP pipeline examples with SpeakCrate, EmbedCrate, GatewayCrate, InferCrate, ParseCrate, and UplinkCrate, see [Self-Hosted AI Stack](https://github.com/hwdsl2/self-hosted-ai-stack).\n\n**Related AI services:**\n\n- **Voice:** [ScribeCrate](https://github.com/hwdsl2/scribecrate), [SpeakCrate](https://github.com/hwdsl2/speakcrate)\n- **Documents & search:** [ParseCrate](https://github.com/hwdsl2/parsecrate), [EmbedCrate](https://github.com/hwdsl2/embedcrate)\n- **Models & gateways:** [InferCrate](https://github.com/hwdsl2/infercrate), [UplinkCrate](https://github.com/hwdsl2/uplinkcrate)\n\n## Usage counts\n\nSee [Usage counts](https://github.com/hwdsl2/gatewaycrate#usage-counts).\n\n## Technical details\n\n- Base image: `python:3.12-slim` (Debian)\n- Runtime: Python 3 (virtual environment at `/opt/venv`)\n- LiteLLM: latest `litellm[proxy]` from PyPI\n- Data directory: `/etc/litellm` (Docker volume)\n- Model storage: `config.yaml` inside the volume — created on first start, preserved on restarts\n- Proxy management REST API: runs on the same port as the proxy\n- Built-in UI: available at `http://:/ui` — log in with username `admin` and your master key as the password\n\n## License\n\n**Note:** The software components inside the pre-built image (such as LiteLLM and its dependencies) are under the respective licenses chosen by their respective copyright holders. As for any pre-built image usage, it is the image user's responsibility to ensure that any use of this image complies with any relevant licenses for all software contained within.\n\nCopyright (C) 2026 Lin Song \nThis work is licensed under the [MIT License](https://opensource.org/licenses/MIT).\n\n**LiteLLM** is Copyright (C) 2023 Berri AI, and is distributed under the [MIT License](https://github.com/BerriAI/litellm/blob/main/LICENSE).\n\nThis project is an independent Docker setup for LiteLLM and is not affiliated with, endorsed by, or sponsored by Berri AI, the creators of LiteLLM.","has_starred","hub_user","immutable_tags_settings",{"_204":11,"_205":206},"enabled","rules",[207],".*","is_automated","permissions",{"_211":22,"_212":11,"_213":11},"read","write","admin","user","storage_size",883538084,"isOrg","reponame","tag","sha256:e293f0abdea53391600eaf426f8304722f8a63a4599aa6a132c1fd4fc45395cc","canDeleteTags","isHubSbxKitsEnabled","currentTab","tagAttestations","displaySecurityMetadata","showSbxLogoOverride","showSbxAgenticPlatformBadge","routes/_layout.layers.$namespace.$reponame.$tag.(images).($imageUrlDigest)._index",{"_27":28,"_157":-7,"_155":-7,"_156":-7,"_29":230,"_35":220,"_233":-7,"_234":11},{"_31":32,"_33":34,"_35":36,"_37":38,"_39":40,"_41":231,"_61":62,"_63":34,"_64":65,"_66":67,"_68":69,"_70":71,"_72":50,"_73":62,"_74":59,"_75":22},[232],{"_44":45,"_35":46,"_47":48,"_49":50,"_51":52,"_53":54,"_55":48,"_56":-5,"_57":38,"_58":59,"_60":-5},"imageWithLayers","hasPermissionToViewRepoSettings","actionData","errors"]

hwdsl2/litellm-server:latest-arm64