Self-hosted container
Configuration
Volumes, environment variables and launch modes for cf-bundle.
Launch modes
The container entrypoint has three modes:
- No arguments: runs the doctor. Diagnoses the host and exits.
CF_CONTEXT_KEYset: serves the recipe named by the context key. The normal path.- Arguments after the image name: full override; bypasses the recipe and passes your arguments to the runtime directly.
Volumes
| Mount | Purpose |
|---|---|
<license file>:/etc/carbonforge/license.key:ro | License (alternative: CARBONFORGE_LICENSE_KEY env) |
hf-cache:/root/.cache/huggingface | Model weights cache; persists across restarts |
carbonforge-logs:/var/log/carbonforge | Runtime and engine logs, trace files |
Environment variables
| Variable | Required | Effect |
|---|---|---|
CF_CONTEXT_KEY | For serving | Recipe to serve; unset runs the doctor |
CARBONFORGE_LICENSE_KEY | For premium recipes | License key value (instead of the file mount) |
HF_TOKEN | Gated models only | Hugging Face auth for weight download |
CF_DRY_RUN | No | Observe-only mode; no control actions |
CF_MAX_MODEL_LEN | No | Override the recipe's context length |
CF_GPU_MEMORY_UTILIZATION | No | Override the recipe's GPU memory fraction |
CF_VLLM_EXTRA_ARGS | No | Extra vLLM flags, space-separated |
Overrides depart from the benchmark. Every
CF_*override makes the container serve a configuration different from the one the recipe was benchmarked on. Performance and energy numbers no longer apply.
Ports
| Port | What |
|---|---|
| 8000 | OpenAI-compatible API |
| 9105 | Prometheus metrics, only with --metrics (off by default) |
Concurrency
The engine enforces a hard ceiling on concurrent sequences
(--max-num-seqs, recipe-dependent). A gateway in front of the container
will happily accept more; the excess queues inside the engine and shows up
as tail latency. Size your client concurrency to the recipe.