Tracing

The scheduler emits one trace record per completed request: queue wait, total latency, token counts, and scheduler decisions. This page covers enabling it on the container; what the records contain at each level is documented in Tracing.

Enabling

Tracing needs a level and at least one sink. A level without a sink logs one warning and stays off; the server still serves.

Tracing flags are runtime arguments, so they are passed as the container command. To keep serving your recipe, include the context key as a flag (the CF_CONTEXT_KEY environment variable is ignored once a command is given):

docker run ... \
  us-central1-docker.pkg.dev/potent-arcade-491015-g7/cf-containers-dev/cf-bundle:v0.1.28-v0.1.4 \
  --context-key "qwen2.5-0.5b:fp8:chat:l4x1" \
  --request-trace standard \
  --trace-file /var/log/carbonforge/traces.jsonl

Each completed request appends one JSON line to the trace file:

{"request_id": "chatcmpl-...", "t_arrival": 1788962815.33, "t_completion": 1788962816.58,
 "prompt_tokens": 30, "output_tokens": 8, "e2e_latency_s": 1.25, "ttft_s": null,
 "queue_wait_s": 0.0067, "scheduler_family": "ltr-promptlen", "priority_score": 96.0}

Or export to an OpenTelemetry collector:

  --request-trace standard \
  --trace-endpoint http://my-collector:4317 \
  --trace-format otlp

Both sinks can be set at once (the trace is written to the file and exported).

Levels and privacy

Warning: full ships your prompts. At full, raw prompt and completion text is written to every configured sink. Use it only against sinks you control, never in production with customer data.

Span shape

OTLP export emits one span per request named llm_request, with gen_ai.usage.prompt_tokens, gen_ai.usage.completion_tokens and scheduler attributes. Time-to-first-token is not instrumented; streaming and non-streaming requests produce the same span shape.

Securing the export

The OTLP exporter currently supports no authentication headers and no TLS. Run the collector on the same host or private network as the container.