Standard level
Timings, token counts and scheduler metadata. No message content.
standard is the production level: it records how each request was
served, never what it said. No prompt text, no output text, no field
that could carry customer content.
The record
One JSON object per completed request, appended to the trace file (or
exported as one OTLP span named llm_request):
| Field | Meaning |
|---|---|
request_id | The engine's request id; matches the id in the API response, so a slow response can be joined to its trace |
t_arrival | Unix timestamp: request arrived at the engine |
t_completion | Unix timestamp: last token generated |
prompt_tokens | Input size, from the engine's own tokenization |
output_tokens | Tokens generated, reasoning included |
e2e_latency_s | t_completion minus t_arrival, in seconds |
ttft_s | Always null today: first-token time is not visible to the scheduler |
queue_wait_s | Arrival to first-scheduled, in seconds: the congestion signal |
scheduler_family | Which scheduler served it (e.g. ltr-promptlen, fcfs) |
priority_score | The scheduler's priority for this request; meaningful only for reordering schedulers |
Example
A short burst against a chat recipe. Note queue_wait_s growing as the
burst saturates the engine's concurrent-sequence ceiling; that pattern,
not any single record, is what capacity planning reads:
{"request_id": "chatcmpl-9f2c41d0", "t_arrival": 1789040112.104, "t_completion": 1789040114.312, "prompt_tokens": 412, "output_tokens": 128, "e2e_latency_s": 2.208, "ttft_s": null, "queue_wait_s": 0.004, "scheduler_family": "ltr-promptlen", "priority_score": -318.0}
{"request_id": "chatcmpl-c07aa1e9", "t_arrival": 1789040112.371, "t_completion": 1789040116.899, "prompt_tokens": 1288, "output_tokens": 256, "e2e_latency_s": 4.528, "ttft_s": null, "queue_wait_s": 0.011, "scheduler_family": "ltr-promptlen", "priority_score": -1104.5}
{"request_id": "chatcmpl-51b8e277", "t_arrival": 1789040112.590, "t_completion": 1789040119.443, "prompt_tokens": 96, "output_tokens": 512, "e2e_latency_s": 6.853, "ttft_s": null, "queue_wait_s": 1.937, "scheduler_family": "ltr-promptlen", "priority_score": -71.2}The same record as an OTLP span
Exported to a collector, each record is one span named llm_request
from service carbonforge-scheduler, with the same facts as
attributes (span shown abridged):
{"name": "llm_request", "kind": 1,
"startTimeUnixNano": "1788450022633673216", "endTimeUnixNano": "1788450022799637248",
"attributes": [
{"key": "cf.request_id", "value": {"stringValue": "chatcmpl-b8af6f4d128fdd4c-b31b5b1f"}},
{"key": "gen_ai.usage.prompt_tokens", "value": {"intValue": "54"}},
{"key": "gen_ai.usage.completion_tokens", "value": {"intValue": "2"}},
{"key": "cf.e2e_latency_s", "value": {"doubleValue": 0.1659}},
{"key": "cf.queue_wait_s", "value": {"doubleValue": 0.0119}},
{"key": "cf.scheduler.family", "value": {"stringValue": "trace"}}
]}Span start and end are arrival and completion, so any OTLP-compatible backend (Tempo, Jaeger, ADOT) renders latency without configuration.
Reading it
- Queueing vs serving:
e2e_latency_s - queue_wait_sis time actually spent in the engine. Ifqueue_wait_sdominates, add capacity or reduce client concurrency; the engine itself is fine. - Token accounting:
output_tokenscounts everything generated, including reasoning on reasoning models, which is why it can be far larger than the visible answer. - Joining to your logs:
request_idequals theidfield your client received, so client-side and server-side views of one request can be lined up exactly.