Trace model calls with OpenTelemetry and read them in Langfuse

Review by

2027-04-30

How to do it

The spans come from the same OpenTelemetry SDK that traces the rest of the app. The GenAI semantic conventions were still at Development status in October 2026, so pin the attribute names you use. Langfuse is a recorded exception to the five-year test, dated 27 October 2026, because its first 1.0.0 SDK came out on 11 August 2023. Its server shipped 4.70.1 on 26 October 2026. The langfuse package had about 7.2 million weekly PyPI downloads, and the repository about 36,600 stars. It accepts OTLP over HTTP at /api/public/otel and reads the gen_ai.* attributes. Self-hosting needs PostgreSQL, ClickHouse, Redis or Valkey, and S3 storage. Prompts and answers hold personal data, so keep them as Keep prompt and answer text briefly and apart says. Choose something else when you only need tokens and cost. Then the spans in Tempo and a Grafana panel are enough.

Examples

Python example

from opentelemetry import trace

tracer = trace.get_tracer("support-bot")

def answer(question: str) -> str:
    with tracer.start_as_current_span("chat support-answer") as span:
        span.set_attribute("gen_ai.operation.name", "chat")
        span.set_attribute("gen_ai.request.model", "support-answer")
        reply = client.chat.completions.create(model="support-answer", messages=[{"role": "user", "content": question}])
        span.set_attribute("gen_ai.usage.input_tokens", reply.usage.prompt_tokens)
        span.set_attribute("gen_ai.usage.output_tokens", reply.usage.completion_tokens)
        return reply.choices[0].message.content

Shell example

OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://langfuse.internal/api/public/otel/v1/traces
OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic ${LANGFUSE_AUTH}"

How to check

Every model call in the code runs inside a span with gen_ai.request.model and gen_ai.usage.input_tokens set. A test call shows up in Langfuse within 60 seconds with its token count.

Sources

• OpenTelemetry semantic conventions for generative AI

Related rules

• STK-62 Use OpenTelemetry to instrument your code in Software choices
• AIF-48 Keep prompt and answer text briefly and apart in AI features