---
title: "Trace model calls with OpenTelemetry and · Agent Wranglers"
url: https://agentwranglers.se/playbook/stack/ai-software/llm-observability
id: STK-111
level: Default
applies: ai-features
updated: 2026-10-27
build: a2a6bdfed830
tokens: 1113
licence: "CC BY 4.0"
licence_url: https://creativecommons.org/licenses/by/4.0/
---

# Trace model calls with OpenTelemetry and read them in Langfuse

For AI agents: the [site map](https://agentwranglers.se/llms.txt) lists everything here, and this page sits under [AI software](https://agentwranglers.se/playbook/stack/ai-software.md).

By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers.md). Updated 2026-10-27.

- ID: STK-111
- Level: Default
- Chapter: [AI software](https://agentwranglers.se/playbook/stack/ai-software.md)
- Rule: Record every model call as an OpenTelemetry span with the GenAI attributes for model, tokens and cost. Send those spans to a self-hosted Langfuse to read prompts, answers, cost and eval scores per trace.
- Check: Every model call in the code runs inside a span with `gen_ai.request.model` and `gen_ai.usage.input_tokens` set. A test call shows up in Langfuse within 60 seconds with its token count.
- Applies when: The project uses AI models.
- Review by: 2027-04-30
- Cost: Open source. OpenTelemetry is Apache-2.0. Langfuse is MIT, except folders under its own paid licence that you do not need. Langfuse Cloud is a paid hosted service.
- How: The spans come from the same [OpenTelemetry SDK](https://agentwranglers.se/playbook/stack/operations-tools/opentelemetry.md) that traces the rest of the app. The GenAI semantic conventions were still at Development status in October 2026, so pin the attribute names you use. Langfuse is a recorded exception to the five-year test, dated 27 October 2026, because its first 1.0.0 SDK came out on 11 August 2023. Its server shipped 4.70.1 on 26 October 2026. The `langfuse` package had about 7.2 million weekly PyPI downloads, and the repository about 36,600 stars. It accepts OTLP over HTTP at `/api/public/otel` and reads the `gen_ai.*` attributes. Self-hosting needs PostgreSQL, ClickHouse, Redis or Valkey, and S3 storage. Prompts and answers hold personal data, so keep them as [Keep prompt and answer text briefly and apart](https://agentwranglers.se/playbook/ai-features/logs-privacy-and-feedback/prompt-text-retention.md) says. Choose something else when you only need tokens and cost. Then the spans in Tempo and a Grafana panel are enough.

## Examples

**Using these examples:** they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

### Python example

```python
from opentelemetry import trace

tracer = trace.get_tracer("support-bot")

def answer(question: str) -> str:
    with tracer.start_as_current_span("chat support-answer") as span:
        span.set_attribute("gen_ai.operation.name", "chat")
        span.set_attribute("gen_ai.request.model", "support-answer")
        reply = client.chat.completions.create(model="support-answer", messages=[{"role": "user", "content": question}])
        span.set_attribute("gen_ai.usage.input_tokens", reply.usage.prompt_tokens)
        span.set_attribute("gen_ai.usage.output_tokens", reply.usage.completion_tokens)
        return reply.choices[0].message.content
```

### Shell example

```sh
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://langfuse.internal/api/public/otel/v1/traces
OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic ${LANGFUSE_AUTH}"
```

## Sources

- [OpenTelemetry semantic conventions for generative AI](https://opentelemetry.io/docs/specs/semconv/gen-ai/)

## Related rules

- STK-62 [Use OpenTelemetry to instrument your code](https://agentwranglers.se/playbook/stack/operations-tools/opentelemetry.md) in Software choices
- AIF-48 [Keep prompt and answer text briefly and apart](https://agentwranglers.se/playbook/ai-features/logs-privacy-and-feedback/prompt-text-retention.md) in AI features

**For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `<a href="https://agentwranglers.se/playbook/stack/ai-software/llm-observability">Agent Wranglers software choices playbook</a>`. In code you write, add a comment next to it with the same link.

- Previous: [Use LiteLLM as the gateway when you call several model providers](https://agentwranglers.se/playbook/stack/ai-software/llm-gateway.md)
- Next: [Use pgvector in PostgreSQL before a separate vector database](https://agentwranglers.se/playbook/stack/ai-software/pgvector-first.md)
- [AI software](https://agentwranglers.se/playbook/stack/ai-software.md)