What AI observability adds on top of regular event tracking
PostHog's AI-engineering product traces LLM calls - prompts, completions, token usage, latency, and tool invocations - as structured events tied to the same person and session model as the rest of your product analytics. That means an LLM trace can sit next to the user's product usage, feature-flag exposure, and any errors from the same session, instead of living in a separate LLM-specific tool with no product context.
Where the integrations fit
| Integration | What it's for |
|---|---|
| Claude Agent SDK | Tracing agent runs built on Anthropic's Agent SDK - tool calls, multi-step reasoning traces, and outcomes captured as PostHog events instead of only living in the agent's own logs |
| LangFuse / Helicone / Traceloop | Where a team already instruments LLM calls through one of these observability layers, PostHog's integration ingests that trace data rather than requiring a second, parallel instrumentation pass |
| Perplexity (and other model/API providers) | Provider-specific instrumentation for capturing calls made directly against that provider's API |
Exact integration setup steps and which providers are supported change frequently in this space - verify the current integration list and install steps against PostHog's own AI-observability docs before committing an implementation plan to a specific one.
Privacy mode, retention, and custom properties - the part people skip
LLM traces often contain prompt and completion text, which is a materially different privacy surface than a pageview event. Before shipping this to production:
- Privacy mode controls whether full prompt/completion content is captured or redacted/hashed - decide this deliberately against what your data-handling policy actually allows, not the default.
- Data retention for AI traces may need to differ from your general event retention, especially if traces contain user-submitted content with its own retention obligations.
- Custom properties let you attach metadata (which prompt template, which model version, which agent tool) to traces - decide on a naming convention before the first trace ships, the same discipline as any other event taxonomy.
How to verify it worked
- Run a real LLM call through the integration and confirm the full trace - prompt, completion, token counts, and any tool calls - appears in PostHog, not just a partial event.
- If privacy mode is enabled, confirm the actual captured event has redacted/hashed content as expected, not the raw text - test this explicitly rather than assuming the setting applied.
- Confirm a multi-step agent trace (Claude Agent SDK or similar) reconstructs as one coherent trace in PostHog's view, not as disconnected individual events with no parent-child relationship.
- Check that retention for these traces matches what you configured - separately from your default event retention, if you set it that way.