Inside the OpenTelemetry GenAI SIG: The Conversations Shaping AI Observability

The companion to this post covered the settled parts of OpenTelemetry's GenAI conventions, the attributes, metrics, and adoption that are stable enough to build on today. This one is about everything that is still being argued out. If you've ever wanted to influence a standard before it locks rather than complain about it afterward, this is the post that tells you where the arguments are happening and how to walk in.

The bottom line: the most consequential decisions in AI observability are being made right now, in public, by a SIG that explicitly wants more practitioner input, and the biggest open questions (evaluation, agentic systems, and content privacy) are exactly the ones that will determine whether your production telemetry is portable in three years. The conventions you depend on are not handed down from a vendor. They're shaped in weekly meetings and a Slack channel that anyone can join.

We'll cover four things: where the community actually meets and talks; the four debates that dominate those conversations; the agentic-systems work that is the largest single effort underway; and how to get involved if any of this touches your stack.

Part one: where the conversation happens

The GenAI SIG runs as an open forum. Its contributor list reads like an industry roll call: Amazon, Elastic, Google, IBM, Microsoft, Langtrace, OpenLIT, Scorecard, and Traceloop are all upstream. That breadth matters, because it means the conventions aren't being shaped to favor one vendor's existing product.

The meetings are public and recorded. General GenAI topics are discussed every Tuesday at 9:00 PT, with an alternating session every other Tuesday at 9:00 UTC+8 to accommodate Asia-Pacific contributors. Agent-specific topics get their own slot every Monday at 9:00 PT. Every meeting has a notes document and a recording.

The day-to-day discussion lives on the CNCF Slack, in the #otel-genai-instrumentation channel. This is the practical entry point. As one SIG member put it, the group is accessible enough that if you care about a specific attribute or metric, you can bring your use case directly and influence the spec before it locks in. The proposals and meeting notes are tracked in the OpenTelemetry community repositories, and the 2026 Semantic Conventions Roadmap is itself an open GitHub issue collecting input from every sub-SIG, explicitly labeled work-in-progress rather than a finished plan.

The reason to care about the venue, not just the output: when a convention is still in Development, the difference between an attribute that fits your workload and one that doesn't is often a single comment from someone who actually runs the system in production. That someone could be you.

Part two: the four debates

Four questions dominate the current conversations. None of them have settled answers, and all four will shape what you can observe.

1. Evaluation and quality scoring. This is the largest gap, and the SIG knows it. OpenTelemetry captures what happened; it does not assess whether what happened was any good. A span can faithfully record that a model returned 1,200 tokens in 850ms, and say nothing about whether those tokens contradicted the source documents. Standardized conventions for embedding evaluation results (faithfulness, groundedness, toxicity, policy compliance) directly into spans are still in early discussion. Until they land, every team bolts on its own evaluation layer with its own attribute names, which is precisely the fragmentation the conventions exist to prevent. Watch this one closely: there's a real prediction circulating in the community that evals will become a CI gate, the way unit tests did a decade ago.

2. Content capture versus privacy. The conventions default to capturing no prompt or completion content for a reason. Prompts routinely contain PII, confidential business data, or regulated information. But debugging an agent often requires seeing what was said. The community's working answer is a three-mode design for content capture and a defense-in-depth posture: capture nothing you don't need at the SDK, redact at the Collector with the redaction and transform processors, and lock down the backend. The guidance the SIG echoes is to filter sensitive data as early in the pipeline as possible. This is less an unsolved problem than a permanently uncomfortable tradeoff that every team has to tune for itself.

3. Span explosion in agentic hierarchies. A single user request to a multi-agent system can generate hundreds of spans. An orchestrator calls three sub-agents, each makes multiple tool calls and LLM invocations, and each of those generates child spans. Without a sampling strategy, this overwhelms both backends and budgets. The practical guidance emerging from the discussions: head-based sampling around 10% for high-volume endpoints, paired with tail-based sampling to capture 100% of error traces. The convention work here is about making spans cheap enough to keep and structured enough to navigate.

4. The gaps nobody has filled yet. Two are explicitly flagged as unsolved. Real-time evaluation of multi-modal outputs (images, audio, video) has no standardized telemetry convention at all. And cost attribution across compound AI systems, where one user request triggers multiple models and tools each with different pricing, remains fragmented. If your products touch either, these are areas where early input carries outsized weight, precisely because so little is settled.

Part three: the agentic-systems effort

The single largest body of in-flight work is the proposal for semantic conventions covering generative AI agentic systems, tracked as a meta-issue in the GenAI conventions repository. It addresses a real shortfall: today's conventions handle LLM completions well but lack coverage for the things that make agents agents.

The proposal defines attributes for tracing tasks, actions, agents, teams, artifacts, and memory, along with the relationships among them. The goal is to standardize telemetry across complex multi-agent workflows so that debugging, evaluation, and reproducibility don't depend on each vendor's custom attribute soup. The initial agent-application convention draws on Google's AI agent whitepaper as a foundational framework, and the agent-creation and agent-invocation spans (create_agent, invoke_agent) are already specified.

Running alongside it is a parallel effort on agent framework conventions, standardizing instrumentation for CrewAI, AutoGen, LangGraph, Semantic Kernel, and others. Because each framework structures agents differently, this is adapter work as much as convention work, and it raises a genuine design question the SIG is still working through: should instrumentation be baked into each framework natively (as CrewAI has done), or provided through external libraries like Traceloop and Langtrace? The honest answer emerging in production is a hybrid of both.

The reason this matters to anyone running agents: agents fail differently from ordinary software. They can return a clean HTTP 200 and a well-formed response that is simply wrong, with no exception thrown and no error logged. The same input can trigger different tool sequences across runs. Observability that only answers "did it run?" is useless here; you need telemetry that answers "what did it decide, and why?" That is exactly what the agentic-systems conventions are trying to make standard.

Part four: how to get involved

If any of the four debates or the agentic work touches what your team builds, the path in is genuinely low-friction. Join the CNCF Slack and find #otel-genai-instrumentation. Drop into a Tuesday general meeting or a Monday agent meeting; they're open to anyone, from seasoned contributors to people who are simply curious. Read the 2026 roadmap issue to see what each sub-SIG is prioritizing. And if you have a use case the current conventions don't serve, bring it as a concrete example rather than a "+1". The SIG triages on real production context.

The core point, now with the detail behind it: AI observability standards are being written in the open, in real time, by a community that wants practitioner input, and the questions still on the table (how to score quality, how to trace agents, how to handle content safely, how to attribute cost) are the ones that will decide whether your telemetry survives the next three years of churn.

For the teams reading this, that's an unusual opportunity. Most of us inherit standards as finished artifacts and spend years working around the parts that don't fit. This is a rare window where the people who run these systems in production can shape the vocabulary before it hardens. The engineers on your team who are wrestling with span explosion or eval fragmentation today are exactly the voices the SIG is asking for.

We covered four things: where the community meets, the four live debates, the agentic-systems effort, and how to participate. The companion post covers the settled half of the picture, the conventions stable enough to instrument against right now. Together they're the full map: what to build on today, and what to help shape for tomorrow.