Abstract
As large language models (LLMs) become a core part of modern software stacks, observability is critical—but traditional tools weren’t built for the complexity of generative AI. OpenLLMetry is an open-source project built on OpenTelemetry, designed to provide native, end-to-end observability for LLM pipelines.
In this talk, we’ll walk through how OpenLLMetry works, how it integrates with popular frameworks like LangChain, OpenAI, and HuggingFace, and how it captures meaningful telemetry—prompts, responses, token usage, latencies, and more—without changing your app code.
We’ll also look ahead: What does standardization of LLM observability look like? What are the challenges in making sense of this new kind of telemetry? And how can we create a shared language and tooling to help developers understand, debug, and optimize generative AI systems at scale?
About the Speaker
Nir Gazit is the CEO and co-founder of Traceloop, and co-creator of OpenLLMetry. Prior to founding Traceloop, Nir served as Chief Architect at Fiverr and led machine learning teams at Google. Alongside his co-founder and CTO, Gal Kleinman, he co-created OpenLLMetry—an open-source project bringing native observability to generative AI workflows. Together, they now lead the Generative AI SIG at OpenTelemetry, working to define the standards for LLM observability.
