Supervising AI: from black box to trust dashboard.

By Samuel Desseaux — CEO of Erythix

Paris Monitoring Day 2025

Large language models (LLMs) are now finding their way into every business environment: copilots, assistants, chatbots, conversational agents… But putting them into production is often done blind. Unlike a traditional application, an LLM is neither deterministic nor fixed: it learns, evolves, drifts… and sometimes hallucinates.

This talk proposes a shift in perspective: moving past the fantasy of autonomous AI to take back control, through an approach built on supervision, observability and business-level readability. Because AI without oversight is a black box — and therefore a risk of drift, cost overruns or critical errors, invisible until it's too late.

From technical monitoring to cognitive observability

Supervising an AI model isn't just about measuring latency or the number of tokens consumed. It means learning to observe the model's behavior and internal logic through new indicators — cognitive SLIs:

• behavioral drift,

• hallucination rate,

• cost per useful token,

• response stability,

• security and compliance scores (GDPR, AI Act…).

This approach puts monitoring back at the heart of trust: understand in order to control, observe in order to steer.

Two angles for regaining control

Samuel Desseaux will explore two complementary dimensions:

🔹 Black box or dashboard? How to make AI systems intelligible and steerable, with explainable metrics, useful alerts and indicators aligned with what truly matters: quality, compliance, performance.

🔹 AI deployment checklist What absolutely must be planned for to avoid classic mistakes? Which oversights end up costing the most? How do you build AI governance that is accessible, useful and lasting?

Real-world cases and tools

Through examples drawn from industry, local government and digital services, Samuel will show how to combine tools like OpenTelemetry, Langfuse, Grafana or Phoenix (Arize) to build supervision tailored to LLMs.

The goal: to build trust dashboards where performance, compliance and human understanding come together.

The key message is clear: deploying an AI without supervision is like running a company without a dashboard.

A readable, steerable, responsible AI

It's time to move past the technological illusion and build AI that is readable, steerable and responsible. In a world where artificial intelligence is becoming a strategic lever, observability is no longer a technical constraint: it's a factor of reliability, compliance and performance.

A loyal pioneer of Paris Monitoring Day

Present since the very beginning, Samuel Desseaux is one of the pioneers of Paris Monitoring Day. Always available, generous in sharing his knowledge, he embodies the spirit of the event: an open, demanding community passionate about the challenges of monitoring and observability.

Don't miss his talk on November 25, 2025 at Paris Monitoring Day in Paris.

📅 Paris Monitoring Day – November 25, 2025 🎟️ Register now → Ticketing

#ParisMonitoringDay #Monitoring #Observability #AI #LLM #MLOps #AIObservability #Erythix #TechForGood #DataReliability