Conversational AI · 2
The strategic and the analytical.
Professional work on a production conversational-AI system: evaluation frameworks, experimentation infrastructure, the strategy around them. Details are abstracted to respect confidentiality; the lessons travel intact.
Case studies
Designing an evaluation framework for a production conversational-AI system
How do you measure whether a production conversational-AI system is actually getting better? An eval taxonomy, a labeling protocol, and the downstream changes it forced.
LLM-as-judgeLabeling protocolGolden setFailure-mode taxonomy
Experimentation and analytics infrastructure for a production conversational-AI system
Instrumentation, BigQuery + dbt, CUPED for variance reduction, and a build-vs-buy memo that kept us out of a multi-year vendor lock-in.
BigQuerydbtCUPEDPower analysis
Say hi.
The old-fashioned way.
Three doors straight to a human — for anything the assistant can't or shouldn't answer.
It's in Madrid · EN / ES / HE · or ask the assistant on the homepage · how it works.