Your Dashboard Is Lying: Goodhart's Law and the Metrics Crisis in AI-Era Drupal Delivery
Your Dashboard Is Lying: Goodhart's Law and the Metrics Crisis in AI-Era Drupal Delivery
Ulka Neil (www.drupal.org/u/ulkaneildrupal)
Your delivery metrics have never looked better. Your AI tools are closing tickets, boosting velocity, and painting every dashboard green. So why did the client just escalate? Goodhart's Law has an uncomfortable answer and this session will make you rethink every metric you report.
Prerequisite
Experience managing or delivering Drupal projects in an agency or client-services context. Working familiarity with common Agile delivery metrics velocity, burndown, SLAs at a practical level, not theoretical. No economics or statistics background needed. If you've ever written a weekly status report or reviewed a sprint dashboard, you have all the context required.
Target Audience
Agency Owners, Project Managers, Account Managers, Scrum Master, Agile Coach, operation Leads
Outline
Your sprint velocity is up. Tickets are closing faster than ever. The dashboard is green. AI tools are generating code, drafting content, and accelerating throughput. Three weeks later, the release is a mess and the client is blindsided. What happened?
Goodhart's Law happened: "When a measure becomes a target, it ceases to be a good measure." The moment your team is judged on velocity, they inflate story points. When ticket close rate becomes a KPI, tickets get closed without real resolution. When SLA response time is tracked, clients get acknowledgment emails that say nothing useful.
AI makes it worse dramatically. AI can generate code that inflates velocity without improving quality. It can close tickets and surface green metrics faster than any human. Your dashboard looks healthy while delivery quietly drifts.
This talk walks through four metrics every Drupal agency tracks velocity, ticket resolution time, SLA response rate, and sprint burndown and shows how each gets gamed when AI enters the workflow. More importantly, it introduces what to measure instead: leading indicators of actual delivery health that resist gaming.
You'll leave with a measurement hygiene framework that separates vanity metrics from signals that actually predict whether your project is heading toward success or a client escalation.
This isn't anti-AI. It's anti-autopilot. AI makes delivery faster. Goodhart's Law asks: faster toward what?
Learning Objectives
Attendees will learn to identify which delivery metrics are vulnerable to gaming especially when AI accelerates the distortion. They'll understand Goodhart's Law as a practical diagnostic tool, not just an economics concept, and see it applied to four common Drupal delivery metrics: velocity, ticket resolution time, SLA response rate, and sprint burndown. They'll leave with a measurement hygiene framework that distinguishes vanity metrics from genuine delivery health signals including leading indicators like decision-log freshness, dependency resolution rate, client question turnaround, and risk-to-mitigation ratio. The goal is not to abandon metrics but to stop trusting the wrong ones.
Experience level
Intermediate