The METR Study: AI Tools Made Experienced Developers 19% Slower
A rigorous RCT found AI coding tools slowed down experienced developers by 19%. The developers themselves believed they were 20% faster. The perception-reality gap changes everything.
Tag
9 articles
A rigorous RCT found AI coding tools slowed down experienced developers by 19%. The developers themselves believed they were 20% faster. The perception-reality gap changes everything.
A junior spent 3 weeks stuck on a 20-minute fix. Remote work killed osmosis learning. Seven practices that rebuild mentorship for distributed teams.
SWE postings down 49% from peak. AI roles up 340%. Junior hiring collapsed 73%. The market is bifurcating and depth sets the price.
Karpathy coined both terms a year apart. One builds $400M startups. The other lost Amazon 6.3 million orders. The difference is about to define which developers thrive.
She got promoted to Staff, spent 18 months in crisis, and left for a Senior role. The staff engineer trap is real — here's how to avoid it.
200 unit tests passed. The chatbot still hallucinated a dentist's phone number. LLM testing needs evals, LLM-as-judge, and regression for non-determinism.
65% of companies use generative AI. Almost none test it properly. Here's the eval framework that caught our $47K hallucination disaster.
Technical debt costs $2.41T/year. But the metaphor itself is the problem. It's a communication failure, not a code problem. Here's what to say instead.
Playwright: 45% adoption, 78K GitHub stars, 2-3x faster than Selenium. Auto-wait killed flaky tests. Migration guide from Cypress and Selenium.