The History of Agile: From Toyota to Cargo Cult Agile

Agile is the most successful methodology in the history of software development. It’s also, in many of its current implementations, barely recognizable as what its founders intended. Understanding how a successful revolution became an abused brand requires knowing where it came from.

What Problem Was Agile Solving?

Before Agile, the dominant model was Waterfall: gather all requirements, design the full system, build it, test it, deploy it. The logic was impeccable. The results were frequently catastrophic.

The Standish Group’s 1994 CHAOS report, the first in a long-running series, found that roughly a third of software projects were cancelled outright, more than half were significantly over budget and behind schedule, and fewer than 17% were delivered on time, on budget, and with the originally specified features. The report’s methodology has drawn criticism since, but the pattern matched what practitioners were seeing. The “specified features” criterion turned out to be the most revealing problem: by the time software was delivered, the requirements had changed enough that what was built wasn’t quite what was needed.

The Waterfall model had a fundamental assumption baked into it: that requirements could be fully and accurately specified before development began. In practice, this assumption was almost always wrong. Requirements changed. Technology constraints emerged. Users discovered what they actually wanted when they saw working software. The model treated these realities as failures of process rather than inherent properties of complex work.

What Came Before Agile?

Several approaches were already attacking this problem before 2001.

Iterative and incremental development had roots going back to the 1950s. Craig Larman and Victor Basili’s “Iterative and Incremental Development: A Brief History” (IEEE Computer, 2003) records NASA’s Project Mercury using iterative development in the early 1960s. Barry Boehm’s Spiral Model in the 1980s introduced risk-driven, iterative planning. These approaches recognized that learning happens during development and that plans should accommodate new information.

Rapid Application Development (RAD), introduced by James Martin in 1991, compressed development cycles and emphasized continuous customer involvement. RAD practitioners were already doing many things that would become Agile best practices.

The Toyota Production System, described by Taiichi Ohno in “Toyota Production System” (1988), contributed ideas that would eventually become Lean software development: eliminating waste, just-in-time production, and the concept that workers closest to the problem have the most relevant knowledge.

What Happened at Snowbird in February 2001?

Seventeen software developers gathered at a ski resort in Snowbird, Utah. As Jim Highsmith’s account of the meeting puts it, they called themselves “organizational anarchists.” They had various methodologies (Extreme Programming, Scrum, Crystal, DSDM, Pragmatic Programming) but shared a frustration with heavyweight, document-driven development processes.

The group included Kent Beck (XP creator), Jeff Sutherland and Ken Schwaber (Scrum), Alistair Cockburn (Crystal), and others who had been thinking seriously about lightweight development approaches. They were practitioners describing what had worked for them, and they had no interest in presenting academic theory.

The result was The Agile Manifesto: four value statements, followed in the next few months by twelve principles, all short enough to fit on a single page. That brevity was deliberate. The authors wanted something that couldn’t be bureaucratized into a methodology without losing its meaning. (What each value means in practice is covered in the Agile Manifesto and its principles.)

Whether that aspiration succeeded is debatable.

How Scrum, XP and Kanban Grew Around the Manifesto

The Manifesto was a values document, not a process document. The frameworks that translated those values into practice had already been developing in parallel and continued to evolve afterward.

Scrum had been developing since 1993 (Jeff Sutherland), and Ken Schwaber and Sutherland presented it at OOPSLA in 1995. By the early 2000s it was the most widely adopted Agile framework, primarily because it was prescriptive enough to implement and lightweight enough to adopt without a full organizational transformation. Its rules are now maintained in the Scrum Guide, last revised in 2020.

Extreme Programming (XP), set out in Kent Beck’s “Extreme Programming Explained” (1999), was arguably the most technically rigorous Agile methodology: Test-Driven Development, pair programming, continuous integration, collective code ownership. XP had significant influence on engineering practices even in teams that didn’t formally adopt XP as a framework.

Kanban’s adaptation to software began in the mid-2000s, most notably through David J. Anderson’s work, summarized in his book “Kanban” (2010). Its origins in Toyota’s manufacturing system gave it a different character from the other Agile frameworks: less prescriptive about roles and meetings, more focused on flow and waste elimination.

Large-scale frameworks emerged as Agile adoption grew into enterprises that needed to coordinate many teams. Large-Scale Scrum (LeSS), the Scaled Agile Framework (SAFe), and Disciplined Agile all appeared from the late 2000s onwards to address this problem, with varying degrees of success and various criticisms about whether they actually preserved Agile values at scale.

What Went Wrong as Agile Spread?

Agile adoption spread remarkably quickly. By 2010, Agile had moved from a software development approach to a general organizational philosophy. Marketing teams did “Agile marketing.” HR departments did “Agile HR.” Agile principles appeared in management consulting proposals to companies that had nothing to do with software.

This success created its own problems.

The certification industry grew rapidly around Scrum and other Agile frameworks. Two-day certifications proliferated. “Certified Scrum Master” became a common LinkedIn credential for people who had attended a training and passed a multiple-choice exam. The certifications signaled exposure to vocabulary. Practical competence was another matter.

Cargo cult Agile emerged at scale: organizations adopted the rituals and terminology of Agile while maintaining the governance, incentives, and culture of traditional management. Daily Scrums happened in front of Jira boards, Sprint Reviews were conducted, retrospectives were held, and nothing changed about how decisions were made, how work was prioritized, or how much autonomy teams had. The annual “State of Agile” survey, run by VersionOne and later by Digital.ai, repeatedly ranked organizational culture among the top barriers to Agile adoption, even among organizations that considered themselves Agile.

“Agile” became a brand applied to anything iterative, anything customer-focused, anything collaborative. This dilution made the term almost meaningless in some contexts.

Where Does Agile Stand Today?

The core ideas of the Agile Manifesto have held up well. Short feedback loops are better than long ones. Working software is a better progress measure than documents. Customer collaboration produces better outcomes than requirements handoffs. Adapting to change is more realistic than following a fixed plan.

The implementations have become more varied and more contested. Scrum remains dominant. Remote work, which accelerated dramatically in 2020, has forced adaptation of practices designed for co-located teams. AI-assisted development is changing what “the work” looks like in ways that current frameworks haven’t fully addressed.

The Values Held Up Better Than the Implementations

The critics of Agile (and there are thoughtful ones) point to real problems: Agile teams can become reactive rather than strategic, sprinting from priority to priority without architectural vision; “responding to change” can become an excuse for poor upfront thinking; the human cost of endless iteration without stable foundations is real.

These are fair criticisms, and they land on implementations. The values themselves still stand. Returning to Waterfall won’t help, because the problems Waterfall failed to solve haven’t become easier. The answer is continued reflection on what the values actually require in any given context.

That’s the part the Manifesto got right that most implementations miss: it called for continuous improvement and never prescribed a permanent methodology. The history of Agile is a story of practitioners trying to solve a hard problem. The next chapter is the same story.