DevOps

DevOps is a new term for a group of concepts that, while not new, are experiencing a Renaissance and rapid spread throughout the technical community.  Like any new buzzword, people have somewhat confused and sometimes contradictory impressions of what it is.  Here’s my take on how DevOps can be usefully defined; I propose this definition as a standard framework to more clearly discuss the various issues DevOps covers.

Definition of DevOps

DevOps is a new term describing what has also been called “agile system administration” or “agile operations” joined together with the values of agile collaboration between development and operations staff.

Effectively, you can define DevOps as system administrators participating in an agile development process alongside developers and using a many of the same agile techniques for their systems work.

For this purpose, “DevOps” doesn’t differentiate between different sysadmin sub-disciplines – “Ops” is a blanket term for systems engineers, system administrators, operations staff, and various other subdisciplines.

DevOps means a lot of different things to different people because “agile development” itself covers a lot of ground.  People say that DevOps is “developer and operations collaboration,” or it’s “treating your code as infrastructure,” or it’s “using automation,” or “using kanban,” or a variety of seemingly loosely related items.  The best way to define it in depth is to compare to the definition of agile development.  Agile development, according to Wikipedia and the agile manifesto, consists of a couple different “levels” of thing.

• Agile Principles – like “business/users and developers working together.”  These are the core values that inform agile, like collaboration, people over process, software over documentation, and responding to change over planning.
• Agile Methods – specific process types used to implement the agile principles.  Iterations, Lean, XP, Scrum.  “As opposed to waterfall.”
• Agile Practices – techniques often found used in conjunction with agile development, not linked to a given method flavor, like test driven development, continuous integration, planning poker, etc.

I believe the different parts of DevOps that people are talking about map directly to these three levels.

• DevOps Principles – How we need to think differently about operations.  Examples include dev/ops collaboration, “infrastructure as code,” and other high level concepts; things like James Turnbull’s 4-part model seem to be spot on examples of trying to define this arena.
• DevOps Methods – Process you use to conduct agile operations – including iterations, lean/kanban, stuff you’d read in Visible Ops.
• DevOps Practices – Specific techniques and tools used as part of implementing the processes, like automated build and provisioning, continuous deployment, monitoring, anything you’d have a “toolchain” for.

The dev2ops blog describes this as “People over Process over Tools,” indicating how, according to agile philosophy, these are supposed to flow down into each other.

History of DevOps

The genesis of DevOps comes from an increasing need for innovation on the systems side of technology work.  The DevOps movement inherits from the Agile System Administration movement and the Enterprise Systems Management (ESM) movement.

ESM, which arose in the mid-2000′s, provided the original impetus of “Hey, running systems seems to still be in a pretty primitive state.  Let’s start talking about doing it better.”  John Willis, whurley, and Mark Hinkle from Zenoss were involved in that, and sponsored a BarCamp around the concept.   I think during this phase, initial enchantment with ITIL as a governance framework was largely overthrown for the “ITIL Lite” Visible Ops approach, as well as a shift from being “large vendor” focused – used to be, the “enterprise frameworks” like HP, IBM, and CA were the only meaningful solutions to end to end systems management, but more open source and smaller vendor stuff was coming out, including Spiceworks, Hyperic, Zenoss, and others.

Also in 2008, the first Velocity conference was held by O’Reilly, focusing on Web performance and operations, which provided a venue for information sharing around operations best practices. In 2009 there were some important presentations about the developer/operations collaboration at large shops and how that promoted safe, rapid change of Web environments.  Provisioning tools like Puppet and Chef had strong showings there.

Somewhat in parallel, as agile development’s growth in the development space was reaching its most fevered pitch and moving from niche to common practice, this turned into thinking about “Agile Systems Administration” especially in Europe.  Gordon Banner of the UK talked about it early on with this presentation.  A lot of the focus of this movement was on process and the analogies from kanban and lean manufacturing processes to IT systems administration.  Then sometime in 2009, Patrick Debois from Belgium started talking up DevOps, and there was a DevOpsDays event there that continued to light the fuse.  The concept started to be talked up more in other venues (I found out about it at OpsCamp Austin) including Velocity and DevOpsDays here in the US.

DevOps has become a bit of a “perfect storm” of these things coming together.  The toolchain approach fed by more good monitoring and provisioning tools, agile processes, dev/ops collaboration – they collided and unconsciously brought together all three layers of what you need for the agile movement (principles, process, and practices) and caught fire.

What is DevOps Not?

It is not “they’re taking our jobs!”  Some folks thinks that DevOps means that developers are taking over operations and doing it themselves.  Part of that is true and part of it isn’t.

It’s a misconception that DevOps is coming from the development side of the house – DevOps, and its antecedents in agile operations, are largely being initiated out of operations teams.  It’s because operations folks (and I speak for myself here as well) are realizing that our existing principles, processes, and practices have not kept pace with what’s needed.  As businesses and development teams need more agility, we’ve often been providing less, and we need a fundamental reorientation to be able to provide systems infrastructure in an effective manner.

Now, as we realize some parts of operations need to be automated, that means that either we ops people do some development, or developers are writing “operations” code, or both.  That is scary to some but is part of the value of the overall collaborative approach.

It’s also not “about the tools.”  One reason why I want to bring this discussion back to the Agile Manifesto discussion is that the risk of having various sub-definitions increases the risk that people will implement the processes or tools of DevOps without the principles in mind, which is definitely an antipattern.  The Agile guys would tell you that just starting to work in iterations without initiating meaningful collaboration is likely to not work out real well.

And that happens in agile development too – there are some teams here at my company that have adopted the methods and/or tools of agile but not its principles, and the results are suboptimal.

And in the end, it’s not exclusionary.  Some people have complained “What about security people!  And network admins!  Why leave us out!?!”  The point is that all the participants in creating a product or system should collaborate from the beginning – business folks of various stripes, developers of various stripes, and operations folks of various stripes, and all this includes security, network, and whoever else.  There’s a lot of different kinds of business and developer stakeholders as well; just because everyone doesn’t get a specific call-out (“Don’t forget the icon designers!”) doesn’t mean that they aren’t included.   The original agile development guys were mostly thinking about “biz + dev” collaboration, and DevOps is pointing out “dev + ops” collaboration, but the mature result of all this is “everyone collaborating”.