The 3 Ways of DevOps
The Forgotten Template to Success
Introduction
DevOps is a term that means different things to different people. For me, it’s less about semantics and more about the concepts. Implementing these 3 fundamental concepts is foundational to making an IT organization efficient.
The Three Ways
The First Way: Flow or Systems Thinking
Flow is about understanding how work is done. You need to be able to trace how earlier tasks lead to later tasks, how hand offs work, which can include forking into parallel tasks and rejoining. Typically you want to illustrate this flow visually. Most depictions move horizontally from left to right, initial tasks on the left followed by later tasks to the right. The whole idea being to understand the big picture and how it flows.
By having the entire picture you get to see the forest from the trees. This helps you avoid localized optimizations, ie: a small portion of an enterprise. The whole gets considered so you don’t miss the highest priority in favor of a silo.
A few practices that work well to get you started understanding flow are Value Stream Mapping and Metrics-Based Process Mapping (MBPM).
The Second Way: Optimizing Feedback Loops
The goal here is to get information back on what you are doing, what you’re building and is it working well or not. Ideally you want this information sooner rather than later so working on speed and frequency of feedback is also important here.
A prerequisite to understanding this concept is that perfection in IT is impossible to obtain or even near perfection is very hard to obtain. If you understand that and you seek quality then you have to take an iterative approach that uses feedback cycles to incrementally get better. The faster you can test your approach and get this information, the faster you will get better.
If you are coming from a waterfall methodology, this is often difficult and counterintuitive. Waterfall methodology holds that if something didn’t turn out well in the design once it was delivered, it was because of poor planning or poor design, and you need to plan or design better next time. This second way stands in stark contrast and holds that you can’t create good solutions without feedback, so if feedback is the ultimate barometer of success, you need to design a process that gets feedback quickly and course correct sooner rather than later.
Metrics
One of the more effective feedback loops can be quantifiable metrics. The highest order of feedback is from your customer or end users.
It is worth pointing out that even subjective metrics can be quantifiable if you put the rating on a scale, think product reviews for instance. You can also pose subjective questions to rate things relative to each other or in rank order.
Business metrics are not only for the business to gauge customer success, this also has technical information. A sudden drop or surge in business could be due to a technical issue. For example if you were an e-commerce website and you typically take 100 orders per hour and suddenly the orders fall to zero, that could be be an indication something with the system isn’t functioning correctly.
Proxy Metrics
Somethings that can be optimized are hidden from the end user, for instance how long is your testing cycle for a release. This does effect the customer but its probably not something they can put their fingers on as to a specific way things can be measured in the customers eyes. For these back office type metrics or proxy metrics it can be important to measure so long as optimizing it serves the end customer. Ideally you do need to make sure the end user is really benefiting from optimizing processes when using proxy metrics and you’re not just justifying it based on a theory of how it could benefit the end user.
One popular method of measuring proxy metrics that are often shown to benefit end users is the DORA metrics from the Accelerate book:
- Deployment Frequency: This measures how often a successful releases to production is done. A higher frequency represents a more mature team or organization. If you are measuring in multiple releases per day, that is highly mature. If you are releasing quarterly or annually you may need some improvement.
- Lead Time for Changes: This measures the time it takes for an Agile story to move from the start to it being live in production. Lower time is considered more mature as users are delivered value quicker and the business can benefit from their investment sooner rather then later.
- Change Failure Rate: This measures the percentage of code changes that result in failures or incidents in production. Lower failure rates are considered more mature when the other metrics are also optimized.
- Mean Time To Recovery (MTTR): This measures the time it takes to restore a service after a failure. A lower MTTR indicates a more resilient system and a more effective problem response process.
The Third Way: Culture of Continual Learning and Experimentation
Once you get the first two ways established in your DevOps journey, you are now ready to move on to perhaps the most powerful of the ways.
If you remember me saying that perfection in IT is impossible to obtain or even near perfection is very hard to obtain, it is because really good answers are not obvious from the start and the ever-shifting IT landscape of technologies is rapidly changing. Between these two factors, you have to allow space for learning and growth.
Not all learning is a training course or reading an O’Reilly book (this is the technical publisher I am referring to). While I encourage this, it’s not a complete solution to learning. Some things are unique to your environment, unique to your customer base and you can’t get that out of generic sources of information.
Experimentation can be a tough thing to allow in some IT cultures. The thinking typically is that a failure is always a waste of money. Also, sometimes failures are also assumed to be because of low knowledge or low competency employees. If these ideas take hold, that causes a culture of caution and hesitancy to take place, as no one wants to do the wrong thing. This generally leads to dated technology as doing nothing is safer then making a change with the chance of something going wrong. It also leads to slow changes as overly cautious culture takes hold causing the organization to fall behind the market over time. IT departments that obsess over getting it right the first time more often than not slowly fail as a whole. There needs to be a balance between avoiding failures and speed of delivery.
Experimentation must be encouraged, failures need to be seen as and taken as a learning experience, having environments, tools and time are all necessary. Also successful experimentation needs to be celebrated to build this culture.