5 Minute DevOps: Continuous Delivery FAQ
There are many misconceptions about CD. There are also many things about CD that are not obvious on the surface. If you are “CD curious”, perhaps this will help.
“CI/CD or CD?”
It’s just “CD”. Just as DevOps encompasses security, business, compliance, user experience, etc., CD encompasses the entire development process.
“There is no objective definition for CI”
There is an objective definition. Some may believe that pulling updates from the trunk daily but only integrating changes after several days of work to make sure the change is “feature complete” is CI. People believe all sorts of things that aren’t true. Believing really hard doesn’t make it more true. The point of CI is team communication. The code integrates very frequently and is tested together. This allows everyone to see everyone else's changes. It reduces the problems with dependency between changes, code conflicts, etc. Smaller is better.
- CI Definition on Wikipedia
- CI practice and tips on MinimumCD.org
- Martin Fowler on CI
- Trunk Based Development patterns
“We automated the build. Are we doing CD?”
Build and deploy automation are important. Test automation is even more important. It takes people, process, and automation to execute CD. Automation is about 10% of the CD problem though. Check out MinimumCD.org’s minimum viable list of problems to solve.
Continuous delivery is continuous product development and continuous quality feedback. The robots enable us to standardize and accelerate delivery to reduce the cost and improve the safety of change. This makes it viable for us to deliver very small changes to get feedback on quality immediately. People frequently underestimate the definition of “small”. We aren’t talking epics, features, or even stories. We are talking hours or minutes worth of work, not days or weeks. A CD pipeline includes the robots, but it starts with the value proposition and ends when we get feedback from production on the actual value delivered so that we can make decisions about future value propositions. This takes teamwork from everyone aligned to the product flow and unrelenting discipline for maintaining and improving quality feedback loops.
“CD is just for unicorn startups?”
CD is a quality process, not an edge case used by dotComs iterating quickly on MVPs. In fact, many startups will grind to a halt and fail because they don’t include “how will we reliably deliver future changes?” in their business plans. Hopefully, they get enough funding to clear the massive tech debt before that occurs.
The reasons for delivering very frequently are:
- Ensure that we can deliver patches quickly and safely in an emergency.
- Establish an efficient and effective quality signal to reduce mean time to detect.
- Reduce the size of changes to reduce the size of the defects in each change.
- Reduce the cost of change to enable more freedom to try new ideas.
CD, when used to drive down batch size and learn from production, acts as a forcing function for stability, efficiency, effective value delivery, and improved quality of life.
“We will focus on CD after we deliver these features”
There’s really no point in building a feature if you cannot reliably test and deliver the feature to validate the value. Also, you have no idea if your tests are valid until you deliver and find out.
- The requirements are probably wrong
- We probably misunderstood them
- They will probably change before we deliver
How much investment do you want to make in development before you find out how many of the above are true?
The correct order of operations is:
- Pipeline to production
- Production delivery of “hello world” to validate the pipeline
- Next change.
1 & 2 should occur on day 1.
“We’ll need to hire smarter people”
No, CD is the tool that grows better engineers and better teams. Solving the problem of “why can’t we deliver this change safely to production today?” in your context is an engineering problem that takes teamwork and disciplined development. If this is the primary focus then skills such as effective use of feature flagging, continuous integration, developer-driven testing, & etc. will grow rapidly. It also improves morale due to the reduced stress of repetitive toil and Fear Driven Development.
“Our users don’t want changes that often”
I assure you that when you have an impacting incident that your users want changes that often. There’s a big difference between generally exposing new features daily and delivering changes daily to verify they do not break. Again, this is a quality process, not a feature delivery process. We need quality feedback from continuous testing in production instead of hoping our next big release doesn’t break anything. Your next big release will break something. See point 3 above.
“We don’t believe these wild claims about CD”
That’s pretty common. Most people don't understand how applying discipline to the development process and focusing on shrinking change-set sizes can make such a dramatic improvement to business outcomes and quality of life. Those of us who live that are trying daily to show that to you. Most of us aren’t paid consultants to tool vendors. We just work this way every day. We are telling you this because we want your lives to be better too and we want to maintain our quality of life if we join your organization. In fact, many of us won’t join an organization where this isn’t allowed and will leave organizations where it is banned.
“How hard is it to start?”
That very much depends on how teams are structured and how applications are structured. I recommend learning about Conway’s Law, Domain-Driven Design, and reading “Team Topologies”. For a team already organized to be small and product-focused, it will take a few months, given the right support and a knowledgeable person or two. For larger, legacy systems it will take the same broader organizational changes your competitors are working on today. The best time to plant a tree is 20 years ago. The second best time is today.
“What happens to teams who are prevented from improving CD?”
Once developers discover that their life is getting better, they will not want to stop. If you are in a management role for a team that is focusing on CD and then decides to send the team on a death march that prevents them from sustainably improving or decide to impose processes that prevent CD, the good ones will resist you by voting with their feet. You’ll be left with the below-average developers who didn’t fully embrace improvement. Good luck with your goals.
I will continue adding to this list as I think about other common questions. Feel free to check back or to comment with questions, challenges, or arguments you have where CD isn’t a valid process.
- 2021–01–23: Initial commit
- 2021–03–28: Added “We automated the build. We are doing CD.”
- 2022–09–02: Added section on CI. Added links to MinimumCD.org project. Minor typo edits.