“Daily Production Releases” Clarified, Part 1

This article is one of the “IT Terminology Clarified” series.

This is abridged version of the original article on my Medium blog, 2022-11-10.

Below is one reader’s comment on my article “Release Early, Release Often” Clarified:

That is fairly common feedback. This article will answer this question and clarify “Daily Production Releases”.

1. “Daily Production Releases” can mean multiple times a day.

The “Daily” in “Daily Production Releases” does not mean exactly one release per day. For example, Facebook “releases twice a day”, for simplicity, people call it “daily releases”, rather “4-hour or 12-hour releases”. So, the ‘daily’ is not a literal date (once every 24 hours).

“Facebook is released twice a day, and keeping up this pace is at the heart of our culture. With this release pace, automated testing with Selenium is crucial to making sure everything works before being released.” — DAMIEN SERENI, Engineering Director at Facebook, at Selenium 2013 conference.

Some people stretch the concept a bit, calling “one release every 2 days” daily-production releases. Personally, I think it is fine too. If there is no code change at all, why bother with another purposeless redeployment?!

The empathize of the “Daily” means a much shorter (~ 1 day) production release cycle (compared to the common 6 months), which will totally change SDLC.

2. “Daily Production Releases” is a capability of regression testing, not a schedule

It is OK to plan a specific time for a 6-month release (by the way, a release means a production release in this article) in Waterfall, but it would be unnecessary to set a specific time for daily production releases, such as 6 AM every day in the morning, at least for most software products. If missing 6 AM, it is not a big deal, we just do it in one hour or two.

So don’t be fixated on the time, as long as it is frequent (from a few hours to a few days).

A production-ready build means the team has high confidence in its quality, specifically, the build passes the quality assurance. A bad release is often much worse than no releases. (Remember Windows Vista?)

So, “Daily Production Releases” is really about the capability of full regression testing, not actual deployment. For all my web apps, deployment is done via one script mina deploy, takes between 15–30 seconds. The full regression testing (running automated end-to-end (UI) tests) in the BuildWise Continuous Testing server takes about 30–50 minutes.

3. "Daily Production Releases" means production deployment on the same day when a change is made.

If there is no change to code and infrastructure, there is no point in doing a production deployment.

Now I can answer the "daily production release is not required in my app or many industries".

• "Not Required"

It is not required because few software companies can achieve that, most IT professionals never witnessed one. Few people thought "high-speed rail" was required until Japan did it in the 60s.

Saying "not required" is really a sign of limited knowledge. Check out the LinkedIn story: The Software Revolution Behind LinkedIn's Gushing Profits (2013). LinkedIn, in Silicon Valley, couldn't get it before successfully luring Kelvin Scott.

Let me put in a sentence that might help understand how rare the real senior test automation engineer is. Image this job ad for Uber Drivers: “a former Grand Prix Winner”.

• “Not for my app or this industry”

Efficient and high-quality product releases are generic, and independent of the type of software. Let’s say, for company A, one production release per year on Jan 1st. On D-day, after deployment, an end-user found a critical defect. The development team quickly acted on and created a fix in 2 hours. Then a test manager said: “sorry, our regression testing takes 2 weeks. We will do deployment next year.”

Code changes happen all the time, and to release to production requires full automated regression testing, which is the pain point. Most (>99%) software teams do not know how to do proper automated regression testing. Quite often, some tech leads blindly thought they knew a bit about test automation, which made things worse. They shall seek professional help from a real test automation coach (rare) instead.

--

Part 2 will cover:

• 4. Automated End-to-end (via GUI) regression testing is the enabler of “Daily Production Releases”
• 5. How “Daily Production Releases” is possible?
• 6. “Daily Production Releases” means a complete change to the software development process.
• 7. Most CIOs, Managers, Tech leads, and Software Engineers chickened out in front of “Daily production releases”