Below are the top 10 questions I try to ensure I have answered before we release a new feature into production.

New Feature Release Checklist

1. What is the release strategy?  Here are 3 options:
   • Big Bang – This is the simplest form of release, where you roll out the feature to your entire user base.  This can either be done with a Feature Gate (or switch), or via the deployment itself.
   • Incremental – This is where you initially roll out the feature to a small subset of users.  You may choose to roll it out to a specific region, user role, or other demographic.  Once you have some confidence after the initial rollout, you increase the user base (at whatever rate you desire) until it eventually gets to 100%.
   • Parallel – This one isn’t always an option, but it’s great to use when possible.  Let’s say you are making some performance improvements to a complex DB query.  Rather than using the big bang and incremental approaches described above, or implementing some crazy regression testing strategy, you can use the parallel live technique.  With this approach, you run the query simultaneously through the legacy and new code paths.  If the results are the same you just return them.  If they are different you return the legacy result to the user and log the difference for review later.  This approach allows you to test out complex refactoring’s with no risk to your users.
2. What about feature gating (a.k.a. feature flags or dark launch)?  How will the feature be enabled in production?  Can it be turned on for additional users without a redeploy?  Check out this Forbes article HERE for some more info on how Google and Facebook leverage feature gating.  Another interesting article from HBR on feature gating HERE.
3. What is the rollback plan?  What happens if this feature causes issues in production?  Can it be disabled with a feature switch?  Or is another deployment necessary?  If so, what’s the estimated downtime?  How will we know if there is an issue?  Will an alarm go off or do we need to manually test?
4. Was the code peer reviewed?  This one might sound like a no-brainer to some, but you’d be surprised how many companies still don’t perform peer reviews.  If you aren’t using github which has reviews built right into their pull request mechanism, there are still countless other tools out there that make peer reviews painless.  One I’ve seen used successfully in the past is Code Collaborator.  Also, it’s a good idea to rotate the reviewers or spot check them, as some developers will treat them with a rubber stamp.
5. Did you write any unit tests?  If you are writing a new feature, you should add in unit tests where possible.  If you are making tweaks or refactoring an existing feature it would be great to add in some tests now while you have spent the time to grok this section of the code base.
6. Was any load testing performed?  Besides standalone load tests (e.g. JMeter, and many others), another great technique to use is HTTP replay, where you replay your production traffic through a staging environment.  We’ve seen countless times where our canned JMeter tests didn’t catch all the corner cases that a production user’s data could.
7. Did QA approve the feature?  Ideally the answer to this question is automated by using some project tracking tool like JIRA.
8. Did the PO (product owner) and UI/UX review the implementation?  I can’t tell you how many times I’ve seen a seemingly simple feature get misinterpreted between the various stakeholders, including product owner, UI/UX, customer, and engineer.  It’s a great idea to just pull this folks over to your desk for a quick review of what is being rolled out.
9. What about DevOps?  Is any new infrastructure necessary for the release?  Are there any release timing dependencies?  What about config settings?
10. Do we need to communicate release notes to customers?  In many companies communication with the customer is handled by product mgmt.  However, if you’re in a startup you may need to handle this one yourself.

 

That’s it!  I try to make sure I have a good idea of the answers to these questions for all new features that get rolled out.  Unfortunately sometimes things slip through the cracks.  However, I’ve noticed that most issues that arise downstream could have been avoided by first getting answers to the questions above before you ship.

EDIT:  I recently created a Production Readiness Checklist that condenses this blog post into a single, good looking sheet of paper that you can pass out to your team or hang on the wall in the office.  You can get a copy of this checklist by signing up for our mailing list on the right!