ManageEngine’s release management framework

Release policy and methodology

Per IT Process Maps, a release policy is defined as “a set of rules for deploying releases into the live operational environment, defining different approaches for releases depending on their urgency and impact.”

What a release policy includes:

1. Release level and identification

Release levels are categorized as major, minor, or emergency based on the objective and urgency of the release. Release identification refers to the number and naming conventions. For example, major releases may be numbered in increments of one (V1.0, V2.0, etc.) and minor releases may be numbered in increments of 0.1 (V1.1, V1.2, etc.). This varies based on the product team or organization.

Level

Definition

Identification example

Major

A release to introduce a new functionality. It can also be an accumulation of previous minor releases.

Launching a new product E.g., Product V1

Minor

A release to makes changes to existing functionalities.

Adding a new feature E.g., Product V1.1

Emergency

A temporary release required to quickly resolve an issue. It’s usually followed by a major or minor update.

Bug fixes E.g., Product V1.1.1

2. Expected frequency of releases

3. Roles and responsibilities (which we’ll talk about in detail later)

4. Documentation requirements

Stages of the release management life cycle

ITIL release management cycle

The release management cycle at ManageEngine.

Stage 1: Submission

The first step of the cycle is to identify a required functionality. Customer-facing teams like pre-sales, support, and product management work together to provide the development team with their input in the form of expected user interfaces and use cases. The submission stage is the primer that allows us to set the expectations for the entire cycle. What are our customers looking for? How will this product or feature enhance the user experience? Once we determine these, we move forward to stating the objectives of the release, including:

  • Business requirements.
  • Priority.
  • Risks and timeline constraints.*
  • Release type, methodology, and policy.
  • Scope of release.
  • Cost (if any).
  • Supporting documents for release requirements.

*Constraints for release deployment: Releases are dependent on multiple factors such as:

  • Service-level agreements (SLAs) or operational-level agreements (OLAs).
  • Resource availability.
  • Budget.
  • Security issues.
  • Corporate policies.

Listing our dependencies allows us to identify maintenance windows, during which releases can be deployed and users can be informed about services that may not be available. This doesn’t apply to emergency releases, which can be released at any time.

After the release requester submits the required details, a release coordinator verifies the information provided and assigns a release engineer. The release engineer is now the owner of the release and they’ll be in charge of the next stage. The owner provides an outline of the release implementation by:

  • Communicating with teams for the release plan.
  • Identifying the development team and the quality assurance (QA) team.
  • Identifying other users for various roles in the release, if any.
  • Assigning due dates for the standardized release process
ITIL release submission stage

Role

Responsibilities

Release requester

  • Defines the release objectives
  • Documents business requirements
  • Sets metric targets
  • Handles communication about the release scope to release team leadership staff or the release coordinator
  • Handles communication to key customer stakeholders (optional; this can be delegated to the release coordinator)

Release engineer

  • Submits the release implementation details and schedules
  • Oversees deployment of the release into the production environment
  • Carries out proper training, if required

Release manager

  • Verifies the planning
  • Ensures user acceptance tests have been carried out
  • Verifies that training has been provided to the affected user community, if required
  • Validates the back-out plan
  • Stages the pilot tests

Development head

  • Plans the release activity and forwards to testing
  • After confirmation from the release manager, builds the release activity with the help of tasks
  • Documents the release activity (timeline, risk, etc.)

Testing or QA head

  • Creates the test plan
  • Identifies issues
  • Approves the test stage after completion
  • Notifies the release engineer if the build can be processed to deployment

UAT owner

  • Tests the functionality of a new release
  • Liaises with the development team to decide on test criteria
  • Performs tests and reports back with issues in the release
  • Approves the UAT stage after completion

RACI matrix:

A responsibility assignment matrix can also be used to map out each member’s task in the entire release management process.

R – Responsible, A – Accountable, C – Consulted, I – Informed

Engineer

Manager

Coordinator

Development head

Reviewer

Testing team

Submission R AR I
Planning R AR AR
Development I I C
Testing I I C R
Deployment R A C C
Training/ support AR
Reviewer AR
Close AR I

Stage 2: Planning

The planning stage is handled by the release engineer. Tasks are distributed for different participants of the release. For example, the QA team members are mapped to the release to test out each build. If you have tasks that often repeat, add them as templates under each stage so you can choose them quickly instead of entering the same details each time.

Add release tasks from template
Release task template

The planning stage requires:

1. Scope: Effects upon:

  • Client-side business areas.
  • IT services.
  • IT infrastructure components.

2. A planned implementation date and time period, itemized according to:

  • Development phase.
  • Test phase.
  • Rollout.
  • Where necessary, reference to more detailed implementation plans.

3. Downtime details:

These consist of different downtime or maintenance windows required for completing the release activity. The downtime is configured here and reviewed later in the review stage.

IT release management downtime

4. Back-out plan:

What happens if the release backfires? We need a plan B that tells us how to undo the release or how the effects can be mitigated. A configuration baseline is a record of the last working state of the service. If the release fails, we can go back to that checkpoint to ensure it doesn’t impact users negatively. The configuration baseline can be defined in the release policy.

The release manager verifies the planning, validates the back-out plan, and approves moving on to the next stage: development.

Stage 3: Development

The development stage is handled by the development team lead. ManageEngine follows a software development life cycle process, which dictates how software is designed, developed, tested, and implemented to satisfy the needs of end users.

A standard template for development consists of sections like:

1. Details:

  • Schedule period (start and end times)
  • Actual time taken (start and end times)
  • Description
  • Attachments

2. Tools used.

3. Tasks:

Work is usually distributed across teams or team members, which are listed here.

4. Approvals:

Any approvals configured for this stage in the workflow are displayed here.

5. Status comments.

You might be wondering why release management has scheduled start and end dates and actual start end dates. A release, or any project for that matter, relies on many variables or has risks that may not always work out the way you’d expect. You might face delays in procuring hardware, or the finance team could delay approval. In such cases, scheduled or baseline dates and actual dates help your team understand if you’re ahead, on time, or behind schedule.

Release engineer to development head

*Tools

There are hundreds of tools available for software development. Here, we’ll list some of the tools preferred by our developers and what they’re used for.

Tool

Purpose

Ansible

IT automation

Ant

Build tool

Zoho Cliq

Communication and collaboration

Docker

Container management

GitHub

Distributed version control management

Jenkins

Open-source platform for CI

Mercurial

Distributed source control management

Selenium

Test automation

ServiceDesk Plus

Help desk and release management

Once the build is completed, the developers will provide the release notes and build URL.

Stage 4: Testing

Once the development team lead confirms the development stage is complete, the QA team starts working on testing the release package based on the tasks assigned to them. This stage is overseen by the QA team lead or testing lead. As the name suggests, the QA team handles planning, validates all testing-related tasks, and maintains standards.

There are three major types of tests:

(a) Functional testing:

This determines whether the code satisfies the desired functional requirements. Regression testing is also a form of functional testing. It verifies that the code doesn’t unsettle existing functionalities.

(b) Integration testing:

This tests if the code works well with integrations.

(c) Performance testing:

This monitors performance indicators. Load testing is a form of performance testing. We add load to the system and look for errors or breaks.

After these checks are performed, the team may perform product-specific checks or secondary tests to ensure satisfaction of business requirements and suitability for deployment.

For this stage, we require the following:

1. A test plan:

This demonstrates how the testing has to be carried out.

2. Testing tasks:

Using tasks, the testing lead distributes work across their team.

3. An issue tracker:

A document with issues recorded by the testing crew.

4. QA stage schedule start and end times:

These can be used by the testing lead to schedule tasks properly.

5. QA stage actual start and end times:

These can be used by the testing lead to record the actual time taken to complete the testing stage.

6. Approvals:

Before allocating the work, the testing lead needs to confirm that the proper build or environment is provided to the team with necessary access and requirements.

Development head to qa head

Apart from these, we also use features like:

1. Testing stage-specific statuses:

These allow the testing lead to keep track of the status of the work happening.

Default statuses used:

  • Pending for acceptance
  • Accepted
  • In progress
  • On hold
  • Rejected

2. Status comments:

When the testing lead specifies a status, they can also provide comments mentioning the reason why it’s chosen. This provides clarity and ensures everyone in the team is on the same page.

3. Notes:

On a timely basis, team members can use notes to interact and collaborate.

Stage 5: User acceptance testing (UAT)

In this stage, the end user will be provided with a test environment to validate that the release meets end users’ expectations as per the release plan. The UAT stage is predominantly handled by the UAT owner(s). They’re responsible for planning and validating UAT-related tasks. Ideally, this should be someone with an in-depth understanding of the process or software. They’ll also be provided with the requirement documents accepted by the QA team.

After the development stage, we select members from the QA team to act as the end users and carry out acceptance testing. The team member will raise issues, if any, in a bug tracking product. These need to be fixed by the development team before release. They also need to verify compliance parameters, which are part of the development cycle. This stage usually takes a couple of weeks.

For this stage, we require:

1. A UAT plan:

This demonstrates how the testing has to be carried out.

2. UAT tasks:

Using tasks, the UAT owner distributes work across the team.

3. An issue tracker:

A document containing issues recorded by the UAT crew.

4. UAT stage schedule start and end times:

These can be used by the UAT owner to schedule tasks properly.

5. UAT stage actual start and end times:

These can be used by the UAT owner to record the actual time taken to complete the UAT stage.

6. Approvals:

Before allocating the work, the UAT owner needs to confirm that the proper build environment is provided to the team with the access and requirements mentioned.

QA head to UAT owner

Apart from these, we also use:

1. UAT-specific statuses:

These allow the UAT owner to keep track of the current status of the work happening.

Default statuses used:

  • Pending for acceptance
  • Accepted
  • In progress
  • On hold
  • Rejected

2. Status comments:

When the UAT owner specifies a status, they can also provide comments mentioning the reason why it’s chosen. This provides clarity and ensures everyone in the team is on the same page.

3. Notes:

On a timely basis, team members can use notes to interact and collaborate.

Stage 6: Deployment

Types of deployment

1. Big bang vs. phased deployment

Using a big bang approach simply means updates are released all at once for all end users, whereas phased deployment involves a release of updates in parts or for different user groups.

Big bang vs. phased deployment

There are pros and cons for both methods.

Factor

Big bang deployment

Phased deployment

Implementation cost

Low

High

Risk

High

Low

Implementation time

Short

Long

Productivity

Low (at first)

High

ROI realization

Fast

Delayed

There’s no definitive answer on which deployment method works better. You should evaluate them based on specific conditions such as budget, productivity, risks, and timeline.

2. Push vs. pull deployment

Push deployment:

An update is pushed from a central location to users simultaneously.

Pull deployment:

Updates are made available for users to download at their convenience or when a user workstation restarts, e.g., mobile app updates.

Push

Pull

Enforces uniformity

Lacks uniformity

Requires accurate configuration data

Doesn’t rely on accurate configuration data

Harder to scale up

Not complex to scale up

3. Automated vs. manual deployment

If your organization has fast release cycles, automated deployment is a far better choice than manual deployment.

Benefits of automated deployment:

  • Ensures repeatability and consistency
  • Less prone to errors
  • Takes significantly less time
  • Supports larger target environments
  • Increases productivity and efficiency

Manual deployment is preferred only in certain cases, like rare releases and releases with limited cycles.

Now, let’s talk about the deployment process. It’s time to take our build from the test environment to the live environment. The deployment stage is handled by the release engineer and overseen by the release manager.

The release owner must document:

  • Downtime for configuration items involved.
  • Schedule start and end times.
  • Actual start and end times.
  • Tasks.

After deployment, the release engineer confirms completion of the stage.

UAT owner to release engineer

Stage 7: Training or early life support (ELS)

ITIL® defines ELS as “a stage in the service life cycle that occurs at the end of deployment and before the service is fully accepted into operation. During early life support, the service provider reviews key performance indicators, service levels, and monitoring thresholds and may implement improvements to ensure that service targets can be met. The service provider may also provide additional resources for incident and problem management during this time.”

Customer satisfaction is key, right? The objective of this stage is to keep all customer-facing teams updated on new functionalities, assess overall performance, and quickly resolve any operational issues that pop up during the initial phase after deployment. ELS promotes increased user satisfaction and a reduction in number of reported incidents.

Requirements that must be documented in advance include:

  • Resources allocated.
  • Time period.
  • SLAs and OLAs.
  • KPIs.
  • Exit criteria.

The development and QA teams are responsible for training support staff who in turn assist end users. The release engineer ensures training and ELS are provided to affected users relating to the release for the time period as per the SLA.

Development, QA to support

Stage 8: Review and close

Review

The review stage helps determine whether the release was successful or not. At this point, we should be able to answer the following questions:

  • Does the release carry out its required function?
  • Are all the relevant teams aware of these changes?
  • Can this build template be replicated easily?
  • Can it be rolled back?
  • Is more training required for users?
  • Is it user-friendly?
  • Have all configuration items and processes been updated in the CMDB?
  • Have all the post-launch issues been resolved?
  • Is the documentation process complete?

The review stage is owned by the reviewer, who is usually a member of the development or QA team and who may not have been involved in the build process. The release reviewer and a team assess The release reviewer and a team assess the release, including the entire release process’ efficiency, cost, and time effectiveness, impact, intended and incidental outcomes, and more. The processes involved in each stage of the release will be thoroughly inspected. The review team also determines whether the release met the specified business requirements while causing little or no disruption to the production environment. If it meets the business requirements, SLA targets, and end users have stopped reporting incidents, they can verify the deployment and provide review comments.

Reviewer to release manager

Next review schedule:

If the reviewer wants to re-verify the deployment due to changes that may have occurred, they can choose another date for it. In that case, the review stage will be moved back to “In progress.” If there are any other minor issues that don’t pose a threat or affect users, these must be documented and addressed in the next release.

Close

The life cycle of any release, be it completed or canceled, ends with the closure stage. In this stage, the release manager associates a closure code for the release and provides their comments, if any. The closure code provides details on whether the release is closed due to cancellation, rejection, completion, or other reasons, which makes it easier for stakeholders to understand too. Based on the comments, the release engineer can finally close the release. Resources taken up by this release can now be allocated to other tasks, i.e, your next release!

Metrics that matter

KPIs put a quantitative value on how effective your release management process is and identify areas of improvement. These metrics usually cover every stage and are best understood when visualized graphically. We add release metrics to our advanced analytics tool to keep track of our progress.

Examples of KPIs for individual releases:

  • Estimated release downtime in minutes or hours
  • Total release downtime in minutes or hours
  • Deployment duration
  • Number of incidents
  • Mean time to repair
  • Difference in estimated and actual delivery date
  • Number of defects

Examples of KPIs for the overall release management process:

  • Number of releases
  • Release duration in days
  • Number of release back-outs
  • Number of releases by priority
  • Number of releases by risk
  • Average cost per release
  • Average release cycle time

Get fresh content in your inbox

By clicking 'keep me in the loop', you agree to processing of personal data according to the Privacy Policy.