About Azure DevOps

Azure DevOps is Microsoft's end-to-end solution for development teams. It provides numerous features, including source code repository, requirements management, project management, automated builds, lab management, as well as release management.

The Azure DevOps platform started life as Team Foundation Server (TFS) in 2005 and has gone through a number of renames since, including Visual Studio Online and Visual Studio Team System/Services (VSTS).

When Octopus is compared to Azure DevOps, it's really about comparing Octopus Deploy to the Release Management features of Azure DevOps, as Octopus doesn't attempt to do bug tracking, source control, or builds.

The release management story in Azure DevOps began when Microsoft acquired InRelease in 2013. Most TFS users found InRelease limited and confusing, and they pushed Microsoft to look at other solutions, including Octopus Deploy, as examples of how release management could be done better. In 2015, Microsoft rebuilt the release management aspects of TFS (then VSTS at the time) on the same core as their new build platform.

In 2018, Microsoft acquired GitHub, and recently Microsoft's focus seems to be moving to GitHub Actions, with a number of key people involved in Azure DevOps moving over to GitHub. While we expect Azure DevOps to be supported for a long time, it's not clear how the roadmap will evolve or how much Microsoft will continue to invest in it.

About Octopus Deploy

Octopus Deploy is a best-in-class platform for release management, deployment automation, and operations runbook automation.

Unlike Azure DevOps, Octopus doesn't provide bug tracking, build automation, or source control. We're focused on building software that does a few things very well and integrating well with other best-in-class tooling.

Octopus Deploy is an independent software vendor. It was founded in 2011 and is bootstrapped, conservatively managed, and profitable. Octopus Deploy's goal is to build the most powerful and capable solution for release management, deployment, and operations automation. Learn more about our company.

Highlights & key differences

While there are many differences between Octopus Deploy and the pipelines/release management aspects of Azure DevOps, the table below summarizes the key differences.

YAML vs Classic pipelines

Azure DevOps implements two distinct pipeline styles: classic and YAML. The Microsoft documentation provides a comparison of the features available for each style.

Classic and YAML pipelines have differences in the steps they can use, the security boundaries they expose, the concepts they use to model deployments, and the high level workflows they facilitate. Teams looking to automate their deployments in Azure DevOps must make the decision early on as to which pipeline style to use, or support both styles.

Microsoft has been careful not to suggest that one pipeline style is recommended or preferred over the other. However, you can find hints in issues, release notes, and public forums that indicate Microsoft's commitment to each pipeline style.

For example, the GitHub issue What's the long term plan for Classic Release Pipelines? has this response:

Currently there are no plans to deprecate Classic pipelines. However, new features are being planned for YAML first.

This sentiment is reflected in the release notes for Azure DevOps 2022, which lists one "feature" for classic builds (Removal of per-pipeline retention policies in classic builds), and six features for YAML pipelines.

Octopus provides a single, unified deployment experience regardless of whether you wish to define your deployment process as code or not, so there is no need to support two distinct styles.

Limitations of orchestrating Continuous Delivery with YAML pipelines

With the bias towards YAML pipelines in mind, it is worth understanding the limitations of multi-stage YAML pipelines when performing deployments.

Continuous Delivery is a largely automated deployment process that includes manual touch points. For example, release managers may be required to initiate a production deployment. Continuous Deployment, on the other hand, is a completely automated workflow where every commit is deployed to production with no manual intervention, so long as all the tests and other automated requirements pass.

While Continuous Deployments have many benefits, most teams we talk to find Continuous Delivery works for them.

Because of the way they were designed, Azure DevOps multi-stage YAML pipelines don't provide a great Continuous Delivery experience. Each execution of a YAML pipeline expects to run to completion, which in a typical implementation is to complete a production deployment, unless a ManualValidation step or Environment approval times out or is rejected. Rejected approvals result in a failed pipeline, despite the fact that most builds are not expected to be deployed to production when practising Continuous Delivery.

The GitHub issue How to just do manual propagation instead of approvals? discusses this issue, with comments like this highlighting the fundamental problem:

I also need the ability to deploy manually to environments. Setting approvers is insufficient because not every build will go to production. This means most builds would be considered failed due to a timeout, which defeats the entire purpose of the "pipeline failure" metric.

Octopus fully supports both Continuous Delivery and Continuous Deployment. Promoting a release to the next environment is as easy as clicking the DEPLOY button, without any implicit requirement that releases always make it to production. Or, if automated deployments to the production environment are required, deployments are automatically triggered on the next environment when a previous deployment completes successfully.

Octopus releases are progressed to new environments as needed.

Classic pipelines lack global environment security controls

Continuous Delivery often requires the ability to restrict production deployments to a small set of trusted individuals. This means implementing security rules that:

1. Can define what the production environment is.
2. Can enforce rules that allow select individuals to perform deployments to the production environment.

Classic releases in Azure DevOps define environments as stages. However, stages are release pipeline specific entities and each have their own security policies. This means that even if two release pipelines define a stage called "Production", both stages are unique entities, and both must be secured individually.

Deployment groups are a second resource used to define environments. However, there are no permissions or approvals that can be applied to deployment groups to control who initiates a deployment.

What this means in practice is that release pipelines must either:

• Individually define security rules on their production stage.
• Individually define approval gates before a production deployment is initiated.

It is easy to see how this process becomes unwieldy when you have dozens or hundreds of release pipelines. Assuming each release pipeline clearly defines a stage representing a production deployment (which is not guaranteed) security teams still need to individually implement or verify the permissions assigned to the production stage or the approval gates.

The Stack Overflow question How [do I] prevent users from deploying to a Deployment group discusses this limitation.

Octopus environments are global resources used to group infrastructure such as accounts and targets. Environments are also placed in lifecycle phases determining the order in which releases are deployed. Critically, environments also provide a security boundary that cross cutting permissions are applied to.

There is no question as to what your production environment is in Octopus: it is literally the environment called "Production". Security rules are applied to environments at a global level providing certainty that your deployments are secure.

Octopus permissions can be scoped to environments and projects at a global level.

Runbooks: automation for Ops too

Modern operations teams often need to automate a huge variety of tasks:

• Routine operations, especially for tasks that don't happen very frequently. For example:
  • Backing up and testing the restore of databases.
  • Requesting and updating SSL certificates.
  • Resetting API keys or rotating passwords.
  • Provisioning or deprovisioning infrastructure.
  • Suspending or taking applications offline for maintenance.
  • Updating a test environment with fresh, sanitized production data.
• Emergency operations, such as tasks you might need to perform at 3 AM after an alert is raised.
  • Failing over to a disaster recovery site, and then back to the main site.
  • Restarting a set of applications, perhaps in a specific order.
  • Restarting a web application that experiences a periodic memory leak that is yet to be resolved.

Whether you use Octopus or Azure DevOps for releases, you'll likely need to tell your release management solution a whole lot of things:

• Access to your Azure/AWS subscriptions.
• The various servers, cloud targets, and other resources your software runs on.
• Certificates and API keys for various other platforms you use.
• Database and other credentials for schema updates.
• Permissions and controls around who can deploy.

If you're going to set all of this up for deployments, you should be able to use it for other automation tasks too.

In addition to automating deployments, Octopus also provides first-class support for Runbooks, which can be used to automate these routine and emergency operations tasks. Runbooks in Octopus belong to the same project as your deployment process, and they have access to the same environments, variables, certificates, and everything else. It's the closest thing to a complete DevOps automation platform.

Runbooks in Octopus Deploy can be used to automate routine tasks from the same platform.

Despite the name, Azure DevOps has no ability to automate operations tasks that are not deployments. Microsoft's recommendation is to use Azure Automation, another solution. Nothing is synchronized between them, so you'll end up duplicating all of this configuration, with no source of truth. The silo between Dev and Ops will remain, each using their own automation solution.

Configuration as code

Octopus and Azure DevOps Releases use a UI to define the individual steps for deployments. Most developers on a team aren't experts in build/deployment automation, and co-ordinating complex deployments across many machines doesn't lend itself to a scripting language. The UI makes it easy for people - especially those who didn't author the original deployment - to figure out what is going on.

Starting in 2018, Azure DevOps Pipelines added support for writing build/CI pipelines in YAML. In fact, for builds, the UI-driven way of automating builds has been renamed "Classic Pipelines", a signal that YAML is the future.

Whether you like the choice of YAML or not, you'll probably agree Configuration as Code is a good idea. So you might be surprised to learn that Releases in Azure DevOps still use the "Classic" UI-driven solution - there's no YAML support for releases. Microsoft originally indicated it would happen, but so far it hasn't, and it's not on their public roadmap.

It seems as though Releases (and Release Management) in Azure DevOps has been ignored for YAML, in favor of "multi-stage Pipelines" which use YAML. However, multi-stage pipelines assume that the entire pipeline will execute (unless there's a failure) all the way, for every commit. This makes sense if you want every commit to go to production as soon as it's built, but not if you have any kind of approval process (e.g., a QA team), or designated times when deployment is safe.

The summary is, in Azure DevOps, you can use YAML, or you can use release management, but not both. And there's nothing on the roadmap suggesting Release Management will get YAML support.

With Config as Code in Octopus, we provide the full power of Git with the usability of Octopus. This means you can see your deployment process in your Git repo next to your application code and evolve them together. When Config as Code is enabled for a project, you can continue to use the Octopus UI as you always have, or you can edit the text files in your favorite editor.

Our Config as Code solution is different. Importantly, the Octopus UI still works, so you don’t need to learn a whole new language to edit your process. We want you to have a great experience using the Octopus Web Portal so you can configure your Git repository, select your branch, create new branches, commit changes, and more, all from the Octopus UI.

We believe branches are Git's superpower, and we wanted to expose that power. You can switch and create branches, in the Octopus UI, allowing you to make changes to your deployment process on a branch. This effectively allows for a draft mode, as you can then create a release from your branch and test your changes. When combined with features from your Git provider, like pull requests, protected branches, and code-owners, this enables a whole new set of workflows.

Our configuration file format is Octopus Configuration Language (OCL), based on HashiCorp's HCL. We wanted our configuration files to be human readable and to support complex documents like deployment and runbook processes. We like HCL and feel it's the right tool for the job, but we've written our own parser/serializer. This means we're not obligated to follow any direction HashiCorp takes HCL, and there's nothing preventing us from making changes.

If you prefer full control, you can edit the underlying configuration files but this isn't required. The rich two-way sync means you can mix and match your use of the Octopus UI and direct editing. We also built a Visual Studio Code extension so you can edit OCL files with syntax highlighting, code snippets and fast navigation.

First-class multi-tenancy

If you deploy a copy of your software for every customer that signs up, you might find yourself with dozens, hundreds, or even thousands of deployment pipelines.

Octopus is the only release management & deployment automation solution with first-class multi-tenancy support, letting you manage deployments to a single customer or a group without duplication. With Azure DevOps, you need to duplicate your release definitions for every customer.

Multi-tenancy is particularly suited to scenarios like:

• White-labeled software.
• SaaS software where every customer gets their own database.
• Managed services.
• Head-office deployments to restaurants, retail stores, and more.

Learn more about Multi-Tenant deployments