A Friendly and Informative Beginner’s Guide to Azure DevOps

Azure DevOps, CodePipeline, Jenkins, Bamboo, and many more. The rise of DevOps has led to the creation of an ever-changing software development lifecycle, which allows organizations to handle any issue or request.

Various tools support DevOps, such as build automation tools, dependency management tools, and repository tools. With the advent of cloud-based technologies, teams can now work seamlessly across various regions and organizations.

This article will describe a cloud-based version of the service, Azure DevOps.

What Is Azure DevOps?

As you may be aware, Azure is Microsoft’s cloud computing platform, which consists of a collection of services provided by Microsoft through its data centers. 

Azure DevOps (formerly VSTS) is one such service, an environment that includes several services aimed at making DevOps integration within your business as simple as possible.

You can use the tools of your choice to extend the Azure DevOps services. So, whether you’re a Java, Node.js, or.Net developer or use Jenkins, Ansible, or Puppet, Azure DevOps lets you set up your end-to-end DevOps chain.

You can find Azure DevOps in 2 variants:

1. The cloud-based Azure DevOps service
2. The Azure DevOps Server

The Azure DevOps Server, originally known as the Team Foundation Server (TFS), is a DevOps server solution targeted for on-premise installations. It consists of all the tools supplied in the cloud-based Azure DevOps service to fuel any DevOps pipeline.

This same server includes a free variant called Azure DevOps Server Express, intended for solo developers and small teams of up to 5-10 team members. It can be installed in any scenario.

Azure DevOps guarantees almost 99.9% availability for all commercial DevOps services, including premium user-centric extensions. It also provides 99.9% availability to execute load testing and build and deploy activities in premium Azure Test Plans (Load Testing Service) and Azure Pipelines.

Registration For Azure DevOps

Registering for Azure DevOps is an uncomplicated and straightforward process that requires only a Microsoft account. Visit this link and click on “Start for free.”

When enrolling, you will need to enter further information such as project name, organization name, version control type, etc.

Organisation refers to the Azure DevOps account name and can contain multiple projects.

Projects allow users to separate tasks, manage access, and partition the code, tests, and pipelines to keep them within the allocated projects. 

For example, a project can be public or private, leveraging Git or Team Foundation server as the version controlling system. Additionally, you can build projects using a work item technique like Scrum or Agile that we can use in Azure Boards to manage the project.

Once you complete the registration process, you will receive a dedicated organization URL with the following notation:

https://<organization name>.visualstudio.com

Users may manage their projects and use the DevOps services by visiting this URL.

Azure DevOps Services

Azure DevOps comprises five services, each of which may be bundled under a project to provide customers with sufficient isolation between tasks that use various technologies and cater to diverse demands.

Following are the Azure DevOps services that you can employ all together or just the ones you need to supplement your current operational processes.

#01 Azure Boards

It allows you to organize, monitor, and discuss your teams’ actions. The project’s management hub is the Azure DevOps Boards service.

Team members can utilize boards to plan, track, and collaborate on projects. For example, the Boards team can use Azure to track all project parts by creating work items, dashboards, Kanban boards, backlogs, and custom reports.

You may also design boards to meet your specific workflow needs and use the built-in reporting and monitoring features to obtain valuable insights. In addition, Azure Boards includes first-party connections with Microsoft Teams and Slack, allowing for effective ChatOps.

#02 Azure Pipelines

It offers a continuous integration and delivery (CI/CD) pipeline that works with nearly any activity, platform, or cloud. You can do ongoing development, testing, and implementation with the help of a Git service.

Pipelines are a CI/CD technology that automates software construction, testing, and deployment. 

It supports any programming language or platform, and users can use cloud-hosted agents to develop pipelines that support Windows, Linux, and macOS.

The extensions available in the marketplace make it simple to extend these pipelines. Additionally, they support complex workflows that can come in use to:

• Multi-phase construction.
• Integrations should be tested.
• Functions for custom reporting.

Additionally, Azure Pipelines offer native container functionality, allowing users to push containers to container registries from the pipeline quickly. 

The pipelines can deploy to various settings, including Kubernetes clusters, serverless functions, and other cloud providers like AWS or GCP.

#03 Azure Repos 

This feature allows you to create an infinite number of Git repositories in the cloud. In addition, people and teams can operate more efficiently with pull requests and advanced file management.

Users can manage their codebases via Azure Repos, which are code repositories. These are private and cloud-based repositories that support both Git and TFVC.

Azure Repos can help with projects of any size, from personal ventures to large-scale developments. They also have the following characteristics:

• It can support any Git client (IDE, Text Editor, CLI)
• Search for semantic codes
• To interact with other team members, use collaboration tools.
• Direct interaction with continuous integration and delivery (CI/CD) tools
• Code quality requirements will be enforced using branch policies.

Platform-agnostic systems like Azure allow repo users to interact with Azure Repos in any operating system using any IDE or tool they are accustomed to.

#04 Azure Test Plans

The Azure Test Plans Manual and exploratory test scripts may be tested and published rapidly and comprehensively.

Test Plans is an Azure DevOps tool that lets customers integrate a cloud-based testing platform to manage all of their testing needs, including:

• Manual testing is being planned.
• Acceptance testing for users (UAT).
• Observational research.
• Getting input from stakeholders.

Users can use Azure Test Plans to develop test plans and test cases within a pipeline. 

You can integrate this with Azure Boards to build a test run from the Kanban boards and collaborative plan and author tests.

Test Plans assist in creating user acceptance testing (UAT) plans and the assignment of people from DevOps systems. 

It also supports the Test and Feedback browser extension, allowing interested parties to do exploratory testing without third-party technologies. 

Additionally, Test Plans allow users to test on any platform while maintaining end-to-end traceability and robust data gathering capabilities to diagnose and rectify any discovered issues.

It is the only Azure DevOps service with a free tier due to its extensive toolkit only being available to commercial users.

#05 Azure Artifacts 

Azure Artifacts Azure Artifacts make exchanging code across teams and organizations. Packages can be shared and added to pipelines.

This is the Azure DevOps artifact library service, which can build, store, and share packages (development artifacts). 

Using Azure Artefacts, users may integrate fully featured package management capability into CI/CD pipelines.

Furthermore, Azure Artifacts allows users to manage various package types, such as npm, Maven, and others, and keep them organized in a single library that is solely relevant to the project at hand.

#06 Extensions Marketplace

Finally, Azure DevOps has an Extensions Marketplace with over a thousand extensions, including Slack, Jenkins, Docker, and Kubernetes, that you can add to your Azure DevOps setup.

Azure DevOps also has a premier cloud-based DevOps service that provides a powerful and feature-rich toolkit for building and managing a comprehensive DevOps process. It gives users the ability to:

• Adapt to every DevOps requirement, independent of programming language, technology, or platform.
• From containers to third-party clouds, you can deploy anywhere.

All of this is now possible by Azure DevOps, which provides unrivaled scalability and availability without the bother of maintaining separate software for different DevOps operations.

Benefits of Azure DevOps

Azure DevOps allows the users to write, deploy, and monitor code without accessing many interfaces. You may manage all of this from one view and deliver ease to the clients.

#01 Automation Testing using Azure DevOps

The employment of automated tests, such as security and compliance tests, identify problems during the testing phase. As a result, we can quickly provision resources and configure the whole production setup in a quick time.

#02 Continuous Integration & Continuous Delivery (CI-CD)

When we make the code dedicatedly for a particular task, it automatically builds and checks for errors, enabling defects discovery early. As a result, business businesses may achieve fast and identical deployment to the production environment at any given time.

#03 Any Platform, Any Language

It supports several platforms and runs on multiple frameworks. For example, the developers utilising Java, Node, PHP, .NET, and Python can conveniently operate on it.

#04 App Insights on Azure DevOps

Azure Application Insights delivers insights through application performance management and quick analytics. In addition, you can monitor infrastructure health with Azure Log Analytics and Azure Monitor.

How To Use Azure DevOps (Automated Testing with Azure DevOps)

Test cases in your test plans can be automated and run directly from Azure Test Plans. Below are a few of the advantages of automated tests:

• A simple method for testers unfamiliar with executing tests in Build or Release procedures.
• The ability to run specific tests on-demand instead of scheduled testing in Build or Release workflows runs all tests that fit the filter criteria.
• You have the option of rerunning a few tests that failed due to test infrastructure concerns, or you have a new build with patches for failed tests.

#01 Prerequisites

Here are a few of the prerequisites that you must have before moving to do automation testing with Azure DevOps:

• You’ll need to join a project. Do a project if you don’t already have one.
• A project must be added to you and a project or team.
• You must have Basic access or higher to see or conduct manual or automated tests.
• A test plan is a collection of automated tests that you’ve linked to automated test methods in Visual Studio 2017, Visual Studio 2015, or earlier.
• A build process that generates test binaries-containing builds.
• A program to evaluate. You can use the app for on-demand testing and deploying it as part of the build and release procedure.
• Create and manage releases, change a release environment, and control deployment with permissions. See Set approvals for release pipelines and Release permissions for further details.

#02 Setting Up The Azure Testing Environment

Here are all the steps that you need to follow to accurately set up the testing environment for doing the automation testing using Azure DevOps:

1. Choose your test plan on the Test Plans page, open the shortcut menu and select Test plan settings.
2. Select the build pipeline that generates builds containing the test binaries in the Test plan settings window. You can then test a specific build number or have the system automatically use the most recent build when tests are run.
3. To execute test plans in Azure Test Plans, you’ll require a release pipeline built using the Run automated tests from the Test Manager template. If you already have a release pipeline built with this template, pick it, and then choose the stage in the pipeline where the tests will be run. Otherwise, select the Establish a new link to create a new release pipeline with a single stage and the Visual Studio Test job already included in the dialogue.
4. As needed, give the release pipeline and stage meaningful names.
5. You’ll need to install the Visual Studio Test Platform on the agent computer. You can skip this step if you have Visual Studio on the agent PC. If this isn’t the case, you’ll need to add the Visual Studio Test Platform Installer task to your pipeline design.
6. Add the Visual Studio Test job and configure it in the release pipeline.
   • Here are the steps that you may follow:
     1. The task settings panel’s version number is displayed in the drop-down list at the top left.
     2. Ensure that you choose the Test run in the Select tests using section.
     3. Select Installed by Tools Installer for the Test platform version selection.
     4. Ensure the agent is set to operate as an interactive process with auto-login enabled if you run UI tests using physical browsers. Before queueing a build or release, you must first set up an interactive agent.
     5. The interactive process setup is unnecessary when performing UI tests on a headless browser.
     6. Choose how the test platform is provided, the version of Visual Studio, and the test platform’s placement on the test machines.
     7. Select the correct settings to file from the build artifacts if your tests require input parameters like app URLs or database connection strings. If you don’t include the settings file in the artifacts, you can use the Publish build artifacts tasks in your build process to publish it at a drop place. The application URL is accessible in the run settings file in the example below. You can override it by implementing the override test run parameters setting to set it to a staging URL.
7. Select the Agent job item and check that the deployment queue is on the one that contains the machines where the tests will be running. 
8. Verify that the build pipeline containing the test binaries is link with the release pipeline as an artifact source on the Pipeline page of the release pipeline.
9. You must save the release pipeline.
10. Return to the browser page displaying your test plan settings if you create a new in the Test plan settings dialogue in step 2 of this example. Select the release pipeline and stage you just saved in the Test plan settings window.

#03 Running The Automated Tests

To run the automated tests using azure DevOps, follow the steps below:

1. Open the test plan on the Test Plans website and choose a test suite that comprises the automated tests.
2. Choose Perform test from the Run menu after selecting the test(s) you want to run.
3. To begin the testing procedure, select OK. The system validates the stage to ensure the Visual Studio Test task is present and has valid parameters, checks the user’s permission to make a release for the chosen release pipeline, creates a test run, and finally triggers the creation of a release to the selected stage.
4. Select View test runs to see the progress and examine the failed tests. The error message, console logs, stack trace, and attachments all comes in the test results, which you can use to debug failed tests.
5. The test results appears on the Runs tab of the Azure Test Plans once the test execution gets complete. The Run summary page provides a high-level overview of the race. In addition, there is a link to the release used to conduct the tests, making it easy to locate the release that performed the tests if you need to go back and analyse the findings later. You can also use this link to open the release and examine the release logs.
6. The results for each test in the test run appears on the Test Results page. For failing tests; select a test to see debugging information such as the error message, stack trace, console logs, and attachments.
7. If your tests changes after test execution; go to the Test Plans tab and choose the test plan to examine the status of your tests. To get the most recent test results, select a test.

Frequently Asked Question

Q1: Why do you do tests in release stages?

A1: Azure Pipelines provides an effective orchestration methodology to receive test binaries as artifacts and run tests. This workflow uses the same concepts as the scheduled testing workflow, making it simple to adapt for users already performing tests in the expected workflow. For example, by cloning an existing scheduled testing release pipeline.

Another significant advantage is the task catalog’s vast number of tasks, which completes various operations before and after running tests. Examples are preparing and cleaning test data, producing and cleaning configuration files, and other tasks.

Q2: How does the Visual Studio Test task version 2’s “Test run” option work?

A2: The test run object is used by the Test management subsystem to pass the list of tests that have been selected for execution.

The test task looks for the test run identification, pulls test execution information such as container and test method names, runs the tests, updates the test run results, and assigns test points to the test results in the test run.

The Visual Studio task offers a trace from past releases and test run identifiers to the tests submitted for on-demand test execution from an auditing standpoint.

Q3: Is it better to operate the agent in interactive mode or as a service?

A3: To allow the agent to start a web browser while running UI tests such as coded UI or Selenium tests, the agent on the test machines must be running in interactive mode with auto-login enabled, not as a service.

The agent can be operated as a service or interactive mode if you use a headless browser like PhantomJS. See Agent pools, Deploy an agent on Windows, and Build and release agents.

Q4: I already have a testing release pipeline in place. Is it possible to utilise the same pipeline for on-demand testing, or do I need to design a new pipeline as described above?

A4: For on-demand automated testing, we recommend using a different release pipeline, and stage from Azure Test Plans because:

• It’s possible that you won’t want to deploy the app every time you want to run a few on-demand tests. Instead, the standard setup for scheduled testing stages is installing the product and running tests.
• For each on-demand run, new releases are triggered. If a large number of testers perform a few on-demand test runs each day; your planned testing release pipeline may become overburdened. Also, challenging to locate releases that were triggered for the pipeline that includes scheduled testing and deployment to production.
• You might want to use a Test run identification as an input to the Visual Studio Test job so you can figure out what caused the release.

Q5: My team has several testers. Can they use the same release process to execute tests from various test suites or test plans in parallel?

A5: If it meets the following conditions, they can use the same release pipeline to execute many tests in parallel:

• The stage’s agent pool contains enough agents to handle several requests concurrently. You can still launch the runs if there aren’t enough agents available. However, releases will start queuing for processing until more agents become available.
• You have enough jobs to allow for simultaneous work. Check parallel jobs in Azure Pipelines or TFS for further details.
• Depending on the execution order, you can not conduct the same tests in tandem. This may overwrite the present results.

You need to set the Pipelines stage trigger option for behavior when several releases are waiting to be deployed as follows to enable multiple distinct test runs to run in parallel:

• Set the option to ‘Allow multiple releases to be push’ simultaneously; if your application supports tests running in parallel from various sources.
• Set the option to Allow only one active deployment at a time; if your application does not support tests running in parallel from separate sources.

Conclusion

All in all, Azure DevOps is one of the newer offerings of Azure that allows development teams to collaborate and manage their assignments and releases.

And by now, we are sure that you must have learned a ton about it in a detailed way; you can now explain it to anyone and start using the Azure DevOps services yourself!

Azure DevOps and the cloud ecosystem is a dynamic and evolving field. Ensure to read about the newest developments in Azure DevOps that will be coming soon.

For more articles, visit our blog.