A Simple Guide for DevOps Beginners

DevOps is a set of practices that combines software development (Dev) and IT operations (Ops). It aims to shorten the development lifecycle and provide continuous delivery with high software quality. In this blog, we’ll cover some of the fundamental concepts you’ll need to get started with DevOps, such as agents, deployment groups, release pipelines (classic and YAML), and much more, in simple terms with detailed examples.

---

What is DevOps?

DevOps is a culture and a set of practices aimed at automating and integrating the processes between software development and IT teams. The ultimate goal of DevOps is to improve collaboration, speed, and quality in the software delivery process.

DevOps involves continuous integration (CI), continuous delivery (CD), and monitoring, ensuring that the software you develop can be quickly and reliably delivered to production.

Key Concepts in DevOps

Let’s dive deeper into some important concepts that you will work with in DevOps:

1. Version Control Integration (Git)

A version control system (VCS) like Git is fundamental to DevOps. It tracks changes in the codebase and allows collaboration between developers. In a DevOps pipeline, version control is tightly integrated to trigger builds, deployments, and even tests whenever there’s a code change.

Example:

In Azure DevOps, you can link your repository (like GitHub or Azure Repos) to the pipeline, so any time there’s a new commit or pull request, the pipeline runs automatically.

trigger:
   branches:
       include:
           - main 

This will trigger pipeline on changes to the ‘main’ branch.

2. What is an Agent Machine in DevOps?

In DevOps, an agent machine (or simply agent) refers to the environment or server where your build and release tasks are executed. The agent interacts with the DevOps platform, such as Azure DevOps, Jenkins, or GitLab, and performs tasks such as compiling code, running tests, and deploying artifacts.

Types of Agent Machines:

• Self-Hosted Agent: You install and configure this agent on your own infrastructure, such as a physical server, a virtual machine (VM), or a container. This provides more control over the environment and allows you to install custom software needed for your build.
• Microsoft-Hosted Agent (Cloud Agent): This is a pre-configured agent provided by cloud platforms like Azure DevOps. It is ready to use and offers pre-installed tools, such as .NET SDK, Node.js, Java, etc. These agents are typically used when you don’t want to manage your own infrastructure.

Example:

If you’re working in Azure DevOps, and you want to build a .NET Core application, you can use a Microsoft-hosted agent, like so:

pool:
   vmImage: 'windows-latest'

For more control, you can use a self-hosted agent:

pool:
   name: 'YourCustomAgentPoolName'  

The key difference is that self-hosted agents can be tailored to meet specific requirements (like installing custom software), while Microsoft-hosted agents are managed by the cloud provider.

Tasks Performed by an Agent:

An agent executes various tasks during a DevOps pipeline:

• Building Code: The agent compiles the source code into a deployable artifact.
• Running Tests: It executes automated tests (unit tests, integration tests).
• Deployment: It deploys the compiled code to different environments, like development, staging, or production.
• Managing Dependencies: The agent installs and configures dependencies required by the project.
• Handling Artifacts: The agent handles artifacts, which are the output (e.g., compiled binaries) of your build process.

3. What are Tasks in DevOps?

A task in a DevOps pipeline represents an individual unit of work that the agent performs. Tasks can include activities like compiling code, running tests, copying files, or deploying the application. These tasks are often defined within the pipeline configuration.

Example:

A typical pipeline might involve tasks such as:

1. Restore Dependencies: This task restores any external libraries or packages that your code needs.
2. Build: This task compiles the code and generates output files (artifacts).
3. Test: Run unit and integration tests to ensure code quality.
4. Publish Artifacts: The agent publishes the build artifacts, which will be used in later stages.
5. Deploy: Deploy the built application to an environment (e.g., staging or production).

Classic Example of Tasks in a YAML Pipeline:

Here’s a simple YAML pipeline where different tasks are defined for a .NET Core application:


trigger:
  branches:
    include:
      - main

pool:
  vmImage: 'ubuntu-latest'

steps:
- task: UseDotNet@2
  inputs:
    packageType: 'sdk'
    version: '5.x'
    installationPath: $(Agent.ToolsDirectory)/dotnet

- script: dotnet restore
  displayName: 'Restore Dependencies'

- script: dotnet build
  displayName: 'Build the Project'

- script: dotnet test
  displayName: 'Run Tests'

- task: PublishBuildArtifacts@1
  inputs:
    artifactName: 'drop'
    targetPath: '$(Build.ArtifactStagingDirectory)'
    publishLocation: 'Container'

In this pipeline:

• UseDotNet is a task to install the .NET SDK.
• dotnet restore, dotnet build, and dotnet test are shell commands that restore dependencies, build the project, and run tests, respectively.
• PublishBuildArtifacts publishes the build output (artifacts).

Each task performs a single step in the pipeline, and the agent executes these tasks in sequence.

4. Variables in DevOps

Variables are values that can be used throughout a pipeline. They help to store and reuse information such as build numbers, project configurations, and environment-specific data.

Secrets management is critical in a DevOps pipeline. Sensitive data such as API keys, passwords, and connection strings need to be securely stored and accessed in the pipeline without exposing them to the codebase.

In Azure DevOps, variable groups can be used to store sensitive information. You can mark these as secrets, and they will be encrypted

Types of Variables:

• Predefined Variables: These are built-in variables provided by the DevOps platform. They contain information like the build number, repository name, or the current branch.
  • Example: Build.BuildId (the unique identifier of the build).
• User-Defined Variables: You can create your own custom variables to store values that you need for the pipeline. These variables can be defined in the pipeline configuration or at runtime.
  • Example: MyVariable: 'value'

Using Variables in YAML Pipeline:

variables:
  buildConfiguration: 'Release'
  myCustomVariable: 'MyValue'

steps:
- script: echo $(myCustomVariable)
  displayName: 'Display Custom Variable'
  
- task: DotNetCoreCLI@2
  inputs:
    command: 'restore'
    projects: '**/*.csproj'
    arguments: '--configuration $(buildConfiguration)'

In this example:

• We define a custom variable myCustomVariable and use it in a script step.
• We also use buildConfiguration to pass a build configuration argument (like Release or Debug) to a .NET command.

Benefits of Using Variables:

• Flexibility: Variables allow you to reuse values, making the pipeline more flexible and easier to maintain.
• Consistency: You can ensure that the same values are used consistently throughout the pipeline.

5. What are Parameters in DevOps?

Parameters are similar to variables, but they are used to pass values to templates or pipeline steps. Parameters are especially useful when you want to create reusable pipeline templates.

Example of Using Parameters in YAML:


parameters:
  - name: environment
    type: string
    default: 'staging'
  
jobs:
- job: DeployJob
  steps:
    - script: echo Deploying to $(environment)
      displayName: 'Deploy to Environment'

In this example:

• The environment parameter is defined and passed into the pipeline. By default, the parameter value is staging, but you can override it when you trigger the pipeline (e.g., use production instead of staging).

Why Use Parameters?

• Reuse: Parameters help in making pipelines more reusable by allowing you to change values without modifying the entire pipeline.
• Modularity: They allow different parts of the pipeline to adapt based on input values.

6. Libraries in DevOps

Libraries in DevOps are collections of shared resources like scripts, templates, variables, or tasks that can be reused across multiple pipelines. They help to standardize and centralize commonly used configurations.

In Azure DevOps, for example, you can create a variable group in a library and use it across multiple pipelines. This is especially helpful when managing environment-specific variables or secrets.

Example: Variable Group in Azure DevOps Library

1. Create a Variable Group in the Library section of Azure DevOps.
2. Define variables like connection strings or API keys in the Variable Group.
3. Reference the variable group in your pipeline YAML:

variables:
  - group: MyVariableGroup

This way, you can centralize your variables and make your pipelines cleaner and more maintainable.

Putting It All Together: A Complete DevOps Pipeline

Here’s an example of a complete YAML pipeline that demonstrates the use of agent machines, tasks, variables, parameters, and libraries:


trigger:
  branches:
    include:
      - main

variables:
   - group: MyVariables
   buildConfiguration: 'Release'

parameters:
  - name: environment
    type: string
    default: 'staging'

pool:
  vmImage: 'ubuntu-latest'

jobs:
- job: BuildAndTest
  steps:
    - script: echo "Restoring dependencies..."
      displayName: 'Restore Dependencies'
    
    - task: UseDotNet@2
      inputs:
        packageType: 'sdk'
        version: '5.x'
        installationPath: $(Agent.ToolsDirectory)/dotnet

    - script: dotnet restore
      displayName: 'Restore NuGet Packages'

    - script: dotnet build --configuration $(buildConfiguration)
      displayName: 'Build the Application'
    
    - script: dotnet test
      displayName: 'Run Tests'

- job: Deploy
  dependsOn: BuildAndTest
  steps:
    - script: echo "Deploying to $(parameters.environment) environment..."
      displayName: 'Deploy to Environment'
    
    - task: AzureWebApp@1
      inputs:
        azureSubscription: 'MyAzureSubscription'
        appName: 'MyAppService'
        package: $(Build.ArtifactStagingDirectory)/**/*.zip

This pipeline:

• Uses self-hosted or Microsoft-hosted agents.
• Defines variables for configuration.
• Accepts parameters for dynamic deployment to different environments.
• Implements tasks to restore dependencies, build the application, run tests, and deploy it.

---

Conclusion

In DevOps Version Control, Agent Machines, Tasks, Variables, Parameters, and Libraries are critical components that work together to automate and streamline software development and deployment. By understanding and using these elements effectively, you can create efficient, flexible, and reusable pipelines.