The Future Belongs to These Open Source DevOps Tools

0
933
DevOps Tools

Organisations are adopting DevOps today for their own good. This article briefly discusses a few open source DevOps tools that will be used a lot in the near future.

DevOps is a culture, and implementing its practices helps manage the application life cycle in an effective manner using automation. It has three pillars — people, processes, and tools. Tools play a critical role in DevOps or culture transformation initiatives. People and processes help in maintaining uniformity and sustainability across business units or projects in an organisation that aims for faster time to market with high quality for end users.

Category-wise list of DevOps tools

Category Tools
Source code management GitLab, Git, GitHub, Atlassian BitBucket,
Code analysis tools SonarQube
CI/CD Jenkins, Microsoft Azure DevOps, TeamCity, GitLab, GitHub, Atlassian Bamboo
Test automation Selenium, Appium, JMeter
Configuration management Ansible, Puppet, Chef
Containerisation and orchestration Docker and Kubernetes
Cloud service providers Amazon Web Services, Microsoft Azure, Google Cloud Platform
Monitoring Prometheus, Grafana
Logging Elastic Search, FluentD, FluentBit, Kibana
Communication and collaboration Microsoft Teams, Slack, Skype and Zoom

I have used a few open source tools and I believe that these are very important for implementing DevOps practices in 2022 and beyond. Let’s take a brief look at them.

GitLab CI/CD pipeline
Figure 1: GitLab CI/CD pipeline

GitLab
This DevOps platform provides the ability to plan, develop, secure, and operate software in a single application. Its essential features include cache and artifact management.

In my experience, GitLab has one of the most futuristic features for implementing DevOps practices.

Initial release 2014
Stable release 14.6.0
Written in Ruby, Go and JavaScript
Licence Community Edition – MIT License
Website about.gitlab.com
GitHub repository https://gitlab.com/gitlab-org/gitlab-foss/
Features
  • Highly scalable
  • Supports collaboration
  • Supports on-premise and cloud installation
How is it useful in implementing DevOps practices? DevOps platform

  • Version control
  • CI/CD
  • Package management
  • Resource management
  • Security
  • Distributed architecture
  • Cloud support
Can we integrate it with Pipeline as Code? Yes. The command is:

Gitlab-ci.yaml
Is a commercial flavour available? Yes
Pricing https://about.gitlab.com/pricing/Free, Premium, Ultimate
Community 2500+ code contributors
Use cases
  • Value stream management
  • Agile development
  • Source code management
  • Continuous integration (CI/CD)
  • Out-of-the-box pipelines
  • Security (DevSecOps)
  • GitOps
  • The DevOps platform
Important Git commands
Description Command
Set a user name
git config --global user.the name “Mitesh Soni”
Set email address
git config --global user.email “xxxxxxxxx@xxxxx.com”
Initialise a Git repository
git init
Clone a repository
git clone https://github.com/xxxxxxxx/Angular9786-DevOps
Verify the status of files in the repository
git status
Add files in the current directory for Commit operation
git add .
Commit files
git commit -m “First commit in the repository”
List your branches
git branch
Switch branch
git checkout
Create a new branch
git branch new-branchname
Merge current branch with the target branch
git merge target-branch
Show all commits
git log
Save modified and staged changes
git stash
Fetch and merge any commits from the remote branch
git pull


Jenkins
Jenkins is an open source tool that provides integration with existing tools used in application life cycle management to automate all activities, starting from build to test and deploy. Many beginners start their journey of automation using Jenkins. It is easy to learn and has a huge community for support.

Jenkins offers a simple way to create a pipeline that includes continuous integration and continuous delivery. Plugin based architecture helps to use existing tools in the pipeline; hence, too many changes in culture can be avoided in one go.

Jenkins Blue Ocean
Figure 2: Jenkins Blue Ocean
Initial release 2011
Stable release 2.324
Written in Java
Licence MIT
Website www.jenkins.io
GitHub repository https://github.com/jenkinsci/jenkins
Features
  • Jenkins installation is supported for Docker, FreeBSD, Gentoo, MacOS X, OpenBSD, OpenSUSE, Red Hat/Fedora/CentOS, Ubuntu/Debian, Windows, and Generic Java package (.war)
  • Multiple views are supported along with folders to manage jobs in an easier manner
  • New UI for Jenkins dashboard that is easy to use
  • New UI to create Pipeline as Code in an easier way
  • Pipeline as Code/multibranch pipeline to manage pipelines easily in repositories
  • Distributed architecture to manage Jenkins effectively
  • Role based access to users based on build jobs/folders/ pipelines/configurations/actions
  • Authentication – Integration with active directory, LDAP and with internal database
How is it useful in implementing DevOps practices? Automation server helps to automate activities involved in application life cycle management
Is a commercial flavour available? CloudBees Jenkins
Awards
  • DevOps Dozen 2017: Best DevOps Open Source Project
  • DevOps Dozen 2016: Best DevOps Open Source Project
  • Bossie Awards 2014: The best open source application development tool
  • Geek Choice Award 2014
  • SDTimes 100, 2014: DevOps & SCM
  • SDTimes 100, 2013: Development Tools & CI
  • SDTimes 100, 2012: Development Tools & CI
  • InfoWorld Bossies Award, 2011
  • O’Reilly Open Source Award, 2011
  • SDTimes 100, 2011: ALM & SCM
  • SDTimes 100, 2010: ALM Platforms
  • GlassFish Community Innovation Award 2008
  • Duke’s Choice Award
Third party integrations
  • More than 1500 plugins to integrate Jenkins with many open source and commercial tools (https://plugins.jenkins.io/)
  • Platforms (iOS development, .NET, Android development, Ruby development)
  • User interface (User Interface, List view column plugins)
  • Administration (Agent controllers, Page decorators, Users and security, Cluster management, CLI extensions)
  • Source code management (SCM connections, SCM related)
  • Build management (Build triggers, Build wrappers, Build notifiers, Deployment plugins, Build parameters, Clean-up actions, Build tools, Build reports, Artifact uploaders)


Docker

Docker is a lightweight DevOps tool that helps to speed up and ease different workflows in your software development life cycle with a combined methodology. A Docker container image is a standalone package that contains all the dependencies required for running an application. It is an open source lightweight tool to create, deploy, and manage containers on a different host operating system using resource isolation features, such as cgroups and Linux kernels.

Docker desktop images
Figure 3: Docker desktop images

A container package can have application code, libraries, and configurations. The container engine is installed on the host OS.

Initial release 2013
Stable release 20.10.11
Written in Go
Licence Apache License 2.0
Docker Community Edition
Website https://hub.docker.com/https://www.docker.com/
GitHub repository https://github.com/docker
Features
  • Easy management of applications
  • Docker hub – public registry
  • Uniform packaging
  • Integrates well with Kubernetes
How is it useful in implementing DevOps practices? OS-level virtualisation, Containers
Can we integrate it with Pipeline as Code? Yes
Is a commercial flavour available? Docker Enterprise Edition
Docker commands
Description Command
Log in to the registry
docker login
Build an image from the Docker file
docker build -t sampleapp :1.0
List all the images
docker images
Delete an image
docker rmi imagename
Create a container
docker run --name containerName dockerImage
Pull an image from the registry
docker pull imageName
Push an image
docker push imagename: tag
Clean an unused image
docker image prune


Kubernetes 

Kubernetes or k8s is an open source, extensible and portable container orchestration platform developed by Google. It offers declarative configuration and automation for deployment. It has become one of the most popular tools for DevOps practices because of its capabilities and features for deployment, management, and scaling applications in Kubernetes clusters. It helps to manage resources effectively.

Google Kubernetes engine
Figure 4: Google Kubernetes engine

We can create a Kubernetes cluster in an on-premise environment as well as on private and public clouds. Public cloud provides managed services for Kubernetes. Google Cloud Platform has Google Kubernetes Engine, AWS has Amazon Elastic Kubernetes Services (EKS) and Azure has Azure Kubernetes Service (AKS) for managing Kubernetes clusters in a cloud environment.

Initial release 2014
Stable release 1.22.2
Written in Go
Licence Apache License 2.0
Website https://kubernetes.io/
GitHub repository https://github.com/kubernetes/Kubernetes
Features
  • Distributed architecture
  • Easy integration with Docker
  • Supports hybrid, on-premise, or cloud environments
How is it useful in implementing DevOps practices? Container orchestration and management
Can we integrate it with Pipeline as Code? Yes
Is a commercial flavour available? Cloud flavours are available
Important Kubernetes commands
Description Command
Get Kubectl version
kubectl version
Get details on Kubernetes cluster
kubectl cluster-info
Get details about a node
kubectl describe nodeName
Get details about a pod
kubectl describe podName
List of all pods
kubectl get pods
List of all services
kubectl get services
List of all deployments
kubectl get deployments

Prometheus 
This is an open source tool for monitoring and alerting. The Prometheus ecosystem has components such as a Prometheus server, client libraries, a push gateway, exporters for services, an alert manager, and support tools.

Prometheus architecture
Figure 5: Prometheus architecture
(Source: https://prometheus.io/docs/introduction/overview/)

Prometheus has features such as a multi-dimensional data model, a flexible query language, and multiple modes of graphing and dashboarding support.

Initial release 2012
Stable release
v2.32.0
Written in Go
Licence Apache License 2.0
Website https://prometheus.io/
GitHub repository https://github.com/prometheus/prometheus
Features
  • A multi-dimensional data model
  • A flexible query language (PromQL)
  • Support for service discovery or static configuration
How is it useful in implementing DevOps practices? Monitoring
Is a commercial flavour available? Prometheus is 100 per cent open source and community-driven
Figure 6: Prometheus dashboard

Terraform
This open source ‘Infrastructure as Code’ tool, developed by HashiCorp, helps in creating immutable infrastructure to avoid the problem of configuration drift. Terraform is a platform-agnostic declarative coding tool that helps developers to use a high-level configuration language to manage infrastructure. It supports different cloud providers like AWS, Azure, Google Cloud Platform, and so on.

SonarQube dashboard
Figure 7: SonarQube dashboard
Initial release 2014
Stable release 1.0.10
Written in Go
Licence Mozilla Public License v2.0
Website https://www.terraform.io/
GitHub repository https://github.com/hashicorp/terraform
Features
  • Manages Infrastructure as Code using declarative config files
  • Dependency graphs
  • State management
How is it useful in implementing DevOps practices? Infrastructure as Code
Can we integrate it with Pipeline as Code? Yes. It can be part of Pipeline as Code, and different strategies can be used to create resources and deploy applications in different cloud platforms.
Important Terraform commands
Description Command
Verify Terraform version terraform –version
Install Terraform modules terraform init
terraform init -input=false
Check configuration and create a plan terraform plan
Execute the plan terraform apply
terraform apply -auto-approve
Delete all resources erraform destroy
terraform plan –destroy
Validate/check the syntax of the Terraform files terraform validate
Tutorials https://learn.hashicorp.com/terraform

SonarQube
SonarQube works like a sanitiser and helps clear bugs, vulnerabilities and code smell in the code. It supports more than 20 programming languages for static code analysis (SCA).

It is much easier to integrate static code analysis using SonarQube in Pipeline as Code. This tool helps to transform the culture of an organisation from manual to automated code review using quality gates and profiles.

Initial release 2006-07
Stable release 8.9
Written in Java
Licence Lesser GNU General Public License
Website https://www.sonarqube.org/
GitHub repository https://github.com/SonarSource/sonarqube
Features
  • Support for the programming languages Java (including Android), C#, C, C++, JavaScript, TypeScript, Python, Go, Swift, COBOL, Apex, PHP, Kotlin, Ruby, Scala, HTML, CSS, ABAP, Flex, Objective-C, PL/I, PL/SQL, RPG, T-SQL, VB.NET, VB6, and XML
  • Quality gates
  • Quality profiles
How is it useful in implementing DevOps practices? Static code analysis
Can we integrate it with Pipeline as Code? Yes
Is a commercial flavour available? Yes

Grafana
This open source tool is fully managed by Grafana cloud, but you can run the enterprise version on your infrastructure with self-managed options.

Grafana is one of the most popular tools when we are using Kubernetes. It is easy to install, configure and manage Grafana and monitoring metrics keeping Prometheus as a data source.

Initial release 2014
Stable release 8.3.3
Written in Go, TypeScript
Licence GNU Affero General Public License, version 3.0
Website https://grafana.com/
GitHub repository https://github.com/grafana/grafana
Features
  • Panels – heatmaps to histograms
  • Plugins – https://grafana.com/grafana/plugins/
  • Alerts
  • Annotations
How is it useful in implementing DevOps practices? Visualisation of monitoring metrics
Is a commercial flavour available? https://grafana.com/pricing/

Ansible
Ansible is an open source IT automation/configuration management tool that provides support for cross-platforms. It is easy to use, and provides security and reliability for safe adoption in an organisation.

Grafana dashboard
Figure 8: Grafana dashboard

It is also popular for deployments and rolling updates in release management activities. Ansible is different from other popular configuration management tools as it is agentless. It can be integrated with Kerberos, LDAP, and other authentication management systems.

Initial release 2012
Stable release 5.0.1
Written in Python, PowerShell, Shell, Ruby
Licence GNU General Public License
Website https://www.ansible.com/
GitHub repository https://github.com/ansible/ansible
Features Distributed architecture
Easy integration with Docker
Supports hybrids, on-premise, or cloud environments
How is it useful in implementing DevOps practices? Configuration management tool
Infrastructure as Code
Can we integrate it with Pipeline as Code? Yes
Is a commercial flavour available? Yes
https://www.ansible.com/products/pricing

 

Ansible
Figure 9: Ansible


Trivy

Trivy is an open source yet simple and comprehensive vulnerability/misconfiguration scanner for containers and other artifacts.

Trivy Docker image analysis
Figure 10: Trivy Docker image analysis

It detects vulnerabilities of the following:

  • OS packages: Alpine, RHEL, CentOS, etc
  • Language-specific packages: Bundler, Composer, npm, yarn, etc.
  • Infrastructure as Code (IaC): Terraform and Kubernetes
Stable release V0.22.0
Written in Go
Licence Apache-2.0 License
Website https://aquasecurity.github.io/trivy/v0.22.0/
GitHub repository https://github.com/aquasecurity/trivy
Features
  • Vulnerability detection for Docker Images, Red Hat Enterprise Linux, CentOS, Oracle Linux, Debian, Ubuntu, Amazon Linux and other OS packages
  • Simple, fast and easy to install
How is it useful in implementing DevOps practices?
  • To find vulnerabilities of Docker images
  • DevSecOps
Can we integrate it with Pipeline as Code? Yes. The pipeline stage can be used to analyse Docker images and publish the report.
Example trivy image python:3.4-alpine
Docker image docker pull aquasec/trivy:0.22.0

LEAVE A REPLY

Please enter your comment!
Please enter your name here