Learn how to navigate the CI/CD landscape with GitHub Actions, Argo, and Flux.
Software developers are always eager to leverage the latest technology to enhance their productivity and performance, and develop high-quality code at a speed never experienced before. Thanks to advancements in self-service internal platform engineering (IPE), it is now possible to bring continuous integration and continuous delivery (CI/CD) together. This practice automates the software development process, ensuring the seamless delivery of high-quality applications. Implemented correctly, it optimises the developer experience and accelerates the delivery of quality business applications. It also helps embrace artificial intelligence (AI), helping organisations compete while generating value.
Innovation and adoption of CI/CD tools have come full circle — from open source to commercial solutions and then to cloud hyperscale providers like AWS, GCP, and Azure building their own tools. Now, open source is at the forefront once again, especially with multi-cloud adoption presenting challenges in mastering proprietary platform tools. Let’s look at how all of this can be simplified using a plug-and-play model, starting with user-friendly IDEs integrating with platform tools like GitHub Actions, Argo, and Flux.
All developers strive for stability at the core with a feature-rich, easy-to-use IDE that gives flexibility and agility at the edge, in order to connect to various critical endpoints for source control of orchestration and CI/CD pipelines. These pipelines automate the build of infrastructure across registry, compute, data, networking, service, observability, and security.
This may sound great, but the challenge is organising it into a scalable internal developer platform. This platform should seamlessly integrate with development environments to establish developer productivity quickly and reliably. The reference architecture diagram in Figure 1 breaks down plug-and-play dev tools for CI/CD across common tools. These can be adapted to your organisation’s setup, providing the ability to work in the multi-cloud development and delivery landscape embraced by most organisations today.
Platform engineering is a hot trend, and is bringing together infrastructure development and software application development under the umbrella of DevOps. The standardisation of the internal developer platform has its benefits, as organisations constantly hire new developers who bring their own expertise on specific productivity tools, making them efficient and, more importantly, happy to contribute, collaborate, and stick with the firm longer. The plug-and-play model of the internal developer platform supports a developer’s abilities to independently run, manage, and develop applications while complying with an organisation’s best practices to ensure reliability and security.
How to get started
If you are just getting started on building an internal platform, begin by establishing reusable, composable, and configurable platform components. This will enhance productivity and help embrace standardisation. Consider treating the internal developer platform as a product where developers are the end users who identify and prioritise technical capabilities, tools, and processes that are most useful. Establish a frequent collaboration route with platform engineers and developers to share feedback on what’s working and what’s not, allowing for constructive dialogue enabling knowledge sharing while progressing towards the common goal of establishing a robust internal developer platform.
AI-augmented development
The use of AI technologies to develop software, and test and deliver applications, is popular with developers working to design tools to code while adopting AI coding assistants like GitHub CoPilot, Code Whisper, and ChatGPT. Augmented testing tools have made huge progress in addressing the gaps in testing data quality, validation of test results, and right sizing of the testing for the given scope of development work. This enables platform engineers and testing experts to collaborate on incorporating the right mix of tools within the internal developer platforms across CI/CD pipelines. Many standards are evolving, including in the US, where the Biden administration has released an executive order on the safe, secure, and trustworthy adoption of AI.
GitHub Actions: Powerful workflow automation tool
GitHub Actions is a widely adopted toolset for the automation of build, test, and deployment pipelines as part of platform engineering. The tool is open source, integrates seamlessly with the GitHub repository, works on-premise, and supports cloud-based development across Linux, Windows, and MacOS operating systems. Ease of use has been one of its biggest plus points as compared to the more familiar Jenkins, which has a steep learning curve for setting up on servers initially. The tool is designed for developers to fully utilise in their CI/CD pipeline, and familiarity with YAML helps.
GitHub Actions can be integrated quickly with existing toolsets, allowing for the build, test, deployment, and scaling of containers. It does not offer platform-specific or granular controls like Azure DevOps, supporting stages defined in the YAML file for creating deployment workflows. This is achieved in GitHub Actions by separating stages into different YAML workflow files, which act as standardisation modularising pipelines for better maintenance and debugging.
Argo: Orchestrating CI/CD workflows on containers
Argo, the container native workflow engine, is designed for orchestrating parallel jobs on Kubernetes. It’s a mature tool that is widely adopted in all types of pipelines. It enables developers to orchestrate complex CI/CD pipelines, providing a declarative workflow definition syntax that ensures clarity and conciseness while maintaining scalability and resource optimisation. Although there are other feature-rich tools like Airflow that are more versatile, Argo excels in its simplicity and quick integration for developing reusable workflows.
Flux: Declaring infrastructure as code
Flux is another open source tool based on GitOps for container apps and infrastructure, allowing the management of Kubernetes resources and workload dependencies. It advances infrastructure as code, enabling developers to declare infrastructure state at the source in Git repositories. This ensures that the actual infrastructure matches the desired state. One significant advantage of using Flux is its continuous monitoring of the infrastructure, automatically reconciling any discrepancies and bringing in reusability with stability in the complex web of cloud applications. Flux demonstrates its power through a seamless process of bootstrapping Flux components on the cluster with monitoring to ensure changes in the source are updated onto the cluster through the reconciliation process.
There are amazing open source tools that serve great purposes across the spectrum of platform engineering. Establishing a reference architecture by layer with a notion of critical functionalities will ensure a robust internal developer platform is established, allowing for flexibility and extensibility in leveraging leading CI/CD pipeline integration products like GitHub Actions, Argo, and Flux. Understanding the design pattern of the developer environment and embracing change with plug-and-play models will go a long way in enhancing developer productivity while improving the quality of work experiences, leading to improved retention of top talent!
