Clouds are pools of virtual resources, storage, applications and services that are managed by software so that on-demand resources can be accessed. Clouds provide the rules and cloud management carries them out. Today’s IT infrastructure demands more control over dynamic and scalable computing environments, and for that strong cloud management is required. Apache CloudStack can help here.
Cloud management is defined as software and technologies designed for operating and monitoring applications, data and services residing in the cloud. It is a combination of software, automation, policies, government and people that determine exactly how cloud computing services will be made available.
Cloud management requires tons of tasks, starting from performance monitoring, security administration, auditing and management to overseeing and maintaining disaster recovery, as well as backup and contingency plans. As cloud computing grows over time and a wide range of public, private, hybrid and even community clouds are adopted by enterprises, strong cloud management tools need to be deployed for managing cloud computing orientations.
Strong and effective cloud management lays a strong foundation for companies to attain the following organisational and technological goals.
- Self-service: This is concerned with the flexibility of operations in handling day-to-day tasks with regard to cloud accessibility, addition of new clouds, monitoring and auditing and, of course, resource management.
- Automation: With cloud management, almost all enterprise operations can run autonomously.
- Analysis: Enterprises can track cloud workflows, and improve the experience of clients and end users.
How cloud management works
The primary objective of cloud management software is to gather information at regular time intervals. This is necessary for making informed decisions with respect to what to keep in a private or public cloud, and to optimise hybrid and community clouds, respectively.
Public cloud providers typically offer their own software tools for monitoring, securing and managing the cost of their cloud offerings. However, these tools rarely offer insights into performance, and instead stick to basic reporting. Third party tools designed to help manage public cloud services become necessary if organisations are using multiple public clouds that all have their own proprietary cloud management tools.
IT administrators can use private cloud management software tools to allocate resources more efficiently. For instance, an IT manager can use a cloud management tool to install a user-based resource quota to ensure that one user does not overwhelm the server with a large workload request. Administrators can also use data gained from resource monitoring to predict and plan for spikes in resource demands.
Cloud management strategy
Cloud management strategy not just depends on effective tools and automation but also on skilled staff with professional hands-on experience. IT and business teams need to collaborate with each other to understand the requirements and maintain a two-fold strategy for organisational objectives.
IT teams must also test cloud application performance, monitor cloud computing metrics, make critical infrastructure decisions, address patch and security vulnerabilities, and update the business rule sets that drive cloud management. Organisations also must rethink their change management policies for the cloud, where consumption of resources can be much more rapid and spread out vis a vis an on-premise IT environment.
Companies that lack skilled IT staff can seek help from third parties. There are third-party apps that support budget threshold alerts that can notify finance and line of business stakeholders so they can monitor their cloud spending. Cloud brokerages often have a service catalogue and some financial management tools. Cloud spending must be monitored early, when apps go into production. Cloud management training should extend beyond IT and into other departments, such as the supply chain and accounting staff.
Apache CloudStack: An overview
Apache CloudStack is open source cloud computing software for creating, managing and deploying infrastructure cloud services. It makes use of existing hypervisor platforms for performing virtualisation like KVM, VMware vSphere including ESXi, vCenter and XenServer. Apache CloudStack also supports Amazon Web Services and Open Cloud Computing Interface.
The features of Apache CloudStack are listed below.
- Effective infrastructure management: Apache CloudStack is an effective tool for managing scalable infrastructure where enterprises have tens of thousands of physical servers distributed across geographical locations without any requirement of cluster-level servers. CloudStack enables easy maintenance with respect to updation of software and hardware infra updates without affecting normal operations.
- Autonomous configuration management: It enables IT admin to automatically configure all sorts of settings in terms of network, security and storage for deployment of virtual machines.
- Easy GUI: CloudStack offers easy and quickly manageable GUI based on the Web for managing and provisioning the cloud and for managing virtual machines.
- API: It provides a REST-like API for operating, utilising and managing the cloud.Less
- downtime: CloudStack has a number of features to increase the availability of the system. The management server itself may be deployed in a multi-node installation where the servers are load balanced. MySQL may be configured to use replication to provide for failover in case of database loss.
Other features include:
- New modern UI (project primate, technical preview)
- Backup and recovery framework
- Backup and recovery provider for Veeam
- VM ingestion
- CloudStack Kubernetes service
- L2 network PVLAN enhancements
- UEFI support
- KVM rolling maintenance
- Enables direct download for systemVM templates
- Template direct download support for local and SharedMountPoint storages
- VR health checks
- Download logs and diagnostics data from SSVM/CPVM/VRs
- Enables additional configuration metadata to virtual machines
CloudStack: Technical architecture
CloudStack deployment is really flexible and scalable; it can be deployed in simple and primary installations and can even accommodate complex installations.
Considering minimal requirements, CloudStack deployment consists of two machines – one running the CloudStack management server and the other acting as the cloud infrastructure. Even a single machine can perform the tasks of both the management server and the hypervisor host.
Figure 1 highlights a simple installation and deployment of CloudStack. This deployment is extensible to multi-node server installations where hundreds and thousands of client nodes can be joined in dynamic fashion.
Management server: The management server is defined as a dedicated server in the network, which controls the allocation of virtual machines to hosts, and performs tasks like storage allocation and IP allocations to instances. It runs in Apache Tomcat Container and uses the MySQL database server for storage operations.
It performs the following tasks:
- Manages all virtual machine storages, backups and instances
- Is a Web interface to clients and administrators
- Monitors the resources, disk images and ISO images
- Supports single point configuration for cloud
- Offers API interface support for CloudStack API and EC2 interface
Cloud infrastructure: All the resources in the cloud are suitably managed and monitored as follows.
- Regions: Geographically proximity zones managed by management servers
- Zones: Single data centre and consists of secondary storage
- Pods: A rack or row of racks that includes a 2-layer switch and one or more clusters
- Host: Client node
- Primary storage: Attached to single cluster running disk images
- Secondary storage: Stores templates, ISO images and snapshots
CloudStack: Implementation guidelines
CloudStack is an open source engineering framework for hybrid cloud development on an IaaS platform service. It can be used on public cloud like AWS, as it supports native EC2 and S3 compatible API to deploy cloud services. It can also be used with private cloud platforms or hybrid cloud solutions.
This framework is a complete package of cloud infrastructure services including containerisation (container-as-a-service), compute orchestration, network-as-a-service, account management, user management, open native API services and command line interface (CLI).
CloudStack can be used with CLI or Web application UI or dashboard services and RESTful API services. It supports all the popular hypervisors like VMware, Citrix XenServer, KVM, Xen, Cloud Platform (XCP), Microsoft Hyper-V and Oracle VM server. It manages and orchestrates pools of storage, network and computer resources to build a public or private IaaS compute cloud.
CloudStack consists of a minimum of two machine installations, in which the primary POD service is called the management server (which manages the entire appliance). The second installation is a hypervisor instance which manages all the cloud services. The hypervisor instance can be one or more nodes and can be part of multi-zone or multi-region deployment.
It can be deployed in some predefined patterns, as listed below.
1. Small-scale deployment: This is suitable for test and trailblazer activities.
2. Large-scale redundant deployment: Suitable for production-ready (pre-production) and production setups.
3. Separate storage network: Used for autonomous storage service on large-scale deployment to manage these services on separate appliances.
4. Multi-node management server: Integrated with hardware load balancers for high-availability of services.
5. Multi-site deployment: Suitable for enterprise scale deployment with high-performing workloads that have multi-node replication.
For the best solution design, it is better to set up a small-scale deployment solution first for the test environment and then replicate the configuration on a large-scale redundant deployment. We also need to monitor the host disk space very frequently to avoid storage failures and monitor VM instances regularly for compute host performance.
CloudStack: Containerisation
As per last year’s Forbes report, container adoption is growing faster than expected. A Gartner report states that by 2023 there will be more applications that run in production with containers, and around 70 per cent of enterprises will run more than two container applications (at least) in production. Hence, there is a growing demand for containerised solutions across industries/topologies/geographies, and across CSPs and cloud adoption models (hybrid and multi-cloud).
StackRox has done extensive research on container adoption strategy and its trends. Six of these trends are shared below.
- Faster container adoption by organisations, which increased by 22 per cent last year. Organisations prefer the head-first approach to containerise applications as part of their digital transformation journey.
- The growth rate of containerised applications moving to the cloud was 32 per cent in the last six months. This indicates that enterprises prefer containers not only for building but also for testing applications.
- AWS container adoption is the most popular among customers across industry, followed by Azure and GCP. AWS leads with a 28-35 per cent share in container adoption.
- Container security, DevOps integration, open source container platforms and container management are the key focus in building a container adoption strategy.
- Security as Code and Infrastructure as Code are the key solution drivers in building a resilient container adoption blueprint to have a ready-to-deploy infra configuration.
Containers need to integrate with existing infrastructure and should not demand higher investment in infrastructure replacement for on-premise or cloud services.
Apache CloudStack was formally developed as a cloud infrastructure management service for hybrid cloud solutions. It later adopted support for container deployment for hybrid cloud environments with the plug-in called CloudStack Kubernetes Service (CKS). CKS has a powerful user interface called the Control Panel or Dashboard service, using which one can build, deploy and monitor container instances quickly.
CKS is a multi-tenant solution, where multiple customers or entities share a common workload environment or platform with highly secured restrictions between their shared infrastructure components. They can build and deploy cloud-native container images and handle container cluster management using advanced cluster management tools.
CloudStack: Cloud service expense management
Cloud service expense management (CSEM) is a new-age requirement from cloud service platforms to provide cost transparency and cost management of cloud services. Typically, they are required to provide a transparent view on all cloud service usage patterns and also offer various manual and automated facilities to manage the costs associated with cloud service. They are also required to enable cost management through budgets, alerts and workflows on infrastructure and platform service provisioning.
CSEM, also termed as FinOps, is expected to offer the core benefit of any cloud platform, which is cost management. Cloud service platforms like Microsoft Azure, Amazon (AWS) and Google Cloud Platform (GCP) provide native cloud management facilities like budgets, alerts, cost advisories and workflow management for provisioning.
Apache CloudStack is a highly scalable platform with a very efficient cloud management service where rule based cost management, like usage limitations and reporting of usage statistics, can be defined. Though explicit CSEM facilities like cost advisory and predictive cost analysis are not explicitly available in the CloudStack platform, it has all the cloud cost and billing APIs available as RESTful service. Hence, it can be integrated with any third party cloud cost management platform like Densify or Apptio Cloudability for flexible CSEM.
