The Complete Magazine on Open Source

Big Data on a Desktop: A Virtual Machine in an OpenStack Cloud

, / 237 0

OpenStack is a worldwide collaboration between developers and cloud computing technologists aimed at developing the cloud computing platform for public and private clouds. Let’s install it on our desktop.

Installing OpenStack using Packstack is very simple. After a test installation in a virtual machine, you will find that the basic operations for creating and using virtual machines are now quite simple when using a Web interface.

The environment
It is important to understand the virtual environment. While everything is running on a desktop, the setup consists of multiple logical networks interconnected via virtual routers and switches. You need to make sure that the routes are defined properly because otherwise, you will not be able to access the virtual machines you create.

On the desktop, the virt-manager creates a NAT-based network by default. NAT assures that if your desktop can access the Internet, so can the virtual machine. The Internet access had been used when the OpenStack distribution was installed in the virtual machine.

The Packstack installation process creates a virtual public network for use by the various networks created within the cloud environment. The virtual machine on which OpenStack is installed is the gateway to the physical network.

Virtual Network on the Desktop (virbr0 interface):
IP address of eth0 interface on OpenStack VM:
Public Virtual Network created by packstack on OpenStack VM:
IP address of the br-ex interface OpenStack VM:

Testing the environment
In the OpenStack VM console, verify the network addresses. In my case, I had to explicitly give an ip to the br-ex interface, as follows:

# ifconfig
# ip addr add dev br-ex

On the desktop, add a route to the public virtual network on OpenStack VM:

# route add -net netmask gw

Now, browse and create a new project and a user associated with the project.
1. Sign in as the admin.
2. Under the Identity panel, create a user (youser) and a project (Bigdata). Sign out and sign in as youser to create and test a cloud VM.
3. Create a private network for the project under Project/Network/Networks:

  • Create the private network with the gateway
  • Create a router and set a gateway to the public network. Add an interface to the private network and ip address

4. To be able to sign in using ssh, under the Project/Compute/Access & Security, in the Security Groups tab, add the following rules to the default security group:

  • Allow ssh access: Custom TCP Rule for allowing traffic on Port 22.
  • Allow icmp access: Custom ICMP Rule with Type and Code value -1.

5. For password-less signing into the VM, under the Project/Compute/Access & Security, in the Key Pairs tab the following:

  • Select the Import Key Pair option and give it a name, e.g., ‘desktop user login’.
  • In your desktop terminal window, use ssh-keygen to create a public/private key pair in case you don’t already have one.
  • Copy the contents of ~/.ssh/ from your desktop account and paste them in the public key.

6. Allocate a public IP for accessing the VM under Project/Compute/Access & Security in the Floating Ips tab, and allocate IP to the project. You may get a value like
7. Now launch the instance under Project/Compute/Instance:

  • Give it a name – test and choose the m1-tiny flavour.
  • Select the boot source as ‘Boot from image’ with the image name ‘cirros’, a very small image included in the installation.
  • Once it is launched, associate the floating ip obtained above with this instance.

Now, you are ready to log in to the VM created in your local cloud. In a terminal window, type:

ssh cirros@

You should be signed into the virtual machine without needing a password.

You can experiment with importing the Fedora VM image you used for the OpenStack VM and launching it in the cloud. Whether you succeed or not will depend on the resources available in the OpenStack VM.


Figure 1: Simplified network diagram


Installing only the needed OpenStack services
You will have observed that OpenStack comes with a very wide range of services, some of which are not likely to be very useful for your experiments on the desktop, e.g., the additional networks and router created in the tests above. Here is a part of the dialogue for installing the required services on the desktop:

[root@amd ~]# packstack
Welcome to Installer setup utility
Enter the path to your ssh Public key to install on servers: 
Packstack changed given value to required value /root/.ssh/
Should Packstack install MySQL DB [y|n] [y] : y
Should Packstack install OpenStack Image Service (Glance) [y|n] [y] : y
Should Packstack install OpenStack Block Storage (Cinder) service [y|n] [y] : n
Should Packstack install OpenStack Compute (Nova) service [y|n] [y] : y
Should Packstack install OpenStack Networking (Neutron) service [y|n] [y] : n
Should Packstack install OpenStack Dashboard (Horizon) [y|n] [y] : y
Should Packstack install OpenStack Object Storage (Swift) [y|n] [y] : n
Should Packstack install OpenStack Metering (Ceilometer) [y|n] [y] : n
Should Packstack install OpenStack Orchestration (Heat) [y|n] [n] : n
Should Packstack install OpenStack client tools [y|n] [y] : y

The answers to the other questions will depend on the network interface and the IP address of your desktop, but there is no ambiguity here. You should answer with the interface ‘lo’ for CONFIG_NOVA_COMPUTE_PRIVIF and CONFIG_NOVA_NETWORK_PRIVIF. You don’t need an extra physical interface as the compute services are running on the same server.

Now, you are ready to test your OpenStack installation on the desktop. You may want to create a project and add a user to the project. Under Project/Compute/Access & Security, you will need to add firewall rules and key pairs, as above.
However, you will not need to create any additional private network or a router.

Import a basic cloud image, e.g., from under Project/Compute/Images.

You may want to create an additional flavour for a virtual machine. The m1.tiny flavour has 512MB of RAM and 4GB of disk and is too small for running Hadoop. The m1.small flavour has 2GB of RAM and 20GB of disk, which will restrict the number of virtual machines you can run for testing Hadoop. Hence, you may create a mini flavour with 1GB of RAM and 10GB of disk. This will need to be done as the admin user.

Now, you can create an instance of the basic cloud image. The default user is fedora and your setup is ready for exploration of Hadoop data.