I guess that almost all desktop systems in the office or home are derived from one prototype, the IBM PC (personal computer), which was built around the concept of an open architecture that IBM introduced 30 years ago. It’s true that the company itself didn’t quite understand how revolutionary open hardware architecture would be a few years later, in the mid-80s. Whatever the case, the fact remains that the PC became popular because many independent manufacturers could produce a wide range of compatible peripherals. This, in turn, allowed a competitive market to develop, eventually making desktop PCs available to everyone.
The open development model of Linux first embraced the desktop market of x86 computers, and then proceeded to “capture” other architectures as well (now more than 20), ranging from x86, ARM and MIPS, to Alpha, SPARC, PowerPC, IA64 and S390.
While “desktop” platforms (x86 and ARM) have many different Linux distributions and other operating systems that can run on them, the “pure” server architectures like IA64, S390 and Power cannot boast of this — the number of distributions is much, much smaller. This fact is understandable. Quality and the cost of server hardware for industrial applications is immeasurably higher than for the usual PC. Moreover, not every Linux distribution vendor can find the resources to build and provide long-term support for major server hardware architecture — even if they are willing to.
Some time ago, I got my hands on an IBM IntelliStation POWER 285 workstation, considered the closest model to large industrial servers. The main brain of the station is a dual-core POWER 5+ processor.
Although announced back in 2005, the system is serious stuff even now. Judge for yourself: 4 GB RAM (max capacity 32 GB), with a disk subsystem that has 2×146 GB SCSI drives (max disk storage 1.2 TB). It has a self-diagnostic system and memory “self-healing” technology known as ChipKill, a remote HMC (Hardware Management Console) and a pair of NICs working at 1 Gbit. This system is unbeatable for an entry-level enterprise server and simply forces you to take another respectful look at IBM — Big Blue creates innovative and striking products.
Well, that’s enough of an introduction. Let’s learn Linux installation and operation on IBM hardware. And later decide how it is possible to effectively exploit Linux on Power.
IBM produces equipment that has OpenFirmware (OF) inside. This is analogous to the traditional PC BIOS used on x86 systems. With OF’s help, you can assign which source should be used to start an OS, perform a number of diagnostic procedures, etc. OF itself uses a ServiceProcessor — a special unit embedded into the big tower. When you plug in the power cord, first the ServiceProcessor starts. That initially sets up a remote access option, and then awaits the user’s response.
The user has to press the hardware Start button on the front panel. So, when plugged in, IntelliStation is never completely turned off — even with the main CPUs off, the ServiceProcessor doesn’t sleep. The scheme is reminiscent of a similar product — iLO, developed by Compaq/HP. iLO also lets one start and control a server remotely.
It’s worth noting that IBM also has additional functions run by HMC (Hardware Management Console), among which are planning and allocation of physical resources to logical partitions (LPAR/DLPAR), splitting main CPU performance onto logical CPUs (Capacity On Demand) and related actions that provide virtualisation management on POWER-based processors. Unfortunately, I can only mention these capabilities, and not go any deeper into the subject, because that would be beyond the scope of this review.
So, in order to establish an access link to the ServiceProcessor and later install Linux, you will need an RS232(F)<->RS232(F) cable and a dedicated workstation, from which you can directly control your IntelliStation 285. As a console, any PC with an installed command-line tool like
surecom (a GUI alternative) will suit. Set the communication speed to 19,200 bit/s, the standard default rate for IBM hardware, plug in the power cord and wait for the login prompt after running
minicom -D /dev/ttyS0 -s 19200 -o. See Figure 1.
To start the whole system and gain access to OF, from where you can select an OS source, choose:
1. Power/Restart Control->
1. Power On/Off System->
8. Power on. After that, in approximately two minutes, you will hear the noise of the fans — the big system has started initialisation and a minute later the hardware will be completely initialised, signalled via two speaker beeps.
At the same time, on the console, you should see a message from SMS (System Management Services) as in Figure 2, where you choose
1 to enter the system OF menu and then select
1. Select Install/Boot Device to choose the boot device (which you will run the installer from).
You should see something like Figure 3, where I chose
4. IDE, because the built-in CD-ROM is connected to the IDE connector. In your version, it might be
In the subsequent menu (Figure 4), I chose
2. Normal Mode Boot, which is suitable for installation, and confirmed exit from SMS. My SLES 11’s Linux kernel correctly identified the IntelliStation 285.
The installation process will proceed as follows:
Welcome to SuSE Linux Enterprise 11! Type "install" to start the YaST installer on this CD/DVD Type "slp" to start the YaST install via network Type "rescue" to start the rescue system on this CD/DVD Welcome to yaboot version r22.8-r1190.SuSE booted from '/pci@800000020000003/pci@2,3/ide@1/disk@0:1,\suseboot\yaboot.ibm' running with firmware 'IBM,SF240_403' on model 'IBM,9111-285', serial 'IBM,0206ABB30', partition '0' Enter "help" to get some basic usage information boot: Please wait, loading kernel... Allocated 01800000 bytes for executable @ 03000000 Elf32 kernel loaded... SuSE Linux zImage starting: loaded at 03000000-0469cd30 (4000000/0/02039a68; sp: 02c9fd50) uncompressing ELF header done. (00000100 bytes) Allocated 0164c090 bytes for kernel @ 04800000 Leave 0122a5ba bytes for initrd @ 034668bb uncompressing kernel done. (00f5a920 bytes) entering kernel at 04810000(34668bb/122a5ba/02039a68)
The installation procedure reminds me of what happens on the x86, with just one difference — in our case, control is via a serial line in text VT102 mode, so some characters may not be displayed properly as you can see in Figure 5.
Several menu items, such as the network address, may require you to return to re-edit these fields. We can use Tab/Alt+Tab to move between text interface fields. These keyboard shortcuts are functional even in this very minimal mode.
Embedded Linux on ServiceProcessor
In addition to control via a serial line, IntelliStation (and in general, all IBM products) offers management via an ordinary Web browser. This Web control mechanism (Advanced System Management — see Figure 6) is provided by a tiny computer within the IntelliStation, with Linux and BusyBox utilities.
This ServiceProcessor also uses Power technology — a PowerPC 405 CPU from IBM [Datasheet PDF] but with a much smaller form factor. This uses a low-power processor FSP (Flexible Service Processor) running at about 200 MHz (see Figure 7).
ASM provides the same controls as the serial-line SMS but in a more usable manner (Figure 8). It lets you manage the system remotely — start/stop it, view logs, allocate resources (like Capacity on Demand parameters), etc.
Let’s briefly see how this subsystem is launched. When you connect the mains power to the box, the ServiceProcessor automatically starts the embedded Linux. After initially testing some components of a larger block, ServiceProcessor displays on a front panel the readiness status for full system start. When you hit the “Power On” button, OpenFirmware loads and ultimately passes control to your Linux boot loader,
Like with the large system, this small subsystem has two independent Ethernet controllers, referred to as HMC 1 and HMC 2. It makes sense to connect them to separate LANs/VLANs for a special management network. First, it helps access restriction and security. Second, you can avoid a network traffic overload on a single LAN.
IP addresses for these ports can be supplied by a DHCP server on a management network. If one isn’t found, default addresses are automatically assigned: 192.168.2.147 (port C7-T1, on HMC1) and 192.168.3.147 (port C7-T2, HMC2). You can log in via telnet, with the default credentials admin/admin.
$ mount /dev/mtdblock/0/3 on / type cramfs (rw) none on /dev type devfs (rw) proc on /proc type proc (rw) /dev/mtdblock/0/1 on /opt/extucode type jffs2 (rw) /dev/mtdblock/0/1 on /etc type jffs2 (rw) /dev/mtdblock/tmpram/tmpram on /tmp type jffs2 (rw) none on /var type tmpfs (rw) /dev/mtdblock/pcscore/pcscore on /core type jffs2 (rw) /dev/mtdblock/pcsram/pcsram on /opt/p1 type jffs2 (rw) shm on /var/shm type shm (rw) /dev/mtdblock/1/1 on /alt/extucode type jffs2 (rw) /dev/mtdblock/2/3 on /opt/fips type cramfs (rw) /dev/mtdblock/nvram/nvram on /opt/p2 type zofs (rw,sync) $ uptime 8:49am up 1 day, 14:53, load average: 0.52, 0.20, 0.07 $ uname -a Linux fsp 2.4.18-rc4 #1 Wed Mar 25 15:01:22 UTC 2009 ppc unknown $ cat /proc/version Linux version 2.4.18-rc4 (root@mcpbuild6) (gcc version 3.2.3 20030401 (prerelease)) #1 Wed Mar 25 15:01:22 UTC 2009
As we see, Embedded Linux uses JFFS/CRAMFS filesystems, which are common in tiny products. An interesting feature is NVRAM memory mounted as storage (
/dev/mtdblock/nvram/nvram). Reading data from its files gives the status of different subsystems. You can also change some values using commands from
/opt/fips/bin — particularly, you can even try to launch several subsystems (or maybe the full system box) with
/opt/fips/bin/powr_fsp_spcn, which checks the current system status, and accordingly launches
/bin/powr_sqh. The ASM Web server (named webs), when you click the button Save settings and power on does the same.
Available distros for POWER
Originally, I planned to use Red Hat Enterprise Linux as the main OS on this system. However, neither RHEL 5.7 nor 6.2 Beta, nor even 6.2 started correctly. RHEL 6.2 Beta and a more recent version 6.2 spews a kernel trap error during initial startup. Yes, the IntelliStation 285 wasn’t in the supported hardware list, but neither was it for SLES 10/11. Still, SLES 10 and 11 support it unofficially. Let us see how the installation process goes, and whether we can bring the system to production mode.
In addition to RHEL and SLES, numerous systems based on the POWER processor even supported Ubuntu 10.04 (LTS edition). However, my experience proved that this model wasn’t suitable for Ubuntu — it cannot start properly. Moreover, newer versions of Ubuntu come without any POWER/PowerPC architecture support at all, unfortunately.
I checked if the most recent release (6.0.3) of Ubuntu’s parent distribution, Debian, could boot. It wasn’t able to start — it drops directly into OpenFirmware. Another failure! Well, let’s not despair. Quite possibly, the Debian community will fix it.
Oddly enough, a completely amateurish project called Crux PPC has a fully working distribution — Crux PPC 2.7a, which not only loads on Power hardware, but also provides video output via correctly-set framebuffer mode. None of the other distributions mentioned can boast of that! All provided only a serial-line text mode. In general, Crux PPC runs as a LiveCD and has no installer, so I advise professionals to use it. Again, this topic is out of the scope of this article, so I will leave it for later.
As I mentioned a bit earlier, IBM products are one level, or a “head above” their competitors. Comparing the very different Power and x86 architectures is hard, because there are no objective criteria for such a comparison. However, from an ordinary user’s perspective, we can try the 7Zip archiver in its benchmark mode. This program is completely open source, so anyone can compile it.
Let us compare IntelliStation based on the POWER5+ processor with 2 cores and 4 GB RAM, manufactured in 2005, with an Intel Core 2 Duo (2 cores with hyper-threading) and 4 GB RAM, but released three years later (2008). First, the IntelliStation:
linux:~/src/p7zip_9.20.1/bin # ./7za b 7-Zip (A) 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,2 CPUs) RAM size: 3632 MB, # CPU hardware threads: 2 RAM usage: 425 MB, # Benchmark threads: 2 Dict Compressing | Decompressing Speed Usage R/U Rating | Speed Usage R/U Rating KB/s % MIPS MIPS | KB/s % MIPS MIPS 22: 1931 99 1891 1878 | 24309 100 2198 2194 23: 1871 100 1916 1907 | 24132 100 2212 2209 24: 1816 99 1964 1953 | 23917 100 2221 2219 25: 1780 99 2043 2032 | 23525 100 2213 2212 ---------------------------------------------------------------- Avr: 99 1954 1942 100 2211 2209 Tot: 100 2082 2076
Next, let us test the Core 2 Duo system:
Intel(R) Core(TM)2 Duo CPU E6550 @ 2.33GHz 7-Zip 9.04 beta Copyright (c) 1999-2009 Igor Pavlov 2009-05-30 p7zip Version 9.04 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,2 CPUs) RAM size: 4006 MB, # CPU hardware threads: 2 RAM usage: 425 MB, # Benchmark threads: 2 Dict Compressing | Decompressing Speed Usage R/U Rating | Speed Usage R/U Rating KB/s % MIPS MIPS | KB/s % MIPS MIPS 22: 2588 138 1831 2518 | 33730 165 1850 3045 23: 2532 136 1903 2580 | 32265 159 1853 2954 24: 2588 142 1958 2783 | 32828 165 1843 3046 25: 2540 144 2011 2900 | 30299 155 1838 2849 ---------------------------------------------------------------- Avr: 140 1925 2695 161 1846 2974 Tot: 150 1886 2834
As you can see, the performance doesn’t differ much. Even a slightly outdated IBM station can compete with a relatively new x86 product.
Despite the fact that equipment based on the Power processor is marketed by IBM as a corporate power-horse for commercial operating systems like AIX and IBM i (produced by IBM itself), we have seen that it can also be used with Linux. Thus, virtually the entire software stack that successfully works on x86 can also be run on Power — Apache, Java, WebSphere, MySQL, etc.
Taking into account the fact that IBM hardware usually comes with unique technologies like the hardware management console and the hardware virtualisation hypervisor, you can easily create an effective solution by combining the best features from both the hardware and software sides — the corporate and the public world.