Here are a series of tutorials for beginners to get started in clustering virtual machines and learn how to parallel program. We will be using VirtualBox and CentOS in this article to simulate a clustered supercomputer.
I have an article on how to install VirtualBox, which is a program that allows you to create and use virtual machines. CentOS is a popular Linux operating. I’m writing these articles to teach the basics on what makes a supercomputer and how to get started with supercomputing and high performance computing.
What makes a supercomputer?
As you may have guessed, supercomputers are computers with high computational capacity compared to your everyday computer. Supercomputers tend to be built as a connected system of smaller computers called nodes. These nodes are connected together through a hub device called a switch where processes can be distributed in parallel via IP or VPI. Less powerful computers/nodes connected to create a supercomputer. With MPI which stands for Message Passing Interface in a distributed system, you can run processes that are passed among the individual nodes and run in parallel to achieve faster computations.
Prequisites
You must first install VirtualBox, which I have written a separate article on. I’ll also leave the link to the VirtualBox download. You will also need to download a Linux operating system ISO for booting the virtual machines with. I will use a minimum ISO of CentOS 7.1 in this example.
- Install VirtualBox
- Download CentOS 7.1
Create two virtual machines in VirtualBox
We create two virtual machines on VirtualBox. For each of the two virtual machines, go to Settings. Inside Settings, go to Storage. For the second item with a disc icon next to the option, select an ISO such as CentOS 7.1 minimal ISO to boot. Make sure that Live CD/DVD is checked. Press OK.
Start both virtual machines. The setup to boot into the operating system appears for installing CentOS. Enter the operating system installation. Use the default configurations for setting up the operating system. At the second setup window, click Installation Destination and click Done. Finally, click Begin Installation.
Click root password and set a password. Wait until the installation is complete and click Reboot. You can boot into CentOS on both machines. To login, the username is root, and the password is what you set the password as.
Setting up Internet Connection and DHCP
Click the X button on both virtual machines. Power off both virtual machines. Open File Explorer on Windows. Go to C:\Program Files\Oracle\VirtualBox. Hold Shift and Right Click white space on this window. Click Open Command Window Here.
We will create an Internal Network with VirtualBox’s built-in command. It will mimic our DHCP server.
VBoxManage dhcpserver add --netname intnet --ip 10.0.1.1 --netmask 255.255.255.0 --lowerip 10.0.1.2 --upperip 10.0.1.200 --enable
Go to Settings on both virtual machines in VirtualBox. First, we go to Storage. We uncheck Live CD/DVD, and we remove the optical disc that is the CentOS 7 minimal ISO. Next, we go to Network. Click on Adapter 2. Select Internal Network as the type. intnet should be automatically assigned. Click OK. Start both virtual machines.
On both machines, setup the Internet Connection.
nmcli d
enps08 enps03
enps08 will be connected, but enps03 will be disconnected.
vi /etc/sysconfig/network-scripts/ifcfg-enp0s3
Change the last line of this file on both machines to ONBOOT=yes.
1 | ONBOOT=yes |
systemctl restart network
Try ping 8.8.8.8. We have access to the Internet. Now, we must get access from one machine to the other machine.
ip addr show
When we check the IP addresses of both machines, we see that:
1 2 | CentOS-1: 10.0.1.2 CentOS-2: 10.0.1.3 |
Each of the machines have an IP address within the Internal Network.
On CentOS-1: ping 10.0.1.3
On CentOS-2: ping 10.0.1.2
We have established the ability to ping from one machine to the other machine. We can communicate within the internal network among machines.
Setting up NFS Server: Machine #1
On CentOS-1, we’ll set the machine as the NFS Server. We will need to install a couple of NFS libraries.
yum install nfs-utils nfs-utils-lib -y
systemctl start rpcbind nfs-server systemctl enable rpcbind nfs-server
Now, we create a new empty folder that will be our shared folder.
mkdir /nfs
vi /etc/exports
We add the following line to /etc/exports. We write the name of the shared folder and the IP addresses that we want the shared folder to be shared.
1 | /nfs 10.0.1.3(rw,sync,no_root_squash,no_subtree_check) |
Now, we load the /etc/exports new changes.
exportfs -a
We have to change the firewall to allow NFS and complimenting services.
firewall-cmd --permanent --zone=public --add-service=nfs firewall-cmd --permanent --zone=public --add-service=mountd firewall-cmd --permanent --zone=public --add-service=rpc-bind firewall-cmd --reload
systemctl restart nfs
Setting up NFS Client: Machine #2
yum install nfs-utils nfs-utils-lib -y
We make a folder where the shared folder from Machine #1 will be mounted on Machine #2.
mkdir -p /nfs
We make sure that we can access CentOS-1, the NFS Server. Make sure that the following two commands do not return any errors.
showmount -e 10.0.1.2 rpcinfo -p 10.0.1.2
mount 10.0.1.2:/nfs /nfs
With df -h, we should see that 10.0.1.2:/nfs mount has been created at the bottom. If we create any file inside /nfs, then all the machines connected can see the same file.
df -h
Now, we test that the shared folder actually works.
cd /nfs
touch microwavesam.txt
On CentOS-1, if we cd /nfs, we will see microwavesam.txt is inside the folder.
Making NFS more automatic
When you restart the two virtual machines, the NFS shared folder will not be there. We need to set a more automatic way for the NFS client to look for the NFS folder.
On the Client, we change a file called /etc/fstab.
vi /etc/fstab
We add the following line:
1 | 10.0.1.2:/nfs /nfs nfs auto,noatime,nolock,bg,nfsvers=3,intr,tcp,actimeo=1800 0 0 |
Every time, we restart the client, we can re-mount the NFS shared folder by typing mount -a.
mount -a
We have connected 2 virtual machines in VirtualBox, and we have created an NFS Server and Client, which is necessary for shared folders.