How to Create Shared Folder on VirtualBox – Windows and Ubuntu

I have a Windows 10 laptop, and I am running VirtualBox. One of my virtual machines has Ubuntu 14.04 64 bit operating system. Now, I want to be able to share folders from my Ubuntu to Windows 10 and vice versa. I’m assuming that you have installed the Ubuntu virtual machine already and ready to set up the shared folder.

Step 1) Make a folder to be shared on your Windows 10 host operating system.

I’ll create a folder called shared on my Windows 10 Desktop.

create-shared-folder

I’ll put a bunch of stuff inside the shared folder to be used by the Ubuntu virtual machine.

a-bunch-of-stuff-in-shared-folder

 

Step 2) Make sure that you have an empty optical drive for your virtual machine.

empty-optical-drive

 

Step 3) Boot your Ubuntu virtual machine. Go to Devices tab and click Insert Guest Additions CD image…

insert-guest-additions-cd-image

 

Step 4) Open up a terminal and install the Guest Additions CD.

We make a directory to mount the Guest Additions CD. The Guest Additions CD contains a file system type that we need to use.

sudo mkdir /media/cdrom

Now, we can mount the Guest Additions CD to the /media/cdrom folder.

sudo mount /dev/cdrom /media/cdrom
mount: block device /dev/sr0 is write-protected, mounting read-only

We should install necessary packages if they have not already been installed.

sudo apt-get install make gcc linux-headers-$(uname -r)

Now, we can install and run the Guest Additions CD.

sudo /media/cdrom/VBoxLinuxAdditions.run

 

Step 5) Click on the Devices tab and click Shared Folder Settings… and integrate the Windows 10 shared folder.

shared-folder-settings

Hit the add shared folder button.

hit-the-add-folder-button

Click on the Folder Path list and find your shared folder on Windows 10. I checked off Read-only, Auto-mount, and Make Permanent. Hit OK.

find-shared-folder-in-shared-folder-settings

 

Step 6) Create a shared folder on Ubuntu virtual machine and mount the shared folder.

You make a directory on the Ubuntu virtual machine that will act as Ubuntu’s shared folder.

mkdir ~/shared-windows10

We mount the shared folder on Windows 10 to the ~/shared-windows10 folder.

sudo mount -t vboxsf shared ~/shared-windows10

shared was the name of the Windows 10 shared folder, and I called the Ubuntu shared folder, ~/shared-windows10, to differentiate between the two.

 

Step 7) Change directory into the shared folder and see all the shared files.

cd ~/shared-windows10
ls

shared-windows10-folder-content

Huzzah! The shared folder is great for transferring files from Windows 10 to the Ubuntu virtual machine and vice versa.

I set the shared folder to Read-only, but you can uncheck the box in the Shared Folder Settings…

How to Setup MPI on CentOS 7 – 2 Connected Virtual Machines Part 2

For this blog post, I will be building on top of the Virtual Machine Clustering and NFS Server Setup in part 1. As a result, before you follow the steps of this post, you will want to have a similar setup like I have instructed in part 1:

http://www.slothparadise.com/how-to-connect-virtual-machines-and-setup-nfs-server-part-1/

MPI stands for Message Passing Interface. MPI isn’t your average networking library. It’s optimized for performance, takes the fastest transport for running parallel programs across machines, and is a usable protocol library implemented on nearly all operating systems.

 

Virtual Machine Information

With the steps from part 1, I have two virtual machines connected together under an internal network with one being the NFS Server and the other being the NFS Client.

 

Setting up SSH Keys

Currently, we can access from machine #1 to machine #2 through SSH. On machine #1,

ssh [email protected]
exit

The problem is that we will always receive a password prompt whenever we SSH across the internal network to the other machine. We want to set up SSH keys to SSH from each machine without passwords.

Let’s make sure that we’re on machine #1. Before we create the SSH key,  create the ~/.ssh folder.

mkdir ~/.ssh
ssh-keygen -t rsa -b 4096 -C "[email protected]"

You can press Enter to leave the next three prompts as default.

Enter file in which to save the key (/Users/you/.ssh/id_rsa): [Press enter]
Enter passphrase (empty for no passphrase): [Type a passphrase]
Enter same passphrase again: [Type passphrase again]
Your identification has been saved in /Users/you/.ssh/id_rsa.
Your public key has been saved in /Users/you/.ssh/id_rsa.pub.
The key fingerprint is:
01:0f:f4:3b:ca:85:d6:17:a1:7d:f0:68:9d:f0:a2:db [email protected]
cd ~/.ssh

We will copy the public key, id_rsa.pub, to authorized_keys to enable this key for access to machine #1.

cp id_rsa.pub authorized_keys

Now, we should send the private key, id_rsa, and public key, id_rsa.pub, from machine #1 to machine #2. We use a command called scp for copying files over machines.

scp ~/.ssh/id_rsa ~/.ssh/id_rsa.pub [email protected]:

On machine #2, we have received the private key and public key. We need to make the ~/.ssh directory on machine #2.

mkdir ~/.ssh

Now, we copy the id_rsa and id_rsa.pub to the ~/.ssh folder.

cp id_rsa id_rsa.pub ~/.ssh

We want to copy id_rsa.pub to the authorized_keys to allow machine #1 to be able to SSH to machine #2 without a password.

cd ~/.ssh
cp id_rsa.pub authorized_keys

We should be able to ssh from machine #1 to machine #2 without a password and vice versa.

exit

 

Installing MPI

First, we’ll need a library called wget. Wget will allow us the ability to download links to the machine. We will download wget on machine #1.

yum install wget -y

We will now download the source of mpich, which is an implementation of MPI. We download inside the shared folder.

cd /nfs
wget http://www.mpich.org/static/downloads/3.1.4/mpich-3.1.4.tar.gz

After we have downloaded mpich, we will install the C compilers, Fortran compiler, and kernel build tools on machine #1 and machine #2.

yum install gcc gcc-c++ gcc-fortran kernel-devel -y (on both machines)
vi ~/.bashrc

To extract the mpich downloaded tar.gz compressed file, we use the following command. The command will create a mpich-3.1.4 folder with all the contents of the extracted compressed file.

tar -xvf mpich-3.1.4.tar.gz

We will make a directory where all the compiled binaries and libraries of mpich will go.

mkdir /nfs/mpich3

Now, we will configure the settings of mpich for installation.

cd /nfs/mpich-3.1.4
./configure --prefix=/nfs/mpich3

Afterwards, we run a couple of commands for finishing the compiling and installation of mpich.

make
make install

If we cd /nfs/mpich3, we will see folders containing the binaries and libraries of mpich. If we cd /nfs/mpich3/bin, we can see mpi binaries like mpirun.

Currently, we won’t be able to use mpirun from anywhere on the machine. We need to change the ~/.bashrc file on machine #1 and machine #2 to globalize the mpi commands.

On both machines:

vi ~/.bashrc

At the bottom of ~/.bashrc, add the following two lines:

export PATH=/nfs/mpich3/bin:$PATH
export LD_LIBRARY_PATH="/nfs/mpich3/lib:$LD_LIBRARY_PATH"

PATH is used for bin folders, and LD_LIBRARY_PATH is used for lib folders. To reload the ~/.bashrc, type the following command on both machines:

source ~/.bashrc

 

Using MPI binaries: Running MPI

Let’s make a folder where we’ll create space for projects.

mkdir /nfs/projects
cd /nfs/projects

We need to create a hosts file that contains the IP addresses of all the IP’s that we want MPI to run.

vi hosts
10.0.1.2
10.0.1.3

MPI relies on ports for TCP and UDP packet communication. We will need to stop the firewalld for the process to hop between machines.

systemctl stop firewalld

We can test mpirun on a Linux command. The -f flag selects the host file that determines the IP addresses where MPI will decide to run the program. The -n flag determines the number of CPU cores that you want the program to run on. You may have to be on the client machine, in this case, 10.0.1.3, for the command to run without any errors.

mpirun -f hosts -n 4 echo "hello world"

The mpirun command prints the following if you have the machines in each other’s known_hosts records.

mpirun has run echo “hello world” 4 times because we chose 4 cores. When mpirun is used on a serial (meant for one machine) command, it will tend to run the command as many times as the number of cores selected as separate instances.

top is a useful Linux command to see what processes are running on the machine. If we check machine #2:

top

And if we run the mpirun command on machine #1 again:

mpirun -f hosts -n 4 echo "hello world"

We will see that top on machine #2 will not see any changes. The process does not jump over. Why? The reason is because when you use a serial Linux command like echo or ls or pwd, these commands are built for a single machine use. MPI does not see the need to jump this command over to another machine. MPI tries the fastest transport, which could be running everything on the one machine. MPI has the choice between the IP addresses specified in the hosts file. As a result, echo “hello world” is run only on machine #1.

Press q to quit top.

On the next blog post, we will write a basic MPI C or Python program and use mpi binaries to show you that the process is run across machines on programs designed to run in parallel. If you use mpirun on programs designed to run serially, the processes will likely run only on that machine because MPI will not see the need to make the jump to another machine, and the program in itself will not be programmed to do so.

How to Connect 2 Virtual Machines and Setup NFS Server Part 1

Here are a series of tutorials for beginners to get started in clustering virtual machines and learn how to parallel program. We will be using VirtualBox and CentOS in this article to simulate a clustered supercomputer.

I have an article on how to install VirtualBox, which is a program that allows you to create and use virtual machines. CentOS is a popular Linux operating. I’m writing these articles to teach the basics on what makes a supercomputer and how to get started with supercomputing and high performance computing.

 

What makes a supercomputer?

As you may have guessed, supercomputers are computers with high computational capacity compared to your everyday computer. Supercomputers tend to be built as a connected system of smaller computers called nodes. These nodes are connected together through a hub device called a switch where processes can be distributed in parallel via IP or VPI. Less powerful computers/nodes connected to create a supercomputer. With MPI which stands for Message Passing Interface in a distributed system, you can run processes that are passed among the individual nodes and run in parallel to achieve faster computations.

 

Prequisites

You must first install VirtualBox, which I have written a separate article on. I’ll also leave the link to the VirtualBox download. You will also need to download a Linux operating system ISO for booting the virtual machines with. I will use a minimum ISO of CentOS 7.1 in this example.

  1. Install VirtualBox
  2. Download CentOS 7.1

 

Create two virtual machines in VirtualBox

We create two virtual machines on VirtualBox. For each of the two virtual machines, go to Settings. Inside Settings, go to Storage. For the second item with a disc icon next to the option, select an ISO such as CentOS 7.1 minimal ISO to boot. Make sure that Live CD/DVD is checked. Press OK.

Start both virtual machines. The setup to boot into the operating system appears for installing CentOS. Enter the operating system installation. Use the default configurations for setting up the operating system. At the second setup window, click Installation Destination and click Done. Finally, click Begin Installation.

Click root password and set a password. Wait until the installation is complete and click Reboot. You can boot into CentOS on both machines. To login, the username is root, and the password is what you set the password as.

 

Setting up Internet Connection and DHCP

Click the X button on both virtual machines. Power off both virtual machines. Open File Explorer on Windows. Go to C:\Program Files\Oracle\VirtualBox. Hold Shift and Right Click white space on this window. Click Open Command Window Here.

We will create an Internal Network with VirtualBox’s built-in command. It will mimic our DHCP server.

VBoxManage dhcpserver add --netname intnet --ip 10.0.1.1 --netmask 255.255.255.0 --lowerip 10.0.1.2 --upperip 10.0.1.200 --enable

Go to Settings on both virtual machines in VirtualBox. First, we go to Storage. We uncheck Live CD/DVD, and we remove the optical disc that is the CentOS 7 minimal ISO. Next, we go to Network. Click on Adapter 2. Select Internal Network as the type. intnet should be automatically assigned. Click OK. Start both virtual machines.

On both machines, setup the Internet Connection.

nmcli d
enps08
enps03

enps08 will be connected, but enps03 will be disconnected.

vi /etc/sysconfig/network-scripts/ifcfg-enp0s3

Change the last line of this file on both machines to ONBOOT=yes.

systemctl restart network

Try ping 8.8.8.8. We have access to the Internet. Now, we must get access from one machine to the other machine.

ip addr show

When we check the IP addresses of both machines, we see that:

Each of the machines have an IP address within the Internal Network.

On CentOS-1: ping 10.0.1.3
On CentOS-2: ping 10.0.1.2

We have established the ability to ping from one machine to the other machine. We can communicate within the internal network among machines.

 

Setting up NFS Server: Machine #1

On CentOS-1, we’ll set the machine as the NFS Server. We will need to install a couple of NFS libraries.

yum install nfs-utils nfs-utils-lib -y
systemctl start rpcbind nfs-server
systemctl enable rpcbind nfs-server

Now, we create a new empty folder that will be our shared folder.

mkdir /nfs
vi /etc/exports

We add the following line to /etc/exports. We write the name of the shared folder and the IP addresses that we want the shared folder to be shared.

Now, we load the /etc/exports new changes.

exportfs -a

We have to change the firewall to allow NFS and complimenting services.

firewall-cmd --permanent --zone=public --add-service=nfs
firewall-cmd --permanent --zone=public --add-service=mountd
firewall-cmd --permanent --zone=public --add-service=rpc-bind
firewall-cmd --reload
systemctl restart nfs

 

Setting up NFS Client: Machine #2

yum install nfs-utils nfs-utils-lib -y

We make a folder where the shared folder from Machine #1 will be mounted on Machine #2.

mkdir -p /nfs

We make sure that we can access CentOS-1, the NFS Server. Make sure that the following two commands do not return any errors.

showmount -e 10.0.1.2
rpcinfo -p 10.0.1.2
mount 10.0.1.2:/nfs /nfs

With df -h, we should see that 10.0.1.2:/nfs mount has been created at the bottom. If we create any file inside /nfs, then all the machines connected can see the same file.

df -h

Now, we test that the shared folder actually works.

cd /nfs
touch microwavesam.txt

On CentOS-1, if we cd /nfs, we will see microwavesam.txt is inside the folder.

 

Making NFS more automatic

When you restart the two virtual machines, the NFS shared folder will not be there. We need to set a more automatic way for the NFS client to look for the NFS folder.

On the Client, we change a file called /etc/fstab.

vi /etc/fstab

We add the following line:

Every time, we restart the client, we can re-mount the NFS shared folder by typing mount -a.

mount -a

We have connected 2 virtual machines in VirtualBox, and we have created an NFS Server and Client, which is necessary for shared folders.

How to Install and Use VirtualBox?

Whoa… VirtualBox! VirtualBox is a fantastic program for creating virtual instances of your everyday regular computer machines. We normally use the program to pop up virtual Linux distribution machines like CentOS and Ubuntu so that even when we’re using a Windows or Mac machine, we can still have a window that is running Linux.

1) Download VirtualBox

You can find the latest virtual box at the following link:

https://www.virtualbox.org/wiki/Downloads

We’ll select the Windows hosts latest option, which is about 106 MB.

 

2) Download Linux distribution image of your choice

You can Google for any Linux distribution. I’ll take Arch Linux!

https://www.archlinux.org/download/

Download the latest iso file.

 

3) Install VirtualBox program

The set-up is quite simple. Proceed through the directions, prompts, and install.

 

 

4) Open VirtualBox

VirtualBox will have any old settings from previous installations of the program.

We will make a New virtual machine by first clicking on the New button at the top left.

 

 

5) VirtualBox settings

We will use the default settings for RAM, but make sure that you give enough hard drive memory for your virtual machine.

By default, the hard drive space for a VirtualBox Disk Image (VDI) will be 8 GB. I’ll expand my VDI to 50 GB.

 

6) After configuring settings, we want to right click our machine and make sure it loads the iso

Right click the virtual machine on the left side and click on Settings (CTRL-S). This will pop up the machineName – Settings.

 

7) Click on Storage option on the left

Underneath Controller: IDE, click on Empty. Make sure that the check box that says Live CD/DVD is checked.

Click on the Disc icon and click on Choose a virtual CD/DVD disk file. For me, I’ll find where I saved the archlinux.iso that I saved earlier.

Click OK.

 

8) Allow access to the internet for your virtual machine

Still in the settings window, click on the Network option on the left sidebar. By default, you’ll see Attached to NAT for Adapter 1. Leave this the same. Click on Adapter 2, enable the Adapter and click on the dropdown for Attached to Bridged Adapter. Leave the Name by default. This setting will enable your virtual machine to use the wi-fi or wired connection of your main machine.

 

9) Click on the machine on the left sidebar and click Start

 

10) After installing your Linux distribution, you’re all set

You can now use your Linux machine on your other operating system. Arch Linux is installed and ready to go!