Build your own high availability home data center, part 1!9 min read
Hello! Welcome to another blog post :) in this one, I’ll be sharing one project that I did at school and really liked. If you are interested in knowing how you could have a highly available website and want to do it, follow along! Just as an fyi, this doesn’t mean that this is what all data centers have, I am a student, I am learning and this was just for fun. This tutorial is not meant for production use, it is meant for learning purposes, people that want to know how to configure, monitor and use different servers and how they interact with each other, for beginners. Here’s a summary of what we’ll be doing:
- Create 4 Ubuntu servers:
- Two will be running Apache Web Server to host any website you’d like!
- One will only be running MariaDB, an open source database that will hold all the website information.
- And lastly, one will be running Nginx. This one will be on top of the web servers performing layer 7 load-balancing to efficiently distribute the incoming HTTP traffic to our backend servers.
- Share a step-by-step of how to set it up.
As I’ll be explaining some thing and going over others, the tutorial will be divided in several parts. This first part is about installing our servers, bringing the network up and just that. Later on we’ll implement the database, the web server, etc…
Of course, all will be done with open source software that is freely available on the web. The following links have the software that we’ll be using, download it for your PC architecture. The tutorial works on my Mac and in my Windows PC and the instructions here apply for both operating systems:
- Ubuntu server 16.04.5 64-bit http://releases.ubuntu.com/16.04/
- VirtualBox 6.0.0-1 https://www.virtualbox.org/wiki/Downloads
A little bit about virtualization
To be able to understand how a data center would work we’ll get into some virtualization stuff which is what powers today’s biggest data centers and allow Google, Amazon, Microsoft and others give those amazing cloud computing services they offer. Virtualization is enabled by using a software called hypervisor, which runs in the computer or server. A hypervisor is basically software that allows a computer (or host) to share its physical resources such as memory, storage, processors, NICs, etc, with one or more virtual machines (also called guests). This makes the guest independent of the underlying hardware and allows mobility for example, which is the ability to move a whole virtual server from one physical server to another. For virtualization, there are two types of hypervisors:
- Type 1 hypervisors: also called ‘bare-metal’. These are created specifically to run directly in the hardware and take all the advantages the server has, for example Hyper-V or VMWare ESXi.
- Type 2 hypervisors: the one we are using here! The hypervisor will run on top of our operating system, so it has to go though the OS to get hardware resources. Not the best for data center environments, that’s why we have type 1, but they are suited for our virtualization needs.
So the main difference between the two is that: a layer. More technically, a type 1 hypervisor will be able to use the resources the host has in a better way than the type 2 hypervisor due to that direct access to the hardware that it has and also that the kernel is specifically created for this purpose.
Back to what we are here for, I’ll proceed to show you my architecture. I did it with LucidChart that has nice graphics. Here it is:
Notice that each server has its own hostname, this will help us identify them and also to have them labeled in VirtualBox. Each name pretty much describes what function the VM does.
Let’s now move to installing Ubuntu server and create our 4 virtual machines. I hope you have now installed VirtualBox! The steps to create the virtual machines are:
- Click on add or new
- Give the VM a name, in this case I’ll use the ones in the above diagram. Select Linux as the type and Ubuntu 64-bit (or 32-bit, depending on what you chose).
- Data centers would have plenty of RAM to give to its VMs but as I just have 8 and this is a basic tutorial, we’ll go ahead and give it 1GB (1024MB) of memory. This is enough for the tutorial and learning purposes.
- Next, select the default storage option (Create a virtual hard disk now->VDI->Dynamically allocate) and give it 10GB of storage.
- Right click on the newly created VM in the left hand side of the VirtualBox screen and click on clone. Change the VM’s name to fit your needs and follow the wizard.
The end result is something like this:
Setup the network adapters
By now we have 4 virtual machines but they don’t have an OS installed yet. We’ll configure that in a moment! First we have to look into an important detail: how are we going to allow access to our website, i.e. networking? VirtualBox and most hypervisors have different configuration options for the network. On most we can do NAT, bridged, internal networking, host-only and NAT with port-forwarding. Which one should we choose? Let’s break it down:
- NAT: when an OS boots, it sends a DHCP request through its NIC to get an IP address assigned. Virtualbox will pick up this DHCP request and send a DHCP offer to the VM with an IP and a default gateway. This IP and default gateway will be the same for all VMs, if we have 10 VMs, they’ll all have the same IP. When the VMs want to send traffic out the network, virtualbox will rewrite the packet and place the host’s IP into the source IP of the packet, this way all traffic that’s going out of the network looks like is being generated from the host OS and there’s no VMs running under. The VM is completely isolated then, which is a good advantage if we are moving between different networks, but it’s a disadvantage if we need the VMs to communicate with each other. As they are using the same IP, they can’t. This is a problem for us as we rely on the communication between servers, we need another option.
- Bridged: what we are looking for! The bridged networking option makes VirtualBox create a vNIC, this virtual NIC will communicate or be “bridged” to the physical NIC. This way, when our VM boots it will send the DHCP request OUT the vNIC, virtualbox will forward that packet through the physical NIC and the router (or server) in our network will provide an IP address to our VM. Bridged networking allows each VM to have its own IP, making the VM act as if it was a standalone PC, allowing us to reach the servers from within our network, allowing server communication and enabling host-to-server communication.
How to change the network settings? Right click on the server from within the VirtualBox GUI, select settings from the popup menu, select network and move the default adapter type to ‘Bridged Adapter’. There are several more settings for this, take a look here for details.
Do the above before moving forward! Don’t install Ubuntu yet, first make sure network adapter has been set to fit your needs.
Install the guest OS
The last part of this first tutorial will be installing the Ubuntu operating system into our virtual machines. Remember to change the network adapters to bridged so we can communicate the VMs with each other! The installation process is very simple, it’s a guided step-by-step that is easy to follow. The only issue is that it has to be done via keyboard and can’t be completed with the mouse, which can be frustrating for some inexperienced users.
- First it asks us for our language, we move through the options with the up/down arrows and select one with the return key.
- We are asked whether we want to install Ubuntu Server or other options, we’ll select the first one
- It will then asks us for our language. Select the one you feel more comfortable with.
- if you’d like the OS to detect your keyboard, select the option to do so and then select the keyboard you want.
- Next we have to give our VM a hostname. I’ll set the ones I have previously mentioned in the diagram. Notice the hostname cannot have special characters, just alphabet letters and numbers.
- Create an account for your use, in this case I typed my name for both accounts that are requested, created a password and confirmed it.
- It also give us the option of encrypting our home directory. I have no needs for this, hence I selected ‘No’.
- The OS has already been assigned an IP based on our network adapter settings. With that IP it calculates the zone we are in and suggests a timezone for the machine. If the timezone is correct, select ‘Yes’.
- The next step is to partition the disk. In this case we’ll select ‘Guided - Use entire disk and setup LVM’. There should be just one partition to use next, so select that one. If multiple, select the appropiate. Confirm the changes to disk and configure LVM by selecting ‘Yes’ in the next option that is given, leave the disk space with 10GB (default) and select ‘Yes’ to confirm the changes to disk.
- If you are using a proxy, enter it here. In my case I am not, I’ll select ‘No’.
- Select No default updates and from the software install options select ‘Standart system utilities’. We’ll install the software we need later on.
- Finally, install the GRUB bootloader by selecting ‘Yes’ when prompted. Select continue and we are done! You’ll be now prompted to enter your user and password. Perform this process for every VM you need and we’ll be done.
The end of this blog post is here! If we have configured everything correctly, setup the network adapters right and performed the installation right, we’ll now be able to ping each VM from our host, from outside of the host (maybe another PC, your phone, your tablet?) and most importantly, VM communication will be enabled which is what we needed to progress with the final result: a highly available website with 4 different servers, a load balancer to efficiently distribute the incoming traffic to our backend servers and a database server where we store and have all our data. Notice in below image that the VMs have a different IP than my host but are within the same subnet, just what we wanted!
I hope you liked this blog post, found it interesting and learned something! For comments, questions or suggestions feel free to send them through the contact me box at the bottom of the screen here. Thanks and see you in the next entry!