Build your own high availability home data center, part 2!

      16 min read  

Hi! Glad you made it here :)

Thanks for continuing with the tutorial. If you reached here but haven’t yet installed VirtualBox, haven’t configured the VMs or have some doubts in the process, please refer to the first part. Else, we’ll continue working on our ‘Home Data Center’ here. In this tutorial I’ll be going over the next things to configure our environment:

Let’s begin.


Configuring SSH

Our first step in this tutorial will be to install SSH to be able to access the servers from the Mac terminal or from PuTTy on Windows. SSH is a cryptographic protocol to securely log into remote devices. It uses TCP port 22 (hence port 22 should be open in the server) and you can install it by running the following commands:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install openssh-server

After ssh is installed, restart the service so it can start working:

sudo systemctl restart ssh

Next, look for ‘Terminal’ in Mac or PuTTy (download link here) on Windows and login to the server via ssh! From terminal in Mac, type:

$ ssh username@server_ip

From PuTTy just enter the server’s ip, select ssh and use your username and password when prompted:

Putty

By installing SSH we are now able to paste all the commands we need into the terminal, no need to type them anymore!


Apache setup

As mentioned in the first part of the tutorial, our main goal is to have a highly available website. This will be done by having a load balancing server configured to efficiently handle the petitions for the website that it gets. The load balancer will get an HTTP request to its IP address and based on its configuration it take the decision of where to send that specific request. Lets move onto the Apache setup. I’ll briefly introduce Apache so you can get a notion of what it is, in case you don’t know.

Apache is a software distributed by the Apache Software Foundation that is freely to use, scalable and the most used web server in the world due to its great performance. Apache runs the http daemon when it runs on Unix systems, such as the ones we are using (Ubuntu). This daemon will be in charge of responding to those http requests that it gets. When the request gets to the server the http daemon will examine the packet. Inside of the ‘Required URL’ part of the packet we’ll find just that, the URL that a client is requesting. Httpd will look for that URL within its directories inside the server and return a response with certain codes. If the site is found then it will return a ‘200 OK’ with the file that was requested, if it didn’t find it then it will return a ‘404 Not found’. The image below shows how this process is done. The image was taken from here and all credits go to them. I wrote a little about HTTP and layer 7 in the OSI model, if you want to read more feel free to follow this link.

http example

Lets proceed with the installation of Apache in both of our servers:

  1. Log into each Apache server
  2. From here, run the commands below. The first one will update our working server, then we’ll instal Apache. The third will disable directory listing which can be harmful and is best to disable it for security measures. Lastly, we restart the apache service and set it to start running whenever the server starts:
sudo apt-get update
sudo apt-get install apache2
sudo sed -i "s/Options Indexes FollowSymLinks/Options FollowSymLinks/" /etc/apache2/apache2.conf
sudo systemctl restart apache2.service
sudo systemctl enable apache2.service

With this finished we are ready to test if Apache installed correctly. Run the following command to get the iP address of the web server, then browse for that IP in your browser and you should get a website similar to the one below, congratulating us for correctly enabling Apache!

Apache successful installation

The file that we are seeing with the ‘Congratulations’ is the index.html that is saved in /var/www/html under our Ubuntu server. Since Ubuntu 14.04 and later, the document root for Apache is found in this directory. When we go to the IP address of the server from within our web browser, we are sending an http request to the server. The httpd picks up this request, browses for the URL requested and sends the website we were looking for. As we are not specifying any site, then it sends us the default website which is index.html:

Apache Index


Analyzing the PC-Server communication

To be able to see this process in a more visual way, I have downloaded a software called WireShark. This is a packet sniffer that allows us to capture the traffic going in and out or NIC and analyze it.

Communication example

From the image above we can see some other steps that take place before the website is requested. I have filtered the messages to only show those that belong to the IP of the server (192.168.1.37). My PC has the 192.168.1.40 IP address.

As we want the data from the PC to reliably get to the server, HTTP uses TCP as its transport layer protocol. This means that even before sending HTTP data, TCP will first establish a connection with the server.

In networking, this is called a 3-way handshake. Let’s break the process down:

  1. The host initiating the connection sends a SYN packet with a sequence number of 0.
  2. The server that gets the packet will check it and see a SYN message, this makes the server allocate resources to create a new connection with a new PC. The server will respond to the SYN with a SYN,ACK; a synchronize message and an acknowledgement of that SYN the PC sent.
  3. The last part of the 3-way handshake is to acknowledge the receival of the server’s SYN, ACK; with that we are done and have established a connection oriented session with the server!

After this has happened, notice the next packet that is sent is an HTTP GET request for the root site of the server (in this case, the index.html). This packet is encapsulated inside a TCP segment that will be delivered over the connection that we just established. Below is an example of how this packet looks like on the wire:

HTTP packet

From this image we can see several important fields in a packet that we as the architects of the data center should be aware of:

  1. The TCP ports we are using are two: our host’s port is an ephemeral port 50090 so that our PC identifies to which application the incoming replies come. The other port is 80 at the server, 80 is the well known port for HTTP traffic.
  2. The HTTP information itself is showing a ‘GET’ request to get the root document ( the ‘/’ next to GET represents this root document, which is the index.html) and then it shows the HTTP version being used (HTTP/1.1). This is the end of the data sent by the PC.

Next, we can see 3 acknowledgment packets being sent from the server to the host acknowledging the receival of the HTTP packet we just saw earlier. This means that the server has received the request, a few things happend onwards:

  1. The packet will be checked by the server, when it reaches the layer 4 data it will see that the destination port is 80, meaning HTTP. The transport layer will then move the packet to the http daemon that handles this in layer 7 and will see that the HTTP GET is to get the root index.html, so httpd will look for that index.html file within the /var/www/html directory and create an HTTP response.
  2. Analyzing the HTTP response packet we can see that the TCP ports have switched, now the destination port will be the ephemeral PC port and the source is the well known http port. Within the layer 4 segment we see the HTTP part, where an HTTP OK response phrase is sent, telling the PC that everything went fine and it adds an HTTP status code 200 as well. We’ll include the HTML as text within the HTTP packet, this is how we transfer the index.html file back to the PC that requested it.

HTTP packet 2


Changing the index.html

To be able to identify which server we are accessing when the load balancer is configured, it will be useful to have a banner with the hostname of the server. Let’s configure this! Before configuring the index.html site, we’ll need to follow some steps to configure FTP access to the server to be able to push files from our local computer to the server. The steps to take are:

sudo apt-get update
sudo apt-get install -y vsftpd
sudo service vsftpd restart

Filezilla example

sudo chown yourusername: /var/www/html/
rm -rf /var/www/html/index.html
  1. First, open the file that has the configuration of FTP. This one is located in /etc directory and the file is called vsftpd.conf
sudo nano /etc/vsftpd.conf
  1. Next, locate the line that has the write access configuration and uncomment it. Just delete the ‘#’ that is in front.
# Uncomment this to enable any form of FTP write command.
#write_enable=YES
  1. After doing this hit ctrl+o, then hit return, then ctrl+x to go out of edit mode. Then, restart the vsftp daemon
sudo systemctl restart vsftpd

The image below shows how it should now look.

FTP configuration

sudo chmod 755 /var/www/html

And lets now code our index! This is gonna be pretty simple. We want our index.html to be just a big title that shows the name of the server we are logged into, this way when we set the load balancer later on we can see the changes between the servers. Below is the code, copy it and paste it into a file and call it index.php. Notice that we are using a .php file instead of .html, this is because we’ll be fetching the server’s hostname via php and then this file will be displayed to the user.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <title>Index</title>
</head>
<body>
    <?php 
        $hostname = gethostname();
        echo "<h1>" . $hostname . "</h1>";
    ?>
</body>
</html>

Save this index.php into your desktop, then look for it inside FileZilla and upload it to the server. Once that is done, remember to set the permissions of the file to 755! The directory where this file should be is /var/www/html/

sudo chmod 755 /var/www/html/index.php

FTP configuration

We are done for now. Let’s move to another part.


Installing PHP

Now that we have changed the index file to a php file, we want the php to be able to run. For this, we have to download and install php into the server.

sudo apt-get install software-properties-common
sudo add-apt-repository ppa:ondrej/php
sudo apt update
sudo apt install -y php7.1 libapache2-mod-php7.1 libapache2-mod-php7.1 php7.1-common php7.1-mbstring php7.1-xmlrpc php7.1-soap php7.1-gd php7.1-xml php7.1-intl php7.1-mysql php7.1-cli php7.1-mcrypt php7.1-ldap php7.1-zip php7.1-curl
sudo nano /etc/php/7.1/apache2/php.ini
file_uploads = On
allow_url_fopen = On
memory_limit = 256M
upload_max_filesize = 64M
max_execution_time = 360
date.timezone = America/Costa_Rica

To go back to the terminal once you are done editting the php.ini file remember the process is:

Now that PHP is installed and configured, go to your web browser and search for the IP address of the server. We should now see the hostname! Remember to do this steps for both web servers, Apache 1 and Apache 2 (or as you named them).

Hostname server

We should be now done with the website hosting in our servers. Apache is handling the incoming HTTP requests and serving the file we wanted. Now, we’ll move to database server where we’ll save our information.


Configuring MySQL database

I’ll finish the tutorial by installing and configuring the database. In this case, we’ll be using MySQL as our database.

MySQL is the most used database in the world as it is powerful, scalable, free and open source. It is great for small and big projects, that’s why it is so used. In this tutorial we’ll be creating a simple CRUD web app that stores students by their first and last name, then displays them in a table. So in this case we”ll just have one table with the student ID, the first name and the last name. The table we’ll create in the database will look like the one below:

Student table

Lets first start by installing MySQL into our database server. Remember to install FTP and SSH to the server as well! We could be using it later.

sudo apt-get update
sudo apt-get -y install mysql-server
sudo systemctl restart mysql
mysql_secure_installation
sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf

Mysql config

sudo mysql_ssl_rsa_setup --uid=mysql
sudo systemctl restart mysql
sudo netstat -anp | grep -mysql

Mysql config

mysql -u root -p
CREATE DATABASE High_School;
USE High_School;
CREATE TABLE students(
   ID INT NOT NULL AUTO_INCREMENT,
   FirstName VARCHAR(255) NOT NULL,
   LastName VARCHAR(255) NOT NULL,
   PRIMARY KEY (ID)
);

Database creation

INSERT INTO students (FirstName, LastName) VALUES ('Gabriel', 'Umana');
INSERT INTO students (FirstName, LastName) VALUES ('John', 'Doe');
SELECT * FROM students;

Data insert

CREATE USER 'teacher'@'192.168.1.37' IDENTIFIED BY 'Y0uRStr0ngP4$swoRD!';
CREATE USER 'teacher'@'192.168.1.56' IDENTIFIED BY 'Y0uRStr0ngP4$swoRD!';
GRANT ALL PRIVILEGES ON High_School.* TO 'teacher'@'192.168.1.56';
GRANT ALL PRIVILEGES ON High_School.* TO 'teacher'@'192.168.1.37';
FLUSH PRIVILEGES
EXIT
sudo ufw allow from 192.168.1.37 to any port 3306
sudo ufw allow from 192.168.1.56 to any port 3306

Testing the database connection

The last part of this tutorial is to test the connection from our web servers to our database server. For this, we’ll need the MySQL client installed in our web servers, next steps show you how to get this done:

sudo apt-get update
sudo apt-get -y install mysql-client

Once installed, test the connectivity to the database server. The following command connects to the database server with the same credentials we used in a few steps back. The IP address I put here is the IP that my database server has, use the command ‘ifconfig’ to get the IP of yours.

mysql -u teacher -h 192.168.1.38 -p

And it should work! I tested it from both web servers, here’s the result in a side by side picture

SQL connection


Final results

This is the end of the tutorial for today. We configured the Apache web servers, SSH, FTP and the database server with all its components. In the next tutorial we’ll be configuring the load balancer and then we’ll configure the web app that shows us the site that we want.

I hope you liked this blog post, found it interesting and learned something! For comments, questions or suggestions feel free to send them through the contact me box at the bottom of the screen here. Thanks and see you in the next entry!

- Gabriel