Build your own high availability home data center, part 2!16 min read
Hi! Glad you made it here :)
Thanks for continuing with the tutorial. If you reached here but haven’t yet installed VirtualBox, haven’t configured the VMs or have some doubts in the process, please refer to the first part. Else, we’ll continue working on our ‘Home Data Center’ here. In this tutorial I’ll be going over the next things to configure our environment:
- Apache, PHP and MySQL installation
- FTP and SSH configuration
- Hosting and serving our website with Apache
- Configuring the database
Our first step in this tutorial will be to install SSH to be able to access the servers from the Mac terminal or from PuTTy on Windows. SSH is a cryptographic protocol to securely log into remote devices. It uses TCP port 22 (hence port 22 should be open in the server) and you can install it by running the following commands:
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install openssh-server
After ssh is installed, restart the service so it can start working:
sudo systemctl restart ssh
Next, look for ‘Terminal’ in Mac or PuTTy (download link here) on Windows and login to the server via ssh! From terminal in Mac, type:
$ ssh username@server_ip
From PuTTy just enter the server’s ip, select ssh and use your username and password when prompted:
By installing SSH we are now able to paste all the commands we need into the terminal, no need to type them anymore!
As mentioned in the first part of the tutorial, our main goal is to have a highly available website. This will be done by having a load balancing server configured to efficiently handle the petitions for the website that it gets. The load balancer will get an HTTP request to its IP address and based on its configuration it take the decision of where to send that specific request. Lets move onto the Apache setup. I’ll briefly introduce Apache so you can get a notion of what it is, in case you don’t know.
Apache is a software distributed by the Apache Software Foundation that is freely to use, scalable and the most used web server in the world due to its great performance. Apache runs the http daemon when it runs on Unix systems, such as the ones we are using (Ubuntu). This daemon will be in charge of responding to those http requests that it gets. When the request gets to the server the http daemon will examine the packet. Inside of the ‘Required URL’ part of the packet we’ll find just that, the URL that a client is requesting. Httpd will look for that URL within its directories inside the server and return a response with certain codes. If the site is found then it will return a ‘200 OK’ with the file that was requested, if it didn’t find it then it will return a ‘404 Not found’. The image below shows how this process is done. The image was taken from here and all credits go to them. I wrote a little about HTTP and layer 7 in the OSI model, if you want to read more feel free to follow this link.
Lets proceed with the installation of Apache in both of our servers:
- Log into each Apache server
- From here, run the commands below. The first one will update our working server, then we’ll instal Apache. The third will disable directory listing which can be harmful and is best to disable it for security measures. Lastly, we restart the apache service and set it to start running whenever the server starts:
sudo apt-get update
sudo apt-get install apache2
sudo sed -i "s/Options Indexes FollowSymLinks/Options FollowSymLinks/" /etc/apache2/apache2.conf
sudo systemctl restart apache2.service
sudo systemctl enable apache2.service
With this finished we are ready to test if Apache installed correctly. Run the following command to get the iP address of the web server, then browse for that IP in your browser and you should get a website similar to the one below, congratulating us for correctly enabling Apache!
The file that we are seeing with the ‘Congratulations’ is the index.html that is saved in /var/www/html under our Ubuntu server. Since Ubuntu 14.04 and later, the document root for Apache is found in this directory. When we go to the IP address of the server from within our web browser, we are sending an http request to the server. The httpd picks up this request, browses for the URL requested and sends the website we were looking for. As we are not specifying any site, then it sends us the default website which is index.html:
Analyzing the PC-Server communication
To be able to see this process in a more visual way, I have downloaded a software called WireShark. This is a packet sniffer that allows us to capture the traffic going in and out or NIC and analyze it.
From the image above we can see some other steps that take place before the website is requested. I have filtered the messages to only show those that belong to the IP of the server (192.168.1.37). My PC has the 192.168.1.40 IP address.
As we want the data from the PC to reliably get to the server, HTTP uses TCP as its transport layer protocol. This means that even before sending HTTP data, TCP will first establish a connection with the server.
In networking, this is called a 3-way handshake. Let’s break the process down:
- The host initiating the connection sends a SYN packet with a sequence number of 0.
- The server that gets the packet will check it and see a SYN message, this makes the server allocate resources to create a new connection with a new PC. The server will respond to the SYN with a SYN,ACK; a synchronize message and an acknowledgement of that SYN the PC sent.
- The last part of the 3-way handshake is to acknowledge the receival of the server’s SYN, ACK; with that we are done and have established a connection oriented session with the server!
After this has happened, notice the next packet that is sent is an HTTP GET request for the root site of the server (in this case, the index.html). This packet is encapsulated inside a TCP segment that will be delivered over the connection that we just established. Below is an example of how this packet looks like on the wire:
From this image we can see several important fields in a packet that we as the architects of the data center should be aware of:
- The TCP ports we are using are two: our host’s port is an ephemeral port 50090 so that our PC identifies to which application the incoming replies come. The other port is 80 at the server, 80 is the well known port for HTTP traffic.
- The HTTP information itself is showing a ‘GET’ request to get the root document ( the ‘/’ next to GET represents this root document, which is the index.html) and then it shows the HTTP version being used (HTTP/1.1). This is the end of the data sent by the PC.
Next, we can see 3 acknowledgment packets being sent from the server to the host acknowledging the receival of the HTTP packet we just saw earlier. This means that the server has received the request, a few things happend onwards:
- The packet will be checked by the server, when it reaches the layer 4 data it will see that the destination port is 80, meaning HTTP. The transport layer will then move the packet to the http daemon that handles this in layer 7 and will see that the HTTP GET is to get the root index.html, so httpd will look for that index.html file within the /var/www/html directory and create an HTTP response.
- Analyzing the HTTP response packet we can see that the TCP ports have switched, now the destination port will be the ephemeral PC port and the source is the well known http port. Within the layer 4 segment we see the HTTP part, where an HTTP OK response phrase is sent, telling the PC that everything went fine and it adds an HTTP status code 200 as well. We’ll include the HTML as text within the HTTP packet, this is how we transfer the index.html file back to the PC that requested it.
Changing the index.html
To be able to identify which server we are accessing when the load balancer is configured, it will be useful to have a banner with the hostname of the server. Let’s configure this! Before configuring the index.html site, we’ll need to follow some steps to configure FTP access to the server to be able to push files from our local computer to the server. The steps to take are:
- Log into each server and run the following commands:
sudo apt-get update
sudo apt-get install -y vsftpd
sudo service vsftpd restart
- Download FileZilla FTP client from here, its free and open source.
- Open FileZilla on your PC, in ‘Host’ type the IP address of the server you want to access, enter your username, password and click connect. If you are prompted with a ‘Server doesn’t allow TLS’, just click OK. You should now be able to see the directories inside the server, move to the /var/www/html directory by typing it inside the server’s window:
- We can now see the default index.html inside the /var/www/html directory. Lets delete it from there and create a new one. If we try to delete it from FileZilla we’ll get a ‘Permission Denied’ error. This is because the index.html file was created by root and we do not have the permissions over that file. Lets change the ownership of index.html to us instead of root and delete it from CLI:
sudo chown yourusername: /var/www/html/
rm -rf /var/www/html/index.html
- To be able to upload our new index.html file to the server, we first need to configure write access in the FTP configuration, the following steps show how to do this.
- First, open the file that has the configuration of FTP. This one is located in /etc directory and the file is called vsftpd.conf
sudo nano /etc/vsftpd.conf
- Next, locate the line that has the write access configuration and uncomment it. Just delete the ‘#’ that is in front.
# Uncomment this to enable any form of FTP write command.
- After doing this hit ctrl+o, then hit return, then ctrl+x to go out of edit mode. Then, restart the vsftp daemon
sudo systemctl restart vsftpd
The image below shows how it should now look.
- Now set the html directory permissions to 755
sudo chmod 755 /var/www/html
And lets now code our index! This is gonna be pretty simple. We want our index.html to be just a big title that shows the name of the server we are logged into, this way when we set the load balancer later on we can see the changes between the servers. Below is the code, copy it and paste it into a file and call it index.php. Notice that we are using a .php file instead of .html, this is because we’ll be fetching the server’s hostname via php and then this file will be displayed to the user.
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
$hostname = gethostname();
echo "<h1>" . $hostname . "</h1>";
Save this index.php into your desktop, then look for it inside FileZilla and upload it to the server. Once that is done, remember to set the permissions of the file to 755! The directory where this file should be is /var/www/html/
sudo chmod 755 /var/www/html/index.php
We are done for now. Let’s move to another part.
Now that we have changed the index file to a php file, we want the php to be able to run. For this, we have to download and install php into the server.
- We’ll use php 7, we need to tell ubuntu from which repository to get php, this can be done by executing in the terminal:
sudo apt-get install software-properties-common
sudo add-apt-repository ppa:ondrej/php
sudo apt update
- Then, we install all the necessary php files. This is one command, make sure to type it correctly!
sudo apt install -y php7.1 libapache2-mod-php7.1 libapache2-mod-php7.1 php7.1-common php7.1-mbstring php7.1-xmlrpc php7.1-soap php7.1-gd php7.1-xml php7.1-intl php7.1-mysql php7.1-cli php7.1-mcrypt php7.1-ldap php7.1-zip php7.1-curl
- Php installation is done. Lets just configure it, first open the php.ini file with nano:
sudo nano /etc/php/7.1/apache2/php.ini
- After opening it, modify the following parameters. Look for the parameters and set them as they are below. I am using my timezone, if you are in a different place than mine refer to this link to get yours.
file_uploads = On
allow_url_fopen = On
memory_limit = 256M
upload_max_filesize = 64M
max_execution_time = 360
date.timezone = America/Costa_Rica
To go back to the terminal once you are done editting the php.ini file remember the process is:
- ctrl+o to save, then hit return
- ctrl+x to exit
Now that PHP is installed and configured, go to your web browser and search for the IP address of the server. We should now see the hostname! Remember to do this steps for both web servers, Apache 1 and Apache 2 (or as you named them).
We should be now done with the website hosting in our servers. Apache is handling the incoming HTTP requests and serving the file we wanted. Now, we’ll move to database server where we’ll save our information.
Configuring MySQL database
I’ll finish the tutorial by installing and configuring the database. In this case, we’ll be using MySQL as our database.
MySQL is the most used database in the world as it is powerful, scalable, free and open source. It is great for small and big projects, that’s why it is so used. In this tutorial we’ll be creating a simple CRUD web app that stores students by their first and last name, then displays them in a table. So in this case we”ll just have one table with the student ID, the first name and the last name. The table we’ll create in the database will look like the one below:
Lets first start by installing MySQL into our database server. Remember to install FTP and SSH to the server as well! We could be using it later.
- Run the following commands to install the database, quick and easy
sudo apt-get update
sudo apt-get -y install mysql-server
- After running the above commands, a dialog message will prompt you to enter a password for ‘root’, give it a strong password and continue. When this is done, restart the service
sudo systemctl restart mysql
- Next, run the following command on the shell. This will start the MySQL script to secure the database, allowing us to secure our content. As we already set a root password, you can skip that step by hitting return. Root access should only be allowed from ‘localhost’ and not from the network, so when asked to disallow root login remotely enter ‘y’. All other questions can be answered easily.
- MySQL is up! We just need to tweak some other configuration settings to allow incoming connections to the sql port, this will permit our web servers to communicate to our database server and be able to perform DML operations in the database. The next step will be just that. We’ll configure the database to listen for incoming connections to mysql, which by default the port used is 3306. Run the following command to open up the MySQL configuration file
sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf
- Next, hit ctrl+w to search and type ‘bind’, then hit enter. This will find the ‘bind-address’ parameter. Change it to 0.0.0.0, this will allow any IP to connect to our database server. After that, type ‘require_secure_transport = on’, this will encrypt the communications between the database server and the web servers, making it harder for anyone in the LAN to be able to get the data that is being sent. The configuration should look like the image below, remember to hit ctrl+o to save, then hit return, then ctrl+x.
- Then, create the keys and certificates MySQL will use to allow SSL connections to the server. This can be done by just entering the following command, MySQL will do the configuration by itself.
sudo mysql_ssl_rsa_setup --uid=mysql
- Now you may be thinking, isn’t it insecure to allow every IP to connect to our database? Yes, it is. But unfortunately, we cannot specify multiple IPs in our bind-address parameter within the mysql configuration. For this reason, we’ll be making this secure by allowing just the IPs that we want to be able to connect to port 3306 through the firewall. The next steps will show this. First go ahead and restart the mysql daemon, then check if it is listening for any IP connections on port 3306
sudo systemctl restart mysql
sudo netstat -anp | grep -mysql
- We are ready! MySQL is now installed, configured and listening for incoming connections to serve. We are just missing some more stuff, which is the database configuration itself (the database that will have the tables with our student into) and the firewall configuration. Lets move on to that. First, connect to the local database with your root user. After typing this command and hitting enter you’ll be asked for that root password that we setup a few minutes ago.
mysql -u root -p
- Once in the MySQL engine, proceed with the database creation, start using that database and create the table we previously mentioned
CREATE DATABASE High_School;
CREATE TABLE students(
ID INT NOT NULL AUTO_INCREMENT,
FirstName VARCHAR(255) NOT NULL,
LastName VARCHAR(255) NOT NULL,
PRIMARY KEY (ID)
- Insert some data and select it just to prove that everything is working right!
INSERT INTO students (FirstName, LastName) VALUES ('Gabriel', 'Umana');
INSERT INTO students (FirstName, LastName) VALUES ('John', 'Doe');
SELECT * FROM students;
- To be able to connect to mysql from our webservers we’ll need to configure a user and the ip that the webservers will be using. The next steps show this. You’ll need the IP addresses of the two web servers, if the IPs change the servers won’t be able to connect to the database. Please keep in mind this, it would be good to set the web servers IPs to static. After configuring the users, give the users some privileges so they can work
CREATE USER 'teacher'@'192.168.1.37' IDENTIFIED BY 'Y0uRStr0ngP4$swoRD!';
CREATE USER 'teacher'@'192.168.1.56' IDENTIFIED BY 'Y0uRStr0ngP4$swoRD!';
GRANT ALL PRIVILEGES ON High_School.* TO 'teacher'@'192.168.1.56';
GRANT ALL PRIVILEGES ON High_School.* TO 'teacher'@'192.168.1.37';
- We are done with the MySQL configuration! We now just need to allow the web server IPs through the firewall:
sudo ufw allow from 192.168.1.37 to any port 3306
sudo ufw allow from 192.168.1.56 to any port 3306
Testing the database connection
The last part of this tutorial is to test the connection from our web servers to our database server. For this, we’ll need the MySQL client installed in our web servers, next steps show you how to get this done:
sudo apt-get update
sudo apt-get -y install mysql-client
Once installed, test the connectivity to the database server. The following command connects to the database server with the same credentials we used in a few steps back. The IP address I put here is the IP that my database server has, use the command ‘ifconfig’ to get the IP of yours.
mysql -u teacher -h 192.168.1.38 -p
And it should work! I tested it from both web servers, here’s the result in a side by side picture
This is the end of the tutorial for today. We configured the Apache web servers, SSH, FTP and the database server with all its components. In the next tutorial we’ll be configuring the load balancer and then we’ll configure the web app that shows us the site that we want.
I hope you liked this blog post, found it interesting and learned something! For comments, questions or suggestions feel free to send them through the contact me box at the bottom of the screen here. Thanks and see you in the next entry!