Wednesday, June 1, 2011

MySQL Master/Slave Replication


1. Setup Overview

Here is a quick setup of a simple MySQL master/slave replication using SLES 11.1 on two 64-bit x86 systems.



2. Common Setup on Master and Slave Servers

MySQL Server 5.0.67 comes pre-installed with the SLES-11.1 operating system.

Firstly, ensure that MySQL server service always runs on both servers in the desired run level. To verify this, execute the following command.

chkconfig --list | grep mysql

If this results in ‘off’ status for all the run levels, then turn on ‘mysql’ to run always in run level 3, 4, and 5, by executing the following command.

chkconfig -s mysql 345

Check if ‘/var/lib/mysql’ folder exists. This is the folder where all the databases will be stored. If yes, ensure that the ownership of this folder if to user ‘mysql’ and group ‘mysql’.

Change ownership of these folders for user ‘mysql’.

chown -Rf mysql:mysql /var/lib/mysql

Start the MySQL service by the running the following command.

/etc/init.d/mysql start

Set the MySQL administrator password for the first time.

mysqladmin -u root password 'mypasswd'

Before proceeding to server specific settings, ensure that the above common configuration is complete on both the servers.

3. MySQL Master Configuration

On the replication ‘master’, binary logging must be enabled and a unique server ID should be established. Edit the MySQL configuration file, ‘/etc/my.cnf’ to reflect the following changes under ‘[mysqld]’ section.

Uncomment line 53 to read as,

log-bin=mysql-bin

Ensure that the line containing ‘server-id’ is uncommented and equals to ‘1’.

server-id=1

After making the changes, restart the service.

/etc/init.d/mysql restart

MySQL ‘slave’ must connect to the ‘master’ using a MySQL user name and password, so there must be a user account on the ‘master’ that the ‘slave’ can use to connect. The account also needs the ‘REPLICATION SLAVE’ privilege, as it is solely used for replication purpose. Issue these statements on the ‘master’.

mysql -p

mysql> CREATE USER 'slvusr'@'%' IDENTIFIED BY 'myslvpwd';

mysql> GRANT REPLICATION SLAVE ON *.* TO 'slvusr'@'%';

To configure replication on the ‘slave’ we must determine the master's current coordinates within its binary log. We will need this information so that when the ‘slave’ starts the replication process, it is able to start processing events from the binary log at the correct point.

To obtain the ‘master’ binary log coordinates, execute the following commands at the 'mysql' prompt.

mysql> SHOW MASTER STATUS;
+------------------+----------+--------------+------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+------------------+----------+--------------+------------------+
| mysql-bin.000001 |      455 |              |                  |
+------------------+----------+--------------+------------------+
1 row in set (0.00 sec)

For clear understanding, paste the above output in a text editor that supports UNIX type output. It will display a proper table structure with the status information.

As it is a fresh MySQL setup that has the default set of databases, there is no need to take a backup or dump any database. We can now configure the ‘slave’ server.

4. MySQL Slave Configuration

On the replication slave, we must establish a unique server ID. Edit the MySQL configuration file, ‘/etc/my.cnf’ to reflect the following changes under ‘[mysqld]’ section.

Ensure that the line containing ‘server-id’ is uncommented and equals to ‘2’.

server-id=2

After making the changes, restart the MySQL service.

/etc/init.d/mysql restart

To set up the ‘slave’ to communicate with the ‘master’ for replication, we must tell the ‘slave’ the necessary connection information. To do this, execute the following statement on the ‘slave’.

mysql -p

mysql> CHANGE MASTER TO MASTER_HOST='master.mydomain.com', MASTER_USER='slvusr', MASTER_PASSWORD='myslvpwd', MASTER_LOG_FILE='mysql-bin.000001', MASTER_LOG_POS=455;

Notice that the ‘MASTER_LOG_FILE’ and ‘MASTER_LOG_POS’ values are obtained from the ‘master’ as shown in the steps above. Now, start the slave thread.

mysql> START SLAVE;

After this procedure is performed, the ‘slave’ should connect to the ‘master’ and catch up on any updates that have occurred since the snapshot was taken. To check the ‘slave’ status, run the following command from the ‘mysql’ prompt.

mysql> SHOW SLAVE STATUS;
+----------------------------------+----------------------+-------------+-------------+---------------+------------------+---------------------+-------------------------+---------------+-----------------------+------------------+-------------------+-----------------+---------------------+--------------------+------------------------+-------------------------+-----------------------------+------------+------------+--------------+---------------------+-----------------+-----------------+----------------+---------------+--------------------+--------------------+--------------------+-----------------+-------------------+----------------+-----------------------+
| Slave_IO_State                   | Master_Host          | Master_User | Master_Port | Connect_Retry | Master_Log_File  | Read_Master_Log_Pos | Relay_Log_File          | Relay_Log_Pos | Relay_Master_Log_File | Slave_IO_Running | Slave_SQL_Running | Replicate_Do_DB | Replicate_Ignore_DB | Replicate_Do_Table | Replicate_Ignore_Table | Replicate_Wild_Do_Table | Replicate_Wild_Ignore_Table | Last_Errno | Last_Error | Skip_Counter | Exec_Master_Log_Pos | Relay_Log_Space | Until_Condition | Until_Log_File | Until_Log_Pos | Master_SSL_Allowed | Master_SSL_CA_File | Master_SSL_CA_Path | Master_SSL_Cert | Master_SSL_Cipher | Master_SSL_Key | Seconds_Behind_Master |
+----------------------------------+----------------------+-------------+-------------+---------------+------------------+---------------------+-------------------------+---------------+-----------------------+------------------+-------------------+-----------------+---------------------+--------------------+------------------------+-------------------------+-----------------------------+------------+------------+--------------+---------------------+-----------------+-----------------+----------------+---------------+--------------------+--------------------+--------------------+-----------------+-------------------+----------------+-----------------------+
| Waiting for master to send event | master.mydomain.com | slvusr   |        3306 |            60 | mysql-bin.000001 |                 455 | mysqld-relay-bin.000004 |           235 | mysql-bin.000001      | Yes              | Yes               |                 |                     |                    |                        |                         |                             |          0 |            |            0 |                 455 |             235 | None            |                |             0 | No                 |                    |                    |                 |                   |                |                     0 |
+----------------------------------+----------------------+-------------+-------------+---------------+------------------+---------------------+-------------------------+---------------+-----------------------+------------------+-------------------+-----------------+---------------------+--------------------+------------------------+-------------------------+-----------------------------+------------+------------+--------------+---------------------+-----------------+-----------------+----------------+---------------+--------------------+--------------------+--------------------+-----------------+-------------------+----------------+-----------------------+
1 row in set (0.00 sec)

For clear understanding, paste the above output in a text editor that supports UNIX type output. It will display a proper table structure with the status information.

5. Verifying the Master Slave Replication

Create a database on the ‘master’ server. We will verify if that database gets replicated over to the ‘slave’ automatically.

mysql -p

mysql> create database vivek;
Query OK, 1 row affected (0.00 sec)

mysql> use vivek;
Database changed

mysql> create table dummy(name varchar(20),id varchar(20));
Query OK, 0 rows affected (0.02 sec)

mysql> insert into dummy values('test','data');
Query OK, 1 row affected (0.00 sec)

mysql> select * from dummy;
+------+------+
| name | id   |
+------+------+
| test | data |
+------+------+
1 row in set (0.00 sec)

Connect to the ‘slave’ server and verify whether the database created on ‘master’ has synchronized.

mysql -p

mysql> use vivek;
Database changed

mysql> select * from dummy;
+------+------+
| name | id   |
+------+------+
| test | data |
+------+------+
1 row in set (0.00 sec)

Configuration of MySQL master slave automatic replication of database transactions is now complete.

Wednesday, May 11, 2011

Tomcat Load Balancer using Apache 'mod_proxy_balancer'


1. Setup Overview:

A simple load balancing setup between two Tomcat web servers using ‘mod_proxy’ and ‘mod_proxy_balancer’ Apache modules.

Both the web servers are hosted on SLES 11.1.





















2. Tomcat Setup

Apache Tomcat doesn’t come pre-installed with SLES-11.1 OS. Follow the steps below to install Tomcat on both the servers under ‘/webserver’ folder.

a. Download Apache Tomcat tarball ‘apache-tomcat-6.0.32.tar.gz’, from the Tomcat site.
b. Extract Tomcat tarball under ‘/webserver’ folder. The extracted folder will be ‘apache-tomcat-6.0.32’.
d. Create a soft link to the extracted tomcat folder in the same location, by executing the following command,

ln -s /webserver/apache-tomcat-6.0.32 /webserver/active-tomcat
e. Go into the bin directory of tomcat and start tomcat by executing the following script,

./startup.sh
f. Tomcat will now be up and running. You can verify this by accessing the following URL from your web browser, ‘http://tomcat-A:8080/’ or ‘http://tomcat-B:8080/’.
g. To auto start Tomcat during boot time, configure it as mentioned below.
(i) Copy the following content into a new file by the name ‘tomcat’ under ‘/etc/init.d’.

#!/bin/bash
#
# description: Start up the Tomcat servlet engine.

RETVAL=$?
CATALINA_HOME="/webserver/active-tomcat"

case "$1" in
start)
if [ -f $CATALINA_HOME/bin/startup.sh ];
then
echo $"Starting Tomcat"
$CATALINA_HOME/bin/startup.sh
fi
;;
stop)
if [ -f $CATALINA_HOME/bin/shutdown.sh ];
then
echo $"Stopping Tomcat"
$CATALINA_HOME/bin/shutdown.sh
fi
;;
restart)
$0 stop
sleep 5
$0 start
;;
*)
echo $"Usage: $0 {start|stop|restart}"
exit 1
;;
esac

exit $RETVAL

(ii) Save the file above.
(iii) Change the permission of this script and make it executable. Run the following command to do so.
chmod 754 /etc/init.d/tomcat
(iv) Enable the script to start in the run levels 3, 4, and 5, by executing the following command.

chkconfig -s tomcat 345

3. Apache Configuration:

You can have Apache configured on a separate server as shown in the diagram above. The Apache version used here is 2.2.10.

Firstly, ensure that Apache service always runs in the desired run level. To verify this, execute the following command.

chkconfig --list | grep apache2

If this results in ‘off’ status for all the run levels, then turn on ‘apache2’ to run always in run level 3, 4, and 5, by executing the following command.

chkconfig -s apache2 345

Start the Apache service by the running the following command.

/etc/init.d/apache2 start

We use Apache's 'mod_proxy_balancer' module to configure the load balancer. This configuration is useful for applications that are stateless and therefore do not require clustering or sticky-sessions.

We need to initially load the 'mod_proxy' and 'mod_proxy_balancer' module. To use the Balancer Manager (for finding out load balancing statistics), the 'mod_status' module is loaded as well.

Execute the following commands to load the above modules.

a2enmod proxy
a2enmod proxy_balancer
a2enmod status

To verify whether the modules are loaded, run the following command.

apache2ctl -t -D DUMP_MODULES

In the output you should be able to locate the above loaded modules.

Later in the Apache2 configuration file, '/etc/apache2/httpd.conf', append the following lines to configure the balancer.

# Load Balancing Tomcat Servers on tomcat-A and tomcat-B
<IfModule mod_proxy_balancer.c>
  <Location "/balancer-manager">
    SetHandler balancer-manager
  </Location>


  <Proxy balancer://mycluster>
    BalancerMember http://tomcat-A:8080/ loadfactor=1
    BalancerMember http://tomcat-B:8080/ loadfactor=1
  </Proxy>


  ProxyPass /lb-test balancer://mycluster
</IfModule>

Now restart Apache to start the load balancer.

/etc/init.d/apache2 restart

The service should be up and running without any errors.


4. Verifying the Load Balancer

To verify the working of the load balancer, change the ‘/webserver/active-tomcat/webapps/ROOT/index.html’ configuration file in Tomcat on both the servers.

For the configuration file on tomcat-A update the following line (~ line 97),

Apache Tomcat

as,

Apache Tomcat - tomcat-A

Similarly, for the configuration file on tomcat-B update the same line as,

Apache Tomcat - tomcat-B

Restart tomcat web server on both the servers, by executing the following commands from their ‘bin’ folders.

./shutdown.sh
./startup.sh

Tomcat should be up and running on port 8080 on the respective server. Access the Tomcat URL on tomcat-A ‘http://tomcat-A:8080/’, from your web browser. This should show up ‘Apache Tomcat - tomcat-A’ on the index page. Similarly when you access ‘http://tomcat-B:8080/’, from your web browser, the index page should show up ‘Apache Tomcat - tomcat-B’.

Access the following URL to verify the load balancer statistics, ‘http://apache-proxy/balancer-manager’. This should show column ‘Factor’ set as ‘1’ and ‘Method’ as ‘byrequests’, that means the load should be evenly balanced in a round-robin fashion. The ‘Elected’ column shows the value as ‘0’ after Apache is configured for Load Balancing and restarted.

Access the balancer URL to verify the Tomcat Load Balancing between ‘tomcat-A’ and ‘tomcat-B’, ‘http://apache-proxy/lb-test/’. Keep refreshing this web page; you will notice that the browser alters displaying pages containing ‘Apache Tomcat - tomcat-A’ and ‘Apache Tomcat - tomcat-B’. To understand the statistics of the load balancer, again access the balancer statistics page, ‘http://apache-proxy/balancer-manager’. This will now show values in the ‘Elected’ column. The value will either be the same for both the worker URL’s or one less than the other. This indicates that the load is getting evenly balanced across both the servers.

Now, stop the Tomcat web server on ‘tomcat-B’, and now check the balancer URL, ‘http://apache-proxy/lb-test/’. The page will display only ‘Apache Tomcat - tomcat-A’, because ‘tomcat-B’ is no more serving the web page. Also verify the balancer statistics page, ‘http://apache-proxy/balancer-manager’. This will show the ‘Status’ column for ‘tomcat-B’ worker URL as ‘Err’, that means the worker is not serving the pages on that URL. If you keep refreshing the balancer URL again and again, the statistics page will show that ‘tomcat-A’ will always get ‘Elected’ to serve the pages as its value keeps increasing.

If you start Tomcat on ‘tomcat-B’ again, the balancer setup will be back to normal and serve as expected.