Call us at 1-888-GTCOMM1
Memcached is, as the name alludes to, a memory caching system used by many high-traffic sites such as Twitter, Reddit and Facebook to cache data in order to speed up their web servers. Memcached is free and open source under the BSD license, so you can install it on your own web server. This guide walks you through the installation process for CentOS 7.
Getting started
For this guide, make sure you have the following:
• 3 Nodes (Cloud Server or Dedicated Server) with a clean installation of CentOS 7.
o 1 small server/cloud instance for HAPROXY
o 2 web servers/cloud instances for Apache, PHP and Memcached
• All commands must be entered in root.
Note: This guide will only show how to set up Memcached on three nodes. One node will be a proxy server, and then there will be two non-replicated apache web servers that each run an instance of Memcached. Each server will be able to access both Memcached instances.
Additionally, to keep this article brief we will use a PHP script in place of a database. A production environment will always include databases.
Step-by-step guide
These are the IP addresses and server names that we’ll be referring to in this guide:
• LB1 173.209.44.219 10.0.0.93 Haproxy
• WEB1 173.209.44.220 10.0.0.94 Apache, PHP, Memcached
• WEB2 173.209.44.221 10.0.0.95 Apache, PHP, Memcached
First, we’ll need to prepare all three servers. Make sure that each is up to date, that Selinux is disabled, and that their firewalls are disabled.
yum -y update
yum -y install nano wget
setenforce 0
sed -i 's/enabled/disabled/' /etc/sysconfig/selinux
systemctl disable firewalld
systemctl stop firewalld
We’ll focus LB1 first. The following basic configuration will suit most applications.
Install the Haproxy package.
yum -y install haproxy
After it’s installed, open the configuration file.
nano /etc/haproxy/haproxy.cfg
Replace its contents with the following:
global
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4096
user haproxy
group haproxy
daemon
stats socket /var/run/haproxy.cmd
defaults
mode http
log global
option httplog
option dontlognull
option httpclose
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 45s
timeout server 45s
timeout check 10s
maxconn 4096
listen http :80
mode http
balance roundrobin
option forwardfor
option httpclose
option http-server-close
timeout http-keep-alive 3000
http-request del-header Proxy
server web1 10.0.0.94:80 weight 1 check inter 3000 rise 2 fall 1
server web2 10.0.0.95:80 weight 1 check inter 3000 rise 2 fall 1
Now start Haproxy and enable it to run on boot.
systemctl enable haproxy.service
systemctl start haproxy.service
Next, we’ll configure the two web servers. First, install the Apache, PHP and Memcached services. Be careful to install php-memecache, not php-memecached.
yum -y install httpd php memcached libmemcached-devel php-pecl-memcache
When that is completed, configure the Apache ServerName directive.
nano /etc/httpd/conf/httpd.conf
For identification purposes, set up a server name that corresponds to your site or infrastructure as follows:
ServerName mysite.com
Finally, verify that PHP was installed properly, and also that the memcache library is installed.
[root@web1 ~]# php -m | grep memcache
memcache
Now it’s time to configure the PHP-memcache module for the distributed PHP sessions. Open the following file in an editor:
nano /etc/php.d/memcache.ini
Replace the contents with:
extension=memcache.so
memcache.maxreclevel=0
memcache.maxfiles=0
memcache.archivememlim=0
memcache.maxfilesize=0
memcache.maxratio=0
memcache.hash_strategy = consistent
memcache.allow_failover = 1
memcache.session_redundancy = 3
Bug alert: The memcache.session_redundancy value should be the sum of your configured memcached instances + 1. Since we are running 2 memcached servers, you must enter 3. This is a known bug in PHP.
We’ll configure PHP to store user sessions in Memcached next. Open your php.ini file:
nano /etc/php.ini
Delete the line “session.save_handler = files” and insert the following lines in its place:
session.save_handler = memcache
session.save_path = "tcp://10.0.0.94:11211, tcp://10.0.0.95:11211"
This next step is a little confusing, but bear with us. On CentOS 7 the RPM package also specifies the session save path in the Apache configuration. It can override our settings from php.ini, so we’ll have to erase it.
nano /etc/httpd/conf.d/php.conf
Delete these lines:
php_value session.save_handler "files"
php_value session.save_path "/var/lib/php/session"
Let’s configure Memcached to listen to each server’s local IP address. We can also configure Memcached’s memory usage at this point. We want each Memcached instance to accept 1024 connections and to have 256MB reserved for cache size. This will allow them to store many thousands of PHP sessions.
Open this configuration file:
nano /etc/sysconfig/memcached
Update the contents like so, making sure that the IP address is the same as what your server has:
PORT="11211"
USER="memcached"
MAXCONN="1024"
CACHESIZE="256"
OPTIONS="-l 10.0.0.95"
Start both services and enable them on boot.
systemctl start httpd
systemctl start memcached
systemctl enable httpd
systemctl enable memcached
Memcached is running, but is it working? We can test our configurations using a script. First, we’ll test the web servers and load balancing to make sure it’s creating sessions and that it’s balancing between the web servers.
Create a new file named session.php inside /var/www/html/session.php with the following content:
cd /var/www/html
nano session.php
<?php
header('Content-Type: text/plain');
session_start();
if(!isset($_SESSION['visit']))
{
echo "Globo.Tech welcomes you as a first time visitor on this server\n";
$_SESSION['visit'] = 0;
}
else
echo "This server has seen you ".$_SESSION['visit'] . " times \n";
$_SESSION['visit']++;
echo "Server IP: ".$_SERVER['SERVER_ADDR'] . "\n";
echo "Client IP: ".$_SERVER['REMOTE_ADDR'] . "\n";
print_r($_COOKIE);
?>
We’ll test PHP session creation and persistence next. By visiting the script we placed earlier via the load balancer IP, we’ll be able to verify that some visits will end up on the server WEB1 while others will end up on WEB2. Check that the session ID remains the same at each load. Also, the amount of visits should increase by one with every refresh.
After visiting the script in your web browser, you can expect the following output:
This server has seen you 30 times
Server IP: 10.0.0.95
Client IP: 10.0.0.93
Array
(
[PHPSESSID] => aeoj7s47i7ke2l0fr7b534v684
)
Upon refresh, you’ll end up on the other web server. The visit counter will become 31 while PHPSESSID remains the same.
This server has seen you 31 times
Server IP: 10.0.0.94
Client IP: 10.0.0.93
Array
(
[PHPSESSID] => aeoj7s47i7ke2l0fr7b534v684
)
Try it out with another browser, too, to verify that this works over multiple sessions.
If we install the utility netcat on LB1, we can query both memcached servers’ stats to see if they’re working correctly and storing sessions.
[root@lb1 ~] yum -y install nmap-ncat
Use this command to see a server’s stats:
[root@lb1 log] echo 'stats' | nc 10.0.0.94 11211
STAT pid 28882
STAT uptime 39
STAT time 1471052900
STAT version 1.4.15
STAT libevent 2.0.21-stable
STAT pointer_size 64
STAT rusage_user 0.016870
STAT rusage_system 0.020244
STAT curr_connections 5
STAT total_connections 104
STAT connection_structures 6
STAT reserved_fds 20
STAT cmd_get 1
STAT cmd_set 195
STAT cmd_flush 0
STAT cmd_touch 0
STAT get_hits 0
STAT get_misses 1
STAT delete_misses 0
STAT delete_hits 0
STAT incr_misses 1
STAT incr_hits 0
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT touch_hits 0
STAT touch_misses 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 10212
STAT bytes_written 2598
STAT limit_maxbytes 268435456
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT threads 4
STAT conn_yields 0
STAT hash_power_level 16
STAT hash_bytes 524288
STAT hash_is_expanding 0
STAT bytes 202
STAT curr_items 2
STAT total_items 195
STAT expired_unfetched 0
STAT evicted_unfetched 0
STAT evictions 0
STAT reclaimed 0
END
And then check the second one:
[root@lb1 log] echo 'stats' | nc 10.0.0.95 11211
STAT pid 28117
STAT uptime 50
STAT time 1471052915
STAT version 1.4.15
STAT libevent 2.0.21-stable
STAT pointer_size 64
STAT rusage_user 0.013569
STAT rusage_system 0.039201
STAT curr_connections 5
STAT total_connections 103
STAT connection_structures 6
STAT reserved_fds 20
STAT cmd_get 97
STAT cmd_set 291
STAT cmd_flush 0
STAT cmd_touch 0
STAT get_hits 96
STAT get_misses 1
STAT delete_misses 0
STAT delete_hits 0
STAT incr_misses 1
STAT incr_hits 96
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT touch_hits 0
STAT touch_misses 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 21822
STAT bytes_written 8479
STAT limit_maxbytes 268435456
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT threads 4
STAT conn_yields 0
STAT hash_power_level 16
STAT hash_bytes 524288
STAT hash_is_expanding 0
STAT bytes 202
STAT curr_items 2
STAT total_items 291
STAT expired_unfetched 0
STAT evicted_unfetched 0
STAT evictions 0
STAT reclaimed 0
END
If you’re following the steps in this guide, you should see that both memcached servers have activity and content stored. Be warned, however, that this is not 100% bulletproof. If one of the two memcached instances ends suddenly, there is no guarantee that both servers contained the same exact sessions. However, the only consequence is that affected users will need to log back in.
The final test for us is to test the failover and replication. Simply shut down either Memcached instance (but not both) and refresh our test script page.
[root@web2 ~] systemctl stop memcached.service
If the visit counter keeps going up after 2-3 refreshs, then failover is active and working.
Let’s next confirm that replication is working. Start up the Memcached instance that you just shut down:
[root@web2 ~] systemctl start memcached.service
Now kill the Memcached instance on the other server.
[root@web1 ~] systemctl stop memcached.service
Re-visit the site. Again, you should see the counter increase every time you refresh. That means replication is also working.
Conclusion
Congratulations on completing the installation! Memcached is an excellent choice for a site that may scale. As your site grows, consider adding more Memcached servers for greater reliability and more caching power. If you found this tutorial useful, feel free to share it with others who may be interested.
