How To Use Varnish As A Highly Available Load Balancer On Ubuntu 20.04 With SSL

by Janeth Kent Date: 25-04-2021 varnish cache linux ubuntu balancer apache


Load balancing with high availability can be tough to set up. Fortunately, Varnish HTTP Cache server provides a dead simple highly available load balancer that will also work as a caching server.

The modern use of SSL/TLS for all traffic has made this a little harder as Vanish has to handle unencrypted traffic to cache it. This means that we will need to terminate and decrypt the HTTPS connections before they are handed off to Varnish. We will do this with Apache2.

This means that the HTTPS requests will arrive at the Varnish server and get terminated by Apache2. Apache2 will then pass them on to the Varnish server for caching and distributing to the web front ends.

This guide will use the following three servers:

Function Name IP Listen Port
Varnish load balancer varnish 1.1.1.1 443
Web server web1 2.2.2.2 80
Web server web2 3.3.3.3 80

You should already have web servers configured to serve your site over HTTP (port 80)on your web backends.

I recommend not attaching the web servers to the internet as they are not using HTTPS. Attach all the server’s onto a private network and configure the webservers to only listen to HTTP traffic on the private interfaces.

Install Varnish and Apache2

Log into your Ubuntu 20.04 server that you want to use as the load balancer and install Varnish and Apache2 with apt :

apt install varnish apache2

Configure Apache2

First, enable the Apache2 modules that we will need:

a2enmod proxy
a2enmod proxy_balancer
a2enmod proxy_http
a2enmod ssl

Then restart Apache2

systemctl restart apache2.service

Next, create a VirtualHost file that will accept the HTTPS connections on the public IP address on port 443. Place this file into /etc/apache2/sites-available :

<VirtualHost *:443>
ServerName             <DOMAIN>
ErrorLog               /var/log/apache2/<DOMAIN>-https_error.log
CustomLog              /var/log/apache2/<DOMAIN>-https_access.log combined
SSLEngine on
SSLCertificateFile     <PATH/TO/CERT>/<DOMAIN>.crt
SSLCertificateKeyFile  <PATH/TO/KEY>/<DOMAIN>.key
ProxyPreserveHost       On
ProxyPass               / http://127.0.0.1:8080/
ProxyPassReverse        / http://127.0.0.1:8080/
</VirtualHost>

You will need to edit this to match your domain.

As you can see, you need to get an SSL certificate for your website. If you already have this then edit the SSLCertificateFile and SSLCertificateKeyFile lines to point to your certificate’s files.

Now, enable the new VirtualHost file:

a2ensite <VIRTUALHOST FILE>

And restart Apache2

systemctl restart apache2

Apache2 is now configured to terminate the HTTPS requests and pass them off to Varnish which will listen on 127.0.0.1:8080 for HTTP requests from Apache2.

Configure Varnish

The first job is to configure Varnish to listen on 127.0.0.1:8080 . This is done by modifying the start up parameters that are given to systemd.

Fist, create the following directory:

mkdir /etc/systemd/system/varnish.service.d

Next, create and edit this file /etc/systemd/system/varnish.service.d/override.conf with the following contents:

[Service]
ExecStart=
ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a 127.0.0.1:8080 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m

Next, reload systemd:

systemctl daemon-reload

Now that Varnish is listening on the correct port and IP you can create the load balancing configuration. Begin by moving to /etc/varnish/ then rename to supplied configuration file:

mv default.vcl default.vcl.origional

Then create and edit a new default.vcl file by opening it with a text editor:

nano default.vcl

Then copy and past the following configuration:

vcl 4.0;
import directors;
backend web1 {
.host = "104.248.172.77";
.port = "80";
.probe = {
.url = "/";
.timeout = 1s;
.interval = 5s;
.window = 5;
.threshold = 3;
}
}
backend web2 {
.host = "165.232.104.211";
.port = "80";
.probe = {
.url = "/";
.timeout = 1s;
.interval = 5s;
.window = 5;
.threshold = 3;
}
}
sub vcl_init {
new balancer = directors.round_robin();
balancer.add_backend(web1);
balancer.add_backend(web2);
}
sub vcl_recv {
set req.backend_hint = balancer.backend();
}

Let’s break down these configuration blocks. The first two sections define the web backends:

backend web1 {
.host = "2.2.2.2";
.port = "80";
.probe = {
.url = "/";
.timeout = 1s;
.interval = 5s;
.window = 5;
.threshold = 3;
}
}

The .host can the web server’s IP address or a domain name that resolves to it. The .probe section is the health check that Varnish performs to determine if the webserver is online. It checks every 5 seconds that it can get an HTTP response within 1 second. If that fails Varnish will consider it offline and route traffic to the other backends.

Varnish will continue to probe the server and when it comes back online Varnish will direct traffic to it again.

The second section:

sub vcl_init {
new balancer = directors.round_robin();
balancer.add_backend(web1);
balancer.add_backend(web2);
}

Tells Varnish to create a load balancer called balancer . The traffic is divided among the backends by round_robin which means that web requests will be sent to the backends in turn.

The last section:

sub vcl_recv {
set req.backend_hint = balancer.backend();
}

routes all inbound traffic to the load balancer.

Finally, restart Varnish:

systemctl restart varnish.service

Testing

First, check that Varnish can communicate with the backends:

$ varnishadm backend.list
Backend name                   Admin   Probe      Last change
boot.web1                      probe   5/5 good   Mon, 07 Dec 2020 14:30:40 GMT
boot.web2                      probe   5/5 good   Mon, 07 Dec 2020 14:30:40 GMT
boot.balancer                  probe   healthy    Mon, 07 Dec 2020 14:30:40 GMT

Stop Apache2 on one of the webservers, wait a few seconds and try again:

$ varnishadm backend.list
Backend name                   Admin   Probe      Last change
boot.web1                      probe   1/5 bad    Mon, 07 Dec 2020 15:09:15 GMT
boot.web2                      probe   5/5 good   Mon, 07 Dec 2020 15:07:15 GMT
boot.lb                        probe   healthy    Mon, 07 Dec 2020 15:07:15 GMT

Varnish has detected that web1 is down and is now ignoring it. You can now restart Apache2 and watch Varnish accept it back into the cluster.

I also recommend putting different index.html pages on the webservers during testing so you can tell where the page has been loaded from.

 
by Janeth Kent Date: 25-04-2021 varnish cache linux ubuntu balancer apache hits : 392  
 
Janeth Kent

Janeth Kent

Licenciada en Bellas Artes y programadora por pasión. Cuando tengo un rato retoco fotos, edito vídeos y diseño cosas. El resto del tiempo escribo en MA-NO WEB DESIGN END DEVELOPMENT.

 
 
 

Related Posts

How to install a Linux partition on a Windows 10 PC

In spite of a past we could say almost confronted, the approach between Windows and Linux is accelerating more and more, drawing a story closer to love than to hate.…

WSL2 is released to run Linux distributions on Windows

If you are reading about this for the first time, the Windows Subsystem for Linux is a kind of virtual machine that allows you to run the Linux terminal on…

How to move Chrome, Firefox or Edge cache to save space

Caching is a technique used by many programs to improve their performance, especially for applications that run over the Internet, such as streaming programs or web browsers. This technique consists…

Linux For Dummies: Permissions

In the previous articles I made a short introduction to the Unix world and in the following article I have dealt with the basic commands for the file system management. Today we are…

Linux for Dummies: Ubuntu Terminal

I introduced in the previous article, available here, the basic concepts concerning the Linux world. Today we are going to have a look to some basic operations that we can perform…

Linux for Dummies: Introduction

If you have thought about migrating from Windows to a Unix operating system, or Linux specifically there are things you should know. The goal is to give essential information (and…

The Best RSS Readers for Ubuntu

Even if most of the tech experts actively claim that RSS (Rich Site Summary) is dead especially after Google Reader was discontinued 5 years ago but it isn’t yet as…

80 Linux Network Monitor Software & Tools for Managing & Monitoring Unix/Linux Systems

It’s hard work monitoring and debugging Linux performance problems, but it’s easier with the right tools at the right time. Finding a Linux Network Monitor tool or Software package for…

How to install Letsencrypt Certificates with Certbot in Ubuntu

In this article we will explain how to install, manage and configure the SSL Security certificate, Let's Encypt in NGINX server used as proxy. This certificate is free but does…

How to Set up a Fully Functional Mail Server on Ubuntu 16.04 with iRedMail

Setting up your own mail server from scratch on Linux is complex and tedious, until you meet iRedMail. This tutorial is going to show you how you can easily and…

GIMP 2.10 released: Features 32-bit support, new UI and A Ton Of Improvements

It's been over a half-decade since the GIMP 2.8 stable debut and today marks the long-awaited release of GIMP 2.10, its first major update in six years. And among other…

Setting Up SFTP on Ubuntu 16.04

I recently had a request to setup SFTP for a customer so they could manage a set of files in their environment through an FTP GUI. Being an avid user…

We use our own and third-party cookies to improve our services, compile statistical information and analyze your browsing habits. This allows us to personalize the content we offer and to show you advertisements related to your preferences. By clicking "Accept all" you agree to the storage of cookies on your device to improve website navigation, analyse traffic and assist our marketing activities. You can also select "System Cookies Only" to accept only the cookies required for the website to function, or you can select the cookies you wish to activate by clicking on "settings".

Accept All Only sistem cookies Configuration