A look into the challenges of supporting a VMOD in different versions of Varnish
Recently 51Degrees took on a project to provide support for as many Varnish versions as possible. In light of Varnish 6.1 being released, we explored some of the issues faced with supporting the newer versions while providing backwards compatibility.
Although Varnish has excellent usage documentation on their site, one of the challenges was the lack of information relating to changes or deprecation of methods between versions. This blog will help you to understand what changes were made to the 51Degrees Varnish API and help others trying to support their own VMOD.
Which Version?
Varnish verisons 4, 5 and 6 are all being updated in parallel. The first step is to pinpoint which versions of Varnish work and which do not. The task is then to identify a fix at each stage and finally provide a solution that collates them all together into a unified build process.
Version 4.1.5+
With the help from the Open Source community it was discovered that the 'Init' method defined in the 51Degrees VCC template is no longer usable and must be changed to an 'Event'. This method of course requires some changes to the C code.
vmod_fiftyonedegrees.vcc
$Init init_function
$Event init_function
An important thing to note for the event is that it will run on more events than just the module's LOAD event. You can specify which events you want your method to run on by using a switch.
vmod_fiftyonedegrees.c
int init_function(
const struct vrt_ctx *ctx,
struct vmod_priv *priv,
enum vcl_event_e e)
{
switch (e) {
case VCL_EVENT_LOAD:
priv->priv = malloc(sizeof(fiftyoneDegreesProvider));
global.provider = (fiftyoneDegreesProvider*) priv->priv;
priv->free = privProviderFree;
return (0);
default:
return (0);
}
return (0);
}
In the example it allows memory allocation for the provider to only be done on LOAD as opposed to other events such as USE, DISCARD or FINI.
Additional investigation will need to be carried out now that a fix has been discovered:
- Which version will this work up to?
- Will the currently supported versions continue to build successfully?
From testing you will find that this fix will work up until 6 and currently supported versions will no longer work. This means the original `INIT` method must be available and executed depending on the version of Varnish installed.
Version Checks
A short Python script can be added to the repository that extracts the Version number installed with Varnish. This can then be passed as a variable to build the VCC template and VMOD C code accordingly.
We now have a method of fixing most later releases as well as a method of collating them into a single VMOD.
Python tools are required to build varnish from source, so it is safe to assume anyone using the API will have it installed.
#!/usr/bin/python
import sys
default = "040100"
if len(sys.argv) > 1:
versionOut = "";
versionIn = sys.argv[1].split('.')
for i in range(3):
if len(versionIn) > i:
if len(versionIn[i]) < 2:
versionOut += "0" + versionIn[i]
else:
versionOut += versionIn[i]
else:
versionOut += "00"
print versionOut
else:
print default
Version 6+
This is a simple fix. There have been some consolidating changes to the header files in Varnish which means that changes are required to the include references for the VMOD’s C code.
`cache.h` now includes a reference to `vrt.h` so it cannot be referenced again in the 51Degrees C file.
#ifdef FIFTYONEDEGREES_VARNISH_VERSION
#if FIFTYONEDEGREES_VARNISH_VERSION < 060000
#include "vrt.h"
#include "vcl.h"
#endif
#else
#error "FIFTYONEDEGREES_VARNISH_VERSION was not defined. Define the format MMmmRR or build using configure which does this automatically."
#endif
#include "cache/cache.h"
With the use of the Python script mentioned above and change to the build process we now have 3 builds working with each other.
The full list of changes made as part of this project can be found within the latest release of our GitHub repository for those interested.