February 1, 2016
Drupal

Drupal Genesis

Image
Tassy János
Cloud Engineer

Five years after the launch of Drupal 7, we were terribly excited about the launch of Drupal 8 set for November 19 2015. As the release date had been announced a few weeks before, we wanted to join the initiative of the worldwide Drupal 8 celebration (#Celebr8D8 - celebratedrupal8.com) and create a unique surprise for the Drupal community.

Image

We got an invite to take part in a surprise video, but we wanted to do more than that.
 


After a quick brainstorming session, we decided to go back to the origins to see how Drupal had evolved during its 15 years, thanks to the tenacious work of the Drupal community. After a full day spent on research we managed to find all of the previous versions of Drupal. We ‘only’ needed to overcome one challenge: to revive the 10-15 year old codes. 

In this blogpost we are going to show you how we created our Drupal Celebr8 website and what challenges we had to face in the process. 

Infrastructure

Our first task was to plan the necessary infrastructure. We decided to install the different Drupal versions on a separate server with virtualized environment to save time and resources. We also wanted to prepare solutions to different security issues, as it was possible to run into known, but not yet fixed, security problems in the previous Drupal versions. Our goal was to lower the chance of security attacks and minimize the risk of successful penetrations that would cause damage.
 

In the end, we decided to launch a 14.04 Ubuntu instance on Linode (8GB RAM, 6 CPU Cores, 192 GB SSD Storage, 8 TB Transfer, 40 Gbps Network In, 1000 Mbps Network Out) with LXC containers. 

But what is LXC, LXD and why did we choose this solution?

We have a lot of experience with Docker, but we thought we could use a different tool to perform the task. Instead of heavyweight virtualization we decided to go with LXC/LXD, as it seemed to be a perfect fit for our goals, and we wanted to do some experimenting with a promising, yet for us still unknown technology. Docker focuses on a single process, so it handles one responsibility per container. We could have solved the problem with Docker, but it seemed to be more comfortable to use Linux Containers (LXC), as it provided an operating-system-level virtualization. With the help of LXD we couldn't have handled these kind of containers.

The simplified LXD install on a host machine is the following:
 

$ apt-add-repository ppa:ubuntu-lxc/lxd-stable
$ apt-get update
$ apt-get install lxd


How to add officially supported LXC image repositories: 
 

$ lxc remote add images images.linuxcontainers.org


Creating and launching a container named debian-squeeze from the official Debian Squeeze image: 
 

$ lxc launch images:debian/squeeze/amd64 debian-squeeze


You can run commands in separate containers with “lxc exec containername -- command”. You can start working in the given container after running a root shell with the following command: 
 

$ lxc exec $container_name -- bash

How did we do it?

Our basic idea was to put each Drupal version into separate LXC containers with a full LAMP stack. We installed Nginx and Varnish on our (host) server on Linode. LXD provides a virtual network that allows containers to communicate with each other and the host as well. We took care of static IPs by editing the /etc/network/interfaces files in each container. The bold part of the below configuration shows the difference between the default and the modified configurations.
 

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
 address 10.0.3.201
 netmask 255.255.255.0
 gateway 10.0.3.1
 dns-nameservers 8.8.8.8


Nginx (and Varnish, that is taking care of caching beforehand) forwards the requests based on host names to the Apache httpd servers in the containers.
 

To host our sites, our goal was to use trusted and continuously maintained software, but it seemed like an impossible mission, as we had to use PHP4 for the older Drupal versions. Luckily, we found a repository for Debian 6 on the internet that provides PHP4 for the still maintained, but not very up-to-date Debian 6. In regular cases, it's not very smart to use a software from an unknown, untrusted repository, but after a short hesitation, we agreed that it wouldn't be too big of a risk due to the severability of the systems. LXD's snapshot function is a great tool to create many similar containers, thus it was easy to make environments for older Drupals that required PHP4.
 

In order to make Drupal work, we needed to install the proper LAMP environment with some previously mentioned modifications in PHP4 in cases of Drupal versions 1-3. Plus, we needed to direct the webserver on the host machine to the listen port of the web servers in the containers with a simple proxy solution, e.g.
 

  1. # /etc/nginx/conf.d/drupal1.conf
  2. 
     
  3. server {
  4.     listen      127.0.0.1:8080;
  5.     server_name 1.drupal.cheppers.com;
  6. 
     
  7.     location / {
  8.         proxy_pass http://10.0.3.201$uri$is_args$args;
  9.         proxy_set_header Host            $host;
  10.         proxy_set_header X-Forwarded-For $remote_addr;
  11.     }
  12. 
     
  13.     access_log  /var/log/nginx/d1.access.log  main;
  14.     error_log   /var/log/nginx/d1.error.log error;
  15. }

Drupal-related development and problems

As you can see, besides the PHP4 version, we didn't really have any big issues with setting up the infrastructure and it was done quickly. Installing the old Drupal versions was a bigger, more interesting challenge for us.
 

The very first problem we faced was how to recover the old Drupal (or Drop, its former name) versions. We finally managed to do this by checking out the proper commit on the repo (thanks for the tip to NeverGone). 

1.0: 6e88265b1f5fd984c7bff8207e214982ec260e3e
2.0: b156f83be82a64d3c4d2a215e47c6420f48a83e0
3.0: b5e551557b35914203c22a9ae4d99f29287d693b


With the first two versions we encountered two curiosites/problems at install and in the database. One was that the database could be reached by an import of a prefabricated sql file, the other was that the database schema defined in the database/database.mysql file was compatible with the MySQL < 4.1 versions, as the MySQL password() function has changed since.
 

The size of the {users}.passwd field is too small as the new PASSWORD() function generates a 41-characters long hash that is shortened to the length of the field by the MySQL INSERT and UPDATE command without an error message. Thus the search cannot give any results later.
 

We had two options.
Set the old_passwords=1 configuration option on the MySQL >= v4.1 server.
Or ALTER TABLE users MODIFY passwd VARCHAR(41);


We went with the second option.


For Drupal 3, our one and only Andor Dávid, who most of you know as Sweetchuck, wrote his and Cheppers' very first Drupal 3 module. Yes, in 2015! :)
 

The module displayed a block on the right side of the page to facilitate the navigation between our different Drupal version sites.


And why was this needed? After a few minutes of digging the admin interface Andor couldn’t find where he could add new blocks, so he thought he could write a module for that. Previously he went through the code and saw how it should be implemented. This module is only turned on the Drupal3 version as on Drupal 1 and Drupal 2 we could find how to turn a block on. By the time we got to Drupal 3 we knew the way how to do that, but by that time the module was already written. It’s interesting that until Drupal 7 the way to implement blocks didn’t really change.
 

  1. <?php
  2. /**
  3.  * @file
  4.  * Custom navigation block.
  5.  */
  6. 
     
  7. /**
  8.  * Implements hook_block().
  9.  */
  10. function cheppers_block() {
  11.  return array(
  12.    array(
  13.      'info' => 'Drupal versions',
  14.      'subject' => 'Primary links',
  15.      'content' => '<ul><li>' . implode('</li><li>', array(
  16.          '<a href="//1.drupal.cheppers.com">Drupal 1</a>',
  17.          '<a href="//2.drupal.cheppers.com">Drupal 2</a>',
  18.          '<a href="/">Home</a>',
  19.          '<a href="//4.drupal.cheppers.com">Drupal 4</a>',
  20.          '<a href="//5.drupal.cheppers.com">Drupal 5</a>',
  21.          '<a href="//6.drupal.cheppers.com">Drupal 6</a>',
  22.          '<a href="//7.drupal.cheppers.com">Drupal 7</a>',
  23.        )) . '</li></ul>',
  24.    ),
  25.  );
  26. }

Content

The content was put together by many of us as a result of exciting research that went long into the night. The first and most important content for every Drupal site was the given version's changelog. Changelogs are really interesting reads as they showed what has changed in Drupal with time and in what intervals. In addition, we wanted to put together content that referred to the date of each Drupal version's launch, e.g. birth of the Druplicon logo, the time when drupal.org was created, or the DrupalCon in Szeged. Our main goal was to show what Drupals and the published content looked like back then, and what each version featured as function. Thus, we used the original Drupal themes on each site (except for Drupal 3 as it unfortunately ruined the site layout due to low resolution). The funniest was to see how Druplicon has changed in the last 15 years.


How did we do it?

Naturally we hoped that our idea would raise a great interest, but we didn't expect this amount of hype. The central site (that linked to the different Drupal versions) had 30.000 page views by cca. 7000 visitors. Most of the visitors came from Drupal Planet and Twitter, but we had a lot of visitors from our Facebook post, from The Weekly Drop newsletter and from Reddit as well. Most of the visitors live in the USA, then Europe with several countries. We were happy to read many comments from the visitors on the central site.

On Twitter we got 24 likes and 50 retweets (it's hard to tell how many retweets we had overall), and Dries’s tweet about our idea received 36 more retweets as well.
 


In the end we got several thousand more clicks on our Facebook post compared to the average numbers, and we received a few hundred likes, which we didn't expect at all.
 

We wanted our work to be reproducable, so we made the LXC containers available for download.

They are easy to use, you can add them to the reachable images by the “lxc image import /some/directory/drupal-lxc-images/dX/$HASH.tar.gz” command from the downloaded exported LXC images, and you can run a container with the “lxc launch $HASH” command:
 

$ curl https://cheppers.com/sites/default/files/static/dr1-8_containers.tar.gz -o dr1-8_containers.tar.gz
$ tar -xf dr1-8_containers.tar.gz
$ lxc image import $HASH.tar.gz
$ lxc image alias create $NICE_ALIAS $HASH # If you want to add an alias to your container
$ lxc launch $NICE_ALIAS
$ lxc exec $NICE_ALIAS -- bash # You can start working in the given container after running this.


If you want to publish them to live, don't forget to change the database passwords!
 

The hackers didn't sleep of course. During the first hours after launching the sites several dozens of people tried to hack the Drupals (many of our friends actually messaged us and asked why we made it so hard for them). They tried to hack the old sites with ancient techniques: sql injection, inserting JS into watchdog log, etc. We expected this and we changed the general security solutions to a rather drastic one: we simply disabled login on the sites :) 
 

Lastly, it’s worth mentioning that our team had a lot of fun during the creation of this interesting project, and it was great to go back to  'ancient' times. 

Related posts

Image
Image
February 22, 2016
Drupal

In the first part of this series, we showed you how to perform a very basic migration without any customisation. In this part, we will show you how to modify some fields during migration.

Image
Image
February 17, 2016
Drupal

Migrations are becoming a crucial part of Drupal 8, especially in connection with the upcoming Drupal 6 EOL. In this first article, we are going to show you how to quickly migrate your site to Drupal 8.