Create a Wikifarm

On Wikifarms

What is a Wikifarm?

Wikifarm and the verb Wikifarming refer to the setup and operation of multiple wikis on a single server. This could be because on a single website you want to run multiple wikis (e.g. http://www.saruman.biz/linuxwiki and http://www.saruman.biz/bsdwiki). Or it could be because you're running multiple websites on a single server (each behind its own IP, or all on the same IP number using virtual hosts), and more than one of these websites needs to have its own wiki.

Types of Wikifarms

There are multiple ways to set up a wikifarm; what is the most suitable way for you depends on how you want to use the farm. We can roughly distinguish the following type of farms:

• Multiple wikis, each running their own code and their own database. This is simple to set up when you install the Wiki code base from source - funny enough, under Debian it's not easy. Also, it's not a recommendable solution from the perspective of maintenance: each wiki must be updated separately when e.g. a security patch is issued for the wiki software.
• Multiple wikis, running on a shared code base, but each with their own database. This is a good solution for multiple wikis for multiple websites.
• Multiple wikis, running on a shared code base, and on a shared database. This could be a good solution for multiple wikis for a single website, because a user can register on one wiki, and automatically have the same account in the other database as well.

We're not going to go into the first type, since the shared codebase is more maintainable. Instead, we're going to show how you can share the Mediawiki code base under Debian, so that a Debian update to the Mediawiki package is automatically applied to all instances of your wikifarm. And we'll see that from there, the choice between separate databases or shared databases is quite trivial.

Wikifarming under Debian

As with most Linux-based software packages, the MediaWiki package mediawiki that Debian offers is modified to suit the Debian policies and needs. This means that it has moved all configuration files to reside under /etc/mediawiki, but also that it offers a somewhat better way to include extensions. The downside of these modifications is that creating a wikifarm under Debian is somewhat more complicated than when you're using vanilla MediaWiki sources - mostly because there isn't much documentation on the Internet about it. In the following we'll be documenting one possible way to create a wikifarm. Note that our method is particularly well-suited to "small" wikifarms (a handful of wikis on a single server), and not exactly fit for "big" wikifarms (thousands of wikis on one or more servers).

General procedure to creat a wikifarm wiki instance

Setting up a wiki under Debian is quite easy - as you might have seen from this wiki's description under entry Basic MediaWiki Installation. This is because the Debian customization includes specific changes to some MediaWiki files, and to the default configuration files. Thus, the usual methods of creating a wiki instance next to the default one run into an additional obstacle: the forementioned Debian customizations. The way around it is this:

• We suppose you've already installed your first wiki. It'll probably be just as described in the Basic MediaWiki Installation.
• Make sure you adapt this first wiki to support virtual hosts, as described in the section on placing MediaWiki inside a Virtual Host.
• If your first wiki is running OK, follow the instructions in the next set of sections to move the wiki from its default place to the wikifarm. As a result, your first wiki should still run OK, but the place where Debian expects your wiki to live is now "empty" again
• Set up your second wiki as if it were the first one, and move it over to the wiki farm.
• et cetera.

Because of the manual work involved, this system is not very well suited to "big" wikifarms; for those instances you'd be better off with one of the scripts that creates the wikifarm instance, and populates it with a default database and default configuration file. Scripts and instructions can be found on the Internet, like this one on jirp.nl.
However, the method is quite doable when you've only got a handful of wikis to create, and remains largely compatible with Debians mechanisms for updating packages.

Debian Mediawiki wikifarm structure

The standard Debian MediaWiki structure

After installation of the mediawiki package, your server will be expanded with the MediaWiki directory structure. Within this structure, there are four main branches (if you haven't installed the mediawiki-extensions package yet):

• /etc/mediawiki contains the configuration files for MediaWiki.
• /usr/share/doc/mediawiki contains the documentation files.
• /usr/share/mediawiki contains the source code. It is not used directly, rather it is linked to from the /var/lib/mediawiki directory.
• /var/lib/mediawiki contains the site, including links to the configuration files in /etc/mediawiki.

Within these branches, we should not delete or move any file or directory, since that will confuse our Debian package manager when an update to the mediawiki package becomes available, so we strive to minimize the changes to this structure. Furthermore, the default configuration files explicitly declare the wiki to "live" in /var/lib/mediawiki, so we'll designate the wiki in that location as the "default wiki" as opposed by "wikifarm instances" that live e.g. under /var/wikifarm/<instancename>.

Extending the structure for a wikifarm

First we need a label <instancename>, which designates the name for each wikifarm instance, so that we can label them as we're creating them. Each name needs to be unique because we're going to create directories based on the labels. Note that this label need not be the Wiki name, as you fill it in when configuring the default wiki instance, although you could of course use that name. In our example, we use the domain name of the website on which the wiki instance is running. Thus, this wiki named "SaruWiki" is instance "saruman.biz" in our little wikifarm.

Furthermore, you need a database name for each wiki instance that gets its own database, and it should be different from the default database name "wikidb" (unless you want to put your wiki instance into this default database, together with the wiki or wikis that are already in there). You could use the label you've chosen, or different ones, or derived ones. Remember to observe the MySQL limits on database names (no more than 64 characters, no slashes or dots in the name, not ending in a space et cetera).

Thus, you could have labels and database names like this:

• saruman.biz and saruman-biz-wiki
• iceditch.nl and iceditchNLwiki
• ann.example.org and wiki-ann
• bob.example.org and wiki-bob
• charles.example.org and wikiCharles

And remember, they are just labels. You could just as easily use user names, numbers or whatever. For maintainability though, we recommend a consistent scheme with recognizable labels. On the other hand, there are a number of labels you must avoid, because they will come to conflict with directories that already exist: most notably config, extensions and images. So before you begin, sit down and write out what your label and database name convention will be. This is trivial if you have just a handful of wikis, but can become very important should you ever grow to hundreds of instances.

Each wikifarm will need its own set of configuration files, so for each new wiki instance we'll create directories under /etc/mediawiki using the chosen wiki labels:

• /etc/mediawiki/saruman.biz
• /etc/mediawiki/iceditch.nl
• et cetera.

Make each of these directories owned by the Apache2 user (chmod www-data:www-data saruman.biz et cetera).

Each wikifarm will need its own code base, so for each new wiki instance we'll create directories under /opt/mediawikifarm using the wiki labels:

• /opt/mediawikifarm/saruman.biz
• /opt/mediawikifarm/iceditch.nl
• et cetera.

The choice for /opt/mediawikifarm is quite arbitrary; one could make a case to use /var/www or /var/lib/mediawikifarm/ or anything like this, but /opt is meant for optional software, which under Debian is software that's not in the standard repositories. And since the wikifarm isn't, it makes sense to create the links to the codebase there.

Each wikifarm may have its own upload directory for images. This directory should not be placed under /opt, but rather in a place where you expect data, e.g. /data if you have that, or /var/www, or something like this. We have chosen for /data/wikifarm/<wikilabel>/images.

Linking to the shared code base

Now we can create links to the code base for each wikifarm instance. You need to create symlinks in your chosen wiki directory (e.g. /opt/mediawikifarm/saruman.biz) to all files and folders in /var/lib/mediawiki, except the directory images and the files AdminSettings.php (if it even exists) and LocalSettings.php. Such a thing can be done simplest with a little script like this: <source lang="bash"> WSRC='/var/lib/mediawiki' WDEST='/opt/mediawikifarm/saruman.biz' cd $WSRC for s in *; do

  ln -s $WSRC/$s $WDEST/$s

done cd $WDEST unlink images unlink AdminSetings.php unlink LocalSettings.php </source> Either run this at the command prompt, or bake it into a little Bash script (it could take the destination directory as a parameter to make it reusable).

Recreating (moving over) your first MediaWiki instance

We'll assume you've followed the other tips in this wiki. Most notably, you've taken all steps described in the article on "how to Place MediaWiki inside a Virtual Host". The steps are simple

Move the Wiki configuration within /etc

Your Wiki config file will be /etc/mediawiki/LocalSettings.php, and the settings for Apache2 will be in /etc/mediawiki/apache.conf. We'll create a subdirectory under /etc/mediawiki that hosts the configuration of your Wiki. Suppose we name the subdirectory after the Wiki we're handling, in this example the SaruWiki. Furthermore, suppose that you also have an AdminSettings.php file. If you don't: no worries! Just skip the steps involved with that file!

cd /etc/mediawiki
mkdir SaruWiki
mv LocalSettings.php SaruWiki
mv AdminSettings.php SaruWiki
mv apache.conf Saruwiki

Note: make sure that after moving the files, the owner and permissions are the same. Moving a file while you are root has the effect of changing ownership to root. LocalSettings.php must be owned by user:group www-data:www-data in order for Apache2 to be able to read the file. The AdminSettings.php file, if it exists, must be owned by root:root and NOT be readable by world (permissions 600) because it should contain the MySQL "root" username and password, with which maintenance scripts can perform their tasks if you instruct them to (while you are root, of course). To keep the Wiki running, we're going to need to tell Apache2 where the two files went to. In this case:

• change the path to apache.conf in the Include statement in the site declaration (e.g. in /etc/apache2/sites-available/000-saruman.biz, change the Include line to

Include /etc/mediawiki/SaruWiki/apache.conf

• change the symlink in /var/lib/mediawiki that points to LocalSettings.php; and if you have one, change the AdminSettings.php symlink as well:

cd /var/lib/mediawiki
unlink LocalSettings.php
ln -s /etc/mediawiki/SaruWiki/LocalSettings.php
unlink AdminSettings.php
ln -s /etc/mediawiki/SaruWiki/AdminSettings.php

If you now restart Apache2, your Wiki will still work - but the two primary configuration files have moved over to a dedicated spot.

Move the Wiki website

As you can see from the MediaWiki directory structure, the code that is actually served in the webserver is in two separate trees: by default the webserver thinks the full tree is in /var/lib/mediawiki, but a large part of what you find in that spot is actually symlinked to files and directories in /usr/share/mediawiki. Actually, the tree /usr/share/mediawiki can be thought of as a source code library, and /var/lib/mediawiki as the instance that we're actually serving. Since an instance needs both source code and some files of its own, we find that /var/lib/mediawiki contains both links to that source code, as well as actual files and directories.

So what we now need to do, is to tell the webserver that the full tree can be served from elsewhere, and in that spot "elsewhere" create symlinks to the right spots. Let's begin with the latter.

Let's assume we wish to move our default Wiki instance to /data/wikifarm/SaruWiki. Other instances could then run under /data/wikifarm/<instancename>. We now first create the links to the /usr/share/mediawiki files as we see them in the /var/lib/mediawiki instance:

cd /data/wikifarm/SaruWiki
for i in /usr/share/mediawiki/*.php; do
> ln -s $i
> done
unlink AdminSettings.php
unlink LocalSettings.php
ln -s /usr/share/mediawiki/*.phtml
ln -s  /usr/share/mediawiki/install-utils.inc
ln -s /usr/share/mediawiki/includes
ln -s /usr/share/mediawiki/languages
ln -s /usr/share/mediawiki/maintenance
ln -s /usr/share/mediawiki/skins

Next, we create the directories that contain the instance-specific files, i.e. the files that belong to this one member of the Wiki farm. This includes the images directory, where users of this wiki instance can upload their images.

mkdir images
chown www-data:www-data images
mkdir config
chown www-data:www-data config
cp -P /var/lib/mediawiki/config/* .