Calendar Server in defiance of the cloud

I suppose these days running your own Internet services would be considered to fall under the banner of a “personal cloud”. I run my own email, for example, something which many nowadays outsource to the likes of Google. The reason I do this is the control it gives me over exactly how the services I use function. I also find it interesting. Many people associate email with one of these hosted “cloud” services and may not even realise that the vast majority of what you use on the Internet you can do yourself.

I have just added a new service to my arsenal – that of a CalDAV server (using the Darwin Calendar Server – this is the open source (Apache Licensed) version of the software available in Snow Leopard Server). The only other major option I could see is DAViCal (which uses php – yuck!)

Setting this up isn’t too difficult once you get past the almost complete lack of documentation. Fortunately the use of XML (Apple’s plist format) for the configuration files allows you to make some pretty good guesses as to what various settings do. The server seems to work around the concept of “principals”. This is a term which refers roughly to objects which can own calendars, e.g. users, groups, resources, locations etc. Principles are identified by a GUID (ick) and a set of DAV properties. The structure beyond that is automatically generated by the server.

Basic Installation

Using Debian, this is easy, just do:

aptitude install calendarserver

Note: This pulls down memcached, so don’t be surprised to see that running.

Other platforms may have this package in their package management systems, if not you may have to install it from source (not too hard given it’s Python-based).

The server makes use of user-level Extended Attributes on the filesystem to store information. If these aren’t switched on for your filesystem you’ll need to enable them. This can be done by adding the “user_xattr” flag to your /etc/fstab, e.g.:

/dev/hda3     /home     ext3     defaults,user_xattr     1 2

Server Setup

The server operates on port 8008 by default without SSL. I like all my services to be encrypted (the default SSL port is 8443), but this is easy to set up in the configuration. You can also set which IP addresses the server listens on. Unfortunately the version included with Debian doesn’t support IPv6 yet – I will have to fix that sometime.

If you look at the server with a web browser (e.g. https://myserver.com:8443) you get an interface which allows you to interrogate the various principals and settings. You can also enable admin via this interface.

    <!--
Permissions
-->


EnableAnonymousReadRoot



EnableAnonymousReadNav



EnablePrincipalListings

The first two settings here determine whether non-authenticated users can view basic root-level information about your CalDAV server, and whether they can view detailed information deeper into the hierarchy. I see no reason to enable these.

The third setting determines whether detailed information is listed for principals (to authenticated users). This is quite useful for diagnostic purposes so I turn this on.

Authentication can be via Basic Auth (good to use with SSL), Digest Auth (useful for non-SSL) or Kerberos. The latter option is overkill for my needs.

There are also settings for your public/private key pair, log files, process management etc. These are all quite straightforward and self-explanatory (if you’ve configured Apache you’ll be fine with these).

A nifty feature I might have to look into further is integration with an XMPP server for notifications. Various Server-to-Server protocols are also available (for distributed meeting planning). I haven’t explored these options yet.

One setting towards the end is important:

    
EnableWebAdmin

With this turned on you can query and set various properties via the web interface. Only principals which have been placed into one of the “special principals” containers can do this, e.g.:

    
AdminPrincipals

<!-- /principals/__uids__/some-GUID/ -->

I don’t see much need to use the web admin interface so I turned it off for the moment.

The other “special principle” groups allow global read access (“ReadPrinciples”) to all other principals (e.g. you can see everyone’s calendars etc.) and “sudo” principals (which can impersonate other principals) – the “sudo” ones are set via a third configuration file, sudoers.plist).

Quotas are supported Data Store settings, the defaults are fairly sensible for a small installation though you’ll probably want to tweak down the default 100MB per user if you have a larger site.

User Setup

The main thing to worry about when configuring the server is to set up your source of principals (e.g. where the user/group etc. information comes from, this is known as a Directory Service). Various sources are supported, the simplest of which is an XML file. OpenDirectory, LDAP etc. work too.

LDAP is something I may explore later to tie my services together, for now I went with the XML option for simplicity.

These principals are specified in the “accounts.xml” file. You define a realm, under which multiple accounts of type user, group, location or resource can live. In a larger site the maintainability of a single XML file is likely to become a problem at which point use of LDAP or OpenDirectory would become a pressing concern.

One big downside to this approach is the need to specify user passwords in cleartext in the configuration file. This is something I try and avoid doing and will be a big motivation to moving to LDAP as a directory backend.

I’d recommend for a small installation specifying non-GUIDy GUIDs for your users. E.g.:



someusername
g_someusername
someusernamepassword
Some User
mailto:someuser@some.domain

This means that instead of getting URLs appearing like:

/principals/__uids__/AE668DD7-D2B8-4D4D-A574-2AA533DF26A4/

You get:

/principals/__uids__/g_someusername/

Which is easier to debug.

More info on users and groups: http://dcswiki.org/dwiki/doku.php?id=users

Using the server

Both iCal (Mac OSX) and the iPhone (iOS 4.1+) can use a CalDAV server. There’s some good info on setting this up here:

http://manuals.kerio.com/kmsug/en/sect-caldav.html

And for BusyCal:

http://www.busymac.com/help/sync/caldav-other.html

Conclusion

Setting this up is really quick and easy if you already have a Debian server available. Being able to sync my calendars wirelessly between my Mac and iPhone (as well as enabling other people to subscribe to view it) is a powerful capability. The next step is to set up a CardDAV server as well to do the same thing for contacts.

Debian kernel upgrade

Process for this is:

1. Look for new kernels:

aptitude search linux-image

2. Install the one you want:

aptitude install linux-image-2.6.32-5-xen-686

3. Edit /boot/grub/menu.lst, e.g.:

#title Xtrahost Xen Kernel (2.6.18-164.9.1.el5xen)
# root (hd0)
# kernel /boot/vmlinuz-2.6.18-164.9.1.el5xen ro root=/dev/xvda1 xencons=xvc
# initrd /boot/initrd-2.6.18-164.9.1.el5xen.img
title Debian Linux Kernel (2.6.32-5-xen-686)
root (hd0)
kernel /boot/vmlinuz-2.6.32-5-xen-686 ro root=/dev/xvda1 xencons=xvc
initrd /boot/initrd.img-2.6.32-5-xen-686

4. Reboot your server:

shutdown -r now

5. Hope it comes back up (it should do if you’re using a stable kernel…)

My Debian box is now running Squeeze, which was released very recently 🙂

Is the cupboard bare?

This really does mark the beginning of the end for IPv4. APNIC announced today that the last two /8s available for general assignment have been assigned to them by IANA. There are actually 5 more /8s remaining, but under a long-standing agreement IANA have to immediately release one of these to each of the 5 RIRs (AfriNIC, ARIN, APNIC, LACNIC and RIPE NCC). This is expected to be announced on Thursday at a press conference.

What does this mean? Well immediately very little, since there are still quite a few IPv4 addresses remaining to be allocated. IANA provides assignments to the 5 regional registries (usually in /8 blocks), the regional registries then assign addresses in smaller blocks to local registries which then assign them to end users. The rate at which the RIRs will run out of blocks of addresses varies between RIRs – according to current consumption levels APNIC is likely to run out before the end of 2011, and its dependant LIRs soon afterwards.

This tool provides a good estimate of when exhaustion will occur for the RIRs:

http://www.potaroo.net/tools/ipv4/index.html

And more information from RIPE about exhaustion:

http://ripe.net/info/faq/IPv6-deployment.html

Eventually as the RIRs and LIRs start to run out of IPv4 addresses it’ll become increasingly hard for end users to get ahold of them. This will lead to all sorts of technical workarounds (such as so-called “Carrier-Grade” Network Address Translation, extended use of DNS SRV records for server services etc., and hopefully widespread adoption of IPv6). The routing information used on the Internet will also start to fragment as smaller routes are announced via BGP and the global routing tables grow in size. Routing is actually one of the key restrictions to IPv4 addressing since it has much more of a restrictive effect on global connectivity than sheer address numbers do.

While not much will change due to the IANA IPv4 exhaustion this announcement should serve as a wake-up call for the Internet-using community that this is a real problem right now. The only true long-term solution for this problem is moving to use of IPv6 as soon as possible. Fortunately this process is underway, and those that adopt it now serve to gain in the long term from being leaders in the field.

APNIC press release:

http://www.apnic.net/publications/news/2011/leading-global-internet-groups-make-significant-announcement-about-the-status-of-the-ipv4-address-pool

Ubiquity begets assumption

A lot of installers these days assume you’re connected to the Internet. This is especially true of Microsoft’s installers. Despite having produced a wonderful (genuinely) installation famework (MSI) Microsoft still insists on distributing a thousand other installation methods, each with subtly different command line parameters. This invariably makes life a pain if your job involves doing automated installs on machines which are specifically not connected to the net.

Most recent example of this is the Silverlight Tools package. This is a 33MB download, which still falls into the “wait a few minutes” category for most people. Despite this they still insist on downloading a file as part of the installation (only one file mind).

In almost all of these cases there’s a simple workaround – but why not make life a little bit easier and use a consistent offline installation method for everything?