Installing Darwin Calendar Server 2.3 on Ubuntu 9.10

Back in my original 'calendarserver on Linux' posting, I wrote about how to set up Apple's Darwin Calendar Server (DCS) on Ubuntu Linux. At the time, I neglected to mention that was Ubuntu 8.10. This turns out to be important, because I have since built a new server using Ubuntu 9.10 and discovered some important differences in the new Ubuntu.

Ubuntu 9.10's calendarserver package is broken

First of all, the calendarserver package provided by Ubuntu is broken and uninstallable. It depends on the python-xml package, which has been removed.

alex@grawp:~$ sudo apt-get install calendarserver
[sudo] password for alex:
Reading package lists... Done
Building dependency tree      
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
  calendarserver: Depends: python-xml but it is not installable
E: Broken packages

There are a few bugs open on this subject. You should check on these before going further. The calendarserver package may be fixed by this time, and you can simply install it rather than building the whole thing yourself as described in the remainder of this post.

https://bugs.launchpad.net/ubuntu/+source/calendarserver/+bug/426778
https://bugs.launchpad.net/ubuntu/+source/calendarserver/+bug/403349

Initial Setup

Install some packages DCS needs to run.

sudo apt-get install memcached subversion curl python2.6 python2.6-dev python-setuptools

One caveat... I'm fairly sure this is not a complete list. Until I get another 9.10 VM started up to test my instructions, this is my best guess on prerequisite packages you'll need.

Create a user to run as. I added my user to the 'ssl-cert' group so I can access the system-wide self-signed certificate. You don't need this if you're not going to serve calendars over HTTPS, or if you have your own certificate stored elsewhere.

useradd caldavd -G ssl-cert

DCS uses filesystem extended attributes to store some data. You'll need to enable the 'user_xattr' flag on the filesystem which will host your calendaring data. See my previous post on DCS (linked above) for instructions on how to do this. That post also has more information on configuration you may find helpful.

Now get a copy of the source code. As of now, version 2.3 is the latest DCS release. You can check http://svn.calendarserver.org/repository/calendarserver/CalendarServer/t... to see if anything newer is now available.

sudo mkdir /usr/local/src/caldavd
sudo chown caldavd:caldavd /usr/local/src/caldavd
sudo su - caldavd
cd /usr/local/src/caldavd
svn co http://svn.calendarserver.org/repository/calendarserver/CalendarServer/tags/release/CalendarServer-2.3

Building and Installing

cd /usr/local/src/caldavd/CalendarServer-2.3
./run -h # this will show you what's possible.
./run -i /home/caldavd

DCS includes its own miniature build system which knows how to download all its prerequisites and build them for you. This command './run -i' will download all these various things (like memcached, PyXML, pydirector, Twisted, etc) to /usr/local/src/caldavd, build them, and install everything into the directory you specify. In my example, this all goes into /home/caldavd.

As you might have noticed, DCS uses memcached for caching. When starting up, caldavd starts its own instance of memcached to use, which you can see in the process list of a running DCS instance. Even though ./run -i will download and build memcached for you, DCS will not start that memcached binary. You'll still need to install the memcached package from Ubuntu. This was weird to me at first. I saw that memcache had been downloaded and complied, but my DCS log file was full of errors about memcache not being available.

Setup

I followed a file layout similar to that used by the existing Ubuntu/Debian calendarserver package. The difference is that I put everything in subdirectories of /home/caldavd rather than using system directories. For example, where the package uses /var/run, I used /home/caldavd/var/run.

I did this since I plan to install the Ubunutu calendarserver once it actually works, and I don't want to worry about file clashes between the two servers at that time. Also, user caldavd already has write access in /home/caldavd, so I don't have to go flitting about the filesystem chmoding in various places.

mkdir -p /home/caldavd/var/spool
mkdir /home/caldavd/var/run
mkdir /home/caldavd/var/data
mkdir /home/caldavd/var/docroot
mkdir /home/caldavd/log

You need three configuration files to get DCS to run. There are examples of all three available in /usr/local/src/caldavd/CalendarSserver-2.3/conf.

  • /usr/local/src/caldavd/CalendarServer-2.3/conf/caldavd.plist
  • /usr/local/src/caldavd/CalendarServer-2.3/conf/sudoers.plist
  • /usr/local/src/caldavd/CalendarServer-2.3/conf/auth/accounts.xml

cp /usr/local/src/caldavd/CalendarServer-2.3/conf/caldavd.plist /home/caldavd/usr/local/caldavd
cp /usr/local/src/caldavd/CalendarServer-2.3/conf/sudoers.plist /home/caldavd/usr/local/caldavd
cp /usr/local/src/caldavd/CalendarServer-2.3/conf/auth/accounts.xml /home/caldavd/usr/local/caldavd

caldavd.plist is the main configuration file. Here is the one I'm using... I think you should be able to use this as-is if you follow the file layout I described above. If not, it's pretty easy to read the file and change paths as desired.

<?xml version="1.0" encoding="UTF-8"?>

<!--
Copyright (c) 2006-2007 Apple Inc. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   <a href="http://www.apache.org/licenses/LICENSE-2.0</p>
<p>Unless" title="http://www.apache.org/licenses/LICENSE-2.0</p>
<p>Unless">http://www.apache.org/licenses/LICENSE-2.0</p>
<p>Unless</a> required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>

  <!--
   Public network address information

   This is the server's public network address, which is provided to
   clients in URLs and the like.  It may or may not be the network
   address that the server is listening to directly, though it is by
   default.  For example, it may be the address of a load balancer or
   proxy which forwards connections to the server.
 -->

  <!-- Network host name -->
  <key>ServerHostName</key>
  <string>grawp.dean</string>

  <!-- HTTP port [0 = disable HTTP] -->
  <key>HTTPPort</key>
  <integer>0</integer>

  <!-- SSL port [0 = disable HTTPS] -->
  <!-- (Must also configure SSLCertificate and SSLPrivateKey below) -->
  <key>SSLPort</key>
  <integer>8443</integer>


  <!--
   Network address configuration information
   This configures the actual network address that the server binds to.
 -->

  <!-- List of IP addresses to bind to [empty = all] -->
  <key>BindAddresses</key>
  <array><string></string></array>

  <!-- List of port numbers to bind to for HTTP [empty = same as "Port"] -->
  <key>BindHTTPPorts</key>
  <array></array>

  <!-- List of port numbers to bind to for SSL [empty = same as "SSLPort"] -->
  <key>BindSSLPorts</key>
  <array></array>


  <!--
   Data Store
 -->

  <!-- Data root -->
  <key>DataRoot</key>
  <!--<string>/var/run/caldavd</string>-->
  <string>/home/caldavd/var/data</string>

  <!-- Document root -->
  <key>DocumentRoot</key>
  <!--<string>/var/spool/caldavd</string>-->
  <string>/home/caldavd/var/docroot</string>

  <!-- User quota (in bytes) -->
  <key>UserQuota</key>
  <integer>104857600</integer><!-- 100Mb -->

  <!-- Attachment size limit (in bytes) -->
  <key>MaximumAttachmentSize</key>
  <integer>1048576</integer><!-- 1Mb -->


  <!--
   Directory service

   A directory service provides information about principals (eg.
   users, groups, locations and resources) to the server.

   A variety of directory services are available for use.
 -->

  <!--  XML File Directory Service -->
  <key>DirectoryService</key>
  <dict>
    <key>type</key>
    <string>twistedcaldav.directory.xmlfile.XMLDirectoryService</string>
 
    <key>params</key>
    <dict>
      <key>xmlFile</key>
      <string>/home/caldavd/usr/local/caldavd/accounts.xml</string>
    </dict>
  </dict>
 
  <!--  Open Directory Service
 <key>DirectoryService</key>
 <dict>
   <key>type</key>
   <string>twistedcaldav.directory.appleopendirectory.OpenDirectoryService</string>
 
   <key>params</key>
   <dict>
     <key>node</key>
     <string>/Search</string>
     <key>requireComputerRecord</key>
     <true/>
   </dict>
 </dict>
 -->


  <!--
   Special principals

   These principals are granted special access and/or perform
   special roles on the server.
 -->

  <!-- Principals with "DAV:all" access (relative URLs) -->
  <key>AdminPrincipals</key>
  <array>
    <!-- <string>/principals/users/admin/</string> -->
  </array>

  <!-- Principals that can pose as other principals -->
  <key>SudoersFile</key>
  <string>/home/caldavd/usr/local/caldavd/sudoers.plist</string>

  <!-- Create "proxy access" principals -->
  <key>EnableProxyPrincipals</key>
  <true/>

  <!--
   Authentication
 -->

  <key>Authentication</key>
  <dict>

    <!-- Clear text; best avoided -->
    <key>Basic</key>
    <dict>
      <key>Enabled</key>
      <false/>
    </dict>

    <!-- Digest challenge/response -->
    <key>Digest</key>
    <dict>
      <key>Enabled</key>
      <true/>
      <key>Algorithm</key>
      <string>md5</string>
      <key>Qop</key>
      <string></string>
    </dict>

    <!-- Kerberos/SPNEGO -->
    <key>Kerberos</key>
    <dict>
      <key>Enabled</key>
      <true/>
      <key>ServicePrincipal</key>
      <string></string>
    </dict>

  </dict>


  <!--
   Logging
 -->

  <key>Verbose</key>
  <false/>

  <!-- Apache-style access log -->
  <key>AccessLogFile</key>
  <string>/home/caldavd/log/access.log</string>
  <key>RotateAccessLog</key>
  <false/>

  <!-- Server activity log -->
  <key>ErrorLogFile</key>
  <string>/home/caldavd/log/error.log</string>

  <!-- Server statistics file -->
  <key>ServerStatsFile</key>
  <string>/home/caldavd/var/spool/stats.plist</string>

  <!-- Server process ID file -->
  <key>PIDFile</key>
  <string>/home/caldavd/var/run/caldavd.pid</string>


  <!--
   SSL/TLS
 -->

  <!-- Public key -->
  <key>SSLCertificate</key>
  <string>/etc/ssl/certs/ssl-cert-snakeoil.pem</string>

  <!-- Private key -->
  <key>SSLPrivateKey</key>
  <string>/etc/ssl/private/ssl-cert-snakeoil.key</string>


  <!--
   Process management
 -->

  <key>UserName</key>
  <string>caldavd</string>

  <key>GroupName</key>
  <string>caldavd</string>

  <key>ProcessType</key>
  <string>Combined</string>

  <key>MultiProcess</key>
  <dict>
    <key>ProcessCount</key>
    <integer>1</integer> <!-- 0 = one per CPU core -->
  </dict>



  <!--
   Non-standard CalDAV extensions
 -->

  <!-- Calendar Drop Box -->
  <key>EnableDropBox</key>
  <true/>

  <!-- Drop Box Notifications -->
  <key>EnableNotifications</key>
  <true/>

  <!--
   Miscellaneous items
 -->

  <key>Twisted</key>
  <dict>
      <key>twistd</key>
      <string>/home/caldavd/usr/local/bin/twistd</string>
  </dict>

  <key>PythonDirector</key>
  <dict>
      <key>pydir</key>
      <string>/home/caldavd/usr/local/bin/pydir.py</string>
      <key>ControlSocket</key>
      <string>/home/caldavd/var/run/caldavd-pydir.sock</string>
  </dict>

  <!-- A unix socket used for communication between the child and master processes. -->
  <key>ControlSocket</key>
  <string>/home/caldavd/var/run/caldavd.socket</string>

  <key>GlobalStatsSocket</key>
  <string>/home/caldavd/var/run/caldavd-stats.sock</string>

  <!-- Support for Content-Encoding compression options as specified in RFC2616 Section 3.5 -->
  <key>ResponseCompression</key>
  <true/>

</dict>
</plist>

Starting DCS

export PYTHONPATH=/home/caldavd/usr/local/lib/python2.6:/home/caldavd/usr/local/lib/python2.6/dist-packages
/home/caldavd/usr/local/bin/caldavd -X -f /home/caldavd/usr/local/caldavd/caldavd.plist -T /home/caldavd/usr/local/bin/twistd

This causes DCS to start up in the foreground (-X). You'll see the server starting up. Watch for any Python exceptions and other errors. As I said, I'm afraid I've not captured all the steps you may need to perform to get DCS running, and any omissions will probably show themselves at this point. Try browsing to https://your.server:8443/ and see what you get! Post comments if you run into errors. I'll be checking my own work soon, and will correct things as needed.

Getting an Init Script

cd /usr/local/src/caldavd
mkdir debian
cd debian
apt-get source calendarserver
cd calendarserver-1.2.dfsg/debian

Have a look at calendarserver.init.d. This requires only very minimal modifications :
# change these
DAEMON=/home/caldavd/usr/local/bin/caldavd
RUNDIR=/home/caldavd/var/run/
SPOOLDIR=/home/caldavd/var/spool/
# add these
DAEMON_OPTS='-f /home/caldavd/usr/local/caldavd/caldavd.plist -T /home/caldavd/usr/local/bin/twistd'
export PYTHONPATH=/home/caldavd/usr/local/lib/python2.6:/home/caldavd/usr/local/lib/python2.6/dist-packages

I also removed the references to /etc/default/calendarserver, and the check_start_daemon() function. After copying the modified script to /etc/init.d/caldavd, I did
update-rc.d caldavd defaults

And now my caldavd will be started and stopped automatically when the machine starts and stops.

Hopefully, Ubuntu or Debian will just fix their package, and these kinds of shenanigans won't be necessary at all. But in case you're like me and want to use Ubuntu 9.10, but also really need DCS to work, I hope this is helpful.

Thanks for you explanation, tried it today, works great so far. you will need the folowing extra packages on a freshly installed system:
curl
python-setuptools
python-kerberos

Hello georg. Glad it worked for you. I'll add curl and python-setuptools to my list of prerequisites. As for python-kerberos, how did you determine this was required? I wonder if this is due to the following configuration (from caldavd.plist)

    <!-- Kerberos/SPNEGO -->
    <key>Kerberos</key>
    <dict>
      <key>Enabled</key>
      <true/>
      <key>ServicePrincipal</key>
      <string></string>
    </dict>

I'm not using Kerberos in my DCS installation, and that config snippet is only hanging around because it was in the example file. I am guessing that if you remove that from your caldavd.plist, you won't need python-kerberos anymore.

Excellent tutorial.

I had an issue where I kept getting:

ImportError: No module named zope.interface

Despite it being in the dist-packages that the server installed itself. I installed python2.6-zope.interface using apt-get install python2.6-zope.interface and it seemed to resolve the issue, though I'm not sure why it wouldn't load the zope.interface libraries it downloaded and installed.

Hi, I got the server to start and I can view the "root" page but get errors when try to create a new calendar or clicking on links on the root page. Have anyone seen it before?

2010-01-09 16:50:51+0100 [-] [caldav-8443] [PooledMemCacheProtocol,client] Exception rendering:
2010-01-09 16:50:51+0100 [-] [caldav-8443] [PooledMemCacheProtocol,client] Unhandled Error
2010-01-09 16:50:51+0100 [-] [caldav-8443] Traceback (most recent call last):
2010-01-09 16:50:51+0100 [-] [caldav-8443] File "/home/magnus/caldavd/usr/local/lib/python2.6/dist-packages/twisted/internet/defer.py", line 267, in errback
2010-01-09 16:50:51+0100 [-] [caldav-8443] self._startRunCallbacks(fail)
2010-01-09 16:50:51+0100 [-] [caldav-8443] File "/home/magnus/caldavd/usr/local/lib/python2.6/dist-packages/twisted/internet/defer.py", line 310, in _startRunCallbacks
2010-01-09 16:50:51+0100 [-] [caldav-8443] self._runCallbacks()
2010-01-09 16:50:51+0100 [-] [caldav-8443] File "/home/magnus/caldavd/usr/local/lib/python2.6/dist-packages/twisted/internet/defer.py", line 326, in _runCallbacks
2010-01-09 16:50:51+0100 [-] [caldav-8443] self.result = callback(self.result, *args, **kw)
2010-01-09 16:50:51+0100 [-] [caldav-8443] File "/home/magnus/caldavd/usr/local/lib/python2.6/dist-packages/twisted/internet/defer.py", line 769, in gotResult
2010-01-09 16:50:51+0100 [-] [caldav-8443] _inlineCallbacks(r, g, deferred)
2010-01-09 16:50:51+0100 [-] [caldav-8443] --- ---
2010-01-09 16:50:51+0100 [-] [caldav-8443] File "/home/magnus/caldavd/usr/local/lib/python2.6/dist-packages/twisted/internet/defer.py", line 747, in _inlineCallbacks
2010-01-09 16:50:51+0100 [-] [caldav-8443] result = result.throwExceptionIntoGenerator(g)
2010-01-09 16:50:51+0100 [-] [caldav-8443] File "/home/magnus/caldavd/usr/local/lib/python2.6/dist-packages/twisted/python/failure.py", line 338, in throwExceptionIntoGenerator
2010-01-09 16:50:51+0100 [-] [caldav-8443] return g.throw(self.type, self.value, self.tb)
2010-01-09 16:50:51+0100 [-] [caldav-8443] File "/home/magnus/caldavd/usr/local/lib/python2.6/dist-packages/twistedcaldav/extensions.py", line 144, in authenticate
2010-01-09 16:50:51+0100 [-] [caldav-8443] creds = (yield factory.decode(authHeader[1], request))
2010-01-09 16:50:51+0100 [-] [caldav-8443] File "/home/magnus/caldavd/usr/local/lib/python2.6/dist-packages/twisted/internet/defer.py", line 749, in _inlineCallbacks
2010-01-09 16:50:51+0100 [-] [caldav-8443] result = g.send(result)
2010-01-09 16:50:51+0100 [-] [caldav-8443] File "/home/magnus/caldavd/usr/local/lib/python2.6/dist-packages/twistedcaldav/directory/digest.py", line 240, in decode
2010-01-09 16:50:51+0100 [-] [caldav-8443] originalMethod)
2010-01-09 16:50:51+0100 [-] [caldav-8443] exceptions.TypeError: __init__() takes exactly 5 arguments (6 given)
2010-01-09 16:50:51+0100 [-] [caldav-8443]

Magnus : Did you install memcached from the Ubuntu repositories? DCS will download & compile memcached, but it still tries to start /usr/bin/memcached rather than the one it built.

Update : I discovered that the file /home/caldavd/usr/local/lib/python2.6/dist-packages/twistedcaldav/config.py is apparently the one which starts memcached. It searches your PATH for the memcached executable. You can either install the Ubunutu package, so /usr/bin/memcached is available, or you could add the location of an alternate memcached binary to your PATH.

It's probably not a good idea to mess around with the memcached-1.2.6 which is downloaded & compiled by DCS. It's configured to install into /usr/local/src (or whereever you build DCS from). Changing this might break the DCS build, and running binaries out of a src directory just seems weird to me.

hello Georg

your tutorial is pretty clear. I get the following when starting up in the foreground. I checked, of course, file ownerships and permission. user_xattr is defined in the relevant filesystem as well. Any suggestion?

2010-01-13 06:00:05+0100 [-] Log opened.
2010-01-13 06:00:05+0100 [-] twistd 8.2.0 (/usr/bin/python 2.6.4) starting up.
2010-01-13 06:00:05+0100 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2010-01-13 06:00:05+0100 [-] [twistedcaldav.upgrade#warn] Upgrading to version 1
2010-01-13 06:00:05+0100 [-] Traceback (most recent call last):
2010-01-13 06:00:05+0100 [-] File "/home/caldavd/usr/local/bin/twistd", line 19, in
2010-01-13 06:00:05+0100 [-] run()
2010-01-13 06:00:05+0100 [-] File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/scripts/twistd.py", line 27, in run
2010-01-13 06:00:05+0100 [-] app.run(runApp, ServerOptions)
2010-01-13 06:00:05+0100 [-] File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/application/app.py", line 694, in run
2010-01-13 06:00:05+0100 [-] runApp(config)
2010-01-13 06:00:05+0100 [-] File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/scripts/twistd.py", line 23, in runApp
2010-01-13 06:00:05+0100 [-] _SomeApplicationRunner(config).run()
2010-01-13 06:00:05+0100 [-] File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/application/app.py", line 425, in run
2010-01-13 06:00:05+0100 [-] self.application = self.createOrGetApplication()
2010-01-13 06:00:05+0100 [-] File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/application/app.py", line 493, in createOrGetApplication
2010-01-13 06:00:05+0100 [-] ser = plg.makeService(self.config.subOptions)
2010-01-13 06:00:05+0100 [-] File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/plugins/caldav.py", line 33, in makeService
2010-01-13 06:00:05+0100 [-] return self._serviceMaker.makeService(options)
2010-01-13 06:00:05+0100 [-] File "/home/caldavd/usr/local/lib/python2.6/dist-packages/calendarserver/tap/caldav.py", line 378, in makeService
2010-01-13 06:00:05+0100 [-] upgradeData(config)
2010-01-13 06:00:05+0100 [-] File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twistedcaldav/upgrade.py", line 472, in upgradeData
2010-01-13 06:00:05+0100 [-] method(config)
2010-01-13 06:00:05+0100 [-] File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twistedcaldav/upgrade.py", line 434, in upgrade_to_1
2010-01-13 06:00:05+0100 [-] migrateResourceInfo(config, directory, uid, gid)
2010-01-13 06:00:05+0100 [-] File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twistedcaldav/upgrade.py", line 274, in migrateResourceInfo
2010-01-13 06:00:05+0100 [-] os.chown(dbPath, uid, gid)
2010-01-13 06:00:05+0100 [-] OSError: [Errno 1] Operation not permitted: '/home/caldavd/Library/CalendarServer/Data/resourceinfo.sqlite'

sorry ... typo in config file. Actually, it works.

Hi Alex,

Thanks a lot for this tutorial!

Actually I also need the module: python-kerberos.
otherwise I got following error when running:
$ /home/caldavd/usr/local/bin/caldavd -X -f /home/caldavd/usr/local/caldavd/caldavd.plist -T /home/caldavd/usr/local/bin/twistd
----------
...
File "/usr/local/src/caldavd/CalendarServer-2.3/twistedcaldav/authkerb.py", line 41, in
import kerberos
ImportError: No module named kerberos
----------

After adding this module I run it again and got following output:

$ /home/caldavd/usr/local/bin/caldavd -X -f /home/caldavd/usr/local/caldavd/caldavd.plist -T /home/caldavd/usr/local/bin/twistd
exec python /home/caldavd/usr/local/bin/twistd -n caldav -f /home/caldavd/usr/local/caldavd/caldavd.plist
/home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/mail/smtp.py:10: DeprecationWarning: the MimeWriter module is deprecated; use the email package instead
import MimeWriter, tempfile, rfc822
/usr/local/src/caldavd/CalendarServer-2.3/calendarserver/tap/caldav.py:49: DeprecationWarning: mktap and related support modules are deprecated as of Twisted 8.0. Use Twisted Application Plugins with the 'twistd' command directly, as described in 'Writing a Twisted Application Plugin for twistd' chapter of the Developer Guide.
from twisted.scripts.mktap import getid
Traceback (most recent call last):
File "/home/caldavd/usr/local/bin/twistd", line 19, in
run()
File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/scripts/twistd.py", line 27, in run
app.run(runApp, ServerOptions)
File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/application/app.py", line 689, in run
config.parseOptions()
File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/application/app.py", line 669, in parseOptions
usage.Options.parseOptions(self, options)
File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/python/usage.py", line 231, in parseOptions
self.subOptions.parseOptions(rest)
File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/python/usage.py", line 241, in parseOptions
self.postOptions()
File "/usr/local/src/caldavd/CalendarServer-2.3/calendarserver/tap/caldav.py", line 268, in postOptions
create=(0750, config.UserName, config.GroupName),
File "/usr/local/src/caldavd/CalendarServer-2.3/calendarserver/tap/caldav.py", line 303, in checkDirectory
% (description, dirpath)
twistedcaldav.config.ConfigurationError: Document root does not exist and cannot be created: /Library/CalendarServer/Documents/

I'm quite new in this area - Do you have any idea?

Regards,
Falk

Falk, I think this indicates that DocumentRoot property in the caldavd.plist file was not changed. The value listed in the complaint in your log looks like a default value for Mac OS X Server.

Have been working on getting this to run. However, whenever I try and create a calendar event, the memcached socket is dying. The system version of memcached is installed and available at /usr/bin/memcached, and there is a process running for memcached.

2010-02-06 21:42:34-0700 [-] [caldav-8443] [-] AMP connection established (HOST:UNIXSocket(None) PEER:UNIXSocket('/home/caldavd/var/run/caldavd.socket'))
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient socket marked dead (timed out)
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_server( ) failed to connect
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_socket() found dead socket
2010-02-06 21:43:50-0700 [-] [caldav-8443] [-] [memcacheclient#error] Memcacheclient _get_server( ) failed to connect

Any suggestions on fixing this?

thanks,
Peter

A few more items about this memcached socket failing.

This is produced when I attempt to write an item to the calendar using iCal, such as for the test user. The access.log at this point shows:
"PUT /calendars/__uids__/d817aaec-7d24-5b38-bc2f-6369da72cdd9/calendar/3CEFA48C-9DF5-440A-9092-9A2F1A3DAFF1.ics HTTP/1.1" 500 201 "-" "DAVKit/4.0.1 (730); CalendarStore/4.0.1 (973); iCal/4.0.1 (1374); Mac OS X/10.6.2 (10C540)" i=8443 t=3148.6 or=1

When I did the build, it didn't download or build memcached.
The memcached version installed by the 'apt-get install' is 1.2.2. (vs. the current 1.4.4)

The daemon is running and I can store information into it using the telnet interface and ascii protocol.

uname -a gives "Linux kepler 2.6.28-16-server #57-Ubuntu SMP Wed Nov 11 10:48:29 UTC 2009 i686 GNU/Linux"

Doesn't seem to work on Jaunty. This was actually a 9.04 machine (Jaunty Jackalope) rather than 9.10, so memcached at 1.2.2.

On a fresh install of Karmic server with only the ssh server installed, following this recipe, python-kerberos is a package needed to avoid errors in startup.

And then ran fine, storing and retrieving events for the 'test' user.

Trying to develop a package for Ubuntu that works from the source. Wiki page now at: https://wiki.ubuntu.com/CalendarServerPackaging

Hopefully others can help contribute.

Shoot,

i get the following after follow youre tutorial any ideas ??

/home/caldavd/usr/local/bin/caldavd -X -f /home/caldavd/usr/local/caldavd/caldavd.plist -T /home/caldavd/usr/local/bin/twistd
exec python /home/caldavd/usr/local/bin/twistd -n caldav -f /home/caldavd/usr/local/caldavd/caldavd.plist
/home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/mail/smtp.py:10: DeprecationWarning: the MimeWriter module is deprecated; use the email package instead
import MimeWriter, tempfile, rfc822
/home/caldavd/usr/local/lib/python2.6/dist-packages/calendarserver/tap/caldav.py:49: DeprecationWarning: mktap and related support modules are deprecated as of Twisted 8.0. Use Twisted Application Plugins with the 'twistd' command directly, as described in 'Writing a Twisted Application Plugin for twistd' chapter of the Developer Guide.
from twisted.scripts.mktap import getid
Traceback (most recent call last):
File "/home/caldavd/usr/local/bin/twistd", line 19, in
run()
File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/scripts/twistd.py", line 27, in run
app.run(runApp, ServerOptions)
File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/application/app.py", line 689, in run
config.parseOptions()
File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/application/app.py", line 669, in parseOptions
usage.Options.parseOptions(self, options)
File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/python/usage.py", line 231, in parseOptions
self.subOptions.parseOptions(rest)
File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/python/usage.py", line 241, in parseOptions
self.postOptions()
File "/home/caldavd/usr/local/lib/python2.6/dist-packages/calendarserver/tap/caldav.py", line 226, in postOptions
config.loadConfig(self["config"])
File "/home/caldavd/usr/local/lib/python2.6/dist-packages/twistedcaldav/config.py", line 649, in loadConfig
configDict = readPlist(configFile)
File "/usr/lib/python2.6/plistlib.py", line 78, in readPlist
rootObject = p.parse(pathOrFile)
File "/usr/lib/python2.6/plistlib.py", line 406, in parse
parser.ParseFile(fileobj)
xml.parsers.expat.ExpatError: syntax error: line 1, column 0

I have installed the server, and that works fine, only with the init script I am having a problem:

sudo service caldavd restart
* Restarting calendarserver caldavd exec python /home/caldavd/usr/local/bin/twistd caldav -f /home/caldavd/usr/local/caldavd/caldavd.plist
Usage: twistd [options]
Options:
--savestats save the Stats object rather than the text output of
the profiler.
-o, --no_save do not save state on shutdown
-e, --encrypted The specified tap/aos/xml file is encrypted.
--nothotshot DEPRECATED. Don't use the hotshot profiler even if it's
available.
-n, --nodaemon don't daemonize, don't use default umask of 0077
-q, --quiet No-op for backwards compatibility.
--originalname Don't try to change the process name
--syslog Log to syslog, not to file
--euid Set only effective user-id rather than real user-id.
(This option has no effect unless the server is running
as root, in which case it means not to shed all
privileges after binding ports, retaining the option to
regain privileges in cases such as spawning processes.
Use with caution.)
-l, --logfile= log to a specified file, - for stdout
-p, --profile= Run in profile mode, dumping results to specified file
--profiler= Name of the profiler to use (profile, cprofile,
hotshot). [default: hotshot]
-f, --file= read the given .tap file [default: twistd.tap]
-y, --python= read an application from within a Python file (implies
-o)
-x, --xml= Read an application from a .tax file (Marmalade
format).
-s, --source= Read an application from a .tas file (AOT format).
-d, --rundir= Change to a supplied directory before running [default:
.]
--report-profile= DEPRECATED.

Manage --report-profile option, which does nothing currently.

--prefix= use the given prefix when syslogging [default: twisted]
--pidfile= Name of the pidfile [default: twistd.pid]
--chroot= Chroot to a supplied directory before running
-u, --uid= The uid to run as.
-g, --gid= The gid to run as.
--umask= The (octal) file creation mask to apply.
--help-reactors Display a list of possibly available reactor names.
--version Print version information and exit.
--spew Print an insanely verbose log of everything that
happens. Useful when debugging freezes or locks in
complex code.
-b, --debug run the application in the Python Debugger (implies
nodaemon), sending SIGUSR2 will drop into debugger
-r, --reactor= Which reactor to use (see --help-reactors for a list of
possibilities)
--help Display this help and exit.
Commands:
ftp An FTP server.
telnet A simple, telnet-based remote debugging service.
socks A SOCKSv4 proxy service.
manhole-old An interactive remote debugger service.
portforward A simple port-forwarder.
web A general-purpose web server which can serve from a
filesystem or application resource.
inetd An inetd(8) replacement.
news A news server.
xmpp-router An XMPP Router server
words A modern words server
toc An AIM TOC service.
dns A domain name server.
mail An email service
manhole An interactive remote debugger service accessible via
telnet and ssh and providing syntax coloring and basic line
editing functionality.
conch A Conch SSH service.

/home/caldavd/usr/local/bin/twistd: Unknown command: caldav
[ OK ]

What causes this?

Arjan : 'service caldavd restart' fails for me as well, though I'm able to 'service caldavd stop' and 'service caldavd start'. I only did some basic hacking on the init script, so it seems there's a bit more polish needed there.

Make sure you've added a PYTHONPATH value to the init script. I formerly had this in the 'start' action, but I've moved it up next to the DAEMON_OPTS at the top of the init script, and updated the Getting An Init Script section of this tutorial. Double-check that, try again, and let us know how it goes.

I'll try to look at this again soon, and maybe get my updated init script up on GitHub or somewhere else so others can just download the final version rather than needing to follow these instructions on what changes to make.

Peter N. Steinmetz : I will try to contribute to the wiki page you've started. I understand RPM packaging, but building debs is a bit beyond me at the moment. Glad you're pushing this forward, though.

I am running the process on a clean 9.10 server installation (only the openSSL option at initial install).
The Calendar Server compilation finished only after the two set of dependencies below are included:

sudo apt-get install memcached subversion curl python2.6 python2.6-dev python-setuptools libkrb5-dev apache2

sudo apt-get install subversion libkrb5-dev attr curl build-essential libssl-dev python-pysqlite2 bzip2 python-pyopenssl python-dateutil python-xattr python-pysqlite2 python-twisted python-vobject python-kerberos python-dev

After the compilation I am running the server with the command:

/home/caldavd/usr/local/bin/caldavd -X -f /home/caldavd/usr/local/caldavd/caldavd.plist -T /home/caldavd/usr/local/bin/twistd

The secure connection fails totally and the simple http request in port 12008 in firefox 3.6 gives me:
The connection to 10.1.1.5:12008 was interrupted while the page was loading.

The output of the server running is:
exec python /home/caldavd/usr/local/bin/twistd -n caldav -f /home/caldavd/usr/local/caldavd/caldavd.plist
/home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/mail/smtp.py:10: DeprecationWarning: the MimeWriter module is deprecated; use the email package instead
import MimeWriter, tempfile, rfc822
/home/caldavd/usr/local/lib/python2.6/dist-packages/calendarserver/tap/caldav.py:49: DeprecationWarning: mktap and related support modules are deprecated as of Twisted 8.0. Use Twisted Application Plugins with the 'twistd' command directly, as described in 'Writing a Twisted Application Plugin for twistd' chapter of the Developer Guide.
from twisted.scripts.mktap import getid
2010-03-22 11:37:19+0200 [-] Log opened.
2010-03-22 11:37:19+0200 [-] twistd 8.2.0 (/usr/bin/python 2.6.4) starting up.
2010-03-22 11:37:19+0200 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2010-03-22 11:37:19+0200 [-] twistedcaldav.accesslog.AMPLoggingFactory starting on "'/home/caldavd/var/run/caldavd.socket'"
2010-03-22 11:37:19+0200 [-] calendarserver.tap.caldav.CalDAVStatisticsServer starting on "'/home/caldavd/var/run/caldavd-stats.sock'"
2010-03-22 11:37:24+0200 [-] [caldav_task] 2010-03-22 11:37:23+0200 [-] Log opened.
2010-03-22 11:37:24+0200 [-] [caldav_task] 2010-03-22 11:37:23+0200 [-] twistd 8.2.0 (/usr/bin/python 2.6.4) starting up.
2010-03-22 11:37:24+0200 [-] [caldav_task] 2010-03-22 11:37:23+0200 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2010-03-22 11:37:24+0200 [-] [caldav_task] 2010-03-22 11:37:24+0200 [-] [calendarserver.sidecar.task.CalDAVTaskServiceMaker#info] Configuring directory service of type: twistedcaldav.directory.xmlfile.XMLDirectoryService
2010-03-22 11:37:24+0200 [-] [caldav-12008] /home/caldavd/usr/local/lib/python2.6/dist-packages/twisted/mail/smtp.py:10: DeprecationWarning: the MimeWriter module is deprecated; use the email package instead
2010-03-22 11:37:24+0200 [-] [caldav-12008] import MimeWriter, tempfile, rfc822
2010-03-22 11:37:24+0200 [-] [caldav-12008] /home/caldavd/usr/local/lib/python2.6/dist-packages/calendarserver/tap/caldav.py:49: DeprecationWarning: mktap and related support modules are deprecated as of Twisted 8.0. Use Twisted Application Plugins with the 'twistd' command directly, as described in 'Writing a Twisted Application Plugin for twistd' chapter of the Developer Guide.
2010-03-22 11:37:24+0200 [-] [caldav-12008] from twisted.scripts.mktap import getid
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] Log opened.
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] twistd 8.2.0 (/usr/bin/python 2.6.4) starting up.
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2010-03-22 11:37:24+0200 [-] [caldav_task] 2010-03-22 11:37:24+0200 [-] [calendarserver.sidecar.task.CalDAVTaskServiceMaker#info] Setting up document root at: /home/caldavd/var/docroot
2010-03-22 11:37:24+0200 [-] [caldav_task] 2010-03-22 11:37:24+0200 [-] [calendarserver.sidecar.task.CalDAVTaskServiceMaker#info] Setting up principal collection:
2010-03-22 11:37:24+0200 [-] [caldav_task] 2010-03-22 11:37:24+0200 [-] [calendarserver.sidecar.task.CalDAVTaskServiceMaker#info] Setting up calendar collection:
2010-03-22 11:37:24+0200 [-] [caldav_task] 2010-03-22 11:37:24+0200 [-] [calendarserver.sidecar.task.CalDAVTaskServiceMaker#info] Setting up root resource:
2010-03-22 11:37:24+0200 [-] [caldav_task] 2010-03-22 11:37:24+0200 [-] set uid/gid 1001/1001
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Configuring directory service of type: twistedcaldav.directory.xmlfile.XMLDirectoryService
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Configuring SudoDirectoryService with file: /home/caldavd/usr/local/caldavd/sudoers.plist
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Configuring authentication for realm: Test Realm
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Setting up scheme: kerberos
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [twistedcaldav.authkerb.NegotiateCredentialFactory#error] getServerPrincipalDetails: ('Cannot get sequence cursor from keytab', 2)
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Could not start Kerberos
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Setting up scheme: digest
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Setting up document root at: /home/caldavd/var/docroot
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Setting up principal collection:
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Setting up calendar collection:
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Setting up root resource:
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Setting up WebCalendar resource: /usr/share/collaboration
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Setting up WebAdmin resource
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Setting up Timezone Cache
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Configuring authentication wrapper
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Setting up service
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Logging via AF_UNIX: /home/caldavd/var/run/caldavd.socket
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Configuring log observer:
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Adding SSL server at 10.1.1.5:8443
2010-03-22 11:37:24+0200 [twistedcaldav.accesslog.AMPLoggingFactory] AMPLoggingProtocol connection established (HOST:UNIXSocket('/home/caldavd/var/run/caldavd.socket') PEER:UNIXSocket(''))
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#error] Unable to set up SSL context factory: [('system library', 'fopen', 'Permission denied'), ('BIO routines', 'FILE_CTRL', 'system lib'), ('SSL routines', 'SSL_CTX_use_PrivateKey_file', 'system lib')]
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#error] Disabling SSL port: 8443
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Adding server at 10.1.1.5:12008
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] twext.web2.channel.http.HTTP503LoggingFactory starting on 12008
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] set uid/gid 1001/1001
2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] AMP connection established (HOST:UNIXSocket(None) PEER:UNIXSocket('/home/caldavd/var/run/caldavd.socket'))

any ideas what might goes wrong

tassos :
You wrote "The connection to 10.1.1.5:12008 was interrupted while the page was loading." What is supposed to be listening on that port? My server doesn't have anything configured to listen there.

Your error log shows you have an SSL-enabled calendarserver on port 8443, but there was an error due to the process being unable to access a file (maybe the SSL private key?). "2010-03-22 11:37:24+0200 [-] [caldav-12008] [-] [calendarserver.tap.caldav.CalDAVServiceMaker#error] Unable to set up SSL context factory: [('system library', 'fopen', 'Permission denied'), ('BIO routines', 'FILE_CTRL', 'system lib'), ('SSL routines', 'SSL_CTX_use_PrivateKey_file', 'system lib')]"

alex
the calendar server was instructed to do http requests at 12008 just as it does https requests in 8443

the ssl certificates were not created, and I try to use this port for the plain text service

I suggest you try removing the HTTPS configuration for port 8443, and focus on getting the HTTP configuration working on port 12008.

You might try 'telnet 10.1.1.5 12008'. This would allow you to send & receive raw HTTP traffic, and might give you a better idea why the connection is being interrupted.

Your /var/log/syslog may contain some clues as well.

Hi there alex,

Thanks for writing up these instructions. I followed them to get Calendar Server 2.4 installed on Ubuntu 9.10. Works like a charm. I did hit a hardcoded bug in upgrade code since I wanted to upgrade all data from 1.2. The bug will affect all linux installations. Anyone wishing to get around this will find the patch at:

http://trac.calendarserver.org/ticket/337

Other than that, the server is running beautifully with access from iCal on both Mac (using /principals/__uids__//) and iPhone (using /principals/users//).

Thanks again alex!

The dependencies should also contain readline6-dev and libzip-dev. Thus the apt-get command should be changed to:

sudo apt-get install memcached subversion curl python2.6 python2.6-dev python-setuptools readline6-dev libzip-dev

Thank you for sharing!

Post new comment

The content of this field is kept private and will not be shown publicly.
CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.