roll your own openstreetmap server - Ubuntu 11.10

Looks like there a few benefits from setting up an OSM server on Ubuntu 11.10 as most of the software used is now at 'good' versions - only mapnik lags behind, but while the standard OSM XML files are still not mapnik 2 compliant, that can be a good thing too.
So here's a complete rundown on getting an OSM server up and running.

getting the server ready

I run this sort of thing as a VM on a baby server upstairs. So I've set up a VM with a 100Gb disc (through LVM) ready to build Ubuntu 11.10 server. This disc size is plenty if you're going to use a subset of the data. Here's how I first fire up the VM from the host machine (also running Ubuntu 11.10).

virt-install -n mapnik11 -r 10000 --vcpus=2 \
--os-variant=ubuntuoneiric \
--disk /dev/bigraid/mapnik11 \
--disk device=cdrom,path=../ubuntu-11.10-server-amd64.iso \
--boot cdrom \
--network bridge=virbr0 \
--graphics vnc \
--check-cpu

If you want to know more about this, go search the web.

Installing and basic setup of Ubuntu

I always start with Ubuntu server, as booting up the server install system is soooo much faster than booting into full GUI desktop (which is how the desktop install starts. Once it's built it also boots (and shuts down) in a few seconds (i.e. 5 or 6).
So do a completely straight install of Ubuntu server, with the relevant settings for you. I built mine with 100Gb of filestore (on an LVM logical volume so I can snapshot it), and let the installer partition automatically on the whole disc. Near the end of the install process I select ssh as the only package to install at this stage.
I could install more packages, but as I have an apt-proxy I don't want to install any more until I have the proxy set up.
Once the VM has rebooted I:

• set up my apt-proxy (irrelevant for you if your not running a proxy)
• install acpid, nfs server setup nfs server.
  • acpid means you can shutdown your VM with virsh or Virtual Machine Manager
  • nfs server is really handy so you can move files around between machines 9as long as their linux)
  • you'll probably want to set NEED_IDMAPD=yes in /etc/default/nfs-common as well if your running nfs4
• If you wanna use webmin or other standard tools, now is the time to do that too.
• If you use a windoze desktop to access things, you probably want to install Samba at this stage, but personally I'm all linux here so I've never used Samba.

sudo apt-get install acpid nfs-kernel-server

Installing database, gis and other things you will need

At this point if your internet connection isn't too fast, start downloading the openstreetmap data to save time later. You can go for the whole thing, or (probably a better idea to start with at least) you can get a regional subset.
I usually use the british isles (about 650Mb). The whole planet is around 20Gb as of now.
Most of the following is derived from this guide, updated to work with Ubuntu 11.10 and where practical to use packages that are standard in Ubuntu 11.10. I have copied the relevant info over from there where relevant.

sudo apt-get install acpid build-essential libxml2-dev libtool libgeos-dev libpq-dev libbz2-dev proj subversion autoconf postgresql postgresql-9.1-postgis

• add the line kernel.shmmax = 268435456 to /etc/sysctl.conf
  • performance tweaks to help the database server
• sysctl -p will force params to be reloaded (and get this changed value setup)
• and there are changes to make to the postgres config file as well: change the following settings in /etc/postgresql/9.1/main/postgresql.conf

shared_buffers = 128MB # 16384 for 8.1 and earlier
checkpoint_segments = 20
maintenance_work_mem = 256MB # 256000 for 8.1 and earlier
autovacuum = off

Restart postgres: sudo /etc/init.d/postgresql restart

Setting up the database

Replace USERNAME with your username in the following:

sudo -u postgres -i
createuser USERNAME # answer yes for superuser
createdb -E UTF8 -O USERNAME gis
exit

And now set up postgis on the new database:
psql -f /usr/share/postgresql/9.1/contrib/postgis-1.5/postgis.sql -d gis
This will generate lots of output, ending with

....
....
CREATE FUNCTION
COMMIT
....
....
....
DROP FUNCTION

Change the permisions to save lots of hassle later!
echo "ALTER TABLE geometry_columns OWNER TO username; ALTER TABLE spatial_ref_sys OWNER TO username;" | psql -d gis
Finally set the Spatial Reference Identifier:

psql -f ~/bin/osm2pgsql/900913.sql -d gis

which should respond with

INSERT 0 1

Loading the database

Assuming your download has finished, now its time to stuff all that luverly data into postgres.... Something like this should do the trick:
./osm2pgsql -S default.style --slim -d gis -C 2048 --number-processes=2 --cache-strategy=dense ~/planet/british_isles.osm.bz2
Put in the right filename and let it go. On my baby VM it takes about 45 minutes to load the british isles......
Once that's done we just need to vacuum things (since we turned autvacuum off earlier....)

• first fire up psql with psql -d gis
• then do this vacuum analyze

that's all for now.