Bigger and better album storage

We have started to outgrow the system we set up 2 years ago for storing your albums, so we decided to set up something new that can handle the ever increasing number of albums that get published to jalbum.net. We were looking for something that could reliably handle the new increased sizes of the Premium and Power accounts. After looking at different options we decided to go for a ZFS based system. ZFS is a file system that was originally developed at Sun where it was released as part of Open Solaris. As we are not too happy about what has happened with Open Solaris after Oracle acquired Sun, we looked at other OSs that include ZFS support. In the end, we decided to to with the Nexenta Core platform.

After doing some satisfactory tests of Nexenta on our lab servers, we started gathering hardware for the production servers. First, we needed a chassi that could handle a lot of disks – with 24 drive bays, a Super Micro RS-9224.6 fully populated with 2 TB drives provides 48 TB of raw data per server. Effective storage will be less though – as we want to maximize both performance and reliability we will get 23 TB of effective storage per server. For the main disks we use SAS drives. As ZFS can take advantage of faster disks to accelerate reads and writes, we use SSD disks for the read cache and – after noticing that SSD did not perform as well for writes as we had hoped – the RAM/SSD hybrid DDRDrive for the write cache. Two quad core processors and 48 GB of RAM finished the setup.

After building and validating our first test server at the office, it was time for it to join our other servers in the atomic bomb proof hosting center Pionen in Stockholm. This hosting center is built in an old bomb shelter, 30 meters below ground and uses old German submarine engines for diesel backed up redundant power. You can read more about it here.

Most of the data from the old storage system can be copied to the new one before switching over to use the new system, but we do have to shut down all uploads while we copy the last data over. This will take a couple of hours and will happen in the morning of Thursday the 14th of April. During those hours no uploads of new albums or updates of old albums will be possible, but viewing existing albums will still be possible. We are sorry for this service interuption, but it is a necessary step to make sure that we can continue to improve our service.

For more photos, visit this album.