ZFS

I was using FreeNAS on an artigo, then something went wrong, perhaps the memory, perhaps the CF. I figure it's time I get something a bit more reliable, specifically with an out of band management card. Since the HP ProLiant MicroServer N54L had the cashback option at the time, it was a no-brainer to get one for £129.

The microserver didn't come with an iLO (Intelligent Lights Out), otherwise known as LOM (Lights Out Management). This is an additional card on this generation of MicroServer, the following generation (G8) include this by default. The expansion card is called a HP MicroServer Remote Access Card Kit, in HP terms, and is part number 615095-B21. If that helps you at all. This model of MicroServer is now discontinued, so it may be completely irrelevant anyway.

Looking at the HP MicroServer BIOS, I don't think it can do serial port redirection, so it isn't possible from what I can tell to get IPMI SOL (Serial Over Lan) to work. That isn't a show stopper though as the Java Console is sufficient and has all the keyboard mappings. Shame it is reliant on a GUI though, which other HP (DL385 etc) servers aren't. Anyway, this is a domestic Microserver, NAS, if you like, so like I say, it's not a huge issue.

Debian amd64 has a working ZFS package I decided it would be ideal to install Debian on the MicroServer and then move the disks to it.

This is really simple to install.

Follow the instructions at zfsonlinux.org, which will give you a working ZFS system.

First things first though, lets make sure this is all possible. I create a VM in virtualbox, install Debian amd64 on that (since I know the i386 version doesn't get you very far with a zpool). To pass the raw disk through to virtualbox, you need this little bit of witchcraft:

VBoxManage internalcommands createrawvmdk -filename ~/VirtualBox\ VMs/deb/zfs.vmdk -rawdisk /dev/sdb

Go to the VM settings and add choose an existing disk to add, then pick the location to the vmdk file from above.

Since I had a pool that wasn't unmounted cleanly the zpool import needed to be issued as:

# zpool import -a -f
(and then)
# zpool export

Pretty simple.

As that works just fine, and it had a new mount log, I took the other disk from this mirror and just went straight ahead and imported that on the MicroServer and plugged the known good disk in later, zpool works the silvering all out. With hindsight it would have been efficient to plug both disks in together, but I wanted to do it staged like this just in case, there are no other copies of this data.

growing a pool

When you upgrade your pool disks, and at some point in the future you will be doing this, you need to set a couple of toggles. This is one annoyance that I have with ZFS though as it requires the pool to be exported (unmounted).

# zpool set autoexpand=on tank
# zpool export
# zpool import
# zpool online -e tank VDEV

write caching

I'm not sure if it is a fault of zfsonlinux or not, but write cache was disabled on the disks that I recently grew the pool onto. I suspect so.
The disks that I inserted into the Microserver did not have write-cache enabled, this is easily rectified, though.

# hdparm -W /dev/sdc

/dev/sdc:
 write-caching =  0 (off)
# hdparm -W1 /dev/sdc
/dev/sdc:
 setting drive write-caching to 1 (on)
 write-caching =  1 (on)

There was a bug in previous versions where having multiple file systems on a device, and write-caching would cause problems with a zpool. I don't believe this to be the case now.

different sector alignment

When replacing some new disks to a pool, I had the odd error message of: cannot replace 4329313178948330079 with /dev/sdb: devices have different sector alignment

What the devil does this mean? Well, it's due to block size differences. The pool was probably created with a blocksize different to the new vdev defaults. Try:

# zpool replace pool0 4329313178948330079 /dev/sdb -o ashift=9 -f

Where 4329313178948330079 is the name of the vdev that you are replacing. Also, use the whole device, don't try and use a partition if there is nothing else sharing blocks since ZFS works best with whole devices.