A quick guide to running a Raspberry Pi from a hard drive

The Raspberry Pi is a fun if limited little single-board computer (were that it had more than one core). That said, running off an SD card is not exactly optimal, with write speeds being the greatest handicap. This is even worse with micro SD cards.

While SD cards with “Ultra” and “Extreme” abound, their claim to fame is speed while writing very large sequential files (some DSLR cameras can push > 40 MB/s of sustained writes in raw video/photography modes). When it comes to small random writes (e.g., the 4K range) the write speed drops precipitously. The maximum bandwidth of the SD card slot is in the 20-25 MB/s range, and finding a card which performs well over the full typical range of writes is difficult (I’ve seen cards that do better at small writes but max out at 15 MB/s). The USB slot can manage 30 MB/s and a small laptop HDD or even an SSD is a very cheap way to provide a serious speed bump across the range!

Prerequisites:

  • A powered USB 2.0 hub can be useful. I personally have a Plugable 7 port hub which has worked quite well for me (it was widely recommended and doesn’t backpower through the USB ports, an important consideration). A powered hub is useful for other things as well, particularly if you add other peripherals (you can easily power the RPi itself as well as Wifi, keyboard and mouse, bluetooth, storage devices and other things). It may be possible to use only the internal RPi USB ports, but it depends on the drive you use and the total power available. Note that USB 3.0 hubs may or may not work and won’t add a speed benefit.
  • A laptop-size hard drive – this will consume less power and can often be powered straight from a USB port. Got a small Western Digital Passport? That’ll work! If you’re repurposing a drive, I had good luck with the Inatek case (it’s handy for other projects and was pretty cheap). Whatever you use, it needn’t be state of the art since this is still I/O bound.
  • A working installation of Raspian on an SD card in a Pi. This example assumes a default two partition setup on the card (boot and root filesystems).

Process:

(Adapted – with important corrections! – from another site)

List the current partitions on all attached drives:

sudo fdisk -l

We should see the original SD Card, e.g.:

Disk /dev/mmcblk0: 15.9 GB, 15931539456 bytes
Device Boot Start End Blocks Id System
/dev/mmcblk0p1 8192 122879 57344 c W95 FAT32 (LBA)
/dev/mmcblk0p2 122880 31116287 15496704 83 Linux

As well as the external drive, e.g.:

Disk /dev/sda: 160.0 GB, 160041885696 bytes
Device Boot Start End Blocks Id System
(Since my drive is unformatted we see no partitions present.)

Use fdisk to partition the new HDD

sudo fdisk /dev/sda
  • Use ‘d’ repeatedly to delete unwanted partitions (typ. all of them)
  • Use ‘w’ to sync and commit the change.
  • Create the root partition: n -> p -> 1 -> (default) -> +64G
  • Create the swap partition: n -> p -> 2 -> (default) -> +4G
  • Set the swap partition type: t -> 2 -> 82
  • Create the ext partition: n -> e -> 3 -> (default) -> (default)
  • Create a spare partition: n -> l -> (default) -> +32G
  • Write, sync and quit: w

Check that things were created OK:

sudo fdisk -l

Run ‘top’ (or free -m) (note that swap is currently a 100 MB on-disk swapfile. Also my main memory is well-short of the 512 MB you’d expect because I’ve reserved 128 MB for video memory in /boot/config.txt (gpu_mem=128):

KiB Mem: 382832 total, 82508 used, 300324 free, 11488 buffers
KiB Swap: 102396 total, 0 used, 102396 free, 42804 cached

For the new root partition, just clone it and size it (you might prefer to do the cloning offline (e.g., using gparted)):

sudo dd if=/dev/mmcblk0p2 of=/dev/sda1 bs=32M conv=noerror,sync

It’s pretty slow! 14.5 MB/s on mine. My SD card is fast but the alternating read-then-write means you’ll be doing this at half the optimal read bandwidth of your card.

sudo e2fsck -f /dev/sda1

Press ‘y’ if fixes are required (e.g., “Free blocks count wrong” is typical)

sudo resize2fs /dev/sda1

Initialize the new swap partition:

sudo mkswap /dev/sda2

Initialize, mount and configure a spare partition (optional):

sudo mkfs.ext4 /dev/sda5
sudo mkdir /mnt/spare
sudo mount /dev/sda5 /spare
sudo chown -R pi:pi /spare

Mount the new rootfs and configure swap and fstab:

sudo mount /dev/sda1 /mnt
sudo vi /mnt/etc/fstab
    # Edit the root paritition: /dev/mmcblk0p2 --> /dev/sda1
    # Because we'll be running on an external drive the 'noatime'
    # SD card wear optimization may be optional now
        /dev/sda1 / ext4 defaults,noatime 0 1
    # Add the swap partition:
        /dev/sda2 none swap sw 0 0
    # Add the spare partition:
        /dev/sda5 /spare ext4 errors=remount-ro 0 1

Configure the boot file:

sudo cp /boot/cmdline.txt /boot/cmdline.orig
sudo vi /boot/cmdline.txt
    # Update: root=/dev/mmcblk0p2 --> root=/dev/sda1

Sync and reboot:

sudo sync
sudo reboot

At this point we are using the new drive for the root fs!

Problems booting? You can mount the boot partition in Linux or Windows to edit files like cmdline.txt when debugging:

  • Adding bootdelay / rootdelay parameters to the cmdline.txt file can help if the HDD is taking too long to spin up and times out
  • You can edit/rename the cmdline.txt file back to what it was originally, allowing you to run from the SD card alone once again

Now, remove the old /var/swap-based swapfile (if present):

sudo dphys-swapfile swapoff
sudo dphys-swapfile uninstall
sudo reboot

Take a look at the output of ‘top’ (or free -m) again. Note the new, larger swapfile:

KiB Mem: 382832 total, 84376 used, 298456 free, 12132 buffers
KiB Swap: 4194300 total, 0 used, 4194300 free, 42552 cached

Speed tests I ran showed read/write performance in the 28-29 MB/s range, which is about as good as you’re going to get (uncompressed) over a USB 2.0 link.

More thoughts:

  • The boot partition is miniscule and has a small impact on speed (and only at boot-time). You could clone the boot partition+MBR to a very small SD card and save the original SD card for other uses (process is outside the scope of this document).
  • Use a short micro SD adapter with a regular model B to contain the boot partition without having a long lever projecting from the device.

Leave a Reply

Your email address will not be published. Required fields are marked *

*