Due to a combination of hoarding (digital and physical), frugality, and laziness I have a large number of external hard drives and cloud services that I use for general storage and backup. I also have an annoying tendency to physically misplace things (adds to the backup paranoia), so I like having some level of basic encryption for removable devices for privacy reasons (my threat model is identity theft, not government agencies).

My (grossly over-engineered) solution involves two fantastic pieces of userland software:

• MergerFS: FUSE filesystem that pools drives into a single mount
• Rclone: Versatile cloud backup and sync tool, also supports local files

The workflow here is roughly as follows:

1. Plug in one or more external drives and mount locally
2. (optional) Use rclone to mount a readonly cloud drive (i.e. Dropbox, Google Drive, etc)
3. Pool the drives into a unified folder using mergerfs
4. Decrypt and mount the pooled content using rclone

After jumping through these hoops, I now have a single folder structure with my files across local and cloud storage. Once mounted, MergerFS tools provides some nice scripts for duplicating (or de-duplicating) content across your drive pools.

This type of thing isn’t useful for many people, but if you’re interested in replicating this arcane setup, here’s how to go about it (instructions assume Linux, should work with minor modifications on MacOS):

1. Choose one or more external drives you’d like to use. There’s no need to clean format the drive (other files can live on the drive as well). Create a folder called encrypted in the root of each drive.

2. (Optional) If you’d like to use Cloud Storage as well, setup one or more providers by following the rclone config instructions. Once setup, use rclone to mount it locally:

   # Replace `cloud` with whatever you named the remote
   # NOTE: Must create `/media/cloud` folder beforehand
   # NOTE: Must create `cloud:/encrypted` folder beforehand via `rclone mkdir`
   # `allow-other` and `read-only` flags are optional
   rclone mount cloud:/encrypted /media/cloud --allow-other --read-only
   

3. Use mergerfs to mount the drives together in a pool:

   # NOTE: Must create `/media/encrypted_pool` if it doesn't exist
   # Replace `/media/driveX` with paths to the removable/cloud drives
   # `allow_other` setting is optional
   mergerfs /media/drive1/encrypted:/media/drive2/encrypted:/media/cloud /media/encrypted_pool \
     -o defaults,fsname=encrypted_pool,allow_other \
     -o moveonenospc=true,category.create=epmfs,func.getattr=newest
   

4. Use rclone to setup the encrypted storage. Use /media/encrypted_pool as the remote path.

5. Mount the decrypted drive:

   # NOTE: Must create `/media/decrypted_pool` if it doesn't exist
   # Replace `pool` with whatever you named the encrypted storage in step 4
   # `allow-other` setting is optional
   rclone mount pool:/ /media/decrypted_pool --no-modtime --allow-other
   

You now have a pooled view of your encrypted folders, congratulations! Of course, they’re completely empty so it is kinda useless.

At this point, you can start copying data into this merged view but I wouldn’t recommend it for a few reasons:

• It won’t be obvious what data is stored on each pooled drive. If you’re always using the same drives (or never disconnect), then maybe this isn’t a concern for you.
• Filesystem tools like cp and rsync are going to have to go through two FUSE layers, which isn’t great for performance.
• For cloud storage, the rclone documentation explains that using rclone sync is the best practice for copying into the cloud (if you mounted your cloud drive with --read-only, this isn’t an issue since you’ll never upload anything).

In order to workaround these issues, I manually add data to individual drives via rclone sync. In order to do this, you need to setup rclone storage for each drive you’re using. You can do this manually via rclone config (use the same passwords you used for the merged drive). Alternatively, it’s easier to manually edit the rclone.conf file and copy the config over per drive.

At this point, your rclone.conf file looks something like:

[cloud]
# `type` varies depending on storage provider
type = s3
client_id =
client_secret =
token = ...

[pool]
type = crypt
remote = /media/encrypted_pool
filename_encryption = standard
password = ...
password2 = ...

Instead of running rclone config a bunch of times, you can copy the [pool] section and just change the remote path, for example:

# Create one of these per drive
[drive1]
type = crypt
remote = /media/drive1/encrypted
filename_encryption = standard
password = ...
password2 = ...

Since the encryption parameters are identical, anything you store in this rclone remote can be read by the pooled mount. To add data to the pool, you can now rclone sync stable_genius_files drive1:/. Do the same setup for your cloud remotes as well.

Running these commands manually is a pain, so if you’re running Linux (and maybe MacOS, I haven’t tested), I’ve created a helper script that automatically takes care of mounting and pooling the drives (also posted as a gist):

#!/usr/bin/env bash
set -euf -o pipefail

# Removable drive names go here, separated by spaces
DRIVE_NAMES=(drive1 drive2 drive3)

# Removable and cloud drives are mounted in this local directory
DRIVE_MOUNT_ROOT="/media"
# Name of folder for root of encrypted storage on drives
ENCRYPTED_DIR_NAME="encrypted"
# Rclone remote name for pooled drives
RCLONE_POOL_NAME="encrypted_pool"
# Local path for mounting decrypted drive pool
DECRYPTED_MOUNT_PATH="/media/decrypted_pool"

# Helper for joining array items with `:`
function semicolon_join { local IFS=':'; echo "$*"; }

# Helper for unmounting on exit
function finish {
  echo "Cleaning up and unmounting ..."

  if mount | grep -q "$DECRYPTED_MOUNT_PATH"; then
    echo "Unmounting merged rclone at $DECRYPTED_MOUNT_PATH"
    fusermount -u "$DECRYPTED_MOUNT_PATH" > /dev/null 2>&1
  fi

  if mount | grep -q "$DRIVE_MOUNT_ROOT/merged"; then
    echo "Unmounting mergerfs at $DRIVE_MOUNT_ROOT/merged"
    fusermount -u "$DRIVE_MOUNT_ROOT/merged" > /dev/null 2>&1
  fi

  for name in "${CLOUD_DRIVE_NAMES[@]}"; do
    mount_path="$DRIVE_MOUNT_ROOT/$name"
    if mount | grep -q "$mount_path"; then
      echo "Unmounting cloud remote $name"
      fusermount -u "$mount_path" > /dev/null 2>&1
    fi
  done
}

trap finish EXIT

VALID_MOUNTS=()
echo "Checking removable drives"
for name in "${DRIVE_NAMES[@]}"; do
  mount_path="$DRIVE_MOUNT_ROOT/$name/$ENCRYPTED_DIR_NAME"
  if [ -d "$mount_path" ]; then
    VALID_MOUNTS+=("$mount_path")
  fi
done

# Mount cloud drives, if any
CLOUD_DRIVE_NAMES=(gdrive)
echo "Checking cloud drives"
for name in "${CLOUD_DRIVE_NAMES[@]}"; do
  mount_path="$DRIVE_MOUNT_ROOT/$name"
  if ! mount | grep -q "rclone_$name"; then
    echo "Mounting rclone remote $name ..."
    rclone mount "$name:/$ENCRYPTED_DIR_NAME" "$mount_path" \
      --allow-other --read-only &
  fi

  VALID_MOUNTS+=("$mount_path")
done
echo "Found drives: ${VALID_MOUNTS[*]}"

echo "Mounting pool via mergerfs"
mergerfs "$(semicolon_join "${VALID_MOUNTS[@]}")" "$DRIVE_MOUNT_ROOT/merged" \
  -o defaults,allow_other,moveonenospc=true \
  -o fsname=encrypted_merged,category.create=epmfs,func.getattr=newest

echo "Mounting decrypted pool"
rclone mount "$RCLONE_POOL_NAME:/" "$DECRYPTED_MOUNT_PATH" \
  --allow-other --no-modtime &

echo "Mounting complete, hit Control-C to unmount and exit"
wait

If you’ve made it this far, congratulations for creating a brittle, complex storage system instead of just buying a bigger drive! Now I assume you can get back to growing your own vegetables, sewing your own clothing, and other inefficient uses of the precious little time you have left before dying alone.