Skip to main content

Infinidisk

Infinidisk turns a bucket into a Linux disk. Format it like any disk (ext4, XFS, btrfs) and use it from any program: there is no size to plan, and you pay only for what is stored. It is open source: github.com/elestio/infinidisk.

How it works​

  • Hot data stays in a local cache on the machine; the rest is read from the bucket.
  • Writes are uploaded to the bucket in the background, in order. If the machine stops suddenly, the writes not sent yet can be lost (usually a few seconds). The filesystem itself is never corrupted: on recovery you get a consistent disk, just missing the very last writes.
  • The disk stays in the bucket. Unmount it, then mount it on another machine with the same commands. One machine at a time: the disk is locked while mounted.
  • A bucket can hold several disks: each one has its own name and place in the bucket.

Requirements​

A Linux machine (amd64 or arm64) with root access, as on every Elestio service, and a Read and write API key for the bucket.

Set it up​

Open the bucket, then the Connect tab, section Use this bucket as a Linux disk.

  1. Enter a disk name and a size (it can be raised later; unwritten space is not stored). Optionally set a local cache size in GB (default 4) and a mount point (any empty directory; default /mnt/infinidisk/<name>).
  2. Copy and run the commands shown:

Install Infinidisk and its runtime:

curl -fsSL https://raw.githubusercontent.com/elestio/infinidisk/main/install.sh | sudo bash

Create and mount the disk, with your Read and write key:

sudo infinidisk create <name> \
  --bucket <bucket> \
  --endpoint https://storage.elestio.com \
  --size <size> \
  --access-key <your access key> \
  --secret-key <your secret key>

Add --cache-gb <N> to size the local cache, or --mountpoint /your/path to mount at a custom empty directory instead of /mnt/infinidisk/<name>.

Mount it again at every start, and unmount when done (writes are flushed to the bucket first):

sudo infinidisk enable <name>
sudo infinidisk umount <name>

The disk is mounted on /mnt/infinidisk/<disk name> (or the mount point you chose).

Performance​

Measured on an Elestio service with 2 CPUs and 4 GB of memory, compared with the local disk of the same machine:

  • Files read again or recently written: as fast as the local disk (they are in the local cache).
  • Files read for the first time: fetched from the bucket, so slower; the cache keeps them for the next reads.
  • PostgreSQL (pgbench, 2 million rows, 8 clients): read-only queries are fast thanks to the cache; read-write transactions reach a few thousand per second, which suits most workloads. Keep very write-heavy databases in mind and test with your own load.

Good practices​

  • Keep a reasonable local cache (the default is fine for most machines). A very small cache makes the disk slower.
  • Check how far behind the disk is with sudo infinidisk status <disk name>: the pending to S3 value is what is not in the bucket yet.
  • Before moving a disk to another machine, or restarting after heavy writes: stop the application, then sudo infinidisk umount <name> drains the remaining writes to the bucket before releasing the disk.
  • For a database, keep regular backups of the database itself.

Cost​

You pay for the data stored, plus about 1,300 write requests per GB written. See Billing and limits.