Mastodon

Proxmox backups on Storm Buckets

There are two ways to get Proxmox backups into a Storm Bucket. If you run
Proxmox Backup Server, give it an S3 datastore pointed at your bucket. If you
run Proxmox VE on its own, let vzdump write to a local directory and send that
directory to the bucket with restic.

Proxmox Backup Server has supported S3 datastores since 4.2. Proxmox VE has no
S3 storage type of its own, which is what the second route is for.

What you need first

  • A bucket for the backups and nothing else. See
    Getting Started.

  • A Read / Write key scoped to that bucket. See
    Managing Access Keys. Proxmox Backup Server
    prunes and garbage-collects, so it needs delete, which Read / Write includes.

  • Your bucket's endpoint hostname, from the Connection panel of your bucket
    page. Proxmox wants the hostname with no https:// in front.

Build your commands

Pick a route and type your bucket name. The builder writes the commands for Storm, and the sections below explain each one.

proxmox-backup-manager s3 endpoint create storm \
--endpoint montreal-1.buckets.stormdevelopments.ca \
--access-key <your key id> --secret-key <your secret> \
--region montreal-1 --path-style true

proxmox-backup-manager s3 check storm your-bucket

proxmox-backup-manager datastore create storm-backups /mnt/s3-cache \
--backend type=s3,client=storm,bucket=your-bucket

Route one: a Proxmox Backup Server S3 datastore

The cache disk

An S3 datastore keeps a local cache, and Proxmox requires it. Proxmox
recommends 64 to 128 GiB on a dedicated disk, partition, or ZFS dataset with a
quota. It cannot be an existing datastore. Add the disk before you start.

Add Storm as an S3 endpoint

In the web UI this is Configuration > Remotes > S3 Endpoints. On the command
line it is the first command the builder writes.

Proxmox Backup Server uses virtual-hosted addressing unless told otherwise, and
Storm is path-style only. The region is your endpoint's name, for example montreal-1. Both values are on the Connection panel of your bucket page. Leave
either line out and the requests fail authentication. See
Troubleshooting for what each mistake looks
like.

The second command, s3 check, confirms the endpoint can see your bucket.

Create the datastore

The third command creates it. /mnt/s3-cache is the cache disk. Add the datastore to Proxmox VE as Proxmox
Backup Server storage and schedule backups to it like any other datastore.

One server per datastore

A datastore belongs to one Proxmox Backup Server. Two servers writing to the
same datastore in one bucket will corrupt each other's view of it. The
datastore name is its prefix inside the bucket, so to reattach after a rebuild,
create it again with the same name:

proxmox-backup-manager datastore create storm-backups /mnt/s3-cache \
  --backend type=s3,client=storm,bucket=your-bucket \
  --reuse-datastore true --overwrite-in-use true

Route two: vzdump and restic

Proxmox VE writes backups with vzdump. Point a backup job at a local directory
storage, then send that directory to your bucket with restic. The builder's
second route writes the repository settings and the backup of /var/lib/vz/dump,
the default local storage. The restic guide
covers the password, checks, restores and the schedule.

Run it after the vzdump job finishes, from the same cron entry or a vzdump hook
script. restic deduplicates between nights, but a compressed archive changes
throughout when one block of the disk changes. Set the backup job's compression
to none and let restic compress and deduplicate instead.

If a backup fails

  • Authentication errors are the region or the addressing style. See
    Troubleshooting.

  • Uploads refused with a 501 mean the conditional upload Proxmox sends was
    rejected. Proxmox has a provider quirk for it:

proxmox-backup-manager s3 endpoint update storm --provider-quirks skip-if-none-match-header
  • Anything else, send it in from the dashboard with the error text. A
    client that fails against Storm is a bug we want to know about.

What this does not cover

  • A key that can prune can delete. Anyone holding this key can destroy the
    backups. Keep it on the backup server and nowhere else.

  • Storage on Storm is single-zone. Keep a second copy of anything you cannot
    afford to lose, as the Terms of Service say.

Next steps