Mastodon

The S3 Protocol

S3 is a protocol, not a product. Amazon S3 was the first implementation, but "S3" today means the API — a standard way for apps to talk to object storage over HTTP. Anything that "speaks S3" works with any S3-compatible server, including Storm Cellar.

If your app already supports Amazon S3, Backblaze B2, Wasabi, or MinIO, it already speaks S3. Point it at our endpoint and it just works.

How it works

Every S3 interaction is an HTTP request to an endpoint, signed with your access key. The server (Storm Cellar, in our case) processes the request and returns a response. That's it.

PUT /my-bucket/notes/today.md HTTP/1.1
Host: s3.garage-one.stormdevelopments.ca
Authorization: AWS4-HMAC-SHA256 Credential=GK...

Your app builds these requests for you. You never see them unless you're debugging.

The four operations

Almost everything an S3 app does is one of four operations:

Operation What it means When apps use it
PUT Upload an object Saving a file, syncing changes
GET Download an object Opening a file, streaming media
HEAD Check if an object exists, fetch metadata Sync clients deciding whether to upload
DELETE Remove an object Cleaning up, applying remote deletions

Plus LIST for enumerating objects in a bucket. That's the whole vocabulary most apps need.

Every one of these shows up in your bucket's Activity feed in Storm Cellar. When Obsidian syncs a vault, you'll see a burst of HEAD requests (checking what's already there) followed by PUT requests (uploading what's new). When PeerTube serves a video, you'll see GET requests as viewers watch. The protocol is transparent — there's no magic.

Authentication

Every request is signed with an access key pair:

  • Access Key ID — public identifier, like a username (GK1234...)

  • Secret Access Key — private signing secret, like a password

Your app uses the secret to sign each request. The server verifies the signature without ever seeing the secret on the wire. This is AWS Signature V4, the same scheme Amazon uses.

Keys are scoped per-bucket in Storm Cellar — a key for my-vault cannot read or write my-photos. See Managing Access Keys.

Path-style vs virtual-hosted

There are two ways to address a bucket in a request URL:

  • Path-style: https://s3.example.com/my-bucket/file.txt

  • Virtual-hosted: https://my-bucket.s3.example.com/file.txt

Storm Cellar uses path-style. AWS deprecated path-style for new buckets but every other S3-compatible server still uses it. Most S3 client libraries default to virtual-hosted, so you'll need to enable a "force path style" option. Every client we document in Connecting an S3 Client shows where that switch is.

For developers building S3 support

If you're adding S3 support to your own app, you don't need to implement the protocol from scratch. Use an official AWS SDK and point it at Storm Cellar:

import boto3

s3 = boto3.client(
    's3',
    endpoint_url='https://s3.garage-one.stormdevelopments.ca',
    aws_access_key_id='YOUR_KEY_ID',
    aws_secret_access_key='YOUR_SECRET',
    region_name='garage',
    config=boto3.session.Config(signature_version='s3v4', s3={'addressing_style': 'path'}),
)

s3.put_object(Bucket='my-bucket', Key='hello.txt', Body=b'Hello, Cellar.')

The AWS SDKs (Python boto3, JavaScript @aws-sdk/client-s3, Go aws-sdk-go-v2, Rust aws-sdk-s3, etc.) all accept a custom endpoint_url. They were built for Amazon but the protocol is open — any S3-compatible server works.

Three settings to remember when configuring any SDK against Storm Cellar:

  1. endpoint_urlhttps://s3.garage-one.stormdevelopments.ca

  2. regiongarage (the value isn't meaningful to us, but the SDK requires it)

  3. addressing_style: 'path' — force path-style URLs

Why this matters

S3 being a protocol — not Amazon's proprietary thing — is what makes Storm Cellar possible. We don't reverse-engineer anything. We run Garage, an open-source S3-compatible server, and your app talks to it the same way it would talk to AWS. If you ever want to leave, every byte you've stored is portable to any other S3-compatible provider.

That's the protocol's quiet superpower: no lock-in.