
S3-compatible
Storage
Source: packages/contentia/src/impl/storage-s3
A StorageRepository on any S3-compatible object store: AWS S3, MinIO, Backblaze B2, DigitalOcean Spaces, Cloudflare R2 over its S3 endpoint, or any other service that speaks the S3 API. On a Cloudflare Worker with an R2 bucket binding, use contentia/storage/r2 instead.
@aws-sdk/client-s3 is not a dependency of contentia. You pass in your own S3Client and the command classes, so code that doesn't use S3 never bundles the AWS SDK.
Usage
import {
DeleteObjectCommand,
GetObjectCommand,
HeadObjectCommand,
ListObjectsV2Command,
PutObjectCommand,
S3Client,
} from '@aws-sdk/client-s3';
import { jsonSerializer } from 'contentia/serializers/json';
import { S3StorageRepository } from 'contentia/storage/s3';
const repo = new S3StorageRepository(
{
client: new S3Client({ region: 'eu-west-1' }),
bucket: 'my-content',
commands: {
HeadObjectCommand,
GetObjectCommand,
PutObjectCommand,
DeleteObjectCommand,
ListObjectsV2Command,
},
},
{ json: jsonSerializer },
'json',
);For MinIO, B2 or any other non-AWS store, set endpoint (and usually forcePathStyle: true) on the S3Client. For R2's S3 endpoint, use region: 'auto' and endpoint: 'https://<account-id>.r2.cloudflarestorage.com'.
Constructor parameters
| Parameter | Position | Required | Default | Description |
|---|---|---|---|---|
config | 1st | yes | — | S3Config, see below. |
serializerRegistry | 2nd | yes | — | Maps file extension → StorageSerializer, same as every other StorageRepository. |
defaultFileExtension | 3rd | yes | — | Extension used for new objects when no other extension is determined. |
ignoreList | 4th | no | ['**/.keep', '**/.DS_Store', '**/Thumbs.db', '**/desktop.ini', '**/.catalog', '**/.contentia'] | Glob patterns excluded from listings. |
determineExtension | 5th | no | defaultDetermineExtension | Overrides how the extension of a new object is chosen. |
S3Config
| Field | Type | Required | Description |
|---|---|---|---|
client | S3ClientLike | yes | An S3Client from @aws-sdk/client-s3, or anything with the same send(command) method. |
bucket | string | yes | The bucket to read from and write to. |
commands | S3Commands | yes | HeadObjectCommand, GetObjectCommand, PutObjectCommand, DeleteObjectCommand and ListObjectsV2Command from @aws-sdk/client-s3. |
keyPrefix | string | no | Stores every key under this prefix, e.g. 'tenant-a' stores docs/x.json as tenant-a/docs/x.json. Slashes are normalised. Callers never see it. |
Behaviour notes
- Folders. S3 has no folders. A folder is a key prefix, and
createFolderwrites an empty.keepobject so an empty folder can still be listed. Zero-bytefolder/placeholders (the kind the AWS console creates) also count as a folder. - Extensions. Objects are stored as
<key>.<extension>, and the public API never shows the extension:getObject('posts/a')finds whichever registered extension exists.createObjectfails withEntryAlreadyExistsErrorif the key exists under any registered extension. - Reads.
getObjectis a singleGetObjectper tried extension. The body,LastModifiedand metadata all come back in one response. createdAtsurvives overwrites. S3 resetsLastModifiedon every PUT, so the creation time is stored as thecontentia-created-atuser metadata (x-amz-meta-contentia-created-at).updateObjectandcreateOrUpdateObjectcarry it forward. Objects written by other tools have no such metadata, so theircreatedAtfalls back toLastModified.- Missing folders.
ListObjectsV2never 404s, so a missing prefix and an empty one look the same. A listing with nothing in it at all (not even a.keepmarker) is treated as a missing folder:listAtoms/listAtomSummariesreturntotal: 0with a recoverableNotFoundError, the same asstorage-fs'slistAtoms, WebDAV, R2, GitLab and Bitbucket. See LCMS-1004. - Deletes. S3's
DeleteObjectsucceeds for keys that don't exist, soremoveAtomsresolves each key first. Missing keys are reported as skipped, with a recoverableNotFoundError. - 403s are errors, not misses. S3 answers 403 instead of 404 for missing keys when the credentials lack
s3:ListBucket. That surfaces as anInternalError, not as "not found", so a permissions problem doesn't look like an empty bucket. Grants3:ListBucketon the bucket pluss3:GetObject,s3:PutObjectands3:DeleteObjecton its objects. - Consistency. AWS S3 has strong read-after-write consistency. Not every S3-compatible store does, so if the readback after a write misses, the repository emits a recoverable warning and returns the object it just wrote instead of failing.
- Capabilities. Pagination is in-memory offset/page slicing over the full listing. Cursor pagination and
subscribeChangesare not supported.
Testing
testing.ts exports InMemoryS3Client and inMemoryS3Commands, an in-memory bucket that follows S3's semantics for the five commands above (404 errors, idempotent deletes, user metadata, ListObjectsV2 delimiters, MaxKeys and continuation tokens). It also exports s3StorageContractCase, which runs the shared storage contract against it. See the storage testkit convention.