Skip to content
GKgkml.dev
← Change mode

The lifecycle rule that increased S3 cost

Design review
AWSSAA-C03MLOpshard

A bucket holds 40 TB of model training artefacts: roughly 9 million objects, most between 20 KB and 200 KB, with a few large checkpoint files. Objects are read heavily in the week after creation and almost never afterwards.

To cut cost, the team added a lifecycle rule transitioning objects to Glacier Flexible Retrieval after 30 days. Versioning is enabled on the bucket.

The following month's S3 bill is 20% higher than before, and a subsequent audit run that needed to read 200,000 old objects took two days and added a further charge.

Explain every mechanism by which a rule intended to reduce cost increased it. Then tell me what the correct lifecycle policy is for this access pattern and why.

Enter sends · Shift+Enter for a new line