Skip to content
GKgkml.dev
← Change mode

Inference cost tripled while traffic stayed flat

Design review
AWSMLOpshard

A batch scoring service reads feature files from S3, scores them, and writes results back to S3. It runs on EC2 instances in private subnets, egressing through a NAT gateway. Instances are covered by a Compute Savings Plan.

Month over month: requests scored is flat within 3%. Total bytes read from S3 is up 4%. The AWS bill for this workload has gone from about $2,100 to $6,400. EC2 instance-hours are unchanged.

Two changes landed last month. The team enabled a new feature group, which added six columns to the feature files. And they moved the scoring fleet from one Availability Zone to three, for resilience.

Instance-hours are flat and bytes read are up only 4%, yet the bill tripled. Where is the money going? Name the line items in order of likely contribution and how you would confirm each from billing data alone.

Enter sends · Shift+Enter for a new line