Ask
92
@carryon_only ·

r2 or s3+cloudfront for 8 tb a month of egress on a $400 budget Egress

Video course platform, small. About 8TB/month goes out to viewers and growing maybe 15% a month. Total infrastructure budget is $400 and right now egress alone would eat all of it on the setup I have.

Currently everything is in S3 and served directly, no CDN, which I now understand is the worst possible configuration. The question is whether to put CloudFront in front of it or move the objects to R2 and accept a migration.

Assets are about 600GB of MP4 that changes slowly. My encoding pipeline writes to S3 and I have a Lambda that triggers on object create for thumbnail generation.

What would you do with those constraints?

9 answers Share
Report

Answering anonymously — a moderator will review it first.

  • @egress_egon · 4mo ago · 3 replies

    At 8TB and growing 15% a month, move. Zero egress is not a marginal saving at your volume, it is the difference between a viable product and one where every new customer makes the maths worse.

    Rough monthly egress at list price for 8,000GB:

    • S3 straight to the internet: around $711
    • S3 behind CloudFront: around $595 after the free tier
    • R2: $0

    You pay storage on R2 either way - 600GB is roughly $9 - plus operations, which for video streaming are Class B and negligible. So your $400 budget goes from impossible to mostly unspent.

    The honest reasons not to move:

    • You need S3 event notifications into an existing AWS pipeline. Your thumbnail Lambda is exactly this. R2 has event notifications too, but they land in a different queue system and you would rewrite that piece.
    • Lifecycle rules into archival storage classes, if you have a lot of cold content.
    • Any AWS-native feature applied on the object path.

    Given you have exactly one Lambda hook, rewriting it is a day of work against $600 a month. That is not a close call.

    One genuinely important detail: the zero-egress model assumes normal serving. Read the terms for your traffic type rather than assuming, and if you are ever unsure whether your usage fits, ask them before you migrate 600GB.

    83
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
    • @quietmargin · 4mo ago

      One more thing to check before committing: whether you need signed URLs with short expiry for paid content, and that whatever you move to supports the exact shape of access control you use today. Getting the storage bill to zero and accidentally making your paid videos scrapeable is a bad trade.

      20
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
    • @carryon_only · 4mo ago

      The thumbnail Lambda is the only AWS-specific thing in the whole pipeline and it is about 60 lines. Sounds like the migration is genuinely smaller than I had built it up to be.

      25
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
  • @durable_ines · 4mo ago

    Before you choose, measure. 8TB of egress for a small course platform sounds high to me and I would want to know why before I optimise the price per GB.

    Things that inflate video egress:

    • no range request support, so a seek re-downloads from the start
    • serving one large MP4 rather than adaptive segments, so a viewer who watches 3 minutes of a 40 minute video still pulls the whole file
    • no caching headers, so a rewatch is a fresh download
    • a preview that autoplays on a listing page

    Fixing the second one alone can cut egress by more than half on a platform where people sample a lot of content and finish little of it. Cheaper per GB is good. Fewer GB is better and it compounds with whatever you pick.

    35
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
  • @quietmargin · 4mo ago · 2 replies

    Third option nobody mentions: keep S3 as the origin and put a commodity CDN in front at around $0.01/GB. That is roughly $80/month at 8TB, you change one hostname, you migrate nothing, and your Lambda triggers keep working exactly as they do today.

    It is not $0, but it is $80 against a $400 budget and it ships this afternoon rather than next month. If video is your core business, the R2 migration is still right eventually. If you are trying to find out whether the business works at all, do the cheap thing now.

    Watch origin egress though - S3 to CDN is still billed egress on the AWS side unless your CDN has an arrangement for it. With a 600GB library and slow-changing content your hit rate should be very high, so origin pulls are small, but check rather than assume.

    51
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
    • @kraut_corner · 4mo ago

      The origin egress caveat is the one that catches people. If your cache hit ratio is 95% you are paying AWS egress on 5% of 8TB, which is around $36. Fine. If you have a long tail of rarely-watched videos that always miss, it can be much worse than that, and video libraries are exactly the shape where a long tail exists.

      18
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
  • @kraut_corner · 4mo ago

    On mechanics of the move: you do not have to do a big-bang copy. There is tooling to lazily migrate objects on first read - requests that miss the new bucket fall through to the old one and get copied on the way - so you can switch reads over immediately and let 600GB drain across however long it takes. Much less scary than a weekend cutover.

    22
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
  • @quiet_stacker · 4mo ago

    Note that CloudFront has a free tier that covers a meaningful chunk of traffic per month, so at 8TB it is not as bad as list price suggests - the first slice is free and the rest is tiered down as you grow.

    10
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
  • @napgapnora · 4mo ago

    Serving 8TB of video directly out of S3 with no CDN is the storage equivalent of leaving the fridge door open.

    5
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report