A chat feature at tens of millions of DAU

Context

A consumer app ships an AI chat feature to tens of millions of daily users. Volume is enormous and compounding, most traffic runs on a cheap model, and the team can keep GPUs busy.

The decision

At this scale the crossover to self host arrives within the first year, pay per token can't compete once utilization is high and volume compounds.

What most miss

The API bill looks fine in the pilot and becomes a scary line item by month nine. The cliff is a function of growth × utilization, not today's invoice.

Stakes

Staying on usage based pricing past the cliff at consumer scale is a seven figure annual overspend.