KalamAI is the tool that turns a keyword into a finished article. Yesterday it kept freezing halfway, and I spent most of the day learning that rate limits and timeouts are not edge cases in an AI pipeline. They are the weather.
The first crack showed up in a step I call R3 embed -- the part that pulls extracted content and turns it into embeddings before drafting. Under load it would start hitting 429s, the too-many-requests wall, and instead of failing cleanly it just hung. Not an error. Not a crash. Just a spinner that never stopped. From the outside the whole tool looked dead.
My first fix was the obvious one: bound the embed step to a wall-clock budget. If it cannot finish in the time I give it, stop, take what you have, and move on. I shipped that, watched it, and it still wedged under a bad enough 429 storm. So I went back and hard-bound it -- a real ceiling measured against the clock, not against retry counts, because retries are exactly what a rate-limit storm feeds on. That one held.
The 60-second wall
Then the drafting side. KalamAI was writing the whole article in one call, and long articles kept wedging right at the 60-second mark. Serverless functions have a ceiling and I was walking straight into it. The fix was to stop pretending an article is one big thing. Draft it section by section. Each section is its own short call, well under the wall, and a long piece is just more sections, not a longer gamble. Slower on paper, but it actually finishes.
The last one stung the most. A single 529 -- the overloaded response from Anthropic -- would fail an entire article. All that crawling, extracting, embedding, drafting, gone because one call landed while the model was busy for a second. That is not a bug in my logic, it is me treating a transient blip as fatal. So I added retries on transient errors. One 529 should never cost you the whole run.
Retries are exactly what a rate-limit storm feeds on -- so the ceiling has to be the clock, not the retry count.
— note to self, buried in a commit message
Somewhere in the middle of all this I also shipped the thing this was all for: a one-hit keyword-to-article flow. Type a keyword, get an article. That was the whole point, and it only felt safe to ship once the pipeline underneath it stopped hanging on me.
What I took away: when you build on top of someone else's model, their bad moments become your reliability problem. A 429, a 529, a slow response -- these will happen, and if your design assumes they will not, the user just sees a frozen screen and leaves. Half of building an AI tool is the AI. The other half is what you do while it is having a bad day.
The article quality still is not where I want it. That is tomorrow's fight. But at least now the machine finishes what it starts.