Your cache dedupes reads that finished, not reads in flight: the hour that took a shared database down

Google wrote to say one of my sites had picked up a new reason it could not be indexed: server error. By the time I went looking, every page it named answered fine. A crawl of 3,060 URLs under Googlebot's own user agent came back with nothing but success codes, and 131 inspection calls reported no server error at all. The failure had already healed itself. It lasted about an hour.

The database's own request log has that hour. Between 10:00 and 11:00 UTC on 2026-08-16 it took 72,109 requests and failed 2,388 of them. The hour before: 5,648 requests, zero failures. Of the two-hour total, 60,502 went to a single table, and they all came from one site.

The failures were not slow queries giving up. They were 503, 522 and 525, which is the database refusing the connection before a query happens. That instance is small and it is shared, so for that hour it was refusing on behalf of every other site sitting on it, to whoever happened to be reading them. A search crawler was walking through at the time, which is the only reason I found out.

The same question, asked nineteen times in the same instant

Each detail page on that site shows the other entries that came from the same source repository. That panel is one query, and the log stores the query string it arrived with:

repo_full_name=eq.<owner>/<repo>&order=score.desc&limit=7

The identical string, character for character, appears 12 to 19 times inside the same window.

Every read on the site already goes through a cache keyed by the request URL, holding answers for an hour. It works, and it was working then. It just has nothing to offer in this situation. A cache can only answer from a result it is holding, and a result exists only once a read has come back. When a build renders hundreds of pages at once and one repository holds a dozen entries, all of those pages ask the same question at the same moment. Every one of them checks, finds nothing, and goes to the database.

Nine lines, and the third one is not the interesting one

function memoized<T>(key: string, run: () => Promise<T[]>): Promise<T[]> {
  const hit = memo.get(key);
  if (hit && Date.now() - hit.at < MEMO_TTL_MS) return hit.p as Promise<T[]>;
  const p = run();
  p.catch(() => memo.delete(key));
  if (memo.size >= MEMO_MAX) memo.delete(memo.keys().next().value as string);
  memo.set(key, { at: Date.now(), p });
  return p;
}

What goes in the map is the unfinished read itself, not its answer, so the second caller through waits on the first one's request instead of opening its own. It stays there for 30 seconds after it resolves, because a burst does not arrive in a single tick.

Line five is the one worth copying. If the read fails, it leaves the map at once. The function underneath this already retries once on a transient failure, and a failure parked in a cache would answer every page for that repository with the same failure, for as long as it sat there. A cache that remembers a bad answer is a slower version of the outage it was added to prevent.

The other exemption is the read marked fresh, which exists to go past every cache on the way to the database, so it goes past this one too.

Nothing about the fix makes the instance bigger. It makes one page's curiosity about its neighbours cost one read instead of nineteen, which was the difference between a normal hour and an hour where the shared database stopped answering anyone.

That directory is SkillWorks, and this is how we built it: https://skillworks.kynth.studio/?utm_source=kynth-hashnode&utm_medium=social&utm_campaign=kynth

One shipped product, taken apart, once a month. What it does, what it cost to build, what the pipeline behind it looks like, and what the numbers did — read off the repository and the live site, not written from memory. Join the list.