Vibe Coding my way to a better cache

I couldn’t shake the feeling our cache wasn’t optimal. We’ve been using it for years and whenever we saw spikes in CPU usage, this was my main scapegoat.

The Problem

Our CloudRun service was burning CPU cycles. Our existing cache solution was handling around 2000 articles cycling through memory every 60 seconds, with TTL and check periods both set to 60 seconds. It worked fine, but it wasn’t optimal and something felt off.

I started looking at alternatives and landed on one of the most popular cache packages. Installed it and had Claude Code set up tests that mirrored our real usage patterns. After comparing them head-to-head, the performance difference? Negligible. Both handled our use case about the same.

But something still felt off. As I said… it was my scapegoat.

The Investigation

Instead of letting it slide and keeping the older package, I asked Claude Code to explain exactly how each approach worked under the hood. That’s when the problems became obvious.

Our current solution stored everything in one massive object. At every check period, it would serialize and deserialize the entire thing, then traverse all 2000 entries checking TTL values. Imagine doing a full table scan on your database every minute just to clean up expired records.

The alternative took the opposite route. It created individual timeout handlers for each cache entry. 2000 articles meant 2000 active timers, each adding its own overhead to the event loop.

Both solutions were solving the wrong problem for us. One was doing too much work per check, the other was creating too many workers. The hunch about the existing cache being suboptimal was right, but the alternative wasn’t much better. CPU-wise or memory-wise.

The Solution

What if I could separate the concerns? One object to track keys and their TTL values, another to store the actual data. Run a single check period that only looks at the TTL object.

When an entry expires, delete from both objects. When you need data, check the TTL object first, then fetch from the data object if it’s still valid.

Yes, it uses marginally more memory. But memory is cheap on CloudRun. CPU cycles are expensive.

I asked Claude Code to build this approach and create a comprehensive test suite. We borrowed test patterns from the existing library to ensure compatibility, then ran everything in random order to catch any hidden dependencies.

The code worked immediately. Claude Code reported some astronomical speedup number (412x faster, which was probably measuring some micro-operation), but the real-world impact was what mattered: CPU usage dropped from 50% to 25%.

This is what vibe coding enables. You have a gut feeling that something’s not right, so you dig deeper instead of accepting the obvious solution. You ask questions about implementation details that are easily skipped. You trust your intuition about performance characteristics and let AI help you explore alternatives rapidly.

Neither approach was bad. Our existing solution had served us well for years. The alternative was popular for good reasons. They just weren’t optimized for this specific pattern. One prioritized simplicity of implementation. The other prioritized individual entry control. Neither prioritized bulk expiration efficiency.

The key was asking the right questions:

• How exactly does TTL checking work?
• Where are the CPU cycles actually going?
• What happens during the check period?
• Can we separate the expensive operations from the frequent ones?

Since the two existing packages performed the same we could have easily just ignored upgrading. But that nagging feeling about optimization was pointing toward something better.

The Lesson

I haven’t released this as a package because it’s deeply integrated into our codebase. It makes assumptions about our specific use patterns and doesn’t handle edge cases that a general-purpose library would need to support.

That’s the trade-off with vibe coding. You can build exactly what you need, optimized for your constraints, but it’s not necessarily portable. Sometimes the perfect solution for your problem isn’t the right solution for everyone else’s problem.

The lesson isn’t that you should always build custom cache solutions. It’s that you should understand your tools well enough to know when they’re not quite right, and have the confidence to explore alternatives when your gut says there’s a better way.