How I almost went insane with CQRS and Event Sourcing

For a while I was convinced I had found the right way to build software. Then I spent far longer undoing it. This is the story of how I used CQRS and event sourcing on a production project, how each reasonable decision led to the next bad one, and what I’d tell myself before starting again.

# It sounded simple

After I learned about Domain-Driven Design, I wanted to try it on something real. I was building an HRIS for the company I worked for, a system meant to dematerialize every HR operation and run in production for actual employees. Not a side project I could abandon when it got hard.

Reading Eric Evans’ blue book and Vaughn Vernon’s Implementing Domain-Driven Design is how I ran into event sourcing and CQRS properly. I had heard both terms for a long time without really understanding them. Now I did, at least to a degree. (The DDD mistakes I made deserve their own post, and I already wrote one: Be careful when adopting Domain-Driven Design.)

Two definitions, since the rest of the story depends on them. CQRS separates the model you use to change state from the model you use to read it. Event sourcing stores state as a sequence of immutable events instead of overwriting rows, so the current state is whatever you get by replaying them.

CQRS looked easy. Separate reads from writes. Rebuild the current state of an aggregate on the write side, serve queries from projections shaped for fast reads. How hard could that be?

# The part that hooked me

Event sourcing fascinated me more than anything else. A system where every value is backed by the full history of events that produced it felt like the right way to model a business. I was so taken by the idea that I never stopped to ask what it would cost.

The examples helped. Banks track balances as a ledger, not as a number you overwrite, and that fit one part of my project perfectly: employee leave balances. So I designed the balance as an immutable list of events, and the current value as whatever those events add up to. It was my first time using event sourcing, and for that use case it was a good call.

Here is the idea with made-up numbers:

LeaveGranted     +1.5   (monthly accrual)
LeaveTaken       -2.0   (approved request)
LeaveCorrected   +0.5   (HR fixed a wrongly counted day)
---------------------
balance           0.0

Nothing is edited or deleted. A mistake is fixed by appending a correction, and anyone asking “why is my balance this number?” gets an answer by reading the list. For leave, that audit trail is a real business requirement, which is exactly why the pattern fits.

Then I started digging deeper. What happens when a ledger grows very large? I found snapshots and a few other techniques for managing long event streams, tried them, and they worked.

So I had a pattern I had finally understood, a use case where it clearly fit, and a few months of it working. I took that as proof. Looking back, this is where I should have stopped and asked what had made it work.

# Why stop at the ledger?

Once the ledger worked, everything else started to look like a ledger. Who wouldn’t want the entire timeline of what led to the current state of the system? There was also time travel: showing what an aggregate looked like on any given date by replaying events up to that point. It seemed fascinating, and I was too excited to ask whether anyone would ever use it.

Employee information was the first thing I moved over. It barely changed and nobody needed its history, but it was an aggregate like any other, and treating it differently felt inconsistent. So it got CQRS and event sourcing too. Then the next module did. I had stopped deciding and started applying.

# The cracks on the write side

The first problems appeared in the event store. I had built a custom one on top of Postgres, and every week the project grew, the list of things I had to worry about grew with it. Event versions, performance as streams got larger, optimistic concurrency control, and a growing list of edge cases.

The core of the store looks harmless:

CREATE TABLE events (
  stream_id  uuid        NOT NULL,
  version    integer     NOT NULL,
  type       text        NOT NULL,
  payload    jsonb       NOT NULL,
  created_at timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (stream_id, version)
);

The primary key on (stream_id, version) is what gives you optimistic concurrency. Two users act on the same aggregate at the same moment, both load it at version 7, both try to append version 8, and one insert fails. Now the application has to decide: retry, reload and re-validate, or tell the user. Multiply that by every aggregate and every command handler and it stops being a detail.

Events are also forever. An event shape I wrote in month one stopped matching what the code needed later, but past events can’t be edited, so old ones have to be handled every time they’re read. And streams that kept growing made loading an aggregate slower.

None of this was unsolvable, and that was the trap. Every problem had a known fix, so none of them felt like a reason to stop. And the write side turned out to be the easy half.

# The read side, where it got worse

I started by storing read models in the same Postgres database, with separate services listening to events and rebuilding the projections. Postgres is structured, though, and that made some read projections harder to design than I wanted. I wanted something more powerful, and Elasticsearch looked like the perfect candidate.

It solved one problem and created several. Its built-in optimistic concurrency control became one more thing to reconcile with mine. Worse, read models are full of related objects: a leave belongs to an employee, an employee has a supervisor, and so on. Changing an employee record meant finding and syncing every document that related to it. A simple edit now fanned out into a chain of updates.

And all of this was eventually consistent. Everything the UI read now sat behind a projector, and that is where the project started to hurt for real.

# When a successful write looks like it failed

A user would submit a form and the write would succeed. The UI then immediately invalidated the affected queries and refetched, but the projection hadn’t caught up yet, so the screen showed the old data. From the user’s side, they did something and nothing happened.

The sequence looked like this:

The user submits the form and the command handler appends an event.
The API returns success and the UI invalidates its queries.
The refetch hits the read model.
The projector hasn’t processed the event yet, so the read model returns the old state.
A moment later the projector catches up, but the screen is already wrong.

The gap lasted a fraction of a second, which is more than enough. Nobody wants to use an app where a successful action isn’t visible right after, and mine was almost unusable.

My first fix was to rebuild the read model synchronously, at the moment the write was committed. It worked, but it hurt performance badly, and it quietly defeated the point of CQRS: the whole reason for a separate read side is that writes don’t wait on it.

The second fix was worse. When eventual consistency had worn me down far enough, I started generating read models on the fly from the events. That meant replaying events on every read, which is exactly what projections exist to avoid. I had built an elaborate read side and then bypassed it.

# The breaking point

There was no single outage or incident. It was slow. Simple adjustments took forever. Adding a feature got more tedious every week, and I kept getting lost in everything that now had to be covered. A change to one concept meant updating:

the event contracts
the value objects
the aggregate
the repository
the projection
the existing records that were now outdated

A change that a normal UPDATE statement would have handled became a chain of edits across half the codebase, with a risk of missing one link.

The complexity wasn’t coming from the HR domain. It was coming from the architecture I had chosen to carry it.

# Pulling back

I accepted that most of the project didn’t need any of this. It took a long time to undo, because the project was already drowned in the mess. I worked alone, and I wrote complex scripts to extract data from the old system and normalize it into the new structure.

Everything moved to a regular Postgres database. I kept events for only a few features, the ones where history is part of the business: the leave balance ledger and the overtime counter.

The refactor was worth it. Features became faster to build, the code became predictable, and my confidence came back with it. I now know when to use event sourcing and CQRS, and when not to.

# What I’d tell myself before starting

Ask who will read the history. Time travel is exciting, but if nobody needs to ask how a value got here, an event stream is a cost with no benefit.
Apply it per aggregate, not per project. The leave ledger and the overtime counter earned it, the employee profiles didn’t.
Treat them as two decisions. CQRS and event sourcing are independent. CQRS on plain tables is much cheaper, and I never separated the two.
Treat eventual consistency as a product decision. If the UI can’t tolerate a stale read, the design has to account for it from the start, not after users complain.
Watch for the workaround that defeats the pattern. Rebuilding read models synchronously or on the fly is a sign you’re no longer getting what you paid for.
Don’t confuse excitement with a requirement. I picked this because it was fascinating, not because the problem needed it.

The project is in production now, and I’d use CQRS and event sourcing again for the right problem. I just wouldn’t start with them. Start with the simplest thing that works, and let a specific part of your domain prove it needs more.