the Room Protocol · concurrency

Concurrency control for a shared plan: distributed locks and per-agent cursors

The whole promise of multi-agent work is agents running in parallel. The moment they do, they can corrupt each other's state. Here's how the Room Protocol stays consistent under concurrency — and why I reached for a Redis lock instead of the fancier options.

In the overview I argued agents should coordinate through a shared room instead of chatting. There's a catch buried in that idea. If the value of multi-agent development is that agents work at the same time, then the shared state they coordinate through is, by definition, under concurrent access. And concurrent access to mutable shared state is where quiet, awful bugs live.

This post is about three of those bugs and the three mechanisms that kill them: a distributed lock for the plan, per-agent cursors for events, and TTL-based presence. None of the mechanisms is exotic. What's worth your time is why each one, over the alternatives that look more sophisticated on paper.

One JSON document, two writers

The plan is a single JSON blob per room: a list of tasks with status, owner, and dependencies. Agents change it with read-modify-write. Read the plan, flip one task to in_progress, write the whole thing back. That is the textbook setup for a lost update.

Agent A reads the plan. Agent B reads the same plan a millisecond later. A marks task 1 done and writes. B marks task 2 done and writes — over A's version, which B never saw. Task 1 silently reverts to pending. No error, no crash. An agent just quietly loses work, and the first anyone notices is when something gets built twice or not at all.

The obvious fixes, and why I skipped them

There are three respectable ways to solve this, and I want to be explicit about why none of them fit before showing the one that did.

The deciding facts: plan writes are infrequent (a handful a minute, not thousands a second), the contended object is one small document, and I already have Redis, which gives me atomic operations for free. When writes are rare and you already own an atomic primitive, a pessimistic lock is the simplest thing that is obviously correct. Simple and obviously correct beats clever every time in infrastructure you have to trust.

“I already have Redis, which gives me atomicity for free” is the whole argument. The lock is just one Redis command used honestly.

SET NX PX, and why both flags matter

The lock is a single Redis SET with two options:

await redis.set(lockKey, agentId, { nx: true, px: ttlMs });
// nx  → set only if the key does NOT exist  → exactly one winner
// px  → auto-expire after ttlMs               → a dead holder can't block forever

nx is what makes it a lock: the operation is atomic inside Redis, so if ten agents fire it at once, exactly one gets "OK" and the rest get nothing. No race, no coordination protocol, no consensus round. px is what makes it a lock I can trust in production: it sets an expiry, so if the agent that holds it crashes, disconnects, or just wanders off, the lock releases itself. A lock without a TTL is a deadlock waiting for its first crash.

Around that primitive is a small acquire-run-release wrapper with bounded, backed-off retries. Every plan mutation runs inside it:

async function withPlanLock(roomId, fn) {
  const lockId = `server:${nanoid()}`;
  let acquired = false;

  for (let attempt = 0; attempt < 5; attempt++) {
    acquired = await acquireLock(roomId, "plan", lockId, 10_000);
    if (acquired) break;
    await sleep(150 * (attempt + 1));   // 150ms, 300, 450, 600 …
  }
  if (!acquired) throw new Error("Plan is locked, retry in a moment.");

  try { return await fn(); }            // the read-modify-write
  finally { await releaseLock(roomId, "plan", lockId); }
}

Three details earn their place here. The retry loop is bounded — five tries, then it gives up and tells the caller to retry, rather than hanging forever. The backoff grows, so two agents colliding don't synchronize into a tight retry loop hammering the same key. And release is in a finally, so the lock comes back even if the work throws. Release also checks ownership before deleting, so an agent can only free a lock it actually holds:

const current = await redis.get(lockKey);
if (current === agentId) { await redis.del(lockKey); return true; }
return false;   // someone else holds it (or it already expired) — don't touch

That ownership check prevents the classic distributed-lock footgun: agent A's lock expires under load, agent B acquires it, then A finishes late and blindly deletes the lock — freeing B's lock out from under it. Checking the holder before deleting closes that window. It's not a fully fenced lock (that would need monotonic tokens), but for a single-Redis coordination server where the TTL is generous relative to the work, it's the right amount of rigor.

In practice that is what lets you point four agents at one backend overnight and trust you wake up to a consistent plan, not a corrupted one. Reliability under parallelism is the entire reason to run multiple agents at all, so it is the one property that can't be “good enough.” The price — one extra Redis round trip on writes and a brief wait under contention — is invisible for a document edited a few times a minute.

Exactly-once events with a per-agent cursor

The event log has the opposite problem. It's append-only, so writes don't conflict — but readsdo, in a subtle way. Every agent needs its own answer to “what's new to me?” and those answers must not interfere. If two agents share a read position, one advancing it hides events from the other.

The tempting fixes are both bad. A shared “last read” pointer breaks the instant a second agent joins. Re-sending the whole recent log every poll and asking the agent to dedupe pushes the problem onto every client and wastes tokens re-reading events it's already seen. The clean answer is a cursor per agent:

const cursor = await getEventCursor(roomId, agentId);   // this agent's position
const recent = await getEvents(roomId, 50);

const unread = cursor === null
  ? recent
  : recent.filter(e => new Date(e.timestamp) > new Date(cursor));

await setEventCursor(roomId, agentId, new Date().toISOString());  // advance
return { events: unread, count: unread.length };

Each agent has its own cursor key. Reading advances only yourcursor and never mutates the shared log, so two agents reading the same stream each get every event exactly once, with zero coordination between them. No locks, because nothing shared is being written. This is the flip side of the lock decision: use the heaviest tool only where writes actually contend, and lean on per-agent state everywhere else.

Presence as a TTL, not a connection

Last one: knowing who is actually online. The instinct from chat apps is to track live connections — a WebSocket per agent, an onDisconnect handler to mark them offline. That instinct is wrong here for one reason: the server is stateless. There is no persistent connection to hang presence on, and I want to keep it that way, because statelessness is what lets any server instance serve any request and restart without ceremony.

So presence is just a key with an expiry. An agent calls heartbeat every ~60s; that writes a key that lives 120s:

// heartbeat: write a key that outlives one missed beat, then expires
await redis.set(heartbeatKey(roomId, agentId), now, { ex: 120 });

// presence: you're online iff your heartbeat key still exists
const lastSeen = await redis.get(heartbeatKey(roomId, agentId));
status = lastSeen !== null ? "online" : "offline";

Stop heartbeating and you fall offline on your own after 120 seconds. No disconnect handler, no cleanup job, no connection state anywhere. The 120s window is deliberately two missed beats, so one dropped request doesn't flap you offline. Liveness becomes a question about the store, not about a socket — which is exactly the property that keeps the server disposable.

The pattern underneath all three

Look at the three mechanisms together and there's one idea running through them: match the coordination cost to the actual contention.The plan is genuinely write-contended, so it gets a real lock. The event log is append-only with per-reader views, so it gets cheap per-agent cursors and no lock at all. Presence has no shared write at all, so it gets a self-expiring key and no bookkeeping. Reaching for a database transaction or a CRDT everywhere would have been more “serious” and strictly worse: more moving parts, more latency, more to operate, to solve problems two of the three cases don't even have.

What this version doesn't do, to be straight about it: the plan lock makes the plan a serialization point, so a room with agents furiously rewriting tasks would bottleneck on it (fine at real usage, but it's the ceiling). The lock isn't fully fenced, so a pathological TTL-expiry-under-load case is mitigated but not provably impossible. And presence and events are poll-based, so “online” and “unread” are accurate to within a poll interval, not instantaneous. Push and fencing are on the roadmap; neither was worth the complexity for the load this handles today.

The lesson I keep relearning: in systems you have to trust, the right amount of cleverness is usually less than you think. One Redis command used honestly, a cursor per reader, and a key that expires on its own. Three small, boring mechanisms, and the room stays consistent no matter how many agents pile in.

— shreyas