Lessons from building an internal tool nobody asked for, that most of my team now uses daily.
Last year I got tired of re-explaining my codebase to my AI coding assistant every morning.
If you use Claude Code, Cursor, or similar tools on a large codebase, you know the ritual: every new session starts cold. You paste in context, re-describe the service you're working on, remind it which conventions your team follows, and re-establish everything it knew perfectly well yesterday. On a small project this is a minor annoyance. Across a sprawling microservices architecture, it's a real tax — paid daily, by every engineer.
So I built a memory layer. Not because anyone asked me to — because I was annoyed. This post is about what I learned building it, presenting it to my team, and experimenting with AI-assisted engineering workflows more broadly.
My first instinct was to reach for the fashionable architecture: vector database, embeddings, semantic search over everything. I'm glad I didn't start there.
The first version was just SQLite.
Sessions get indexed into a plain relational store. An LLM summarizes each session's context — what was worked on, what decisions were made, what the relevant files and tickets were. When you want that context back, you run a single command — /recall {ticket or keyword} — and the summarized context gets injected into your current session. A background watcher polls for new activity so the index stays current without anyone thinking about it.
That's it. No embeddings, no retrieval pipeline, no reranking. And for the dominant use case — "give me back the context for the thing I was working on" — keyword lookup over structured summaries turned out to be entirely sufficient, because engineers almost always recall work by ticket number or feature name, not by fuzzy semantic similarity.
I'm adding a vector store (ChromaDB) now — but for a genuinely different use case: ad-hoc questions where you don't know the keyword ("have we ever dealt with rate-limit issues on callbacks before?"). The lesson wasn't "vector databases are overrated." It was: match the retrieval mechanism to how people actually ask. Exact-recall questions deserve an index. Exploratory questions deserve embeddings. Starting with the simple one meant I shipped in days, not weeks, and learned from real usage before adding complexity.
Tools in this space already existed — claude-mem, for instance, solves a similar problem for Claude Code. But our coding assistant is an internal tool, so nothing off the shelf could plug into it. If we wanted memory, someone had to build it. So I did, initially just for myself.
When I presented it to the team, it landed harder than I expected — and the reason taught me something about the problem I hadn't fully articulated. It wasn't just the morning cold-start ritual. It was multi-ticket work. Everyone juggles several tickets at once; a ticket gets blocked, you switch away, and a week later it unblocks — and you've forgotten what was built, what was decided, and why. The context isn't just expensive to re-establish, it's often gone. A memory layer doesn't just save typing; it makes blocked work resumable.
Today around 70% of the engineers on my team use it daily. The corollary I now believe strongly: internal tools are products, and your colleagues are unforgiving customers. They will not file feature requests; they will just quietly stop using it. The features that survived are the ones that removed friction (the background watcher exists because nobody — including me — remembered to manually index sessions). The features that died are the ones that asked users to change their habits.
Something I only understood over time: the summarisation step isn't a compromise on the way to "real" memory — it's doing quiet, important work.
Injecting a compact summary instead of raw session history saves a lot of tokens, obviously. But the bigger effect is on output quality. Past a certain context size, more information stops helping and starts hurting: the model has too much to attend to, the relevant details get diluted by noise, and answers drift or fixate on the wrong things. Curated summaries keep the signal-to-noise ratio high. (To be precise, this is less about hallucination in the making-things-up sense and more about attention — the model misses or misweights what matters when it's buried. And summarisation is lossy, so you're trading fine-grained recall for precision. For resuming work on a ticket, that's the right trade.)
Small context, deliberately chosen, beats big context passively accumulated. That principle now shapes everything I build in this space.
Alongside the memory tool, I spent months experimenting with AI-assisted engineering workflows — skill-based agent instructions (in the style of the community "skills" repos), heavier agentic patterns, and various degrees of human-in-the-loop.
I went in with an ambitious target: PRD to PR. Hand the agent a product requirement, get back a reviewable pull request. It's the obvious dream, and plenty of demos make it look close. Working on a real, aged codebase, I ran into walls that demos don't show:
The QA boundary. An agent needs a feedback loop to converge on working code — but letting it run and verify its own changes starts overstepping into QA's scope. Where the agent's self-verification ends and human QA begins turned out to be an organisational question as much as a technical one, and nobody has a clean answer yet.
PRDs aren't specs. Real PRDs are written for humans who fill gaps with context. Handed to an agent, "unclear" becomes "unimplementable" — the requirements often can't be matched to the system as it technically exists.
Tech debt is agent poison. Old APIs that still exist but shouldn't be used, multiple flows that do almost the same thing for historical reasons — things a tenured engineer navigates on instinct — reliably confused the agent into building on the wrong foundation.
So I came out convinced that hunting for the workflow is the wrong goal. What actually predicts whether an AI workflow helps comes down to three variables: the engineer's seniority (seniors catch agent mistakes in review, so the agent is leverage; juniors can end up shipping code they can't defend), familiarity with the codebase (in a codebase you know, AI speed is safe because you'll smell anything wrong; in one you don't, the same speed just gets you lost faster), and the type of ticket (a well-specified bug fix, a vague exploration, and a cross-service refactor are three different jobs; a workflow tuned for one is often counterproductive for another).
The cross-service case was painful enough that I built a set of open-source skills for working across microservices — tracing an API call across service boundaries, generating per-service architecture specs agents can consume as context, and keeping a fleet of repos in sync. Notably, the /ms-spec skill exists precisely because of the tech-debt problem above: if the agent is going to be confused by legacy flows, the least you can do is document which flows are legacy.
If you're leading a team through AI adoption: stop looking for the workflow, and start matching workflows to the (engineer, codebase, task) triple.
One more lesson, from a different project: I once took a one-line brief ("explore whether an AI chatbot could help with user onboarding") and had a working demo a week later. The demo worked. It also surfaced genuine security considerations, and the team decided not to proceed.
The fashionable take here is "fast prototyping is great, killed prototypes are wins." I'm honestly not sure I buy it as a general strategy — a week is cheap, but a culture of constantly prototyping things that get killed has real costs too. What I did take away is narrower and, I think, more defensible:
First, sometimes people need a concrete thing to reject. An abstract debate about whether to build a chatbot could have circled for a long time. A working demo with visible security implications ended the discussion in a week — the prototype's job wasn't to win, it was to make the decision cheap and grounded.
Second, the learning survives the project. Building it taught me how this class of chatbot actually works end to end, and how to create one from scratch — knowledge I've since repurposed for automated testing, which is arguably worth more than the original idea was. The prototype died; the capability didn't.
The project I'm most excited about now is a separate one, inspired by Graphify — the open-source tool that turns a codebase into a queryable knowledge graph, so agents traverse a compact graph instead of reading raw files into context. That's a token-efficiency win of the same species as summarise-and-recall, applied to the codebase itself. And it's the claude-mem situation all over again: Graphify ships as a skill for twenty-plus coding assistants, but ours is internal and isn't one of them — so I'm building something that plugs in. I'm starting to think this is the job, in this era: the open ecosystem moves fast, internal platforms can't consume it directly, and the valuable work is rebuilding the idea in a form that fits.
The plan goes further than a port, though. Context tooling like mine is fundamentally local today — every engineer keeps their own index, their own picture of the system. I want the graph to be team-based: exposed over MCP, so the whole team queries one continuously-updated graph instead of maintaining private copies — and so online agents can consume it as context too, not just humans at terminals. From there it naturally extends to a chat interface: when you're away from your laptop and need a fast answer about the system, you should be able to just ask.
Meanwhile the memory layer keeps evolving — semantic retrieval for exploratory queries is in progress, because at daily-driver usage levels, context efficiency stops being an optimization and starts being the feature.
If you're building similar internal AI tooling, I'd love to compare notes — I'm at github.com/lsscecilia.
Views my own; all details described in general terms.