architecture
Against Infinite Context
Agentic workflows become more reliable when context is treated as a designed constraint rather than an unlimited memory dump. The goal is not to make the agent know everything. The goal is to make the right context easy to find, use, and discard
There is a lot of agentic AI advice online right now.
Some of it is useful. Some of it is genuinely thoughtful. Some of it has helped me understand and keep up to date with agentic workflows.
But a lot of it also feels like engagement bait dressed up as workflow design.
A terminal screenshot. A dramatic new AI service. A phrase like “this changes everything” or “this got me working as a 10x engineer.” There is no way half of these people actually use what they are advertising. I am looking at you, Sprout AI.
I do not say that to dismiss the whole space. I use these tools almost every day. I think they are useful enough that they have changed how I work.
But using them often has also made me more careful about what advice I trust.
The thing that has mattered most in my own workflow has not been a specific prompt, model, or CLI command.
It has been context.
More specifically, it has been learning that good agentic workflows are not about giving an agent everything. They are about giving it the smallest useful slice of context it needs to do the task correctly.
That distinction has become one of the main ideas behind how I structure projects now.
Memory is not everything
When people talk about agents, memory often gets framed as long-term project awareness.
The dream version is obvious: an agent that remembers the whole codebase, every decision, every bug, every preference, and every previous conversation.
That sounds useful. Sometimes it is.
But I am more interested in a different question:
What is the least this agent needs to know to complete this task fully and correctly?
That question changes the workflow.
It moves the focus away from maximum memory and toward useful memory.
An agent does not need the entire history of a project to fix one bug, adjust one component, or implement one bounded feature.
It needs the task, the relevant files, the constraints, the acceptance criteria, and enough surrounding context to avoid doing something careless.
Anything beyond that might help.
It might also get in the way.
Context is not free
A lot of agentic work is built around the context window.
How much can the model hold? How long can the session stay alive? How much history can be preserved before the output starts getting worse?
The practical lesson is simple:
Context is not free.
A full context window is not the same thing as a good context window.
More information does not automatically mean better information. It can also mean more noise, more stale assumptions, and more chances for the agent to solve a bigger problem than the one you actually asked for.
This is where long-running agent sessions can start to feel heavy. The agent begins sharp, but after enough detours, logs, corrections, failed attempts, and partial explanations, the same session can start carrying too much.
That is the thing people sometimes call context rot.
The session has not literally gone bad, but the useful signal is now buried under too much old material.
That has made me care less about preserving as much context as possible and more about preserving the useful part of the context window.
The question I keep asking
The question I keep coming back to is:
What should the agent know right now?
Not what could it know.
Not what might be useful.
What does it need for this task?
If the agent is fixing a bug, implementing a feature, or updating documentation, it should know the source of truth, the intended audience, and what changed.
This sounds obvious, but I still catch myself violating it.
When an agent struggles, the instinct is to give it more. More background, more docs, more copied errors, more explanation.
Sometimes that helps.
Sometimes it just turns a narrow task into a messy one.
So the better question is not:
Does the agent need more context?
It is:
Did I give it the right context, or just more context?
That difference matters because “more” is easy. “Right” takes design.
Context as the interface
I have started to think about context less like a storage bin and more like an interface.
A good interface does not expose everything at once. It exposes the right controls at the right level of detail.
Agent context should work the same way.
The agent should not have to read the whole project to make one responsible change. It should have a way to enter the project, understand the current task, follow links to relevant documentation, inspect the necessary files, verify the result, and then stop.
This is where documentation becomes more than documentation.
In my own projects, markdown files have slowly become part of the agent interface. They are not just notes for me to read later. They are how the agent finds the right amount of context without me pasting the whole project into a chat.
A good project setup gives the agent a starting point.
That starting point should explain the repo at a high level, point to a documentation index, and let the agent follow the trail based on the task in front of it.
The important part is not that the agent reads everything.
The important part is that it can find what it needs.
When the agent gets lost
One useful thing I have noticed is that agents expose weak documentation structure very quickly.
If I give an agent a task and it starts searching everywhere, guessing at architecture, opening unrelated files, or asking for context that should have been obvious, that is usually not just an agent problem.
It is often a documentation problem.
Maybe the project has no clear entry point. Maybe the docs are not linked together well. Maybe the source of truth is split across too many files. Maybe the agent cannot tell which document is current and which one is stale.
That feedback is useful.
It is one reason I often spend the first few passes on a new repo building documentation scaffolding before I care too much about feature work.
It can feel slow at first, but those are the foundational steps to get correct early. The agent has somewhere to start, the docs have a shape, and everything else flows better after that.
Small does not mean shallow
To be clear, when I say I want to give agents less context, I do not mean sending them off into the world with no help.
That would be useless.
Small context still needs to be rich.
A good task packet might include:
- the task
- the affected files
- the relevant docs
- the constraints
- the acceptance criteria
- the verification steps
That is not minimal in the sense of being tiny.
It is minimal in the sense of being intentional.
The agent should have enough context to act confidently, but not so much that the task loses its edges.
That is the balance I am trying to design for:
Enough context to act.
Not enough context to wander.
I want to write a more practical note later showing what this looks like in an actual before-and-after task packet. Something like the difference between dumping an entire project into a chat versus giving the agent a scoped task, two source files, one doc link, and a clear way to check its work.
For now, the important part is the pattern:
A task packet should make the next action obvious.
If the agent has to guess the goal, guess the files, guess the constraints, and guess how success will be checked, the context is not small and useful.
It is just incomplete.
Easy to use, easy to discard
The part I am still thinking through is not only how to give the agent useful context.
It is also how to make that context easy to discard.
That sounds strange, but I think it matters.
A lot of agent work gets messy because old context keeps hanging around after it has stopped being useful. A debugging session turns into a refactor. A refactor turns into a design discussion. A design discussion turns into documentation cleanup.
Suddenly the agent is carrying five different versions of the task.
So I want context to have a clear job.
This context is for this task.
This doc is the current source of truth.
This command verifies the change.
This session can end when the work is done.
That is what I mean by easy to discard. Not disposable in a careless way, but bounded. The agent should be able to use the context, complete the work, and leave without dragging every previous assumption into the next task.
Why the process matters
The more I use agents, the less I think of them as magic collaborators and the more I think of them as workers inside a process.
That process matters.
If the process is loose, the agent becomes loose. If the docs are unclear, the agent improvises. If the task is too broad, the agent drifts. If the context is stale, the agent reasons from old assumptions. If there is no verification step, the agent may sound finished before the work is actually done.
This is why I do not think the interesting part of agentic development is only the model.
The more interesting part is the work around the model: how we structure tasks, shape context, and design processes around the fact that an LLM is fundamentally predicting text.
And yes, very impressively 'predicting' code.
That does not make the tools useless. It makes the surrounding process more important.
Takeaway
Agentic workflows become more reliable when context is treated as something to design, not something to dump.
The goal is not to make the agent know everything about the project forever.
The goal is to make the right context easy to find, easy to use, and easy to leave behind when the task is done.
That is the difference between an agent that is merely informed and an agent that is actually useful.
In future notes, I want to unpack the practical side of this: how I structure project documentation, what a useful task packet looks like, why markdown has become central to my workflow, and how task loops can turn scoped context into repeatable work.