The README-as-orders example lands because the system prompt says 'never expose secrets' and the agent probably complies — 'probably, most of the time' is not a control. The review question shift is the right one: what can actually happen if the model gets talked into something. Do you think the three-hop trace catches the third failure mode on its own, or does least-privilege on tokens have to be a separate gate?
The three-hop trace should catch it, but I’d still treat least privilege as a separate gate. The trace helps identify where the risk is, while least privilege limits the damage if something slips through. Even if the review misses the injected instruction entirely, a read-only token can't approve or merge a PR. I think you need both: trace the path, then make sure the permissions don't exceed what the task actually needs.
That's the cleanest version of the argument — the read-only token turns a review miss from a breach into a non-event, which is exactly what a separate gate should do. The trace finds the risk, the gate caps the damage, and neither one has to be perfect for the pair to hold. One question: does the trace change the permissions you grant up front, or is it mostly auditing them after the fact?
Ideally both, but I see it primarily as a design-time exercise first. The trace should help determine the minimum permissions the agent needs before deployment, not just document what it already has. Then you keep auditing those permissions as the agent evolves and new tools get added. Otherwise, least privilege tends to slowly turn into "whatever made the last feature work."
That's the right ordering — trace first to set the floor, then audit as tools get added, because otherwise every new feature quietly widens the permissions. The drift you describe is the real failure mode: least privilege decays into 'whatever made the last feature work' unless someone re-checks it on a schedule. Do you re-run the trace per release, or only when a new tool lands?
I'd do both, but with different levels of depth. A new tool should always trigger a review because it can change the agent's reach immediately. For releases, I'd make a lighter check part of the normal process, especially if prompts, workflows or credentials changed. The important thing is not to let permission changes slip through as ordinary feature updates without anyone noticing.
Backend-leaning Full-Stack Developer building reliable software, APIs, and intelligent systems. Interested in software architecture, DevSecOps, and AI engineering.
I build Kelvra is a : Information flow control for AI agents — declare what an agent may know in a file an auditor can read, and get a signed record of what it actually did. Spec stage.
Interesting approach. I like the idea of making both the allowed information flow and the audit trail explicit rather than relying on prompt instructions. One thing I'd be curious about is how you handle enforcement at runtime. Does Kelvra also prevent an agent from acting on information it shouldn't have access to, or is the focus currently on defining and recording those boundaries?
Backend-leaning Full-Stack Developer building reliable software, APIs, and intelligent systems. Interested in software architecture, DevSecOps, and AI engineering.
Good question. Both, but with a caveat. emit() checks the flow before data leaves, and the MCP gateway refuses disallowed tool
calls before they reach the server, so prevention is real at those boundaries. The record then logs what was allowed and what was denied.What's missing is framework adapters, so today it only enforces where you wire it in yourself. If you build agents and want to poke at it, contributions are very welcome. And a star helps more than you'd think.
That makes sense. I like that you're enforcing at both the data output and tool-call boundaries, rather than just logging violations after the fact. The framework adapter gap is an important one, though. The more manual the integration, the easier it is for teams to leave a path uncovered. I'd be interested to see how you approach making those enforcement points harder to bypass as the project matures. Good direction overall!
Backend-leaning Full-Stack Developer building reliable software, APIs, and intelligent systems. Interested in software architecture, DevSecOps, and AI engineering.
I'd be happy to take a closer look. The idea is interesting, especially the enforcement at both the information flow and tool-call boundaries. I'll check out the repo and see where I could contribute. I think getting the integration and enforcement boundaries right will be really important for making this useful in real-world agent setups.
'Telling isn't enforcing' is going on a sticky note. One source I'd add to your hop-one list is the tool metadata itself. With MCP, the tool descriptions and parameter schemas from each server land in the context before the agent reads a single PR, and the server can change them between sessions without anything in your repo changing. A description that quietly asks the model to pass the contents of ~/.ssh as a 'context' argument sits outside your policy table entirely, unless validate_args happens to reject it. A cheap check is to snapshot each server's tools/list response, keep its hash in version control and refuse to start the agent when it drifts, which pairs nicely with the manifest diff idea further down this thread. Do you treat tool descriptions as untrusted input in your reviews today?
Good point. I’d treat tool descriptions and schemas as untrusted input too. They’re part of what shapes the model’s decisions, and a server-side change can introduce new behavior without any repo diff. Hashing the tools/list response and checking it against a reviewed version sounds like a practical way to catch that drift before startup. I’d still keep validate_args and the tool server’s authorization checks as the enforcement layer, though. A matching description hash tells you what changed, not whether a call is safe.
Completely agree, a matching hash only tells you nothing changed, and validate_args plus the server's own authorization still decide whether a call is allowed. I think of them as working at different moments: the hash check runs once at startup and catches drift you'd otherwise never notice, and the enforcement layer runs on every single call. One small addition that keeps the hash check from getting noisy is storing it per server and version, so an expected upgrade asks for a quick re-review instead of looking like an attack. Thanks for the thoughtful reply, and for such a clear write-up!
This is an important distinction when designing secure agentic workflows. At IT Path Solutions, we see authorization and information-flow control as complementary layers. An agent may be allowed to access certain data, but the system should still control where that data can be sent, stored, or exposed. Enforcing these boundaries independently of model instructions makes the overall architecture more resilient.
Well put. Authorization answers what the agent is allowed to access or do, while information-flow control governs where the data it accesses can go. You need both. A read permission shouldn't automatically mean the agent can send that data to any tool or external destination. Keeping those controls independent of model instructions is what makes the boundary hold.
Analyst focused on AI security
Adversarial evaluation of local LLMs, censorship and rejection rate, prompt injection testing, log analysis and defensive automation. Visit: sammideblas.com
The authorization framing is the useful one. Context from PR descriptions and CI logs is untrusted input you cannot sanitize, so tool scope and a deterministic check before the write matter more than the wording of the system prompt.
Exactly. You can't realistically sanitize every piece of content an agent reads, and trying to do so creates a false sense of security. I'd focus on making sure untrusted input can never expand tool permissions and that every consequential action passes through an independent policy check.
Founder of ZANCTA - privacy-first PDF and image tools that run entirely in your browser. No uploads, no server-side processing. Full-stack: Next.js, TypeScript, Node.js, PostgreSQL.
The tool-metadata point in the discussion adds an important source-to-model edge: a server can change schemas without a repository diff. Pinning reviewed tool definitions helps detect drift, while the server still needs to authorize every call independently. That separation keeps a hash check from becoming the only security control.
Exactly. Pinning the definitions gives you visibility into what changed, but it doesn't make the definitions trustworthy or the calls safe. I'd treat the hash check as a drift detector, then keep authorization at the server boundary on every call. Detection and enforcement need to stay separate.
Founder of ZANCTA - privacy-first PDF and image tools that run entirely in your browser. No uploads, no server-side processing. Full-stack: Next.js, TypeScript, Node.js, PostgreSQL.
Right. A pinned definition or hash can reveal drift, but it cannot establish that a call is safe. Authorization still needs to be checked for the current principal, resource, and action on every call; keeping that boundary explicit makes the audit trail useful without overclaiming what the hash proves.
Exactly, and I like that framing around the current principal, resource, and action. That's what makes the authorization decision meaningful at call time. The hash tells us whether the tool contract changed; the runtime check decides whether this specific caller can perform this specific action. They complement each other, but neither replaces the other.
Exactly: the decisive question is not whether the model recognizes hostile text, but whether that text can obtain authority it was never granted. Treating every external instruction as data, constraining tools to narrow capabilities, and requiring a durable receipt for side effects creates a boundary that does not depend on perfect classification.
Agreed. The durable receipt is an important part of this too. You need to be able to reconstruct what was requested, which policy was applied, and who approved the action. Otherwise, even if the authorization boundary works, investigating what happened afterward becomes much harder.
This reframes it well — the model proposing something isn't the same as it being allowed to do it. For anything that touches real work, the check that matters is a person approving the action before it goes out, not just a better prompt.
Exactly. A better prompt can help the model make better suggestions, but it doesn't establish authority. For consequential actions, approval needs to happen outside the model, through a policy check or an explicit human decision. The model can recommend the action, but it shouldn't be the one granting itself permission to carry it out.
Good question. I'd make the tool manifest machine-readable and diff it in CI rather than rely on an author flag. Any new capability — write access, network egress, shell, or secret access — should fail the check until an owner approves it. The author's note explains intent, but the manifest diff makes permission drift visible and auditable.
Exactly. I'd also make the approval explicit and tied to the specific permission change, rather than a general PR approval. That way, adding a new tool or widening an existing scope can't hide inside an otherwise routine change. The machine-readable diff gives you something concrete to enforce in CI and something useful to audit later.
Completely agree with framing this as an authorization boundary issue rather than purely trying to sanitize text prompts. If an agent has ambient authority, no amount of prompt engineering can reliably prevent privilege escalation.
Exactly. Prompt instructions can help guide the agent, but they can't define the limits of what it is allowed to do. Those limits need to be enforced outside the model, so even if untrusted content influences its decisions, it still can't gain authority it wasn't given.
Analyst focused on AI security
Adversarial evaluation of local LLMs, censorship and rejection rate, prompt injection testing, log analysis and defensive automation. Visit: sammideblas.com
Agreed. Permissions should be tied to the specific tool call and the action being requested, not inherited broadly from the session. I'd add that the authorization decision should use the current policy and the relevant resource, so content from a poisoned read can influence the agent's reasoning but can never expand what it's allowed to do.
Analyst focused on AI security
Adversarial evaluation of local LLMs, censorship and rejection rate, prompt injection testing, log analysis and defensive automation. Visit: sammideblas.com
Exactly. The key is that permissions shouldn't be something the agent can negotiate or change based on what it reads. Untrusted content can influence its decisions, but it should never be able to expand its authority. That's a boundary the application needs to enforce from the start.
I write about Next.js + TypeScript + AI engineering, with a focus on making sites discoverable to LLMs and AI search. Long-form notes at mudassirkhan.me
the summarizer with a write scope token is the one i see most in real setups, because whoever wired it up grabbed the token they already had. least privilege there makes the injected merge instruction harmless, which no prompt wording can promise.
the second one we hit was tool output flowing straight back into context. we tag anything from fetched content as untrusted and put any write call that follows it behind a separate check, which cut the scary cases a lot.
are you enforcing that gate in the harness or at the tool server?
I'd put the actual enforcement at the tool server. The harness can handle context tagging and decide when approval is needed, but the tool server should independently validate the action, scope and credentials before executing it. That way, even if something goes wrong in the harness, the enforcement boundary still holds.
Exactly. Instructions can guide the agent, but permissions define what it can actually do. Even if it misunderstands a request or follows malicious content, the system should still prevent actions outside its allowed scope.
This is so true. We had an issue with an automated supplier webhook at The Printing World where weird data in an invoice file almost messed up a material reorder. Code-level permissions definitely beat just asking the model nicely.
We're a place where coders share, stay up-to-date and grow their careers.
The README-as-orders example lands because the system prompt says 'never expose secrets' and the agent probably complies — 'probably, most of the time' is not a control. The review question shift is the right one: what can actually happen if the model gets talked into something. Do you think the three-hop trace catches the third failure mode on its own, or does least-privilege on tokens have to be a separate gate?
The three-hop trace should catch it, but I’d still treat least privilege as a separate gate. The trace helps identify where the risk is, while least privilege limits the damage if something slips through. Even if the review misses the injected instruction entirely, a read-only token can't approve or merge a PR. I think you need both: trace the path, then make sure the permissions don't exceed what the task actually needs.
That's the cleanest version of the argument — the read-only token turns a review miss from a breach into a non-event, which is exactly what a separate gate should do. The trace finds the risk, the gate caps the damage, and neither one has to be perfect for the pair to hold. One question: does the trace change the permissions you grant up front, or is it mostly auditing them after the fact?
Ideally both, but I see it primarily as a design-time exercise first. The trace should help determine the minimum permissions the agent needs before deployment, not just document what it already has. Then you keep auditing those permissions as the agent evolves and new tools get added. Otherwise, least privilege tends to slowly turn into "whatever made the last feature work."
That's the right ordering — trace first to set the floor, then audit as tools get added, because otherwise every new feature quietly widens the permissions. The drift you describe is the real failure mode: least privilege decays into 'whatever made the last feature work' unless someone re-checks it on a schedule. Do you re-run the trace per release, or only when a new tool lands?
I'd do both, but with different levels of depth. A new tool should always trigger a review because it can change the agent's reach immediately. For releases, I'd make a lighter check part of the normal process, especially if prompts, workflows or credentials changed. The important thing is not to let permission changes slip through as ordinary feature updates without anyone noticing.
I build Kelvra is a : Information flow control for AI agents — declare what an agent may know in a file an auditor can read, and get a signed record of what it actually did. Spec stage.
Kelvra-depot
Interesting approach. I like the idea of making both the allowed information flow and the audit trail explicit rather than relying on prompt instructions. One thing I'd be curious about is how you handle enforcement at runtime. Does Kelvra also prevent an agent from acting on information it shouldn't have access to, or is the focus currently on defining and recording those boundaries?
Good question. Both, but with a caveat. emit() checks the flow before data leaves, and the MCP gateway refuses disallowed tool
calls before they reach the server, so prevention is real at those boundaries. The record then logs what was allowed and what was denied.What's missing is framework adapters, so today it only enforces where you wire it in yourself. If you build agents and want to poke at it, contributions are very welcome. And a star helps more than you'd think.
That makes sense. I like that you're enforcing at both the data output and tool-call boundaries, rather than just logging violations after the fact. The framework adapter gap is an important one, though. The more manual the integration, the easier it is for teams to leave a path uncovered. I'd be interested to see how you approach making those enforcement points harder to bypass as the project matures. Good direction overall!
How about contributing to the project? I would be truly delighted to have contributors and people who are interested in the project.
I'd be happy to take a closer look. The idea is interesting, especially the enforcement at both the information flow and tool-call boundaries. I'll check out the repo and see where I could contribute. I think getting the integration and enforcement boundaries right will be really important for making this useful in real-world agent setups.
'Telling isn't enforcing' is going on a sticky note. One source I'd add to your hop-one list is the tool metadata itself. With MCP, the tool descriptions and parameter schemas from each server land in the context before the agent reads a single PR, and the server can change them between sessions without anything in your repo changing. A description that quietly asks the model to pass the contents of ~/.ssh as a 'context' argument sits outside your policy table entirely, unless
validate_argshappens to reject it. A cheap check is to snapshot each server'stools/listresponse, keep its hash in version control and refuse to start the agent when it drifts, which pairs nicely with the manifest diff idea further down this thread. Do you treat tool descriptions as untrusted input in your reviews today?Good point. I’d treat tool descriptions and schemas as untrusted input too. They’re part of what shapes the model’s decisions, and a server-side change can introduce new behavior without any repo diff. Hashing the
tools/listresponse and checking it against a reviewed version sounds like a practical way to catch that drift before startup. I’d still keepvalidate_argsand the tool server’s authorization checks as the enforcement layer, though. A matching description hash tells you what changed, not whether a call is safe.Completely agree, a matching hash only tells you nothing changed, and validate_args plus the server's own authorization still decide whether a call is allowed. I think of them as working at different moments: the hash check runs once at startup and catches drift you'd otherwise never notice, and the enforcement layer runs on every single call. One small addition that keeps the hash check from getting noisy is storing it per server and version, so an expected upgrade asks for a quick re-review instead of looking like an attack. Thanks for the thoughtful reply, and for such a clear write-up!
This is an important distinction when designing secure agentic workflows. At IT Path Solutions, we see authorization and information-flow control as complementary layers. An agent may be allowed to access certain data, but the system should still control where that data can be sent, stored, or exposed. Enforcing these boundaries independently of model instructions makes the overall architecture more resilient.
Well put. Authorization answers what the agent is allowed to access or do, while information-flow control governs where the data it accesses can go. You need both. A read permission shouldn't automatically mean the agent can send that data to any tool or external destination. Keeping those controls independent of model instructions is what makes the boundary hold.
Can you please check my post and give any comments or reviews for that
The authorization framing is the useful one. Context from PR descriptions and CI logs is untrusted input you cannot sanitize, so tool scope and a deterministic check before the write matter more than the wording of the system prompt.
Exactly. You can't realistically sanitize every piece of content an agent reads, and trying to do so creates a false sense of security. I'd focus on making sure untrusted input can never expand tool permissions and that every consequential action passes through an independent policy check.
The tool-metadata point in the discussion adds an important source-to-model edge: a server can change schemas without a repository diff. Pinning reviewed tool definitions helps detect drift, while the server still needs to authorize every call independently. That separation keeps a hash check from becoming the only security control.
Exactly. Pinning the definitions gives you visibility into what changed, but it doesn't make the definitions trustworthy or the calls safe. I'd treat the hash check as a drift detector, then keep authorization at the server boundary on every call. Detection and enforcement need to stay separate.
Right. A pinned definition or hash can reveal drift, but it cannot establish that a call is safe. Authorization still needs to be checked for the current principal, resource, and action on every call; keeping that boundary explicit makes the audit trail useful without overclaiming what the hash proves.
Exactly, and I like that framing around the current principal, resource, and action. That's what makes the authorization decision meaningful at call time. The hash tells us whether the tool contract changed; the runtime check decides whether this specific caller can perform this specific action. They complement each other, but neither replaces the other.
Exactly: the decisive question is not whether the model recognizes hostile text, but whether that text can obtain authority it was never granted. Treating every external instruction as data, constraining tools to narrow capabilities, and requiring a durable receipt for side effects creates a boundary that does not depend on perfect classification.
Agreed. The durable receipt is an important part of this too. You need to be able to reconstruct what was requested, which policy was applied, and who approved the action. Otherwise, even if the authorization boundary works, investigating what happened afterward becomes much harder.
This reframes it well — the model proposing something isn't the same as it being allowed to do it. For anything that touches real work, the check that matters is a person approving the action before it goes out, not just a better prompt.
Exactly. A better prompt can help the model make better suggestions, but it doesn't establish authority. For consequential actions, approval needs to happen outside the model, through a policy check or an explicit human decision. The model can recommend the action, but it shouldn't be the one granting itself permission to carry it out.
Good question. I'd make the tool manifest machine-readable and diff it in CI rather than rely on an author flag. Any new capability — write access, network egress, shell, or secret access — should fail the check until an owner approves it. The author's note explains intent, but the manifest diff makes permission drift visible and auditable.
Exactly. I'd also make the approval explicit and tied to the specific permission change, rather than a general PR approval. That way, adding a new tool or widening an existing scope can't hide inside an otherwise routine change. The machine-readable diff gives you something concrete to enforce in CI and something useful to audit later.
Completely agree with framing this as an authorization boundary issue rather than purely trying to sanitize text prompts. If an agent has ambient authority, no amount of prompt engineering can reliably prevent privilege escalation.
Exactly. Prompt instructions can help guide the agent, but they can't define the limits of what it is allowed to do. Those limits need to be enforced outside the model, so even if untrusted content influences its decisions, it still can't gain authority it wasn't given.
Scope permissions per tool call, not per session. A poisoned read then cannot widen the agent authority mid-task.
Agreed. Permissions should be tied to the specific tool call and the action being requested, not inherited broadly from the session. I'd add that the authorization decision should use the current policy and the relevant resource, so content from a poisoned read can influence the agent's reasoning but can never expand what it's allowed to do.
Yes. Permissions should be set before the agent reads anything, so injected text cannot widen them.
Exactly. The key is that permissions shouldn't be something the agent can negotiate or change based on what it reads. Untrusted content can influence its decisions, but it should never be able to expand its authority. That's a boundary the application needs to enforce from the start.
the summarizer with a write scope token is the one i see most in real setups, because whoever wired it up grabbed the token they already had. least privilege there makes the injected merge instruction harmless, which no prompt wording can promise.
the second one we hit was tool output flowing straight back into context. we tag anything from fetched content as untrusted and put any write call that follows it behind a separate check, which cut the scary cases a lot.
are you enforcing that gate in the harness or at the tool server?
I'd put the actual enforcement at the tool server. The harness can handle context tagging and decide when approval is needed, but the tool server should independently validate the action, scope and credentials before executing it. That way, even if something goes wrong in the harness, the enforcement boundary still holds.
Good point. Limiting what an agent can do seems more reliable than just telling it what not to do.
Exactly. Instructions can guide the agent, but permissions define what it can actually do. Even if it misunderstands a request or follows malicious content, the system should still prevent actions outside its allowed scope.
This is so true. We had an issue with an automated supplier webhook at The Printing World where weird data in an invoice file almost messed up a material reorder. Code-level permissions definitely beat just asking the model nicely.