"Agent" has become a marketing noun. In production it is a control loop: observe, decide, act, verify — with explicit rules for when the loop may not act. This note is how Hyper Mind Tech designs those loops so they are useful without becoming an unsupervised intern with production credentials.
If you are automating back-office Microsoft processes, also read our enterprise back-office AI agents playbook. If Studio is your build surface, keep Copilot Studio production governance open beside this note. The principles below are product-agnostic; the failure modes are not.
Prefer a boring workflow with a smart edge
The pattern that keeps working:
- Deterministic orchestration for the skeleton (Temporal, Azure Durable Functions, or a well-tested state machine).
- LLM decisions only at the branches where language or unstructured input is the hard part.
- Tools that are small, authenticated, and idempotent where possible.
LangGraph (or the OpenAI Agents SDK, or Azure AI Foundry agent patterns) can own the reasoning graph. It should not own your payment ledger. If the business process has a clear sequence — intake → enrich → decide → write-back — put that sequence in a durable workflow and let the model fill slots and draft, not invent new steps at 3 a.m.
// Fail-closed agent loop
[Event / user task] --> [Workflow skeleton]
[Workflow] --> [LLM plan step]
[Plan] --> [Tool allow-list check]
[Allowed tool] --> [Execute + idempotency key]
[Result] --> {needs HITL?}
{yes} --> [Human queue + timeout policy]
{no} --> [Verify + write durable state]
[Any policy fail] --> [Stop / escalate / no side effect]
Tool surfaces are security surfaces
Every tool is an API with a blast radius. We design tools as if they will be called by a confused, creative principal — because that is what an LLM is under prompt injection.
Rules we rarely break:
- Least privilege per tool, not one mega-tool with admin graph access.
- No free-form SQL or shell from the model in production paths.
- Typed arguments with server-side validation; the model proposes, the tool rejects garbage.
- Idempotency keys on writes so retries do not double-charge or double-ticket.
- Human-in-the-loop before irreversible side effects: money movement, access grants, external email to customers, production config changes.
HITL is not a checkbox in a slide. It is a queue, a timeout policy, and a UI someone will actually use. If approvals pile up unread, you do not have HITL — you have latency and a false sense of safety.
| Tool class | Example | Default autonomy | HITL |
|---|---|---|---|
| Read internal | Get ticket fields, fetch invoice PDF text | Allowed | No |
| Read external | Vendor status page scrape | Case-by-case | Often yes for untrusted HTML |
| Draft only | Write reply draft into CRM | Allowed | Review before send |
| Write internal | Update ticket status, add note | Narrow fields only | Optional after trust |
| Write money / access | Refund, license assign, ACL change | Denied by default | Always |
| External comms | Email customer / vendor | Denied by default | Always |
| Admin / config | Change production policy | Denied | Always + change control |
Memory: intentional or it becomes sludge
Agent "memory" fails in two opposite ways: amnesia (every turn starts cold) and hoarding (the model's scratchpad becomes an unqueryable second database of PII).
We separate:
- Working context — this request, this session, retrieved facts with sources.
- Durable state — in your real systems of record (CRM, ticket, order), updated through tools.
- Long-term preference memory — only if product requires it, with retention, export, and delete paths.
Summaries written by the model into a vector store are not a substitute for structured state. They are a cache with opinions. Treat them accordingly.
| Memory type | Store | Retention | Delete path |
|---|---|---|---|
| Working context | Session / trace store | Days | Automatic expiry |
| Retrieved facts | Ephemeral in span | With trace policy | Redaction jobs |
| Durable business state | CRM / ERP / ticket | System of record policy | System of record |
| Model scratch summary | Optional cache | Short; never sole source of truth | Purge job |
| User preferences | Product DB if needed | Policy + consent | Export + delete API |
Evaluate the loop, not just the final sentence
Chat evals are not enough for agents. You need:
- Trajectory checks: did it call the right tools in a sensible order?
- Side-effect audits: in a sandbox, did it attempt disallowed tools?
- Injection cases: malicious content in retrieved docs or tickets trying to exfiltrate data or escalate tools.
- Cost and step caps: max tool calls, max tokens, wall-clock budget — hard stops, not polite suggestions.
We keep a golden set of multi-step tasks with expected tool traces. When someone "improves the prompt" and the agent starts skipping verification tools, CI should go red. The harness philosophy matches RAG evals before the feature flag; agents add trajectory scoring on top.
// Agent eval layers
[Golden multi-step tasks] --> [Expected tool trace]
[Sandbox run] --> [Trajectory score]
[Injection library] --> [Attack success rate]
[Cost/step caps] --> [Budget pass/fail]
[All gates] --> [Release / block]
| Eval dimension | What we measure | Gate style |
|---|---|---|
| Task success | Final business outcome correct | Block if drop exceeds threshold |
| Trajectory | Right tools, order, no extras | Block on critical skips |
| Disallowed tools | Attempts in sandbox | Block any success on write tools |
| Injection | Attack success on fixed pack | Block above residual rate |
| HITL compliance | Wrote without approval when required | Block any occurrence |
| Cost | Tokens + tool calls per success | Warn; block if runaway |
| Latency | p95 including HITL wait (split metrics) | Per SLA |
For a fuller red-team cadence once agents are in production, see LLM red teaming and evals for enterprise Copilot systems.
Observability or you do not operate it
Production agents without traces are undebuggable. Minimum:
- One trace ID per user-visible run.
- Spans for model calls, tool calls, retrieval, and HITL waits.
- Prompt and graph version tags.
- Ability to replay a failed run in a safe environment with redacted payloads.
If your on-call cannot answer "which tool failed and what did the model see?" you are not ready to turn the flag on for paying customers.
On-call questions that must be answerable in five minutes
- Which agent version and prompt version ran?
- What was the user or system trigger?
- Which tools were proposed vs executed?
- Did HITL fire, skip, or timeout?
- What durable writes succeeded (with IDs)?
- How do we disable this agent for all tenants or one tenant?
If any answer requires "we would need to pull logs from three systems and guess," fix observability before scale.
HITL that is real, not decorative
| HITL design choice | Healthy | Unhealthy |
|---|---|---|
| Queue ownership | Named role with SLA | "Someone in the team" |
| Timeout | Escalate or fail closed | Silent auto-approve |
| UI | One click approve/reject with diff | Email thread archaeology |
| Audit | Who approved what, when | No record |
| Load | Sized to peak volume | Hero evenings |
| Bypass | Change-controlled break-glass | Shadow env with prod tools |
We measure HITL queue age the same way finance measures invoice cycle time. A growing queue means either volume exceeded staffing or the agent is escalating too much — both are design bugs, not "people problems."
What we tell founders who want "a swarm"
Multi-agent swarms look impressive in videos. In enterprise and B2B SaaS, we usually ship one competent agent with a tight tool list first. Parallel specialists can come later when you have evals and ownership for a single loop. Complexity is not a strategy; it is a multiplier on every failure mode above.
| Pattern | When we use it | When we refuse |
|---|---|---|
| Single agent + tools | Most back-office and support flows | — |
| Workflow + LLM slots | Clear process, language at edges | When process is undefined |
| Multi-agent | Proven single agent, clear handoffs | Day-one architecture for demos |
| Fully autonomous writes | After months of HITL metrics | First pilot |
Security and injection assumptions
Assume untrusted text enters every loop: PDFs, email bodies, ticket comments, web pages. Design tools as if the model will eventually be tricked — because something will try. Map findings to OWASP LLM Top 10 language. For Microsoft Graph–adjacent agents, align with Copilot security, DLP, and oversharing controls so seat programs and custom agents do not invent two different data stories.
Staffing past the pilot hero
| Role | Pilot signal | At three processes |
|---|---|---|
| Process owner (business) | 0.2 FTE | 0.2 per process |
| Automation / agent engineer | 0.5–1.0 | 1–2 shared |
| HITL reviewers | Measured from queue | Sized to SLA |
| Security / risk | 0.1 | 0.2 |
| Product / program | 0.3 | 0.5 |
| On-call rotation | Named | Documented roster |
If the pilot only works because one engineer lives in the traces every evening, you have not reduced toil — you have relocated it.
Field vignette (anonymized)
An accounts-payable pilot let the model "update vendor bank details" through a broad ERP tool "to save clicks." Prompt injection via a PDF invoice note never needed a sophisticated jailbreak; it needed a tool that should not have existed. We replaced the mega-tool with a read-only extract tool and a separate write tool that only ran after dual-control HITL and a bank-detail change policy. Cycle time rose slightly. Residual fraud risk fell through the floor. Audit preferred the second design.
Another team measured only final answer quality on a chat UI while the agent also created CRM records. Final answers scored well; the CRM filled with duplicate opportunities because retries lacked idempotency keys. Trajectory evals and write-tool contracts fixed what chat goldens never saw.
Kill switch and staged autonomy
Autonomy is earned in stages, not declared at kickoff.
| Stage | Agent may | Still requires |
|---|---|---|
| 0 — Draft only | Propose text and tool arguments | Human executes writes |
| 1 — Assisted writes | Execute low-risk internal writes | HITL on external/money/access |
| 2 — Narrow autonomy | Execute allow-listed tools within caps | Spot audit + eval gates |
| 3 — Broad autonomy | Rare; multi-tool without HITL | Explicit risk accept + red team |
Promotion between stages needs metrics: HITL reject rate, disallowed-tool attempts, injection pack results, and process cycle-time actually improving — not just "users like the demo." Demotion is as important as promotion: a spike in reject rate or a Sev1 attempt returns the agent to stage 0 until fixed.
The kill switch must work at three scopes: global disable, single-tenant disable, and single-tool disable. Practice each in a drill. An untested kill switch is a slide, not a control.
Closing stance
Agent loops that fail closed are not anti-automation. They are how automation earns the right to touch money, access, and customers. Narrow tools. Durable workflow underneath. Memory that respects systems of record. HITL with a queue and a timeout. Evals on trajectories. Traces for on-call. Kill switch practiced.
Ship one competent loop. Earn autonomy with metrics. Leave swarms for the video — until your first loop is boringly reliable.
If you are facing this
Send the process you want automated, the systems of record involved, and which steps are irreversible. We will tell you whether you need an agent, a workflow with light LLM assist, or just better APIs — and we will say so in writing before anyone draws a boxes-and-arrows shrine to autonomy. Get in touch.