The smarter the model gets, the wider the leaks
In Episode 5, we talked about how to size an automation budget by company scale. Once the quote is on the table, there is one question that consistently comes back from the approver''s chair: "So if we plug this in, is our company data actually safe?" If the budgeting guide by scale in Episode 5 was a finance conversation, Episode 6 is about the final gate that quote has to pass through — security and governance.
To be precise, the problem is not that "LLMs are dangerous." The moment a single LLM node joins your automation workflow, data that used to live inside your internal systems starts flowing out to an external API, and a single judgment call turns into an approval, a send, or an upload — with no human in between. This piece walks through the four new risks that structure creates, and the checkpoints an approver should at minimum verify.
Why this conversation got urgent in 2026
Three currents have converged. First, Korea''s AI Framework Act has entered its full enforcement phase. Second, in OWASP''s latest ranking of LLM application threats, "sensitive information disclosure" jumped from #6 to #2 — a signal that attacks extracting personal data from conversation logs via prompt injection have grown that quickly. Third, as automation tooling shifts to SaaS, every workflow a practitioner wires up adds one more data egress path.
Honestly, it is rare for a company adopting automation for the first time to cover all three. The more common pattern looks like this: someone on the general affairs team builds a chatbot in a workflow tool, pipes customer inquiry data straight to the OpenAI API, and a few days later legal asks, "Hold on — did that data go out without a data processing agreement?" — and the entire project stalls. Organizational accidents like this happen far more often than technical ones.
PII masking — before the data ever touches the model
The first line of defense is very simple: do not pass personally identifiable information (PII) to the model. Just automatically catching the big five — resident registration numbers, card numbers, phone numbers, emails, and names — will prevent roughly half of the incidents you would otherwise see.
Implementation happens across two layers. Right before the prompt leaves for the model, a regex-based filter substitutes tokens like [MASKED_PHONE] or [MASKED_NAME]; that is layer one. After the response comes back, the original values are restored; that is layer two. With this round trip in place, no raw PII lingers in logs, vector databases, or prompt caches.
For fields that regex cannot catch — names being the classic example — the emerging standard is to put a small NER model or a commercial LLM Gateway service (Presidio, OpenGuardrails, etc.) in front. It is not 100% perfect, but it is far more sustainable than the unrealistic alternative of "have a human review every request."
Audit logs — what can you reconstruct after an incident?
The piece of governance that always gets addressed last is the audit log. For every automation run, four things need to persist: when (timestamp), who or what triggered it (user or system), what prompt went out (input, post-masking), what response came back (output), and which external systems were called as a result (tool_calls).
From an operations standpoint, these logs matter for two reasons. One is error tracing. When an automatically sent quote email goes out with the wrong amount, without logs all you can say is "the model got weird." The other is audit response. When a regulator or a customer''s security audit asks, "Whose data was processed by which pipeline in March of last year?", you need to answer that within thirty minutes.
One practical tip: keep audit logs in an app-level event stream (a dedicated log table or an observability tool) separate from the execution history that your workflow tool stores internally. If the SaaS vendor retains logs for only 30 days, that will not match your regulatory retention window.
Output guardrails — JSON schema validation and retries
If a model response flows into the next node as free text, something in the workflow will break — guaranteed. So the first principle of output guardrails is this: "Never trust free text. Validate against a schema."
The most common production pattern is JSON schema enforcement. You specify JSON as the response format on the request, pipe the response through a schema validator (Pydantic, Zod, Guardrails AI, etc.), and on validation failure feed the error message back into the prompt for a retry. Once this retry loop is in place, the roughly 8% format-error rate you tend to see in production usually converges to a clean response within one or two retries.
There is one trap in retries, though: you cannot retry every failure. Format errors and transient API errors are fair game; business rule violations (e.g., an amount above the approval limit) are not — those need to escalate to a human. If you fail to draw that line, the model starts behaving as though it has been trained to produce "whatever answer gets through" on the third attempt.
Five checkpoints an approver should verify
You can go infinitely deep on the technical detail, but from an approver or CIO perspective, five questions to the team are enough. "For this automation workflow —" ① what PII goes where, and how is it masked? ② How many months of audit logs are retained for every execution? ③ How is the response format validated, and how are failures handled? ④ Where is the human-in-the-loop point that cannot be skipped? ⑤ Is there a signed data processing agreement with the external LLM vendor?
An automation without answers to those five is, regardless of scale, not yet ready for production. Conversely, once those five are clear, the team can fill in the remaining implementation details on their own.
Today''s action items
- Start with five PII categories: designate resident numbers, card numbers, phone numbers, emails, and names as masking targets, and get a regex filter in front of the workflow this week.
- Minimum audit log schema: create a single log table with five fields — timestamp, trigger, input (masked), output, tool_calls. Existence beats perfection.
- Enforce JSON schemas: find one place where free text is handed to the next node, and drop in a validation layer with Pydantic or Zod.
- Name your human-in-the-loop points: for any auto-completing flow, put an approval gate in front of nodes that touch amounts, contracts, or outbound sending.
- Request a security design review: if you need an outside read, you can Free consultation. We will put together a pre-launch risk matrix with you.
The next episode (Ep. 7) will cover "The five failure patterns of automation projects — and a checklist to avoid them." We will lay out the scenes that actually play out on the ground when the governance we discussed today is absent — five patterns we see repeatedly in real engagements.
Frequently Asked Questions
Doesn''t PII masking degrade model response quality?
A simple substitution approach ([MASKED_NAME]) does erode some of the conversational naturalness. But if you add a post-processing step that restores the original values before the response reaches the end user, the quality difference is almost imperceptible from the user''s point of view. It also has a useful side effect: it prevents the failure mode of "the model misremembers a name and calls the person the wrong thing."
Where is the safest place to store audit logs?
We recommend dual storage — the workflow tool''s internal execution history plus a separate log store (an observability tool or your own database). It is the minimum safeguard for staying audit-ready even if a SaaS vendor sunsets its service or changes its retention policy. The storage location itself is fine as long as it is a domestic region or a region with a clearly defined jurisdiction under contract.
For a small team, covering all of this feels heavy. Where should we start?
We recommend the order: PII masking → audit logs → schema validation. Masking can go in within a day, logging in two, and schema validation takes a few days depending on the number of nodes involved. More practical than aiming for perfect governance is passing this bar first: "When an incident happens, can we trace the cause and prevent a repeat?"