Security, observability and auditability of AI. (3/8)

When determinism is not given, the traditional guardrails fail.

Fine grained security controls.

Everything is Action. Read. Write. API call. Tool invocation. LLM call. Review.

The security policy needs to be fine grained to grant specific action rights within context of a task, not blanket resource permissions based upon role.

Permissions need to be defined for people, worker AI agents and reviewers (human or agents, as applicable), for each task.

Metadata should define sensitive data behaviour.

Logic for handling sensitive data needs to be coded at the data layer. Delegation to higher layers imply inviting unintended consequences, when it is already being injected in a web/mobile/ text interface or being sent to an AI prompt.

Data integrity and sensitivity are reasons, why thin AI layers on existing applications would find it overly complex to maintain trust, while undergoing multiple releases, over long periods.

Without problem addressed at source, each release gate certification requires significant repetitive effort.

Did we hear that lots of vibe code is waiting in release pipeline?

Data audits and observability got more complicated.

Who took the action? Human or AI.

What was the output of work originally performed?

Did a human override it?

Was the output inconsistent with another AI peer, which reviewed it?

I'll leave system engineers think through the implication of these questions on the underlying enterprise data model. Work diffs (just like code diffs) might become common place, when human and AI peer review become a standard constructs.

The old database schema may need a significant upgrade and yet may not perform, if not designed carefully.

Security, observability and audit policy enforcement can no longer be an afterthought. This is the gap between demo to production, that AI apps are often missing.