Non-Human Identity for AI Agents on the Lakehouse

In One Minute

A non human identity is any credential that lets software act without a person signing in: a service principal, an API token, an OAuth secret. AI agents run on these identities, so they are now your access perimeter. On a lakehouse, the answer is one service principal per agent, short-lived tokens, attribute-based access and a recurring access review.

• What changes: agents read and act at machine speed through whatever identity you hand them.
• Biggest risk: long-lived secrets and shared accounts that nobody owns or retires.
• Effort: an identity inventory first, then scoping, then a quarterly review.
• Tooling: most of the controls already exist in Unity Catalog; the gap is process.

The OWASP Non-Human Identities Top 10, a 2025 incubator project, ranks improper offboarding as risk number one and long-lived secrets as number seven. Neither is a clever attack. Both are housekeeping that nobody was assigned to do, and agents multiply the number of identities that need it.

The gap is easy to see in the numbers on your own platform. On Databricks, an OAuth machine-to-machine access token is valid for one hour, while the OAuth secret behind it can be set to last up to 730 days. The token is short-lived by design. The secret that mints it is the thing an attacker wants, and the thing most teams forget to rotate.

Add AI agents and the problem compounds. Each new agent gets a credential, often borrowed from whoever built it, and almost none of them show up in the access review your auditors already ask about for people.

What Counts as a Non-Human Identity When the User Is an Agent?

For a data team, the list is shorter than the vendor marketing suggests. It covers service principals in your data platform, OAuth client secrets and personal access tokens used by scripts, cloud IAM roles attached to compute, API keys for external models and tools, and the credentials stored inside connectors and MCP servers.

An AI agent is not a new identity type. It is a new consumer of these existing ones, with three habits that change the risk. It runs continuously rather than on a schedule. It decides at runtime which tables and tools to touch. And it can be steered by text it reads, so the identity behind it is only as safe as the content it ingests.

That is why we treat agent identity as a data engineering problem rather than a separate security purchase. The grants, the tables, the lineage and the logs all live on the lakehouse already.

Why Do Agent Identities Slip Through Access Reviews?

Quarterly access reviews were designed around people with managers and HR records. Agents have neither, and five patterns follow from that.

• No owner of record. The engineer who built the agent moves teams, and the identity keeps running with nobody accountable for it. OWASP’s top-ranked risk, improper offboarding, is exactly this.
• Borrowed credentials. A prototype ships on a developer’s personal token. When the developer leaves, either the agent breaks or, worse, the token is kept alive to stop it breaking.
• Shared service accounts. Five agents run under one identity, so a log entry cannot tell you which agent read the payroll table. OWASP lists this as NHI reuse.
• Grants sized for convenience. Access is granted at catalog level because the agent’s needs were not written down. Overprivileged NHI is a named OWASP category for good reason.
• Secrets with no expiry. A two-year secret created during a pilot outlives the pilot, the budget and the person who approved it.

None of these needs a new product to fix. Each needs a written rule and someone who runs it every quarter.

How Do Unity Catalog Service Principals Become the Agent’s Identity?

Databricks defines a service principal as a specialized identity designed for automation and programmatic access. The documentation points to the reason it beats a user account for automated work: jobs keep running when employees leave or groups change. Data access is granted through Unity Catalog the same way it is for a person.

For agents, the rule is simple: one service principal per agent, per environment. A development agent and its production twin get separate identities, so a test run can never read production data. Two separate agents never share a principal, so every log line names the agent that acted.

Two account roles matter here. The Service Principal Manager role controls who can manage a principal’s roles. The Service Principal User role lets someone run a job as the principal rather than as themselves. Keep both lists short and review them alongside the grants, because the person who can run as an agent inherits everything the agent can reach.

Authenticate with OAuth rather than personal access tokens. Databricks describes OAuth as the preferred protocol for service principal authentication outside the UI, and its docs recommend binding each secret to only the scopes its integration needs. Set the shortest secret lifetime your rotation process can actually sustain, not the 730-day maximum. Our Unity Catalog guide covers the grant model these identities plug into.

How Do ABAC and Governed Tags Shrink What an Agent Can Read?

Table-by-table grants do not scale when you run a dozen agents against hundreds of tables. Attribute-based access control does. Databricks announced that ABAC policies for row filtering and column masking, governed tags and automated data classification reached General Availability on 13 May 2026.

The pattern for agents works like this. Classification tags sensitive columns such as personal data or salary. A policy masks any column tagged that way for every principal in an “agents” group, unless a specific agent has a documented reason to see it. A row filter limits a regional support agent to its own region. New tables inherit the protection the moment they are tagged, so a new source does not silently widen what an agent can see.

This turns the access review from a spreadsheet of grants into a short list of policies and exceptions, which is something a reviewer can actually read.

Where Does Unity AI Gateway Fit in the Credential Chain?

Service principals and ABAC govern data. Agents also call models, tools and MCP servers, and those calls need the same identity discipline. Databricks made Unity AI Gateway generally available on 4 August 2026, describing it as extending Unity Catalog identity, permissions, policies and observability to agents, models, MCP servers, skills and tools. Smart Routing remained in Beta at that announcement.

The gateway documentation describes AI assets registered as Unity Catalog securables, granted and revoked with the same privileges as tables, plus consumption limits on model and MCP services and usage tracked in system tables. That means the agent’s service principal becomes the key for every call, and revoking it closes data, model and tool access in one step.

The credential underneath should be short-lived. Configure the agent’s principal to authenticate with OAuth so each access token expires after an hour, keep external provider keys in the gateway rather than in agent code, and confirm in current Databricks docs which credential options apply to each external MCP server you connect. Do not let a long-lived personal token be the thing the gateway trusts.

What Does an Access Review for Agents Look Like?

Treat every non human identity an agent uses like an employee account: run the review on the same calendar as your human review, with the same sign-off. The table shows the questions and the evidence each one produces.

Fix the process before you automate the review. A review that nobody owns produces a signed spreadsheet and no change. Define the owner, the cadence and the deactivation rule first, measure how many identities fail each check in the first cycle, and only then script the evidence collection.

How Do You Prove What an Agent Did Afterwards?

An access review shows what an agent could do. Logs show what it did. Identity events, data access, gateway usage and pipeline runs each produce their own records, and they only become evidence once they are joined on the same identity and clock.

Normalizing those records to a common schema is what makes the join reliable. Our guide to OCSF shows how security and identity events from different sources land in one gold layer on the lakehouse. For the wider list of agent risks this identity work contains, see our map of the OWASP Top 10 for Agentic Applications, where identity and privilege abuse is the risk with the widest blast radius.

Which Identity Plan Fits a Team Like Yours?

Start from where your team actually is. Four profiles cover most mid-market data teams we meet.

• If your agents run on personal tokens today, stop adding agents for two weeks. Create one service principal per agent, move each to OAuth, and retire the personal tokens before anything else.
• If you have service principals but no inventory, build the inventory first: principal, owner, purpose, grants, secret age. The first pass usually surfaces identities nobody can explain, and those go to the deactivation list.
• If you grant access table by table, tag sensitive columns, write ABAC masking policies for an agents group, and replace individual grants with policies plus documented exceptions.
• If agents call external models and MCP servers, route those calls through a gateway under the agent’s own identity, apply rate limits and spend caps per agent, and keep provider keys out of code.

This article is general information, not legal advice, and your counsel or a qualified assessor makes any compliance call.

If you want this built rather than planned, our lakehouse security work covers agent identities, ABAC policy design and the logging that proves both to an auditor.

Frequently Asked Questions (FAQs)

What is a non human identity?

It is any credential that lets software authenticate without a person signing in, such as a service principal, an API key, an OAuth client secret or a cloud role. AI agents, pipelines, CI/CD jobs and integrations all run on them.

Should every AI agent have its own service principal?

Yes. One principal per agent, per environment, lets you scope access to the agent’s task, attribute every log entry to it and revoke it without breaking anything else. Shared principals make all three impossible.

How often should we review agent access?

At least as often as human access, and on the same calendar. Also trigger a review whenever an agent gains a new tool or data source, because that is when grants tend to drift.

Do we need a dedicated NHI security product?

Not to start. On a lakehouse, service principals, OAuth, ABAC, a gateway and system-table logging cover the core controls. A dedicated product earns its place when identities span many platforms beyond the data layer and you need discovery across all of them.