Databricks Lakewatch: What the Agentic SIEM Changes for Security Data

Key Takeaways

Databricks Lakewatch is an agentic SIEM, launched in Private Preview on 24 March 2026, that runs detection and investigation directly on the lakehouse. Security telemetry stays in Delta Lake or Apache Iceberg in your own cloud storage, normalised to OCSF and governed by Unity Catalog, with AI agents helping author detections and correlate signals.

• Cost: Databricks claims up to 80% lower cost from separating storage and compute; no public price list exists yet.
• Effort: the product removes SIEM plumbing, not the work of normalising and owning your log sources.
• Risk: Private Preview means access by invitation and features that can still change.
• Fit: strongest where security, IT and business data already share one Databricks lakehouse.

Security teams have been fighting the same budget line for a decade: the SIEM bill grows with log volume, so the logs that would answer an investigation are often the ones nobody can afford to keep. A third-party pricing guide from Expanso estimates ingest-based Splunk Cloud at $130 to $180 per GB per day on an annual commitment. That is an estimate, not Splunk’s list price, but it explains why retention windows keep shrinking.

On 24 March 2026 Databricks entered that market directly. Its Lakewatch announcement describes a SIEM that unifies security, IT and business data in one governed environment, and it came with two acquisitions reported by TechCrunch: Antimatter, focused on authentication and authorisation for AI agents, and SiftD.ai, founded by the creator of Splunk’s Search Processing Language.

For a security leader the practical questions are narrower than the headlines. What does Lakewatch actually do, what does Private Preview mean for your roadmap, and how much of the hard work, the parsing, normalising and owning of log sources, does it take off your plate? This evaluation answers those from a data engineering chair, because cybersecurity at this scale is a data engineering problem.

What Databricks Lakewatch Is

Lakewatch is Databricks’ own SIEM product, built on the Databricks lakehouse rather than on a proprietary log store. Databricks describes it as a new agentic SIEM designed to help defend against increasingly sophisticated agent-driven attackers, combining detection with investigation and response.

The architectural shift is where the data lives. A traditional SIEM ingests logs into its own indexed store and charges for the privilege. Lakewatch reads security telemetry held in Delta Lake or Apache Iceberg in your own cloud storage. Because storage and compute are separate, Databricks positions it for retaining petabytes of full-fidelity telemetry instead of sampling or dropping it to control cost.

One naming note, since we publish on this topic ourselves. Databricks Lakewatch is Databricks’ product. Our own Lake Watch work is a prototype on dummy data for pipeline monitoring. It is not related to the Databricks product, and we have no partnership on it.

How It Is Built: OCSF, Unity Catalog and Lakeflow Connect

Three existing lakehouse components carry most of the design, and each one will look familiar to anyone who has built a security data lake by hand.

• Schema. Lakewatch is built on the Open Cybersecurity Schema Framework (OCSF), the open standard that started in 2022 with AWS, Splunk, IBM and others and moved to the Linux Foundation in November 2024. Normalising to OCSF in silver is what lets one detection query run across firewall, identity and endpoint data. Our guide to OCSF covers the mapping work in detail.
• Governance. Unity Catalog applies access control at table, row, column and attribute level. For security data that matters: an analyst can hunt across authentication logs without seeing HR fields in the same join.
• Ingestion. Databricks says Lakeflow Connect handles ingestion and normalisation of major security sources, naming AWS, Okta and Zscaler as examples.

None of these is new. What Lakewatch adds is the SIEM workflow on top of them: detections, investigation and response built for security analysts rather than data engineers.

The Agentic Layer

The “agentic” label covers several specific capabilities in the launch material:

• Genie automates parsing of new log sources, drafts net-new detections and translates plain-language questions into SQL.
• Genie Spaces support multi-step threat hunting in natural language.
• Detections are managed as code in YAML, with the logic written in SQL or Python.
• Custom machine learning detections use MLflow and the Feature Store.

Databricks also states that Anthropic’s Claude models help power Lakewatch, using Claude’s reasoning to correlate signals across security, IT and business data. The honest read: the agents speed up the slowest parts of detection engineering, especially onboarding a new source and writing the first version of a rule. A human still reviews what ships to production.

Where Lakewatch Sits Against the Alternatives

On cost, Databricks claims Lakewatch reduces costs by up to 80% by decoupling compute from storage. Treat that as the vendor’s claim until you model your own volumes. The launch named Adobe and Dropbox as Private Preview customers, but no figures from either were published, so there is no independent result yet to benchmark against.

Most independent commentary we have read frames the first phase as augmentation rather than rip-and-replace: keep the existing SIEM for hot alerting and case management, move long-retention and high-volume telemetry to the lakehouse, and shift detections over as they prove out. That is also the lower-risk sequence for a team that cannot afford a blind spot during migration.

The Hard Parts the Launch Post Leaves Out

1. Your sources still need owners. An agent can draft a parser, but someone must confirm what each field means, which system is authoritative, and what happens when a vendor changes its log format.
2. OCSF mapping is judgement, not configuration. Two sources can map the same event to different classes. Without a documented mapping standard, detections silently miss half the data.
3. Detection quality needs measurement. Faster rule authoring produces more rules. Without tracking true and false positive rates per detection, alert volume grows and trust falls.
4. Private Preview limits planning. Access is by invitation, features can change, and there is no published price. A budget cycle cannot rest on a product you cannot yet buy on standard terms.
5. Coexistence takes integration work. Running Lakewatch next to an existing SIEM means deciding which system raises which alert, and routing cases so analysts do not work the same incident twice.

Each of these is a process problem before it is a tooling problem. In Lean Six Sigma terms, they are the Define and Measure steps: agree what a good detection is, baseline today’s false positive rate, then automate.

Readiness Signals: Are You Ready for Lakewatch Yet?

You are ready when:

• Security telemetry already lands in a Databricks lakehouse, or a documented plan puts it there within two quarters.
• Your main log sources are normalised to OCSF in silver, with a written mapping standard and named source owners.
• You track detection performance, at least alert volume and false positive rate per rule, and can compare before and after.
• Retention is a known constraint: investigations regularly fail because the logs were dropped to control SIEM cost.

You are not ready when:

• Logs live in a dozen tools and nobody can list which sources feed which detections.
• The security team has no data engineering capacity, and none is planned.
• The case for change rests only on the vendor’s cost claim, with no model of your own ingest volumes.
• You need a generally available product with contract terms this budget year.

If you land in the second list, the work that pays off regardless of which SIEM you run is the data layer: ingestion, OCSF normalisation and governed gold tables. That is the same foundation described in our piece on data security posture management, and it is where our Databricks consulting engagements on security data start.

Frequently Asked Questions (FAQs)

What is Databricks Lakewatch?

Lakewatch is Databricks’ agentic SIEM, announced on 24 March 2026. It runs detection, investigation and response on the Databricks lakehouse, with security data kept in Delta Lake or Apache Iceberg in your own storage, normalised to OCSF and governed by Unity Catalog.

Is Databricks Lakewatch generally available?

No. Databricks launched it in Private Preview, naming Adobe and Dropbox among the early customers. There is no published GA date and no public price list, so plan around it as an option under evaluation rather than a committed purchase.

Does Lakewatch replace Splunk?

It is positioned as a full SIEM, but most teams will start by running it alongside an existing SIEM: long-retention and high-volume data on the lakehouse, hot alerting where it runs today. Replacement becomes a decision only after detections prove out on your own data.

Which AI models does Lakewatch use?

Databricks states that Anthropic’s Claude models help power Lakewatch, correlating signals across security, IT and business data. Genie features handle log parsing, detection drafting and natural-language queries in SQL, with detections kept as code for review.