The Essentials
Under California’s CCPA regulations, a business that uses automated decision-making technology (ADMT) to make a significant decision about a consumer must, according to law-firm summaries of the final text, give notice, honor opt-outs and answer access requests by 1 January 2027. For the data team, that means a system inventory, lineage on decision inputs, and a durable record of every decision.
- Scope: decisions that provide or deny lending, housing, education, employment or compensation, or healthcare. Targeted advertising alone is excluded.
- Effort: most of the work is inventory, plumbing and record-keeping, not rewriting models.
- Risk: the failure mode is an access request you cannot answer because the inputs sit in five systems with no trail between them.
- Fit: if decision inputs already land in one governed lakehouse, the build is mostly tables, grants and scheduled jobs.
The clock started earlier than most teams noticed. California’s regulations on ADMT, risk assessments and cybersecurity audits took effect on 1 January 2026, and Paul Hastings reports that businesses using ADMT for significant decisions must provide notice by 1 January 2027. That leaves one planning cycle, and most of it belongs to data engineering.
California is not alone. Colorado’s replacement AI law, SB 26-189, requires developers and deployers to retain records needed to demonstrate compliance for at least three years. Two states, two record-keeping expectations, one question from both: can you show what the system saw and what it decided?
Here is the problem I see in almost every assessment. The scoring model lives with one team, the applicant data lives in a CRM, the overrides live in a spreadsheet, and the notice text lives in a marketing tool. Nobody can rebuild a single decision end to end without a week of emails. That is a data silo problem wearing a privacy label.
What Counts as a Significant Decision
Law-firm summaries of the final regulation describe a significant decision as one that results in the provision or denial of financial or lending services, housing, education enrollment or opportunities, employment or independent contracting opportunities or compensation, or healthcare services. Targeted advertising on its own is explicitly outside it.
That definition does the scoping work for you. A churn model that decides which customers get a retention email is out. A model that ranks job applicants before a recruiter sees them is in. A credit pre-screen that routes applications to an automatic decline queue is in. Start the inventory by walking your decision points, not your model registry, because plenty of ADMT is a rules engine or a vendor score that never touched your data science team.
The three consumer-facing obligations reported for the 1 January 2027 date are notice when ADMT is used for a significant decision, an opt-out unless an exception applies, and a response to access requests about the ADMT. Each one needs data you can find, trust and retrieve on demand.
The Dates on the Calendar
The California Privacy Protection Agency finalized the package in September 2025. The dates below come from consistent law-firm reporting of the final text, so read them as reported dates and confirm the detail with counsel.
| Obligation | Reported date | What the data team has to produce |
|---|---|---|
| Regulations take effect | 1 January 2026 | A scoping list of every automated decision point |
| ADMT notice, opt-out and access for significant decisions | 1 January 2027 | Decision records, opt-out flags that pipelines honor, access-request extracts |
| Risk assessments for processing that began before 2026 and continues | Completed by 31 December 2027 | Data inventories, purpose and flow documentation per processing activity |
| Summary risk-assessment information submitted to the CPPA | 1 April 2028, then annually | Repeatable reporting, not a one-time spreadsheet |
| Cybersecurity audit certification, businesses over $100M revenue | 1 April 2028 (later dates for smaller businesses) | Access-control evidence and audit logs |
Notice that the later dates reuse the earlier artifacts. The inventory you build for ADMT feeds the risk assessment. The audit logs you keep for access requests feed the cybersecurity audit. Build once on a governed platform and each deadline becomes a report, not a project.
Why This Lands on the Data Team
Privacy teams own the policy. Legal owns the notice wording. But every obligation above resolves to a query: which consumers were scored by which system, on which inputs, with which outcome, and when. If the answer spans a CRM, an HR platform, a vendor score delivered by SFTP and a lending system, you cannot run that query. You run a scavenger hunt.
This is where the difference between a data lake and a lakehouse stops being academic. A lake gives you a place to put the files. A lakehouse gives you tables with schemas, transactions, a catalog, grants and lineage on top of them. Only the second produces evidence.
The Four Pieces of Data Work
1. An inventory of decision systems
Build a table, not a document. One row per decision point: the business process, the system (model, rules engine or vendor score), the owner, the significant-decision category, the data sources it reads, and the notice text version in force. In a Unity Catalog lakehouse, store it as a governed table and tag the source tables it references with governed tags so the relationship is queryable in both directions.
2. Lineage on the decision inputs
Unity Catalog captures lineage automatically for queries run on Databricks, down to the column level, across all workspaces attached to the metastore. That answers “which source columns fed this feature table” without anyone drawing a diagram. The same page is honest about the gaps: column-level lineage is not captured for UDFs, and tables referenced by path rather than by name lose column lineage. If your feature code relies on either, fix the code before you rely on the lineage. Our Unity Catalog guide covers the setup.
3. Decision records as tables
Every significant decision writes one row to an append-only decision log. A workable schema:
| Column | Why it exists |
|---|---|
| decision_id, decided_at | The key an access request resolves to |
| consumer_key | A pseudonymous key, with the identity mapping held in a separately governed table |
| system_id, model_version | Joins back to the inventory; proves which logic ran |
| input_snapshot_ref | Points to the exact feature row or table version used |
| outcome, outcome_category | What was decided, mapped to the significant-decision categories |
| notice_version, opt_out_status | Shows the consumer was told and whether they had opted out |
| human_reviewer, review_outcome | Populated when a person reviewed or overturned the result |
Only the service principal that runs the scoring job gets write access. Analysts get read access to a masked view. People never edit rows by hand.
4. Opt-out and access-request plumbing
An opt-out is worthless if the scoring pipeline never reads it. Land opt-out events from the preference center into a bronze table, conform them in silver, and make the scoring job join against the current opt-out state before it scores anyone. For access requests, a parameterized job in Lakeflow Jobs (formerly Workflows) pulls every decision row for a consumer, joins the inventory for plain-language descriptions, and writes a reviewed extract for the privacy team. Lakeflow Jobs supports branching and for-each loops, so one job handles requests that touch several decision systems.
Access Control and Audit Trails Around the Records
The decision log is sensitive data about sensitive decisions, so the controls around it matter as much as its contents. Unity Catalog attribute-based access control, which Databricks announced as generally available on 13 May 2026, lets you define row filters and column masks as policies driven by governed tags. Tag consumer_key and outcome as sensitive once and the masking follows the tag to every table that carries it.
For who-touched-what evidence, the audit log system table, system.access.audit, records the identity, action and time of each request. It is in Public Preview, with a default 365-day retention. Treat it as a strong evidence source and keep a copy in your own governed table if your retention policy runs longer.
The same caution applies to lineage. The lineage system tables keep a rolling one-year window. If your records policy runs past a year, snapshot the lineage rows for in-scope tables into your own table on a schedule. Our write-up on Databricks system tables shows the query patterns.
Fix the Decision Process Before You Automate the Record
I run every one of these engagements through AIM-IT: Assess, Innovate, Model, Implement, Track. The Assess step usually finds that nobody agrees where a decision is actually made. The model scores, a supervisor overrides, and the override lives in email. If you automate the logging of that process as it stands, you log the score and miss the decision.
So map the value stream first. Who decides, on what inputs, with which handoffs. Then put the decision log at the real decision point, which is often after the human step, not after the model.
Where ADMT Programs Stall
The model nobody owns
Vendor scores are the usual orphans. The vendor built it, procurement signed it, and the business uses it daily, but no one inside owns the inventory row. Without an owner there is no one to answer an access request about how it works. Assign an owner before you assign an engineer.
Inputs that arrive by spreadsheet
A decision that reads a manually maintained lookup file has no lineage past the file. Move the lookup into a governed table with an owner and a change history, or accept that part of the decision cannot be explained from records.
Records that outlive the logs
Platform logs have their own retention windows, and your policy set with counsel will not match them. Teams discover this during the first real request, when the lineage for a decision from fourteen months ago has already rolled off. Copy what you need into tables you control from day one.
Opt-outs that never reach the pipeline
The preference center records the opt-out. The scoring job runs from a nightly extract taken before the opt-out arrived. The consumer is scored anyway. Make the opt-out join part of the scoring job and test it with a real opt-out before go-live.
Your First 90 Days Toward the 2027 Deadline
Split the work into three blocks. Each one ends with an artifact counsel can review.
- Weeks 1 to 2: inventory and owners. Walk every business process that can grant or deny credit, housing, education, employment or healthcare. List each automated decision point, its owner and its data sources in one governed table. Flag vendor scores separately.
- Weeks 3 to 6: land and trace the inputs. Bring the source data for in-scope decisions into bronze and silver layers of the lakehouse, register every table in Unity Catalog, tag sensitive columns, and confirm lineage renders end to end. Replace spreadsheet inputs with governed tables.
- Weeks 7 to 12: records and plumbing. Stand up the decision log, wire opt-out state into every scoring job, build the access-request job, apply masking policies, and schedule the audit and lineage snapshots. Close with a dry run: answer a mock access request from records alone and time it.
If you want a team that has built this kind of governed decision layer on Databricks, our AI governance and compliance work starts with exactly that 90-day scope. This article is general information, not legal advice, and counsel or a qualified assessor makes the compliance call on your specific systems.
Frequently Asked Questions (FAQs)
Does the CCPA ADMT rule apply to every AI model we run?
No. The reported scope covers ADMT used to make a significant decision: providing or denying financial or lending services, housing, education, employment or compensation, or healthcare. Targeted advertising alone is excluded. A recommendation model for product suggestions sits outside it.
When do the CCPA automated decision-making requirements start?
The regulations took effect on 1 January 2026. Law-firm summaries of the final text report that notice, opt-out and access obligations for ADMT used in significant decisions apply by 1 January 2027, with risk-assessment and audit dates running into 2028 and later.
Can Unity Catalog lineage answer an ADMT access request on its own?
No. Lineage tells you which tables and columns fed a feature, but not what a specific consumer’s inputs were or what was decided. You need a decision log keyed to the consumer, with lineage and audit logs as supporting evidence.
Do we need a lakehouse to meet these rules?
No tool makes you compliant. What a governed lakehouse gives you is one catalog, automatic lineage, policy-based masking and audit logs in one place, which turns each obligation into a query instead of a multi-team search.