SB 1047 August 15 Author Amendments Overview
By Nathan Calvin, Senior Policy Counsel at Center for AI Safety Action Fund
From its earliest stages, California’s SB 1047 – the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act – has been designed to be light-touch, common-sense legislation that will protect the public interest while ensuring developers can continue to innovate.
Based on productive conversations with industry, academic, and open-source stakeholders during the legislative process, SB 1047 has been amended by Sen. Scott Wiener. These amendments aim to address concerns of stakeholders who seek to protect California’s thriving and competitive AI ecosystem while ensuring safety at the frontier of generative AI.
Below is an overview of those amendments. To see the entirety of the amendments, please see the bill’s text, which will be updated in the coming days.
Enforcement
Limitation of civil penalties for violations that do not result in harm or imminent risk. In order to avoid stakeholder concerns about premature enforcement, developers can now face civil penalties only for violations that actually cause harm or imminent risk or threat to public safety. This clarifies and narrows the scope of liability in a manner that still protects public safety.
Elimination of penalty of perjury. The bill imposes no new criminal liability. Certifications of compliance are now called statements of compliance and are submitted to the Attorney General rather than the FMD. Statements of compliance under SB 1047 conform with existing law on the submission of false documents to a public agency.
Simplification of injunctive relief. SB 1047 has been simplified to make the conditions for a court granting injunctive relief more closely align with existing California law. Under existing California law, injunctive relief is only available for wrongful acts that threaten to cause irreparable injury, so the bar for an injunction is high but reachable when truly necessary. The bill also no longer explicitly calls out full shutdown or deletion as remedies.
Regulatory Body
Elimination of the Frontier Model Division. The Frontier Model Division has been removed and its duties have been scaled down in response to feedback from stakeholders who worried about excessive institutional overhead. Instead, its three most important responsibilities – evolving the threshold for compute, issuing safety guidance, and issuing regulations for auditors – have been moved to the already existing Government Operations Agency. The Board of Frontier Models remains to oversee the work of the Government Operations Agency with respect to this bill.
Expansion of the Board of Frontier Models. Moves from 5 members to 9, and adds more expertise in chemical, biological, radiological, and nuclear weapons, cybersecurity of critical infrastructure, and AI safety.
Open-source
Addition of Permanent Fine-Tuning Threshold. To further protect the open-source ecosystem, a permanent fine-tuning threshold was added to the bill. If a person uses less than $10 million worth of compute to fine-tune an existing covered model, that person is not considered a “developer” under the bill and thus has no obligations. No government agency is able to change this threshold, so smaller fine-tuning runs will never produce obligations under the bill.
Clarifying fine-tuning rules for open source models. Open source models may have been fine-tuned for an unknown amount of compute prior to their release. This amendment makes clear that one cannot have obligations under SB 1047 unless one fine-tunes oneself for the entire fine-tuning compute threshold amount. Common law legal standards will prevent people from coordinating their fine-tuning runs to intentionally evade this provision.
CalCompute moved to the UC system. CalCompute, a public cloud computing resource, is being moved to the University of California system to support open-source and academic AI research that may lack access to large-scale compute resources.
Reasonable Care
“Reasonable assurance” modified to “reasonable care.” Knowing that models can behave in unexpected ways and that not all potential harms could be foreseen, references to “reasonable assurance” have been changed to “reasonable care” throughout the bill. This helps clarify the focus of the bill on testing and risk mitigation and uses the term “reasonable care,” which is the most common standard in existing tort liability. Reasonable care means the developer has taken the care that a reasonable person would have taken in that situation.
Reasonable care factors. To ensure that developers took reasonable care as required by SB 1047, SB 1047 now specifies that factors like the quality of a developer’s safety and security protocol and its investigation of risks are relevant to make this determination.
Public Reporting & Transparency
Addition of public reporting. Previously, developers were required to send full safety and security protocols and audit reports to the Frontier Model Division. With the removal of the FMD, the bill now requires them to publicly release protocols and reports and transmit them to the Attorney General, with redactions for public safety or confidential information. The developer is required to provide the Attorney General access to the unredacted materials upon request. Auditing will separately help ensure that companies use redactions appropriately.
Removal of uniform pricing requirements. In response to feedback from startups and larger companies about unintended consequences of the provision as drafted, SB 1047 no longer requires a uniform price schedule for developers and cloud providers.
Testing
Requiring specific tests pre-deployment instead of pre-training. Previously, the bill required developers to detail specific tests they planned to run prior to training. But testing can be an iterative process, and it might not be possible to specify tests upfront. This amendment requires specific tests to be recorded and retained prior to deployment instead of prior to training.
Auditing Requirements & Penalties
Binding regulations for auditors. The Government Operations Agency is now tasked with creating binding regulations for auditors to ensure the integrity, independence, efficiency, and effectiveness of the auditing process. Previously, the Frontier Model Division issued guidance for auditors, but it was not mandatory.
Civil penalties for auditors who make misrepresentations. The amendments allow the Attorney General to seek civil penalties for auditors who knowingly make material misrepresentations on an audit report.
Whistleblower Requirements
Narrowing whistleblower requirements for contractors. Developers are now able to allow their contractors working on covered models the ability to use the developer’s own internal process rather than contractors needing to have their own internal whistleblowing programs or give their employees notice of their rights under the bill.
Publicly-released whistleblower reports. The Attorney General and Labor Commissioner may release whistleblower reports publicly, with appropriate redactions.
Other Clarifications
Only harms caused or materially enabled by a developer are in scope. If a safe covered model is modified by a bad actor to be highly dangerous, is the original developer responsible? If the malicious actor could have just as easily made a dangerous model from scratch and access to the developer’s model didn’t help, then ordinary tort law would not consider the developer to have “caused” or “materially enabled” that harm. With this amendment, SB 1047 clarifies there is no intention to override this background standard. The developer would only be held liable if their model materially enabled the malicious actor.
Clarifying exceptions for critical harms caused with publicly accessible information. There was previously an exception for critical harms that were caused by the provisioning of publicly-accessible information. SB 1047 now clarifies that information must be reasonably publicly accessible by an ordinary person in order to be excluded – not only by people with highly specialized skills. This amendment clarifies that, for example, the expert synthesis of obscure virology papers to assemble and release a bioweapon would constitute a critical harm.