California's new legislative proposal, SB 1047, is sending ripples of concern through Silicon Valley.
The bill, introduced by state Senator Scott Wiener, seeks to implement “common sense safety standards” for companies developing large-scale artificial intelligence (AI) models that exceed certain size and cost thresholds.
If passed, the bill would require the tech giants to take steps to prevent AI systems from causing “significant harm,” give them the ability to shut down the systems if necessary, and require them to disclose their compliance efforts to a new “Frontier Models Division” within the California Department of Technology. Failure to comply could lead to civil penalties and potential lawsuits, raising the risk for Silicon Valley's AI innovators.
The bill is part of a wave of AI regulations being proposed and debated around the world as lawmakers grapple with AI's rapid advancements and their potential risks and benefits.
From the European Union's AI Act to the White House's AI Bill of Rights, governments are seeking to establish frameworks for the responsible development and deployment of AI technologies. SB 1047 is California's attempt to address these issues at the state level, the outcome of which could have a significant impact on the future of AI regulation in the United States and beyond.
The bill has garnered support from Geoffrey Hinton and Yoshua Bengio, two AI pioneers who have been vocal about the potential existential threats posed by artificial intelligence. Hinton praised the bill, saying it “takes a very sensible approach to balancing these concerns.”
But the bill has also drawn significant backlash, foreshadowing attacks on the national stage if Congress moves forward with tougher legislation.
The impact of AI on companies and innovation
Observers say the potential impact of SB 1047 extends beyond the technology industry, as AI is becoming a key component of commerce across a range of sectors.
From retail and healthcare to finance and transportation, businesses are using AI to optimize operations, personalize customer experiences, and drive innovation, so regulations that affect the development and adoption of AI technologies can have far-reaching economic implications.
Critics say that if the bill's requirements are too onerous or vague, they could slow business adoption of AI and put California-based companies at a competitive disadvantage compared to those in other states and countries with less stringent regulations.
The bill could have a major impact on AI companies, especially small startups and open source developers.
“Some safeguards make sense, but regulations need to be precise so as not to stifle innovation in this developing field,” Sunil Rao, CEO and co-founder of AI company Tribble, told PYMNTS.
“Excessive regulation risks hurting smaller businesses trying to push the boundaries of what's possible with AI. We need to think carefully to strike the right balance. Having basic safety standards is sensible, but imposing too heavy a compliance burden will enable tech giants to build AI systems,” he added. “Otherwise we risk going the same way as what happened with nuclear energy in the US, where heavy regulation stifled innovation.”
On the upside, companies can have more confidence that the AI tools they use have been developed to higher safety standards, Bob Rogers, data scientist and CEO of California-based supply chain AI company Oii.ai, told PYMNTS.
“Unfortunately, there are more drawbacks than benefits,” he added. “Businesses will likely face higher prices for AI tools and services as AI developers pass on the extra costs associated with compliance to their customers. Regulation could also impact innovation if AI companies feel the need to be extra careful. Business users themselves may also feel burdened if they have to develop strategies to ensure the AI tools they use are compliant.”
The bill would impose new safety requirements and oversight on the development of large-scale, advanced AI models.
“Companies will have to assess risk, implement safeguards and report compliance to regulators in the new states or risk penalties,” Steven Kowski, field CTO at generative AI security company SlashNext, told PYMNTS.
For companies using AI, this bill creates a lot of uncertainty.
“It is still unclear what technical requirements companies must meet and how the government will assess compliance. That ambiguity could discourage companies from adopting cutting-edge AI technologies,” Rao said.
“Some high-level safety and transparency standards could help increase public trust in AI systems,” he added, “but these standards need to be implemented in a way that allows access to innovative AI tools that help companies innovate and compete. Policymakers should actively engage with the business community to understand their needs and concerns as they refine this legislation.”
Two key aspects of the bill are what it requires companies to do and what happens if they don't comply. Narayana Papp, CEO of Zendata, a San Francisco-based data security and privacy compliance solutions provider, compared it to California's privacy law, the CCPA, which has seen only two enforcement actions from the state since the bill was passed nearly four years ago, while privacy class action lawsuits have grown exponentially, with more than 100 in 2023 alone.
Papp also pointed out potential shortcomings of the bill: “One thing that's missing from this bill is a transparency requirement for the Frontier Model. This would have been a good intermediate step toward a risk mitigation requirement.”
Tarun Tummala, CEO of AI company PressW, told PYMNTS that the bill could unintentionally stifle innovation and make AI systems less safe.
“Unfortunately, while this bill is well-intentioned, I believe it will do more harm than good,” Sammala said. “In many ways, this bill will not only stifle the growth and innovation of AI, but it could also lead to the development of less safe and secure AI.”
The proposed bill would put the burden on software developers to prevent the misuse of AI services, which Sammala said could especially hurt smaller companies in the industry.
“This bill will likely create a world where only large, resource-rich companies can develop these AI models,” he explained. “Open source development is important not only for technological advancement and development, but also for democratizing access to these powerful technologies.”
Tummala believes the bill's approach could backfire and ultimately lead to AI systems being less secure. As an alternative, he proposed legislation that focuses on regulating specific AI applications rather than the development process itself.
“I think laws that focus on use, not development, would better protect our society,” Tummala said. “By regulating the actual applications that are developed and establishing clear boundaries between what is and isn't acceptable AI applications, developers will be free to continue developing and only those who use it to cause harm will be punished.”
Navigating the Legislative Environment
While federal AI legislation is still underway, a patchwork of state-level bills, such as one proposed by California, are emerging.
“California has a huge influence in the tech sector, so its approach could be a model for other states to follow,” Rao said. “I agree with Senator Wiener that federal legislation would be ideal, but the reality is that states may act first. However, a fragmented regulatory environment will be a nightmare for AI companies. It's crucial that states coordinate with each other and be as consistent as possible.”
“The tech industry needs to proactively engage with policymakers at the state and federal levels to help craft smart, innovation-friendly legislation,” he continued. “We have an opportunity now to get the policy framework right and solidify the United States as a leader in AI, but over-regulation could jeopardize that.”
As SB 1047 makes its way through the legislative process, the debate surrounding the bill highlights the tension between innovation and regulation in the rapidly evolving world of AI. All eyes in the tech industry are on California awaiting the outcome of its AI regulations.
