AI systems with ‘unacceptable risk’ are now banned in the EU

As of Feb. 2, EU regulators can ban the use of AI systems they deem to pose “unacceptable risk” or harm. 

Feb. 2 was the first compliance deadline for the EU’s AI Act, the comprehensive AI regulatory framework that the European Parliament finally approved last March after years of development. The act officially went into effect Aug. 1. What’s now following is the first of the compliance deadlines. 

The act is designed to cover a myriad of use cases where AI might appear and interact with individuals, from consumer applications through to physical environments. 

Under the bloc’s approach, there are four broad risk levels:  

• Minimal risk (e.g., email spam filters) will face no regulatory oversight 

• Limited risk, which includes customer service chatbots, will have a light-touch regulatory oversight 

• High risk — AI for health care recommendations is one example — will face heavy regulatory oversight 

• Unacceptable risk applications — the focus of this month’s compliance requirements — will be prohibited entirely 

Some of the unacceptable activities include: 

• AI used for social scoring (e.g., building risk profiles based on a person’s behavior) 

• AI that manipulates a person’s decisions subliminally or deceptively 

• AI that exploits vulnerabilities like age, disability, or socioeconomic status 

• AI that attempts to predict people committing crimes based on their appearance 

• AI that uses biometrics to infer a person’s characteristics, like their sexual orientation 

• AI that collects “real time” biometric data in public places for the purposes of law enforcement 

• AI that tries to infer people’s emotions at work or school 

• AI that creates — or expands — facial recognition databases by scraping images online or from security cameras 

Companies that are found to be using any of the above AI applications in the EU will be subject to fines, regardless of where they are headquartered. They could be on the hook for up to €35 million (~$36 million), or 7% of their annual revenue from the prior fiscal year, whichever is greater. 

The fines won’t be effective for some time even though organizations are expected to have been fully compliant by Feb. 2. 

Last September, over 100 companies signed the EU AI Pact, a voluntary pledge to start applying the principles of the AI Act ahead of its entry into application. As part of the Pact, signatories — which included Amazon, Google, and OpenAI — committed to identifying AI systems likely to be categorized as high risk under the AI Act. 

There are exceptions to several of the AI Act’s prohibitions. 

For example, the act permits law enforcement to use certain systems that collect biometrics in public places if those systems help perform a “targeted search” for, say, an abduction victim, or to help prevent a “specific, substantial, and imminent” threat to life. This exemption requires authorization from the appropriate governing body, and the act stresses that law enforcement can’t make a decision that “produces an adverse legal effect” on a person solely based on these systems’ outputs. 

The act also carves out exceptions for systems that infer emotions in workplaces and schools where there’s a “medical or safety” justification, like systems designed for therapeutic use. 

The European Commission, the executive branch of the EU, said that it would release additional guidelines in “early 2025,” following a consultation with stakeholders in November. However, those guidelines have yet to be published.