EU Leads the Way: The AI Act’s Crucial Ban on Dangerous Technologies

Amnesty International Calls for the European Union to Ban Dangerous AI Technologies in the AI Act

Amnesty International has urged the European Union (EU) to ban dangerous AI-powered technologies in the upcoming AI Act. The AI Act aims to establish the world’s first comprehensive rulebook for AI regulation. Amnesty International argues that numerous states worldwide have deployed unregulated AI systems that assess welfare claims, monitor public spaces, and determine the likelihood of someone committing a crime. These technologies, often presented as “technical fixes” for structural issues like poverty and discrimination, use massive amounts of data and automated systems to make decisions about individuals’ access to housing, benefits, healthcare, and education. However, instead of improving welfare access, many AI systems have exacerbated racism, inequality, and human rights violations.

The Harms Amplified by AI Systems

Mher Hakobyan, Amnesty International’s Advocacy Advisor on AI Regulation, explains that these AI systems do not serve to support people’s access to welfare but rather to cut costs. Moreover, when combined with existing systemic racism and discrimination, these technologies significantly amplify harm against marginalized communities. Amnesty International has documented cases such as the Dutch childcare benefits scandal, where an AI system wrongly accused thousands of parents from low-income and immigrant backgrounds of benefit fraud. This false accusation resulted in immense debt and poverty for these families. Batya Brown, one of the victims, expressed her dismay at being falsely accused and highlighted the ongoing mistreatment they endure even years later.

Amnesty International argues that EU lawmakers should prioritize addressing existing problems, particularly the use of AI technologies that make discriminatory decisions undermining access to basic human rights. Social scoring, profiling, and risk assessment systems should be banned in the AI Act, especially when used to police welfare recipients, predict crime probabilities, or decide on asylum claims.

Intrusive Surveillance and Facial Recognition

Under the pretext of national security, facial recognition systems have become prevalent tools for excessive surveillance by governments. Law enforcement agencies deploy these systems in public spaces to identify individuals suspected of committing crimes, despite the risk of wrongful arrest. Amnesty International, along with a coalition of more than 155 organizations, calls for a complete ban on live and retrospective facial recognition in publicly accessible places, including border areas and detention facilities. Amnesty International has documented how facial recognition systems exacerbate existing systems of control and discrimination in various places, including New York, Hyderabad, and the Occupied Palestinian Territories.

Moreover, Amnesty International highlights the use of facial recognition in the Occupied Palestinian Territories by Israeli authorities to police and control Palestinians, restricting their freedom of movement and access to basic rights. Amnesty International’s research also reveals the involvement of Dutch company TKH Security in supplying surveillance cameras used in occupied East Jerusalem. It is essential, Amnesty International argues, that the EU bans not only the use but also the manufacturing and export of such intrusive surveillance technologies to countries where they contribute to human rights violations.

Abuse of Migrants and Refugees

EU member states have increasingly turned to opaque and hostile technologies to facilitate abuses of migrants, refugees, and asylum seekers at their borders. Amnesty International calls for the ban of racist profiling and risk assessment systems that unfairly label migrants and asylum seekers as threats. The use of forecasting technologies that predict border movements and deny individuals their right to asylum should also be prohibited. Alex Hanna, Director of Research at the Distributed AI Research Institute, emphasizes the personal and existential consequences of AI decisions that limit individuals’ life chances and livelihoods.

Avoiding Self-Regulation by Big Tech

Amnesty International cautions against granting Big Tech companies the power to self-regulate and introduce loopholes in the AI Act’s risk classification process. According to Amnesty International, this would undermine the main objective of the AI Act, which is to protect people from human rights abuses. Amnesty International suggests going back to the original proposal of the European Commission, which provides a clear list of scenarios where the use of an AI tool should be considered high-risk.

Editorial and Recommendations

Addressing the Human Rights Impact of AI

The concerns raised by Amnesty International regarding the use of AI technologies are significant and should not be taken lightly. The impact of these technologies on marginalized communities cannot be overstated, as they have perpetuated racism, discrimination, and human rights abuses. It is crucial that the EU takes concrete action to address these issues and ensure that the AI Act prioritizes human rights protection.

Ensuring Comprehensive Regulation

The AI Act should not focus solely on hypothetical “existential threats” but instead address the real harm caused by existing AI systems. EU lawmakers must recognize and act upon the fact that these technologies are prone to making unjust and discriminatory decisions that undermine access to basic human rights. By explicitly banning social scoring, profiling, and risk assessment systems, the EU can set a strong precedent for the responsible and ethical use of AI in Europe.

Protecting Privacy and Preventing Surveillance Abuse

To protect individuals’ privacy and prevent abuses of facial recognition technology, a complete ban on live and retrospective facial recognition in publicly accessible areas is needed. This ban should extend to border regions and detention facilities. Additionally, EU lawmakers must ensure that companies within their jurisdictions do not profit from exporting intrusive surveillance technologies to countries where they contribute to human rights violations.

Safeguarding the Rights of Migrants and Refugees

The AI Act should explicitly prohibit the use of racist profiling, risk assessment systems, and forecasting technologies that unfairly target migrants, refugees, and asylum seekers. The EU has a responsibility to ensure that its member states treat individuals crossing borders with dignity and respect their fundamental rights, rather than using opaque and hostile technologies to facilitate abuses.

Avoiding Conflicts of Interest

To prevent potential conflicts of interest and the undue influence of Big Tech companies, the EU should reject the idea of self-regulation. Granting these companies the power to decide the risk classification of their own technologies undermines the goal of protecting people from human rights abuses. The EU should adhere to the original proposal by the European Commission, which includes a clear and independent process for determining high-risk AI systems.

Conclusion

Amnesty International’s call for a ban on dangerous AI technologies in the AI Act is a crucial step toward safeguarding human rights in the era of AI. By prioritizing the protection of marginalized communities, addressing the impact of AI on welfare access, intrusive surveillance, and the rights of migrants and refugees, the EU can set a global standard for responsible AI regulation. It is imperative that EU lawmakers heed Amnesty International’s recommendations and enact comprehensive regulations to prevent further harm and ensure that AI technologies serve the betterment of society rather than perpetuating discrimination and inequality.

ArtificialIntelligence-wordpress,AIAct,EU,technology,ban,dangeroustechnologies