A political influence operation in Malaysia has raised concerns about the use of AI in elections, as Anthropic revealed how Claude AI was used to create a fake news outlet and manipulate social media accounts. The operation, which involved about 1,000 X accounts and a fake news site called 'Malaysia Pulse, ' aimed to target voters in recent state elections by exploiting racial, religious, and royal tensions. The AI-generated content was designed to spread disinformation and inflame social divisions, with some experts warning that AI could be weaponized in future elections across south-east Asia. Dr Ben Loh of Monash University Malaysia noted that AI can reinforce existing biases, making it easier to manipulate public opinion. Anthropic said it disrupted the activity and shared intelligence with authorities, but the incident has sparked alarm about the potential for AI to be used in political campaigns. Experts like Katie Harbath warn that AI could be used to flood information streams with false content, especially in regions with limited media diversity.
The operation involved AI-generated content that was tailored to specific voter groups, using data on census and electoral information to create targeted messages. The fake news outlet scraped legitimate Malaysian reporting and republished it under fabricated bylines, while also incorporating content from Russian and Chinese state-aligned sources. Experts caution that without proper safeguards, AI could be used to manipulate elections and deepen social divisions.
The case in Malaysia serves as a warning about the potential for AI to be used in political manipulation, particularly in regions where media freedom is limited and public opinion is easily swayed by disinformation.