Cheap online fake accounts make misinformation a ‘thriving underground market’, study finds

Researchers have mapped how much it costs to set up fake online accounts in every country in the world, as governments and regulators grapple with online misinformation and fraud.

The University of Cambridge on Thursday launched the Cambridge Online Trust and Safety Index (COTSI), a website, which it says is the first global tool to track real-time prices for verifying fake accounts across more than 500 platforms, including TikTok, Instagram, Amazon, Spotify and Uber.

These accounts are often used to build “bot-armies,” which are designed to mimic real people and shape online public debate. Authors of the study say they can be deployed to flood online conversations, promote scams or products, or push political messages in a coordinated way.

The study comes at a critical moment for online trust, as major social media platforms have scaled back content moderation efforts and begun paying users for engagement, potentially incentivising reliance on fake interactions.

Earlier this week, the United Kingdom sanctioned Russian and Chinese firms suspected of being 'malign actors' in information warfare.

The study also found that the rise of generative artificial intelligence (genAI) has made the problem more acute.

“We find a thriving underground market through which inauthentic content, artificial popularity, and political influence campaigns are readily and openly for sale,” Jon Roozenbeek, a senior author of the study and a computational social psychologist at the University of Cambridge, wrote in a statement

“This can be done by simulating grassroots support online, or generating controversy to harvest clicks and game the algorithms,” he added.

Vendors operating banks of thousands of SIM cards and millions of ready-made verifications can generate fake accounts for only a few cents.

Based on the supplier data researchers tracked for a year shows that verification is cheapest in the United Kingdom, the United States, and Russia, and markedly higher in Japan and Australia, where stricter SIM rules increase costs.

According to the analysis, SMS verification for a single fake account costs an average of $0.08 (€0.06) in Russia, $0.10 (€0.086) in the UK, and $0.26 (€0.22) in the US, compared with $4.93 (€4.25) in Japan.

Platforms with the lowest global prices for fake accounts include Meta, Shopify, X, Instagram, TikTok, LinkedIn and Amazon.

Some sellers provide customer support, bulk deals and services to inflate likes, comments and followers, according to the study.

“Generative AI means that bots can now adapt messages to appear more human and even tailor them to relate to other accounts. Bot armies are getting more persuasive and harder to spot,” said Roozenbeek.

The study also noted strong links to Russian and Chinese payment systems, and said the grammar on many supplier websites suggests Russian authorship.

Election-linked price surges

The study also found evidence that political influence campaigns may be driving spikes in the fake account market, with rising demand for “influence operations”.

“Misinformation is subject to disagreement across the political spectrum. Whatever the nature of inauthentic online activity, much of it is funnelled through this manipulation market, so we can simply follow the money,” said Anton Dek, a research associate at the Cambridge Centre for Alternative Finance.

Prices for fake accounts on Telegram and WhatsApp rose sharply in countries about to hold national elections, increasing by 12 per cent and 15 per cent respectively in the 30 days before polls opened.

Because these messaging apps display phone numbers, influence operators must register accounts locally, pushing up demand.

No similar trend was found for platforms such as Facebook or Instagram, where fake accounts created cheaply in one country can be used to target audiences elsewhere.

The team behind the study, which includes misinformation and cryptocurrency experts, believe that regulating SIM cards and enforcing ID checks would raise the cost of producing fake accounts and help curb the market.

They say the new tool can also be used to test policy interventions in countries around the world.

“The COTSI index shines a light on the shadow economy of online manipulation by turning a hidden market into measurable data,” said Sander van der Linden, co-author of the study and a professor of social psychology at the University of Cambridge.

“Understanding the cost of online manipulation is the first step to dismantling the business model behind misinformation.”

Earlier this year, the UK became the first country in Europe to outlaw SIM farms, and the Cambridge team says COTSI will now help measure the impact of that policy.