Gotta Catch 'Em All: How Pokémon Go covertly captured your data for years to train a massive AI model (2024)

Players of "Pokémon Go" — an augmented reality (AR) mobile game that took the world by storm upon its release in 2016 — have been unknowingly training an artificial intelligence (AI) model to map the planet at street level.

Niantic, the company behind the popular game, has revealed that it will use data scraped from its AR apps to construct a "large geospatial model" (LGM) that would enable robots and other devices to better navigate the physical world — even if they only have limited information.

The announcement, made Nov. 12 in a blog post on Niantic’s website, reveals that the company has drawn data from more than 10 million scanned locations worldwide, with users adding around 1 million more new scans each week.

This data has already been used to train 50 million local neural networks (collections of machine learning algorithms structured like the human brain) to operate in more than a million locations worldwide, the company said.

"In our vision for a Large Geospatial Model (LGM), each of these local networks would contribute to a global large model, implementing a shared understanding of geographic locations, and comprehending places yet to be fully scanned," Niantic staff scientist Eric Brachmann and chief scientist Victor Adrian Prisacariu wrote in the post. "The LGM will enable computers not only to perceive and understand physical spaces, but also to interact with them in new ways, forming a critical component of AR glasses and fields beyond, including robotics, content creation and autonomous systems."

Just as Large Language Models (LLMs) such as ChatGPT consume vast quantities of text to accurately guess the most probable words to complete a sentence, LGMs gorge on geodata to infer what buildings in physical space should look like.

Related: 'I'd never seen such an audacious attack on anonymity before': Clearview AI and the creepy tech that can identify you with a single picture

This might seem like a strange task. For humans, our existence in the physical world has already exposed us to innumerable examples that helped us to build a robust spatial understanding.

"But for machines, this task is extraordinarily difficult. Even the most advanced AI models today struggle to visualize and infer missing parts of a scene, or to imagine a place from a new angle," Niantic representatives wrote in the post.

Niantic’s LGM is built on its Visual Positioning System, which uses a single smartphone camera image to pinpoint an object’s position and orientation down to the centimeter (0.4 inches).

As for "Pokémon Go" fans, many seem largely unfazed and unsurprised that their data has been scraped for use by an AI system. Yet critics fear that some of the potential applications of Niantic’s technology could be far from benign.

"It's so incredibly 2020s coded that Pokémon Go is being used to build an AI system which will almost inevitably end up being used by automated weapons systems to kill people,” Elise Thomas, a senior intelligence analyst at the Institute for Strategic Dialogue, a political advocacy organization, wrote on X.