Google Street View has long served as a nostalgic tool for many users, allowing them to wander through familiar locales from their past. However, an exciting update is on the horizon, promising a more engaging experience: the ability to virtually change aspects like weather and time to visualize various scenarios in familiar settings.
This elevated functionality is made possible by Google DeepMind’s integration of Street View with Project Genie. Unveiled during the recent Google I/O developer conference, this project introduces a versatile world model capable of creating dynamic, interactive environments.
“The utility is immense for robotics as well as human users,” said Jack Parker-Holder, a research scientist on DeepMind’s open-endedness team, in an interview with ToolsMixAi.
As an illustrative case, he referenced a new robot being tested in London, a city known for its gray skies. Genie could mimic those rare sunny moments, ensuring the robot isn’t taken by surprise when the sun shines upon the city’s Victorian architecture.
“Imagine stating, ‘I’m off to New York City, but I want to see it in a snowy landscape,’” he added. “Genie enables users to project various weather conditions onto specific locations.”
For two decades, Google has gathered immense amounts of Street View data using camera-equipped vehicles and “tracker backpacks” worn by people. This archival includes over 280 billion images from 110 countries across all seven continents.
“Street View provides an extensive collection of imagery globally,” Jack noted. “The potential to merge this vast amount of real-world information with world-simulation capabilities is significant.”
Last August, Google debuted the research preview of its latest world model, Genie 3, extending access to Google AI Ultra subscribers in the United States in January. This tool enables users to produce interactive landscapes from simple text or images, aiming for applications in education, gaming, and robot training.
Genie 3 is integral to one of Waymo’s simulators, which equips self-driving cars to handle infrequent occurrences like tornadoes or unexpected wildlife. Integrating Street View data could further assist Waymo’s expansion into new urban markets worldwide.
Waymo has developed its simulation system to cover 11 U.S. cities and assess its AI driver in additional locations. Parker-Holder emphasized that unlike its existing simulator—focused solely on vehicular perspectives—Street View facilitates a broader range of viewpoints, allowing for perspectives from both humans and robots.
Google is beginning to roll out Street View integration in Genie to select Ultra users in the U.S. today, with plans for broader global access in the upcoming weeks, according to the company.
Diego Rivas, a product manager at DeepMind, emphasized the intention to make this new feature widely available. However, he acknowledged that the technology, both Street View and Genie, is still in a developmental phase, requiring enhancements for accuracy.
During a demonstration, I observed models that included an underwater simulation of a neighborhood from my past. While the results were visually striking, they were more akin to a video game than a photo-realistic depiction. Additionally, the models lacked genuine physics; for instance, a woman traversing a snowy Joshua Tree simply passed through nearby cacti.
In contrast, Google’s image generator Nano Banana can create flawless text in infographics, while its video generator Veo accurately simulates natural phenomena like current effects on paper boats or smoke dispersion. Such physics aren’t directly programmed; instead, these models acquire knowledge through observation, mimicking natural learning processes.
“This model potentially lags behind video quality by six to 12 months,” Parker-Holder stated. “We anticipate resolving these issues soon.”
Jonathan Herbert, who has been with the Google Maps team for 12 years, shared that while Genie hasn’t yet achieved intricate street reconstructions, the real advancement lies in its AI’s spatial continuity. It accurately recalls and simulates surroundings as users rotate, enabling the creation of new environments from a solid foundation.
“Our long-standing vision has been to leverage Street View data for richer representations of the world,” Herbert remarked. “We have consistently aimed to explore innovative applications of Maps Data in AI research.”
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.
#Googles #Genie #world #model #simulate #real #streets #Street #View