MIT scientists have developed a method that generates satellite images of the future to describe what a region would look like after a potential flood. The method combines a generative artificial intelligence model with a physics-based flood model to create realistic bird’s eye images of a region, showing where flooding is likely to occur given the strength of the flood. ‘an impending storm.
The work is published in the journal IEEE Transactions on Geoscience and Remote Sensing.
As a test, the team applied the method to Houston and generated satellite images depicting what certain locations in the city would look like after a storm comparable to Hurricane Harvey, which hit the region in 2017. The team compared these generated images with real satellite images taken in the same regions after Harvey. They also compared AI-generated images that did not include a physics-based flood model.
The team’s method, enhanced by physics, generated satellite images of future floods that were more realistic and accurate. The AI-only method, on the other hand, generated images of flooding in places where flooding is not physically possible.
The team’s method is a proof of concept, intended to demonstrate a case in which generative AI models can generate realistic and reliable content when combined with a physics-based model. In order to apply the method to other regions to depict flooding from future storms, it will need to be trained on many more satellite images to learn what flooding would look like in other regions.
“The idea is: One day we could use it before a hurricane, where it would provide an additional layer of visualization to the public,” says Björn Lütjens, postdoctoral researcher at the Department of Earth, Atmospheric and Environmental Sciences. of Planets at MIT, who led the research while he was a doctoral student in the MIT Department of Aeronautics and Astronautics (AeroAstro). One of the biggest challenges is encouraging people to evacuate when they are at risk. Perhaps this could be another visualization to help increase that readiness.
To illustrate the potential of the new method, which they dubbed the “Earth Intelligence Engine,” the team made it available as an online resource for others to try.
Co-authors of the study at MIT are Brandon Leshchinskiy; Aruna Sankaranarayanan; and Dava Newman, AeroAstro professor and director of the MIT Media Lab; as well as collaborators from several institutions.
The new study is an extension of the team’s efforts to apply generative AI tools to visualize future climate scenarios.
“Providing a hyper-local perspective on climate appears to be the most effective way to communicate our scientific results,” says Newman, lead author of the study. “People relate to their own postcode, to their local environment where their family and friends live. Providing local climate simulations becomes intuitive, personal and relevant.
For this study, the authors used a conditional generative adversarial network, or GAN, a type of machine learning method that can generate realistic images using two competing neural networks (“adversarial”). The first “generator” network is trained on pairs of real data, such as satellite images before and after a hurricane. The second “discriminator” network is then trained to distinguish real satellite imagery from that synthesized by the first network.
Each network automatically improves its performance based on feedback from the other network. The idea, then, is that such contradictory back-and-forth should ultimately produce synthetic images that are indistinguishable from reality. However, GANs can still produce “hallucinations” or factually incorrect features in an otherwise realistic image that should not be there.
“Hallucinations can mislead viewers,” says Lütjens, who began to wonder if such hallucinations could be avoided, so that generative AI tools could be trusted to help inform people, especially in risk-sensitive scenarios. “We were wondering how we could use these generative AI models in the context of climate impact, where it is so important to have reliable data sources? »
Related News :