Satellites, Supercomputers and Big Data Help Refine Storm Forecasts

As Hurricane Ida and other recent storms illustrate, real-time weather data collection and accurate forecasting are essential for protecting lives and property, as well as verifying the inevitable insurance claims when natural disasters occur.

Environmental data collection using small satellites is the latest refinement to building better, more accurate weather modeling, feeding information into supercomputing data centers around the world.

Today, NOAA has 40 petaflops of supercomputing capacity for weather predicting and research with HPC facilities located in Colorado, New Jersey, Mississippi, Tennessee, and West Virginia. NOAA’s latest computer, Hera, was built by HPE/CRAY and has 63,840 processors along with scratch disk capability of 18.5 Petabytes with a top performance of 3.27 petaflops.

Europe and Japan operate similar large scale HPC data centers dedicated to global forecasting (predicting the weather) and climate reanalysis (checking how well predictions matched what happened), with all national organizations constantly planning upgrades to increase processing speeds and data collection and storage.

At least 18 of the world’s top 500 supercomputers are dedicated to weather and climate research, according to the Top 500 list, with another three dedicated to weather forecasting.

Weather predictions are only as good as the data available to feed into increasingly complex software models, with information collected from ground stations, commercial aircraft traffic, instrument-laden balloons, and satellite data.

In the spring of 2020, forecasting accuracy took a hit due to the drastic decline in commercial aircraft flights with the COVID-19 lockdowns and the concurrent loss of data from those aircraft. Under normal conditions, aircraft around the world generate over 800,000 weather observations per day, but measurements dropped anywhere from 75 to 90 percent.

In the spring of 2020, forecasting accuracy took a hit due to the drastic decline in commercial aircraft flights with the COVID-19 lockdowns.

Satellite data provides a true worldwide snapshot of weather conditions independent of other sources, with the United States in cooperation with other nations operating a fleet of large satellites using state-of-the-art instruments to collect twice-daily global measurements of atmospheric, terrestrial, and oceanic conditions, including sea and land surface temperatures, clouds, rainfall, atmospheric temperature, and water vapor.

Large amounts of data are collected by these satellites, with the JPSS-1 operated by NOAA generates 4.5 terabytes of data per day while the Suomi-NPP satellite producing up to 3.9 TB/day.

How Commercial Satellites Support Weather Models

“A seven-day forecast can accurately predict the weather about 80 percent of the time and a five-day forecast can accurately predict the weather approximately 90 percent of the time. However, a 10-day—or longer—forecast is only right about half the time.” — NOAA’s SciJinks website.

A lot happens on the surface of the Earth in the twelve hours between government weather satellite passes, opening opportunities for commercial companies to provide additional satellite observations for supplementing and increasing weather data. The National Oceanic and Atmospheric Administration (NOAA) was authorized by Congress in 2016 to pilot purchases of commercial weather data for use in its weather modeling and forecasting, awarding initial contracts under $1 million each to GeoOptics and Spire Global.

Over the past two years, NOAA has steadily increased the amount of data observations per day it is purchasing and the number of months it will commit to purchasing the data under an Indefinite Delivery Indefinite Quality (IDIQ) contract to buy up to $23 million dollars of data from the firms. NOAA started out by purchasing 500 GPS radio occultation (GPS-RO) profiles per day for thirty days from Spire Global last year. In the latest award on September 1, 2021, NOAA agreed to purchase 3,000 profiles per day from Spire over a six month period.

Founded in 2021, Spire operates a constellation of over 110 LEMUR-2 satellites, each one roughly the size of a wine bottle box at 30 cm x 10 cm 10 cm. The LEMUR-2 has a sensitive radio receiver capable of picking up ship and aircraft tracking messages as well as the constant sounds of global positioning satellite signals broadcast out by U.S., Russian, and European constellations.

According to Spire’s SEC SPAC filing, the company collects a total of 5 terabytes of data per day, an amount of information that no doubt makes Amazon AWS, its cloud provider of record, quite happy.

Spire uses global GPS signal traffic to collect two sorts of environmental data. GPS radio occultation (GPS-RO) measures the density of atmosphere by the delays of radio signals, providing temperature, pressure, and water vapor information. GNSS reflectometry is a newly introduced feature using reflected GPS signals from the planet’s surface to provide information on soil moisture and sea ice.

Spire’s fleet of satellites currently generate up to 10,000 GPS-RO profiles per day. “The radio occultation measurements on a daily basis produce gigabytes of data,” said Dallas Masters, Earth Intelligence / GNSS Director for Spire. The weather forecasting community would like to get up to 20,000 GPS-RO profiles per day as an initial goal, but Spire believes there’s plenty of room to scale upward to meet the needs of global weather forecaster.

“Spire’s going to get up to 100,000 profiles a day at some point in the future,” Masters said, with the company increasing the number of satellites it has in orbit when the market wants more data. “We’ve come over the hump of getting the (GPS-RO) data tested by weather agencies and seeing the maturity of those data and its impact. The expectations is those agencies are going to see those benefits and then continue to increase those purchases in terms of volume.”

Using Satellite Data to Guide Policy Decisions

Start-up Orbital Micro Systems (OMS) is just starting to grow its efforts to generate and sell commercial weather data. The company has flown a prototype satellite over the past two years, using a customized microwave radiometer instrument to capture 3D temperature and moisture profiles of the atmosphere. Two second-generation satellites are being built for launch later this year and in 2022, with the company focusing on the insurance sector along with data sales to NOAA, the Department of Defense, and other organizations.

“We set out to form this company around the idea of the idea of improving and coming up with new ideas for managing environment risk,” said Michael Hurowitz, OMS Chief Executive Officer and Chief Technology Officer. “Just as important as the (satellite) enhanced observation system are the data analytics, data processing tools, ability to extract information from raw sensor information and using it to drive business and government policies.”

A single OMS satellite will generate 5 GB/day, with a pair of satellites able to provide detailed environmental information about a location on the planet’s surface up to four times a day under ideal conditions. Hurowitz eventually expects to have a constellation of up to 50 satellites providing global coverage and the ability to provide updated information every 15 to 30 minutes. OMS is an Amazon AWS customer, but its data systems are built to be cloud-agnostic, so it can provide customer-specific deployments on other services and for private deployments.

Insurance Analysis via Space Data

Working with the financial industry, Hurowitz sees OMS supporting parametric insurance policies, with supplemental coverage policies coming into play with specific weather events. For example, a homeowner could purchase supplemental insurance for wind or rainfall damage. Satellite data would be used to verify the triggering event for the policy coming into force and paying out if you had hurricane-force winds or excessive rain in your area.

Customized weather products are likely to become a data center growth driver in the years to come across multiple verticals, including maritime, agriculture, finance, energy, and insurance sectors. In the future, Spire, OMS, and other companies anticipate future growth moving beyond “Level 1” data providers feeding minimally processed information into other people’s weather models into more value-added Level 2 processed data and Level 3 application services available via subscription.

Already Spire is teaming up with third-parties to build customized data products and applications for clients such as enterprise clients like Chevron and start-up companies such as PredictWind.com providing sport sailors accurate open ocean wind speed and direction. OMS anticipates building its own applications in partnership with vertical partners.