Weather reanalyses and forecasts are usually plotted on grids of ~25km, but the people's decisions about weather-related actions happen much more locally! Within a single coarse grid cell the land surface isn't uniform, so the weather at a given spot can depart from the cell average quite significantly in ways that terrain and land cover largely determine.
We've shown that a general purpose Earth observation embedding (Tessera in our case) closes some of that gap and enables much more accurate local forecasting. In Earth observation embeddings are effective sub-grid descriptors for probabilistic weather downscaling we compressed a patch of embeddings at 10m into a per-location latent and gave it to a convolutional conditional neural process downscaling ERA5.
Across five climatically diverse regions and 9000+ stations held out in both space and time, this improved CRPS skill by 11.5% for 2m temperature and 6.2% for 10m wind speed. We have ablations and hypotheses about why this happens. Topography explains more of the sub-grid structure of temperature prediction variability, while the embedding itself carries surface information that matters more for wind (e.g. is there a canyon?). I've jotted down an explainer with more of our thoughts about the paper results.
So we know using satellite embeddings helps forecastign, and not much about why they do. When it comes to forecasting, it would be ideal if we know more about the causative factors behind particular temperature variations, in case it helps with climate adaptation efforts. This project aims to help figure out the "why" behind some of this data-driven prediction magic. You would first investigate and pick one of these; the difficulty level here is fairly high.
1 Which parts of the embedding carry the downscaling skill?
Hypothesis: The weather prediction skill rests on only a few interpretable surface properties, and the rest of the Tessera embedding is inert for a given task like temperature or wind prediction.
The interpretability experiments in our paper looked only at the VAE-encoded patch latent, and not at the original embedding features it was compressed from. We therefore know a little about the summary and nothing about what survived the compression.
An interprebility experiment would be to attribute skill back to the raw per-pixel features, and to ask whether the embedding dimensions that are significant correspond to anything that a person would recognise. These might include canopy structure, built roughness, water or slope aspects. This may work particularly well with Tessera v2 which has Matryoshka embeddings whose first dimensions pack in more information than later ones.
2 Where should the next weather station go?
Hypothesis: The embedding space knows which places a network already represents, so it should also know where a new station would provide the most prediction gain for a larger region.
This builds on Rich Turner's environmental sensor placement work about figuring out how to get the best coverage of a large (and often remote) region. Consider the Norwegian network deployment experiment in the paper. If a given station teaches the downscaler about the surfaces "near it" in embedding space, then station coverage is dependent on which representative surfaces we have instrumented, rather than of the overall surface geography.
What if we instead make this an objective; i.e., "where do I put X stations to guarantee Y level of downscaling skill over this region?". We could then test how close a greedy or submodular selection over the embedding space gets to this goal. We're trying to make all stations "reachable" within the embedding space based on which representative surfaces we deploy at, thus maximising the overall surface coverage.
3 Who this suits
This is a good project for an MPhil student who wants to get into data-driven earth systems modelling. You would work alongside Pedro Sousa who is doing a PhD in this space, and many of the models and data are already available from his experiments. Comfort and interest with probabilistic modelling will help a lot, but you can pick this up as you go along too.