Increasing coverage of commercial and public satellites allows us to monitor the pulse of the Earth in ever-shorter frequency (Zhu et al., 2017). Together with the rise of deep learning in artificial intelligence (AI) (LeCun et al., 2015), the field of AI for Earth Observation (AI4EO) is growing rapidly. However, many supervised deep learning techniques are data-hungry, which means that annotated data in large quantities are necessary to help these algorithms reach their full potential. In many Earth Observation applications such as change detection, this is often infeasible because high-quality annotations require manual labeling which is time-consuming and costly.  

Self-supervised learning (SSL) can help tackle the issue of limited label availability in AI4EO. In SSL, an algorithm is pretrained with tasks that only require the input data without annotation. Notably, Masked Autoencoders (MAE) have shown promising performances recently where a Vision Transformer learns to reconstruct a full image with only 25% of it as input. We hypothesize that the success of MAEs also extends to satellite imagery and evaluate this with a change detection downstream task. In addition, we provide a multitemporal DINO baseline which is another widely successful SSL method. Further, we test a second version of MAEs, which we call GeoMAE. GeoMAE incorporates the location and date of the satellite image as auxiliary information in self-supervised pretraining. The coordinates and date information are passed as additional tokens to the MAE model similar to the positional encoding. 
The pretraining dataset used is the RapidAI4EO corpus which contains multi-temporal Planet Fusion imagery for a variety of locations across Europe. The dataset for the downstream task also uses Planet Fusion in pairs as input data. These are provided on a 600m * 600m patch level three months apart together with a classification if the respective patch has changed in this period. Self-supervised pretraining is done for up to 150 epochs where we take the model with the best validation performance on the downstream task as a starting point for the test set. 

We find that the regular MAE model scores the best on the test set with an accuracy of 81.54% followed by DINO with 80.63% and GeoMAE with 80.02%. Pretraining MAE with ImageNet data instead of satellite images results in a notable performance loss down to 71.36%. Overall, our current pretraining experiments can not yet confirm our hypothesis that GeoMAE is advantageous compared to regular MAE. However, in similar spirit, Cong et al. (2022) recently introduced SatMAE which outlines that for other remote sensing applications, the combination of auxiliary information and novel masking strategies is a key factor. Therefore, it seems that a combination of location and time inputs together with adapted masking may also hold the most potential for change detection. There is ample potential for future research in geo-specific applications of MAEs and we provide a starting point for this with our experimental results for change detection. 

How to cite: Kondmann, L., Senaras, C., Asano, Y. M., Rana, A. S., Wania, A., and Zhu, X. X.: Geography-Aware Masked Autoencoders for Change Detection in Remote Sensing, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-2843, https://doi.org/10.5194/egusphere-egu23-2843, 2023.

New models are emerging from Artificial Intelligence (AI) and its sub-fields, in particular, Machine Learning and Deep Learning that are being applied in different application areas including geography (e.g., land cover identification and traffic volume forecasting based on spatial data). Different from well-known datasets often used to develop AI models (e.g., ImageNet for image classification), spatial data has an intrinsic feature, i.e., spatial heterogeneity, which leads to varying relationships across different regions between the independent (i.e., the model input X) and dependent variables (i.e., the model output Y). This makes it difficult to conduct large-scale studies with a single robust AI model. In this study, we draw on the idea of modular learning, i.e., to decompose large-scale tasks into sub-tasks for specific sub-regions and use multiple AI models to achieve these sub-tasks. The decomposition is based on the spatial characteristics to ensure that the relationship between independent and dependent variables is similar in each sub-region. We explore this approach for forecasting COVID-19 cases in Germany using spatiotemporal data (e.g., weather data and human mobility data) as an example and compare the prediction tasks with a single model to the proposed decomposition learning procedure in terms of accuracy and efficiency. This study is part of the project DAKI-FWS which is funded by the Federal Ministry of Economic Affairs and Climate Action in Germany to develop an early warning system to stabilize the German economy.

How to cite: Cheng, X., Arndt, J., Marquez, E., and Ma, J.: Decomposition learning based on spatial heterogeneity: A case study of COVID-19 infection forecasting in Germany, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-3267, https://doi.org/10.5194/egusphere-egu23-3267, 2023.

The iMagine project is devoted to developing and delivering imaging data and services for aquatic science. Started in September 2022, the project will provide a portfolio of image data collections, high-performance image analysis tools empowered with Artificial Intelligence, and best practice documents for scientific image analysis. These services and documentation will enable better and more efficient processing and analysis of imaging data in marine and freshwater research, accelerating our scientific insights about processes and measures relevant to healthy oceans, seas, and coastal and inland waters. By building on the European Open Science Cloud compute platform, iMagine delivers a generic framework for AI model development, training, and deployment, which researchers can adopt for refining their AI-based applications for water pollution mitigation, biodiversity and ecosystem studies, climate change analysis and beach monitoring, but also for developing and optimising other AI-based applications in this field. The iMagine AI development and testing framework offers neural networks, parallel post-processing of extensive data, and analysis of massive online data streams in distributed environments. The synergies among the eight aquatic use cases in the project will lead to common solutions in data management, quality control, performance, integration, provenance, and FAIRness and contribute to harmonisation across RIs. The resulting iMagine AI development and testing platform and the iMagine use case applications will provide another component to the European marine data management landscape, valid for the Digital Twin of the Ocean, EMODnet, Copernicus, and international initiatives. 

How to cite: Fava, I., Thijsse, P., Sipos, G., and Schaap, D.: Using AI and ML to support marine science research, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-4929, https://doi.org/10.5194/egusphere-egu23-4929, 2023.

Eddies are circular rotating water masses, which are usually generated near the large ocean currents, e.g., Gulf Stream. Monitoring eddies and gaining knowledge on eddy statistics over a large region are important for fishery, marine biology studies, and testing ocean models.

At mesoscale, eddies are observed in radar altimetry, and methods have been developed to identify, track and classify them in gridded maps of sea surface height derived from multi-mission data sets. However, this procedure has drawbacks since much information is lost in the gridded maps. Inevitably, the spatial and temporal resolution of the original altimetry data degrades during the gridding process. On the other hand, the task of identifying eddies has been a post-analysis process on the gridded dataset, which is, by far, not meaningful for near-real time applications or forecasts. In the EDDY project at the University of Bonn, we aim to develop methods for identifying eddies directly from along track altimetry data via a machine (deep) learning approach.

Since eddy signatures (eddy boundary and highs and lows on sea level anomaly, SLA) are not possible to extract directly from along track altimetry data, the gridded altimetry maps from AVISO are used to detect eddies. These will serve as the reference data for Machine Learning. The eddy detection on 2D grid maps is produced by open-source geometry-based approach (e.g., py-eddy-tracker, Mason et al., 2014) with additional constraints like Okubo-Weiss parameter. Later, Sea Surface Temperature (SST) maps of the same region and date (also available from AVISO) are used for manually cleaning the reference data. Noting that altimetry grid maps and SST maps have different temporal and spatial resolution, we also use the high resolution (~6 km) ocean modeling simulation dataset (e.g., FESOM, Finite Element Sea ice Ocean Model). In this case, the FESOM dataset provides a coherent, high-resolution SLA and SST, salinity maps for the study area and is a potential test basis to develop the deep learning network.

The single modal training via a Conventional Neural Network (CNN) for the 2D altimetry grid maps produced excellent dice score of 86%, meaning the network almost detects all eddies in the Gulf Stream, which are consistent with reference data. For the multi-modal training, two different training networks are developed for 1D along-track altimetry data and 2D grid maps from SLA and SST, respectively, and then they are combined to give the final classification output. A transformer model is deemed to be efficient for encoding the spatiotemporal information from 1D along track altimetry data, while CNN is sufficient for 2D grid maps from multi-sensors.

In this presentation, we show the eddy classification results from the multi-modal deep learning approach based on along track and gridded multi-source datasets for the Gulf stream area for the period between 2017 and 2019. Results show that multi-modal deep learning improve the classification by more than 20% compared to transformer model training on along-track data alone.

How to cite: Abulaitijiang, A., Bolmer, E., Roscher, R., Kusche, J., and Fenoglio-Marc, L.: Eddy identification from along-track altimeter data with multi-modal deep learning, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-6818, https://doi.org/10.5194/egusphere-egu23-6818, 2023.

Machine learning models are able to capture highly complex, nonlinear relationships, and have been used in recent years to accurately predict crop yields at regional and national scales. This success suggests that the use of ‘interpretable’ or ‘explainable’ machine learning (XAI) methods may facilitate improved scientific understanding of the compounding interactions between climate, crop physiology and yields. However, studies have identified implausible, contradicting or ambiguous results from the use of these methods. At the same time, researchers in fields such as ecology and remote sensing have called attention to issues with robust model evaluation on spatiotemporal datasets. This suggests that XAI methods may produce misleading results when applied to spatiotemporal datasets, but the impact of model evaluation strategy on the results of such methods has not yet been examined.

In this study, machine learning models are trained to predict simulated crop yield, and the impact of model evaluation strategy on the interpretation and performance of the resulting models is assessed. Using data from a process-based crop model allows us to then comment on the plausibility of the explanations provided by common XAI methods. Our results show that the choice of evaluation strategy has an impact on (i) the interpretations of the model using common XAI methods such as permutation feature importance and (ii) the resulting model skill on unseen years and regions. We find that use of a novel cross-validation strategy based on clustering in feature-space results in the most plausible interpretations. Additionally, we find that the use of this strategy during hyperparameter tuning and feature selection results in improved model performance on unseen years and regions. Our results provide a first step towards the establishment of best practices for model evaluation strategy in similar future studies.

How to cite: Sweet, L., Müller, C., Anand, M., and Zscheischler, J.: Model evaluation strategy impacts the interpretation and performance of machine learning models, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-8479, https://doi.org/10.5194/egusphere-egu23-8479, 2023.

Predictive learning from data usually is formulated as a problem of finding the best connection between input and output spaces by optimizing well-defined cost or risk functions.

In geo-environmental studies input space is usually constructed from the geographical coordinates and features generated from different sources of available information (feature engineering), by applying expert knowledge, using deep learning technologies and taking into account the objectives of the study. Often, it is not known in advance if the input space is complete or contains redundant features. Therefore, unsupervised learning (UL) is essential in environmental data analysis, modelling, prediction and visualization. UL also helps better understand the data and phenomena they describe as well as in interpreting/communicating modelling strategies and the results in the decision-making process.

The main objective of the present investigation is to review some important topics in unsupervised learning from environmental data: 1) quantitative description of the input space (“monitoring network”) structure using global and local topological and fractal measures, 2) dimensionality reduction, 3) unsupervised feature selection and clustering by applying a variety of machine learning algorithms (kernel-based, ensemble learning, self-organizing maps) and visualization tools.

Major attention is paid to the simulated and real spatial data (pollution, permafrost, geomorphological and wind fields data).  Considered case studies have different input space dimensionality/topology and number of measurements. It is confirmed that UL should be considered an integral part of a generic methodology of environmental data analysis. Comprehensive comparisons and discussions of the results conclude the research.

How to cite: Kanevski, M.: On Unsupervised Learning from Environmental Data, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-9437, https://doi.org/10.5194/egusphere-egu23-9437, 2023.

Unsupervised classification techniques are becoming essential to extract information from the wealth of data that Earth observation satellites and other sensors currently provide. These datasets are inherently complex to analyze due to the extent across multiple dimensions - spatial, temporal, and often spectral or band dimension – their size, and the high resolution of current sensors. Traditional one-dimensional cluster analysis approaches, which are designed to find groups of similar elements in datasets such as rasters or time series, may come short of identifying patterns in these higher-dimensional datasets, often referred to as data cubes. In this context, we present our Clustering Geodata Cubes (CGC) software, an open-source Python package that implements a set of co- and tri-clustering algorithms to simultaneously group elements across two and three dimensions, respectively. The package includes different implementations to most efficiently tackle datasets that fit into the memory of a single machine as well as very large datasets that require cluster computing. A refining strategy to facilitate data pattern identification is also provided. We apply CGC to investigate gridded datasets representing the predicted day of the year when spring onset events (first leaf, first bloom) occur according to a well-established phenological model. Specifically, we consider spring indices computed at high spatial resolution (1 km) and continental scale (conterminous United States) for the last 40+ years and extract the main spatiotemporal patterns present in the data via CGC co-clustering functionality.  

How to cite: Nattino, F., Ku, O., Grootes, M. W., Izquierdo-Verdiguier, E., Girgin, S., and Zurita-Milla, R.: Clustering Geodata Cubes (CGC) and Its Application to Phenological Datasets, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-11601, https://doi.org/10.5194/egusphere-egu23-11601, 2023.

Industrial pollution remains a major challenge in spite of recent technological developments and purification procedures. To effectively monitor atmosphere contamination, data from air quality networks should be coupled with advanced spatiotemporal statistical methods.

Our previous studies showed that standard interpolation techniques (like inverse distance weighting, linear or spline interpolation, kernel-based Gaussian Process Regression, GPR) are quite limited for the simulation of a smoke-like narrow-directed industrial pollution in the vicinity of the source (a few tenths of kilometers). In this work, we try to apply GPR, based on statistically estimated covariances. These covariances are calculated using СALPUFF atmospheric pollution dispersion model for a one-year simulation in the Kryvyi Rih region. The application of GPR permits taking into account high correlations between pollution values in neighboring points revealed by modeling. The result of the GPR covariance-based technique is compared with other interpolation techniques. It can be used then in the estimation and optimization of air quality networks.

How to cite: Sokolov, A., Delbarre, H., Boldyriev, D., Bulana, T., Molodets, B., and Grabovets, D.: Industrial Atmospheric Pollution Estimation Using Gaussian Process Regression, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-12773, https://doi.org/10.5194/egusphere-egu23-12773, 2023.

Land ecosystems dampen the increase of atmospheric CO₂ by storing carbon in soils and vegetation. In order to estimate how long carbon stays in land ecosystems, a detailed knowledge about the distribution of carbon in different vegetation components is needed. Current Earth observation products provide estimates about total above-ground biomass but do not further separate between carbon stored in trees, understory vegetation, shrubs, grass, litter or woody debris. Here we present an approach in which we link several Earth observation products with a ground-based database to estimate biomass in various vegetation components. Therefore, we use information about the statistical distribution of biomass components provided by the North American Wildland Fuels Database (NAWFD), which are however not available as geocoded data. We use ESA CCI AGB version 3 data from 2010 as a proxy in order to link the NAWFD data to the spatial information from Earth observation products. The biomass and corresponding uncertainty from the ESA CCI AGB and a map of vegetation types are used to select the likely distribution of vegetation biomass components from the set of in-situ measurements of tree biomass. We then apply Isolation Forest outlier detection and bootstrapping for a robust comparison of both datasets and for uncertainty estimation. We use Random Forest and Gaussian Process regression to predict the biomass of trees, shrubs, snags, herbaceous vegetation, coarse and fine woody debris, duff and litter from ESA CCI AGB and land cover, GEDI canopy height, Sentinel-3 LAI and bioclimatic data. The regression models reach high predictive power and allow to also extrapolate to other regions. Our derived estimates of vegetation carbon stock components provide a more detailed view on the land carbon storage and contribute to an improved estimate of potential carbon emissions from respiration, disturbances and fires.

How to cite: Kinalczyk, D., Wessollek, C., and Forkel, M.: Estimating vegetation carbon stock components by linking ground databases with Earth observations, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-12933, https://doi.org/10.5194/egusphere-egu23-12933, 2023.

High resolution in weather and climate was always a common and ongoing goal of the community. In this regards, machine learning techniques accompanied numerical and statistical methods in recent years. Here we demonstrate that artificial intelligence can skilfully downscale low resolution climate model data when combined with numerical climate model data. We show that recently developed image inpainting technique perform accurate super-resolution via transfer learning using the HighResMIP of CMIP6 (Coupled Model Intercomparison Project Phase 6) experiments. Its huge data base offers a unique training opportunity for machine learning approaches. The transfer learning purpose allows also to downscale other CMIP6 experiments and models, as well as observational data like HadCRUT5. Combined with the technology of Kadow et al. 2020 of infilling missing climate data, we gain a neural network which reconstructs and downscales the important observational data set (IPCC AR6) at the same time. We further investigate the application of our method to downscale quantities predicted from a numerical ocean model (ICON-O) to improve computation times. In this process we focus on the ability of the model to predict eddies from low-resolution data.

An extension to:

Kadow, C., Hall, D.M. & Ulbrich, U. Artificial intelligence reconstructs missing climate information. Nature Geoscience 13, 408–413 (2020). https://doi.org/10.1038/s41561-020-0582-5

How to cite: Witte, M., Filippou, D., Plésiat, É., Meuer, J., Thiemann, H., Hall, D., Ludwig, T., and Kadow, C.: From Super-Resolution to Downscaling - An Image-Inpainting Deep Neural Network for High Resolution Weather and Climate Models, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-13196, https://doi.org/10.5194/egusphere-egu23-13196, 2023.

To assess the impact of anthropogenic and natural causes on land use and land use cover change, mapping of spatial and temporal changes is increasingly applied. Due to the availability of satellite image archives, remote sensing (RS) data-based machine learning models are in particular suitable for mapping and analysing land use and land cover changes. Most often, models trained with current RS data are employed to estimate past land cover and land use using available RS data with the assumption that the trained model predicts past data values similar to the accuracy of present data. However, machine learning models trained on RS data from particular locations and times may not be well transferred to new locations and time datasets due to various reasons. This study aims to assess the spatial-temporal transferability of the RS data models in the context of agricultural land use mapping. The study was designed to map agricultural land use (5 classes: maize, grasslands, summer crops, winter crops, and mixed crops) in two regions in Germany (North Hesse and Weser Ems) between the years 2010 and 2018 using Landsat archive data (i.e., Landsat 5, 7, and 8). Three model transferability scenarios were evaluated, a) temporal - S1, b) spatial - S2 and c) spatial-temporal - S3. Two machine learning models (random forest - RF and Convolution Neural Network - CNN) were trained. For each transferability scenario, class-level F1 and macro F1 values were compared between the reference and targeted transferability systems. Moreover, to explain the results of transferability scenarios, transferability results were further explored using dissimilarity index and area of applicability (AOA) concepts. The average macro F1 value of the trained model for the reference scenario (no transferability) was 0.75. For assessed transferability scenarios, the average macro F1 values were 0.70, 0.65 and 0.60, for S1, S2, and S3 respectively. It shows that, when predicting data from different spatial-temporal contexts, the model performance is decreasing. In contrast, the average proportion of the data inside the AOA did not show a clear pattern for different scenarios. In the context of RS data-related model building, spatial-temporal transferability is essential because of the limited availability of the labelled data. Thus, the results from this case study provide an understanding of how model performance changes when the model is transferred to new settings with data from different temporal and spatial domains.

How to cite: Wijesingha, J., Dzene, I., and Wachendorf, M.: Spatial-temporal transferability assessment of remote sensing data models for mapping agricultural land use, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-14716, https://doi.org/10.5194/egusphere-egu23-14716, 2023.

Machine learning algorithms have become popular tools for the analysis of spatial data. However, a number of studies have demonstrated that the application of machine learning algorithms in a spatial context has limitations. New geographic locations may lie outside of the data range for which the model was trained, and estimates of model performance may be too optimistic, when spatial autocorrelation of geographic data is not properly accounted for in cross-validation. We here use artificially created spatial data fields to conduct a series of experiments to further investigate the potential pitfalls of random forest regression applied to spatial data. We provide new insights on previously reported limitations and identify further limitations. We demonstrate that the same mechanism that leads to overoptimistic estimates of model performance (when based on ordinary random k-fold cross-validation) can also lead to a deterioration of model performance. When covariates contain sufficient information to deduce spatial coordinates, the model can reproduce any spatial pattern in the training data even if it is entirely or partly unrelated to the covariates. The presence of spatially correlated residuals in the training data changes how the model utilizes the information of the covariates and impedes the identification of the actual relationship between covariates and response. This reduces model performance when the model is applied to data with a different spatial structure. Under such conditions, machine learning methods that are sufficiently flexible to fit to autocorrelated residuals (such as random forest) may not be an optimal choice. Better models may be obtained using less flexible but more transparent approaches such as generalized linear models or additive models.

How to cite: Heinke, J., Müller, C., and Gerten, D.: Limitations of machine learning in a spatial context, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-16096, https://doi.org/10.5194/egusphere-egu23-16096, 2023.

Levee systems are designed to reduce the risk of water-related natural hazards (e.g., flooding) in areas behind levees. Most levees in the U.S. are designed to protect people and facilities against the impacts of the 100-year floods. However, the current climate change is increasing the probability of the occurrence of 500-year flood events that in turn increases the likelihood of economic loss, environmental damage, and fatality that disproportionately impacts communities of color and low-income groups facing socio-economic inequities in leveed areas. The increased frequency and intensity of flooding is putting extra pressure on emergency responders that often require diverse, multi-dimensional data originating from different sources to make sound decisions. Currently, the integration of these heterogeneous data acquired by diverse sensors and emergency agencies about environmental, hydrological, and demographic indicators requires costly and complex programming and analysis that hinder rapid disaster management efforts. Our domain ‘Levee System Ontology (LSO)’ resolves the data integration and software interoperability issues by semantically modeling the static aspects, dynamic processes, and information content of the levee systems by extending the well-structured, top-level Basic Formal Ontology (BFO) and mid-level Common Core Ontologies (CCO). LSO’s class and property names follow the terminology of the National Levee Database (NLD), allowing data scientists using NLD data to constrain their classifications based on the knowledge represented in LSO. In addition to modeling the information related to the characteristics and status of the structural components of the levee system, LSO represents the residual risk in leveed areas, economic and environmental losses, and damage to facilities in case of breaching and/or overtopping of levees. LSO enables reasoning to infer components and places along levees and floodwalls where the system requires inspection, maintenance, and repair based on the status of system components. The ontology also represents the impact of flood management activities on different groups of people from an environmental justice perspective, based on the principles of DEI (diversity, equity, inclusion) as defined by the U.N. Sustainable Development Goals.

How to cite: Davarpanah, A., Nguy Robertson, A. L., Lipscomb, M., McCord, J. w., and Morris, A.: Knowledge Representation of Levee Systems - an Environmental Justice Perspective, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-16768, https://doi.org/10.5194/egusphere-egu23-16768, 2023.

Large-scale climate variability is analysed, modelled, and predicted mainly based on general circulation models and low-dimensional association analysis. The models’ equational basis makes it difficult to produce mathematical analysis results and clear interpretations, whereas the association analysis cannot establish causation sufficiently to make invariant predictions. However, the macroscale causal structures of the climate system may accomplish the tasks of analysis, modelling, and prediction according to the concepts of causal emergence and causal prediction’s invariance.

Under the assumptions of no unobserved confounders and linear Gaussian models, we examine whether the macroscale causal structures of the climate system can be inferred not only to model but also to predict the large-scale climate variability. Specifically, first, we obtain the causal structures of the macroscale air-sea interactions of El Niño–Southern Oscillation (ENSO), which are interpretable in terms of physics. The structural causal models constructed accordingly can model the ENSO diversity realistically and predict the ENSO variability. Second, this study identifies the joint effect of ENSO and three other winter climate phenomena on the interannual variability in the East Asian summer monsoon. Using regression, these causal precursors can predict the monsoon one season ahead, outperforming association-based empirical models and several climate models. Third, we introduce a framework that infers ENSO’s air-sea interactions from high-dimensional data sets. The framework is based on aggregating the causal discovery results of bootstrap samples to improve high-dimensional variable selection. It is also based on spatial-dimension reduction to allow of clear interpretations at the macroscale.

While further integration with nonlinear non-Gaussian models will be necessary to establish the full benefits of inferring causal structures as a standard practice in research and operational predictions, our study may offer a route to providing concise explanations of the climate system and reaching accurate invariant predictions.

How to cite: He, S., Yang, S., and Chen, D.: Inferring Causal Structures to Model and Predict ENSO and Its Effect on Asian Summer Monsoon, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-102, https://doi.org/10.5194/egusphere-egu23-102, 2023.

The future evolution of tropical cyclones (TCs) in a warming world is an important issue, considering their potential socio-economic impacts on the areas hit by these phenomena. Previous studies provide robust responses about the future increase in intensity and in the global proportion of major TCs (Category 4–5). On the other hand, high uncertainty is associated to a projected future decrease in global TCs frequency and to potential changes in TC tracks and translation speed.

Risk management and regulatory actions require more robust quantification in how the climate change affects TCs dynamics.  A probabilistic hybrid TC model based upon statistical and climate models, physically coherent with TCs dynamics, is being built to investigate the potential impacts of climate change. Here, we provide preliminary results, in terms of present climate reconstruction (1980-2021) and future projections (2022-2060) of cyclogenesis locations and TC tracks, based on different statistical models, such as logistic and multiple linear regressions and random forest.  Physical predictors associated with the TC formation and motion and produced by reanalysis (ERA5) and the Community Earth System Model (CESM) ensemble are considered in this study.

How to cite: Ingrosso, R. and Boudreault, M.: Toward a hybrid tropical cyclone global model, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-239, https://doi.org/10.5194/egusphere-egu23-239, 2023.

Climate models are primary tools to investigate processes in the climate system, to project future changes, and to inform decision makers. The latest generation of models provides increasingly complex and realistic representations of the real climate system while there is also growing awareness that not all models produce equally plausible or independent simulations. Therefore, many recent studies have investigated how models differ from observed climate and how model dependence affects model output similarity, typically drawing on climatological averages over several decades.

Here, we show that temperature maps from individual days from climate models from the CMIP6 archive can be robustly identified as “observation” or “model” even after removing the global mean. An important exception is a prototype high-resolution simulation from the ICON model family that can not be so  unambiguously classified into one category. These results highlight that persistent differences between observed and simulated climate emerge at very short time scales already, but very high resolution modelling efforts may be able to overcome some of these shortcomings.

We use two different machine learning classifiers: (1) logistic regression, which allows easy insights into the learned coefficients but has the limitation of being a linear method and (2) a convolutional neural network (CNN) which represents rather the other end of the complexity spectrum, allowing to learn nonlinear spatial relations between features but lacking the easy interpretability logistic regression allows. For CMIP6 both methods perform comparably, while the CNN is also able to recognize about 75% of samples from ICON as coming from a model, while linear regression does not have any skill for this case.

Overall, we demonstrate that the use of machine learning classifiers, once trained, can overcome the need for multiple decades of data to investigate a given model. This opens up novel avenues to test model performance on much shorter times scales.

How to cite: Brunner, L., Sippel, S., and Voigt, A.: Separation of climate models and observations based on daily output using two machine learning classifiers, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-492, https://doi.org/10.5194/egusphere-egu23-492, 2023.

In climate research we often want to focus on a specific region and the most prominent processes affecting it, but how exactly do we select the borders of that region? We also often need to use long-term in situ observations to represent a larger area, but which area exactly are they representative for? In ocean sciences we usually consider basins as separate regions or even simpler, just select a rectangle of the ocean, but that does not always correspond to the real, physically relevant borders. As alternative, we use an unsupervised classification model, Gaussian Mixture Model (GMM), to separate the northwestern European seas into regions based on the sea level variability observed by altimetry satellites.

After performing a principal component (PC) analysis on the 24 years of monthly sea level data, we use the stacked PC maps as input for the GMM. We used the Bayesian Information Criterion to determine into how many regions our area should be split because GMM requires the number of classes to be selected a priori. Depending on the number of PCs used, the optimal number of classes was between 12 and 18, more PCs typically allowing the separation into more regions. Due to the complexity of the data and the dependence of the results on the starting randomly chosen weights, the classification can differ to a degree with every new run of the model, even if we use the exact same data and parameters. To tackle that, instead of using one model, we use an ensemble of models and then determine which class does each grid point belong to by soft voting, i.e., each of the models provides a probability that the point belongs to a particular class and the class with the maximal sum of probabilities wins. As a result, we obtain both the classification and the likelihood of the model belonging to that class.

Despite not using the coordinates of the data points in the model at all, the obtained classes are clearly location dependent, with grid points belonging to the same class always being close to each other. While many classes are defined by bathymetry changes, e.g., the continental shelf break and slope, sometimes other factors come into play, such as for the split of the Norwegian coast into two classes or for the division in the Barents Sea, which is probably based on the circulation. The North Sea is also split into three distinct regions, possibly based on sea level changes caused by dominant wind patterns.

This method can be applied to almost any atmospheric or oceanic variable and used for larger or smaller areas. It is quick and practical, allowing us to delimit the area based on the information we cannot always clearly see from the data, which can facilitate better selection of the regions that need further research.

How to cite: Poropat, L., Heuzé, C., and Reese, H.: Finding regions of similar sea level variability with the help of a Gaussian Mixture Model, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-753, https://doi.org/10.5194/egusphere-egu23-753, 2023.

Understanding the forcing of regional sea level variability is crucial as many people all over the world live along the coasts and are endangered by the sea level rise. The adding of fresh water into the oceans due to melting of the Earth’s land ice together with thermosteric changes has led to a rise of the global mean sea level (GMSL) with an accelerating rate during the twentieth century, and has now reached a mean rate of 3.7 mm per year according to IPCCs latest report. However, this change varies spatially and the dynamics behind what forces sea level variability on a regional to local scale is still less known, thus making it hard for decision makers to mitigate and adapt with appropriate strategies.

Here we present a novel approach using machine learning (ML) to identify the dynamics and determine the most prominent drivers forcing coastal sea level variability. We use a recurrent neural network called Long Short-Term Memory (LSTM) with the advantage of learning data in sequences and thus capable of storing some memory from previous timesteps, which is beneficial when dealing with time series. To train the model we use hourly ERA5 10-m wind, mean sea level pressure (MSLP), sea surface temperature (SST), evaporation and  precipitation data between 2009-2017 in the North Sea region. To reduce the dimensionality of the data but still preserve maximal information we conduct principal component analysis (PCA) after removing the climatology which are calculated by hourly means over the years. Depending on the explained variance of the PCs for each driver, 2-4 PCs are chosen and cross-correlated to eliminate collinearity, which could affect the model results. Before being used in the ML model the final preprocessed data are normalized by min-max scaling to optimize the learning. The target data in the model are hourly in-situ sea level observations from West-Terschelling in the Netherlands. Using in-situ observations compared to altimeter data enhances the ability of making good predictions in coastal zones as altimeter data has a tendency to degrade along the coasts. The sea level time series is preprocessed by tidal removal and de-seasoned by subtracting the hourly means. To determine which drivers are most prominent for the sea surface variability in our location, we mute one driver at a time in the training of the network and evaluate the eventual improvement or deterioration of the predictions.

Our results show that the zonal wind is the most prominent forcing of sea level variability in our location, followed by meridional wind and MSLP. While the SST greatly affects the GMSL, SST seems to have little to no effect on local sea level variability compared to other drivers. This approach shows great potential and can easily be applied to any coastal zone and is thus very useful for a broad body of decision makers all over the world. Identifying the cause of local sea level variability will also enable the ability of producing better models for future predictions, which is of great importance and interest.

How to cite: Carlstedt, L., Poropat, L., and Heuzé, C.: Drivers of sea level variability using neural networks, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-849, https://doi.org/10.5194/egusphere-egu23-849, 2023.

How to cite: Landt-Hayen, M., Rath, W., Wahl, S., Niebaum, N., Claus, M., and Kröger, P.: Data-driven Attributing of Climate Events with Climate Index Collection based on Model Data (CICMoD), EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-984, https://doi.org/10.5194/egusphere-egu23-984, 2023.

How to cite: Besel, V., Todorović, M., Kurtén, T., Rinke, P., and Vehkamäki, H.: Curation of High-level Molecular Atmospheric Data for Machine Learning Purposes, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-1135, https://doi.org/10.5194/egusphere-egu23-1135, 2023.

There is growing use of machine learning algorithms to replicate sub-grid parametrisation schemes in global climate models.  Parametrisations rely on approximations, thus there is potential for machine learning to aid improvements.  In this study, a neural network is used to mimic the behaviour of the non-orographic gravity wave scheme used in the Met Office climate model, important for stratospheric climate and variability.  The neural network is found to require only two of the six inputs used by the parametrisation scheme, suggesting the potential for greater efficiency in this scheme.  Use of a one-dimensional mechanistic model is advocated, allowing neural network hyperparameters to be trained based on emergent features of the coupled system with minimal computational cost, and providing a test bed prior to coupling to a climate model.  A climate model simulation, using the neural network in place of the existing parametrisation scheme, is found to accurately generate a quasi-biennial oscillation of the tropical stratospheric winds, and correctly simulate the non-orographic gravity wave variability associated with the El Nino Southern Oscillation and stratospheric polar vortex variability.  These internal sources of variability are essential for providing seasonal forecast skill, and the gravity wave forcing associated with them is reproduced without explicit training for these patterns.

How to cite: Hardiman, S., Scaife, A., van Niekerk, A., Prudden, R., Owen, A., Adams, S., Dunstan, T., Dunstone, N., and Seabrook, M.: Machine learning for non-orographic gravity waves in a climate model, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-1244, https://doi.org/10.5194/egusphere-egu23-1244, 2023.

Transfer learning is a technique wherein information learned by previously trained models is applied to new learning tasks. Typically, weights learned by a network pretrained on other datasets are copied or transferred to new networks. These new networks, or downstream models, are then are then used for assorted tasks. Foundation models extend this concept by training models on large datasets. Such models gain a contextual understanding which can then be used to improve performance of downstream tasks in different domains. Common examples include GPT-3 in the field on natural language processing and ImageNet trained models in the field of computer vision.

Beyond its high rate of data collection, satellite data also has a wide range of meaningful applications including climate impact modelling and sustainable energy. This makes foundation models trained on satellite data very beneficial as they would reduce the time, data, and computational resources required to obtain useful downstream models for these applications.

However, satellite data models differ from typical computer vision models in a crucial way. Because several types of satellite data exist, each with its own benefits, a typical use case for satellite data involves combining multiple data inputs in configurations that are not readily apparent during pretraining of the foundation model. Essentially, this means that the downstream application may have a different number of input channels from the pretrained model, which raises the question of how to successfully transfer information learned by the pretrained model to the downstream application.

This research proposes and examines several architectures for the downstream model that allow for pretrained weights to be incorporated when a different number of input channels is required. For evaluation, models pretrained with self-supervised learning on precipitation data are applied to a downstream model which conducts temporal interpolation of precipitation data and requires two inputs. The effect of including perceptual loss to enhance model performance is also evaluated. These findings can be used to guide adaptation for applications ranging from flood modeling, land use detection, and more.

How to cite: Simumba, N. and Tatsubori, M.: Adapting Transfer Learning for Multiple Channels in Satellite Data Applications, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-1502, https://doi.org/10.5194/egusphere-egu23-1502, 2023.

Unprecedented flash floods (FF) in urban regions are increasing due to heavy rainfall intensity and magnitude as a result of human-induced climate and land-use changes. The changes in weather patterns and various anthropogenic activities increase the complexity of modelling the FF at different spatiotemporal scales: which indicates the importance of multi-resolution forcing information. Towards this, developing new methods for processing coarser resolution spatio-temporal datasets are essential for the efficient modelling of FF. While a wide range of methods is available for spatial and temporal downscaling of the climate data, the multi-temporal downscaling strategy has not been investigated for ungauged stations of streamflow. The current study proposed a multi-temporal downscaling (MTD) methodology for gauged and ungauged stations using Adaptive Emulator Modelling concepts for daily to sub-daily streamflows. The proposed MTD framework for ungauged stations comprise a hybrid framework with conceptual and machine learning-based approaches to analyze the catchment behavior and downscale the model outputs from daily to sub-daily scales. The study area, Peachtree Creek watershed (USA), frequently experiences flash floods; hence, selected to validate the proposed framework. Further, the study addresses the critical issues of model development, seasonality, and diurnal variation of MTD data. The study obtained MTD data with minimal uncertainty on capturing the hydrological signatures and nearly 95% of accuracy in predicting the flow attributes over ungauged stations. The proposed framework can be highly useful for short- and long-range planning, management, and mitigation measurements, where the absence of fine resolution data prohibits flash flood modeling.

How to cite: Budamala, V., Wadhwa, A., and Bhowmik, R. D.: Multi-Temporal Downscaling of Streamflow for Ungauged Stations/ Sub-Basins from Daily to Sub-Daily Interval Using Hybrid Framework – A Case Study on Flash Flood Watershed, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-1855, https://doi.org/10.5194/egusphere-egu23-1855, 2023.

Aerosols in the climate system have a direct link to the Earth’s energy balance. Aerosols interact directly with the solar radiation through scattering and absorption; and indirectly by changing cloud properties. The effect aerosols have on climate is one of the major causes of radiative forcing (RF) uncertainty in global climate model simulations. Thus, reducing aerosol RF uncertainty is key to improving climate prediction. The objective of this work is to understand the magnitude and causes of aerosol uncertainty in the chemical transport model TM5.

Perturbed Parameter Ensembles (PPEs) are a set of model runs created by perturbing an ensemble of parameters. Parameters are model inputs, for this study we focus on parameters describing aerosol emissions, properties and processes, such as dry deposition, aging rate, emissions to aerosols microphysics. PPEs vary theses parameters over their uncertainty range all at once to study their combine effect on TM5.

Varying these parameters along with others through their value range, will reflect on TM5 outputs. The TM5 outputs parameters we are using in our sensitivity study are the cloud droplet number concentration and the ambient aerosol absorption optical thickness at 550nm.

Here we discuss the design of the PPE, and one-at-a-time sensitivity studies used in this process. The PPE samples the parameter space to enable us to use emulation. Emulating is a machine learning technique that uses a statistical surrogate model to replace the chemical transport model. The aim is to provide output data with more dense sampling throughout the parameter space. We will be using a Gaussian process emulator, which has been shown to be an efficient technique for quantifying parameter sensitivity in complex global atmospheric models.

We also describe plans to extend this work to emulate an aerosol PPE for EC-Earth. The PPE for EC-Earth will also contain cloud parameters that will vary over their uncertainty range together with the aerosol parameters to examine the influence of aerosol parametric uncertainty on RF.

How to cite: Bouchahmoud, M., Bergman, T., and Williamson, C.: Towards understanding the effect of parametric aerosol uncertainty on climate using a chemical transport model perturbed parameter ensemble., EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-2289, https://doi.org/10.5194/egusphere-egu23-2289, 2023.

Global climate models use parameterizations to represent the effect of subgrid scale processes on the resolved state. Parameterizations in the atmosphere component usually include radiation, convection, cloud microphysics, cloud cover, gravity wave drag, vertical turbulence in the boundary layer and other processes. Parameterizations are semi-empirical functions that include a number of tunable parameters. Because these parameters are loosely constraint with experimental data, a range of values are typically explored by evaluating model runs against observations and/or high resolution runs. Fine tuning a climate model is a complex inverse problem due to the number of tunable parameters and observed climate properties to fit. Moreover, parameterizations are sources of uncertainties for climate projections, thus fine tuning is a crucial step in model development.

Traditionally, tuning is a time-consuming task done manually, by iteratively updating the values of the parameters in order to investigate the parameter space with user-experience driven choices. To overcome such limitation and search efficiently through the parameter space one can implement automatic techniques. Typical steps in automatic tuning are: (i) constraining the scope of the study (model, simulation setup, parameters, metrics to fit and corresponding reference values); (ii) conducting a sensitivity analysis to reduce the parameter space and/or building an emulator for the climate model; and (iii) conducting a sophisticated grid search to define the optimum parameter set or its distribution (e.g., rejection sampling and history matching). The ICOsahedral Non-hydrostatic (ICON) model is a modelling framework for numerical weather prediction and climate projections. We implement a ML-based automatic tuning technic to tune a recent version of ICON-A with a spatial resolution typically used for climate projections. We evaluate the tuned ICON-A model against satellite observations using the Earth System Model Evaluation Tool (ESMValTool). Although automatic tuning technics allow to reach the optimum parameter values in less steps than with the manual tuning, they still require some experience-driven choices throughout the tuning process. Moreover, the performances of the tuned model is limited by the structural errors of the model, inherent to the mathematical description of the parameterizations included in the model.

How to cite: Bonnet, P., Iglesias-Suarez, F., Gentine, P., Giorgetta, M., and Eyring, V.: Machine learning based automated parameter tuning of ICON-A using satellite data, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-2541, https://doi.org/10.5194/egusphere-egu23-2541, 2023.

Clouds and their interdependent feedback mechanisms remain a source of insecurity in climate science. This said, overcoming relating obstacles especially in the context of a changing climate emphasizes the need for a reliable database today more than ever. While passive remote sensing sensors provide continuous observations of the cloud top, they lack vital information on subjacent levels. Here, active instruments can deliver valuable insights to fill this gap in knowledge.

This study sets on to combine the benefits of both instrument types. It aims (1) to reconstruct the vertical distribution of volumetric radar data along the cloud column and (2) to interpolate the resultant 3D cloud structure to the satellite’s full disk by applying a contemporary Deep-Learning approach. Input data was derived by an automated spatio-temporally matching between high-resoluted satellite channels and the overflight of the radar. These samples display the physical predictors that were fed into the network to reconstruct the cloud vertical distribution on each of the radar’s height levels along the whole domain. Data from the entire year 2017 was used to integrate seasonal variations into the modeling routine.

The results demonstrate not only the network’s ability to reconstruct the cloud column along the radar track but also to interpolate coherent structures into a large-scale perspective. While the model performs equally well over land and water bodies, its applicable time frame is limited to daytime predictions only. Finally, the generated data can be leveraged to build a comprehensive database of 3D cloud structures that is to be exploited in proceeding applications.

How to cite: Brüning, S., Niebler, S., and Tost, H.: Deep learning-based generation of 3D cloud structures from geostationary satellite data, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-3404, https://doi.org/10.5194/egusphere-egu23-3404, 2023.

Modelling the transfer of radiation through the atmosphere is a key component of weather and climate models. The operational radiation scheme in the Icosahedral Nonhydrostatic Weather and Climate Model (ICON) is ecRad. The radiation scheme ecRad is accurate but computationally expensive. It is operationally run in ICON on a grid coarser than the dynamical grid and the time step interval between two calls is significantly larger. This is known to reduce the quality of the climate prediction. A possible approach to accelerate the computation of the radiation fluxes is to use machine learning methods. Machine learning methods can significantly speed up computation of radiation, but they may cause climate drifts if they do not respect essential physical laws. In this work, we study random forest and neural network emulations of ecRad. We study different strategies to compare the stability of the emulations. Concerning the neural network, we compare loss functions with an additional energy penalty term and we observe that modifying the loss function is essential to predict accurately the heating rates. The random forest emulator, which is significantly faster to train than the neural network is used as a reference model that the neural network must outperform. The random forest emulator can become extremely accurate but the memory requirement quickly become prohibitive. Various numerical experiments are performed to illustrate the properties of the machine learning emulators.

How to cite: Bertoli, G., Schemm, S., Ozdemir, F., Perez Cruz, F., and Szekely, E.: Building a physics-constrained, fast and stable machine learning-based radiation emulator, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-3418, https://doi.org/10.5194/egusphere-egu23-3418, 2023.

How to cite: Swaminathan, R., Quaife, T., and Allan, R.: Evaluating Vegetation Modelling in Earth System Models with Machine Learning Approaches, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-3457, https://doi.org/10.5194/egusphere-egu23-3457, 2023.

Tropical cyclones are high-impact weather events which have large human and economic effects, so it is important to be able to understand how their location, frequency and structure might change in a future climate.

Here, a lightweight deep learning model is presented which is intended for detecting the presence of tropical cyclones during the execution of numerical simulations for use in an online data reduction method. This will help to avoid saving vast amounts of data for analysis after the simulation is complete. With run-time detection, it might be possible to reduce the need for some of the high-frequency high-resolution output which would otherwise be required.

The model was trained on ERA-Interim reanalysis data from 1979 to 2017 and the training concentrated on delivering the highest possible recall rate (successful detection of cyclones) while rejecting enough data to make a difference in outputs.

When tested using data from the two subsequent years, the recall or probability of detection rate was 92%. The precision rate or success ratio obtained was that of 36%. For the desired data reduction application, if the desired target included all tropical cyclone events, even those which did not obtain hurricane-strength status, the effective precision was 85%.

The recall rate and the Area Under Curve for the Precision/Recall (AUC-PR) compare favourably with other methods of cyclone identification while using the smallest number of parameters for both training and inference. 

Work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344. LLNL-ABS-843612

How to cite: Galea, D., Kunkel, J., and Lawrence, B.: TCDetect: A new method of Detecting the Presence of Tropical Cyclones using Deep Learning, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-3619, https://doi.org/10.5194/egusphere-egu23-3619, 2023.

The Southern Ocean acts as one of Earth's major carbon sinks, taking up anthropogenic carbon from the atmosphere. Earth System Models (ESMs) are used to project its future evolution. However, the ESMs in the Coupled Model Intercomparison Project version 6 (CMIP6) disagree on the biogeochemical representation of the Southern Ocean carbon cycle, both with respect to the phasing and the magnitude of the seasonal cycle of dissolved inorganic carbon (DIC), and they compare poorly with observations.

We develop a framework to investigate model biases in 10 CMIP6 ESMs historical runs incorporating explainable artificial intelligence (xAI) methodologies. Using both a linear Random Forest feature relevance approach to a nonlinear self organizing map - feed forward neural network (SOM-FFN) framework, we relate 5 drivers of the seasonal cycle of DIC in the Southern Ocean in the different CMIP6 models. We investigate temperature, salinity, silicate, nitrate and dissolved oxygen as potential drivers. This analysis allows us to determine dominant statistical drivers of the seasonal cycle of DIC in the different models, and how they compare to the observations. Our findings inform future model development to better constrain the seasonal cycle of DIC.

How to cite: Heubel, P., Keppler, L., and Iliyna, T.: Explainable AI for oceanic carbon cycle analysis of CMIP6, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-3875, https://doi.org/10.5194/egusphere-egu23-3875, 2023.

Motivation. Ice melting in Greenland and Antarctica has increasingly contributed to rising sea levels. Yet, the exact speed of melting, existence of abrupt tipping points, and in-detail links to climate change remain uncertain. Ice shelves essentially prevent the ice sheet from slipping into the ocean and better prediction of collapses is needed. Meltwater at the surface of ice shelves indicates ice shelf collapse through destabilizing ice shelves via fracturing and flexural processes (Banwell et al., 2013) and is likely impacted by a warming climate ( Kingslake et al., 2017). Maps of meltwater have been created from in-situ and remote observations, but their low and irregular spatiotemporal resolution severely limits studies (Kingslake et al., 2019).

Research Gap. In particular, there does not exist daily high-resolution (< 500m) maps of surface meltwater. We propose the first daily high-resolution surface meltwater maps by developing a deep learning-based downscaling method, called DailyMelt, that fuses observations and simulations of varying spatiotemporal resolution, as illustrated in Fig.1. The created maps will improve understanding of the origin, transport, and controlling physical processes of surface meltwater. Moreover, they will act as unified source to improve sea level rise and meltwater predictions in climate models. 

Data. To synthesize surface meltwater maps, we leverage observations from satellites (MODIS, Sen-1 SAR) which are high-resolution (500m, 10m), but have substantial temporal gaps due to repeat time and cloud coverage. We fuse them with simulations (MAR) and passive microwave observations (MEaSURE) that are daily, but low-resolution (6km, 3.125km). In a significant remote sensing effort, we have downloaded, reprojected, and regridded all products into daily observations for our study area over Greenland’s Helheim glacier. 

Approach and expected results. Within deep generative vision models, diffusion-based models promise sharp and probabilistic predictions. We have implemented SRDiff (Li H. et al., 2022) and tested it on spatially downscaling external data. As a baseline model, we have implemented a statistical downscaling model that is a local hybrid physics-linear regression model (Noel et al., 2016). In our planned benchmark, we expect a baseline UNet architecture that minimizes RMSE to create blurry maps and a generative adversarial network that minimizes adversarial loss to create sharp but deterministic maps. We have started with spatial downscaling and will include temporal downscaling. 

In summary, we will create the first daily high-resolution (500m) surface meltwater maps, have introduced the first diffusion-based model for downscaling Earth sciences data, and have created the first benchmark dataset for downscaling surface meltwater maps.

References.

Banwell, A. F., et al. (2013), Breakup of the Larsen B Ice Shelf triggered by chain reaction drainage of supraglacial lakes, Geophys. Res. Lett., 40 

Kingslake J, et al. (2017), Widespread movement of meltwater onto and across Antarctic ice shelves, Nature, 544(7650)

Kingslake J., et al. (2019), Antarctic Surface Hydrology and Ice Shelf Stability Workshop report, US Antarctic Program Data Center

Li H., et al. (2022), SRDiff: Single image super-resolution with diffusion probabilistic models, Neurocomputing, 479

Noël, B., et al. (2016), A daily, 1 km resolution data set of downscaled Greenland ice sheet surface mass balance (1958–2015), The Cryosphere, 10

How to cite: Lütjens, B., Alexander, P., Antwerpen, R., Cervone, G., Kearney, M., Luo, B., Newman, D., and Tedesco, M.: DailyMelt: Diffusion-based Models for Spatiotemporal Downscaling of (Ant-)arctic Surface Meltwater Maps, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-4044, https://doi.org/10.5194/egusphere-egu23-4044, 2023.

The availability of reliable, high-resolution climate and weather data is important to inform long-term decisions on climate adaptation and mitigation and to guide rapid responses to extreme events. Forecasting models are limited by computational costs and, therefore, often generate coarse-resolution predictions. Statistical downscaling can provide an efficient method of upsampling low-resolution data. In this field, deep learning has been applied successfully, often using image super-resolution methods from computer vision. However, despite achieving visually compelling results in some cases, such models frequently violate conservation laws when predicting physical variables. In order to conserve physical quantities, we develop methods that guarantee physical constraints are satisfied by a deep learning downscaling model while also improving their performance according to traditional metrics. We compare different constraining approaches and demonstrate their applicability across different neural architectures as well as a variety of climate and weather data sets, including ERA5 and WRF data sets.

How to cite: Harder, P., Ramesh, V., Hernandez-Garcia, A., Yang, Q., Sattigeri, P., Szwarcman, D., Watson, C., and Rolnick, D.: Physics-Constrained Deep Learning for Downscaling, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-4350, https://doi.org/10.5194/egusphere-egu23-4350, 2023.

Photosynthesis model parameters represent vegetation properties or sensitivities of photosynthesis processes. As one of the model uncertainty sources, parameters affect the accuracy and generalizability of the model. Ideally, parameters of ecosystem-level photosynthesis models, i.e., gross primary productivity (GPP) models, can be measured or inversed from observations at the local scale. To extrapolate parameters to a larger spatial scale, current photosynthesis models typically adopted fixed values or plant-functional-type(PFT)-specific values. However, the fixed and PFT-based parameterization approaches cannot capture sufficiently the spatial variability of parameters and lead to significant estimation errors. Here, we propose a Simultaneous Parameter Inversion and Extrapolation approach (SPIE) to overcome these issues. 

SPIE refers to predicting model parameters using an artificial neural network (NN) constrained by both model loss and ecosystem features including PFT, climate types, bioclimatic variables, vegetation features, atmospheric nitrogen and phosphorus deposition and soil properties. Taking a light use efficiency (LUE) model as an example, we evaluated SPIE at 196 FLUXNET eddy covariance flux sites. The LUE model accounts for the effects of air temperature, vapor pressure deficit, soil water availability (SW), light saturation, diffuse radiation fraction and CO₂ on GPP using five independent sensitivity functions. The SW was represented using the water availability index and can be optimized based on evapotranspiration. Thus, we optimized the NN by minimizing the model loss which consists of GPP errors, evapotranspiration errors, and constraints on sensitivity functions. Furthermore, we compared SPIE with 11 typical parameter extrapolating approaches, including PFT- and climate-specific parameterizations, global and PFT-based parameter optimization, site-similarity, and regression methods using Nash-Sutcliffe model efficiency (NSE), determination coefficient (R²) and normalized root mean squared error (NRMSE).

The results in ten-fold cross-validation showed that SPIE had the best performance across various temporal and spatial scales and across assessing metrics. None of the parameter extrapolating approaches reached the same performance as the on-site calibrated parameters (NSE=0.95), but SPIE was the only approach showing positive NSE (=0.68) in cross-validation across sites. Moreover, the site-level NSE, R², and NRMSE of SPIE all significantly outperformed per biome and per climate type. Ranges of parameters were more constrained by SPIE than site calibrations.

Overall, SPIE is a robust parameter extrapolation approach that overcomes strong limitations observed in many of the standard model parameterization approaches. Our approach suggests that model parameterizations can be determined from observations of vegetation, climate and soil properties, and expands from customary clustering methods (e.g., PFT-specific parameterization). We argue that expanding SPIE to other models overcomes current limits in parameterization and serves as an entry point to investigate the robustness and generalization of different models.

How to cite: Bao, S., Carvalhais, N., Alonso, L., Wang, S., Gensheimer, J., De, R., and Shi, J.: Towards Robust Parameterizations in Ecosystem-level Photosynthesis Models, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-5431, https://doi.org/10.5194/egusphere-egu23-5431, 2023.

Remote sensing has played a fundamental role for land cover mapping and change detection at least since the launch of the Landsat satellite program in 1972. In 1995, the Central Intelligence Agency of the United States of America released previously classified spy imagery taken from 1960 onwards with near-global coverage from the Keyhole programme, which includes the CORONA satellite mission. CORONA imagery is a treasure because it contains information about land cover 10 years before the beginning of the civilian Earth observation and has a high spatial resolution < 2m. However, this imagery is only pan-chromatic and usually not georeferenced, which has so far prevented a large-scale application for land cover mapping or other geophysical and environmental applications.

Here, we aim to harvest the valuable information about past land cover from CORONA imagery for a state-wide mapping of past land cover changes between 1965 and 1978 by training, testing and validating various deep learning models.

To the best of our knowledge, this is the first work to analyse land cover from CORONA data on a large scale, dividing land cover into six classes based on the CORINE classification scheme. The particular focus of the work is to test the transferability of the deep learning approaches to unknown CORONA data.

To investigate the transferability, we selected 27 spatially and temporally distributed study areas (each 23 km²) in the Free State of Saxony (Germany) and created semantic masks to train and test 10 different U-shaped neuronal network architectures to extract land cover from CORONA data. As input, we use either the original panchromatic pixel values and different texture measures. From these input data, ten different training datasets and test datasets were derived for cross-validation.

The training results show that a semantic segmentation of land cover from CORONA data with the used architectures is possible. Strong differences in model performance (based on cross validation and the intersection over union metric, IOU) were detected among the classes. Classes with many sample data achieve significantly better IOU values than underrepresented classes. In general, a U-shaped architecture with a Transformer as Encoder (Transformer U-Net) achieved the best results. The best segmentation performance (IOU 83.29%), was obtained for forests, followed by agriculture (74.21%). For artificial surfaces, a mean IOU of 68.83% was achieved. Water surfaces achieved a mean IOU of 66.49%. For the shrub vegetation and open areas classes only IOU values mostly below 25% were achieved. The deep learning models were successfully transferable in space (between test areas) and time (between CORONA imagery from different years) especially for classes with many sample data. The transferability of deep learning models was difficult for the mapping of water bodies. Despite the general good model performance and successful transferability for most classes, the transferability was limited especially for imagery of very poor quality. Our approach enabled the state-wide mapping of land cover in Saxony between 1965 and 1978 with a spatial resolution of 2 m. We identify an increase in urban cover and a decrease in cropland cover

How to cite: Kugler, L., Marrs, C., Kosczor, E., and Forkel, M.: Harvesting historical spy imagery by evaluating deep learning models for state-wide mapping of land cover changes between 1965-1978, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-5487, https://doi.org/10.5194/egusphere-egu23-5487, 2023.

Accurately interpreting past climate variability can be a challenging task, particularly when it comes to distinguishing between forced and unforced changes. In the  case of large volcanic eruptions, ice core records are a very valuable tool but still often not sufficient to link reconstructed anomaly patterns to a volcanic eruption at all or to its geographical location. In this study, we developed a convolutional neural network (CNN) that is able to classify whether a volcanic eruption occurred and its location (northern hemisphere extratropical, southern hemisphere extratropical, or tropics) with an accuracy of 92%.

To train the CNN, we used 100 member ensembles of the MPI-ESM-LR global climate model, generated using the easy volcanic aerosol (EVA) model, which provides the radiative forcing of idealized volcanic eruptions of different strengths and locations. The model considered global sea surface temperature and precipitation patterns 12 months after the eruption over a time period of 3 months.

In addition to demonstrating the high accuracy of the CNN, we also applied layer-wise relevance propagation (LRP) to the model to understand its decision-making process and identify the input data that influenced its predictions. Our study demonstrates the potential of using CNNs and interpretability techniques for identifying and locating past volcanic eruptions as well as improving the accuracy and understanding of volcanic climate signals.

How to cite: Meuer, J., Timmreck, C., Fang, S.-W., and Kadow, C.: Identifying and Locating Volcanic Eruptions using Convolutional Neural Networks and Interpretability Techniques, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-5583, https://doi.org/10.5194/egusphere-egu23-5583, 2023.

Machine Learning (ML) and AI techniques, especially methods based on Deep Learning, have long been considered as black boxes that might be good at predicting, but not explaining predictions. This has changed recently, with more techniques becoming available that explain predictions by ML models – known as Explainable AI (XAI). These have seen adaptation also in climate science, because they could have the potential to help us in understanding the physics behind phenomena in geoscience. It is, however, unclear, how large that potential really is, and how these methods can be incorporated into the scientific process. In our study, we use the exemplary research question of which aspects of the large-scale atmospheric circulation affect specific local conditions. We compare the different answers to this question obtained with a range of different methods, from the traditional approach of targeted data analysis based on physical knowledge (such as using dimensionality reduction based on physical reasoning) to purely data-driven and physics-unaware methods using Deep Learning with XAI techniques. Based on these insights, we discuss the usefulness and potential pitfalls of XAI for understanding and explaining phenomena in geosciences. 

How to cite: Scher, S., Trügler, A., and Abermann, J.: Potentials and challenges of using Explainable AI for understanding atmospheric circulation, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-5967, https://doi.org/10.5194/egusphere-egu23-5967, 2023.

Air pollution contributes to millions of deaths worldwide every year. The concentration of a particular air pollutant, such as ozone, is controlled by physical and chemical processes which act on varying temporal and spatial scales. Quantifying the strength of causal drivers (e.g. temperature) on air pollution from observational data, particularly at extrema, is challenging due to the difficulty of disentangling correlation and causation, as many drivers are correlated. Furthermore, because air pollution is controlled in part by large scale atmospheric phenomena, using local (e.g. individual grid cell level) covariates for analysis is insufficient to fully capture the effect of these phenomena on air pollution. 

Access to large spatiotemporal datasets of air pollutant concentrations and atmospheric variables, coupled with recent advances in self-supervised learning, allow us to learn reduced representations of spatiotemporal atmospheric fields, and therefore account for non-local and non-instantaneous processes in downstream tasks.

We show that these learned reduced representations can be useful for tasks such as air pollution forecasting, and crucially to quantify the causal effect of varying atmospheric fields on air pollution. We make use of recent advances in bounding causal effects in the presence of unobserved confounding to estimate, with uncertainty, the causal effect of changing atmospheric fields on air pollution. Finally, we compare our quantification of the causal drivers of air pollution to results from other approaches, and explore implications for our methods and for the wider goal of improving the process-level treatment of air pollutants in chemistry-climate models.

How to cite: Hickman, S., Griffiths, P., Nowack, P., and Archibald, A.: Using reduced representations of atmospheric fields to quantify the causal drivers of air pollution, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-6061, https://doi.org/10.5194/egusphere-egu23-6061, 2023.

A promising approach to improve cloud parameterizations within climate models, and thus climate projections, is to train machine learning algorithms on storm-resolving model (SRM) output. The ICOsahedral Non-hydrostatic (ICON) modeling framework permits simulations ranging from numerical weather prediction to climate projections, making it an ideal target to develop data-driven parameterizations for sub-grid scale processes. Here, we systematically derive and evaluate the first data-driven cloud cover parameterizations with coarse-grained data based on ICON SRM simulations. These parameterizations range from simple analytic models and symbolic regression fits to neural networks (NNs), populating a performance x complexity plane. In most models, we enforce sparsity and discourage correlated features by sequentially selecting features based on the models' performance gains. Guided by a set of physical constraints, we use symbolic regression to find a novel equation to parameterize cloud cover. The equation represents a good compromise between performance and complexity, achieving the highest performance (R^2>0.9) for its complexity (13 trainable parameters). To model sub-grid scale cloud cover in its full complexity, we also develop three different types of NNs that differ in the degree of vertical locality they assume for diagnosing cloud cover from coarse-grained atmospheric state variables. Using the game-theory based interpretability library SHapley Additive exPlanations, we analyze our most non-local NN and identify an overemphasis on specific humidity and cloud ice as the reason why it cannot perfectly generalize from the global to the regional coarse-grained SRM data. The interpretability tool also helps visualize similarities and differences in feature importance between regionally and globally trained NNs, and reveals a local relationship between their cloud cover predictions and the thermodynamic environment. Our results show the potential of deep learning and symbolic regression to derive accurate yet interpretable cloud cover parameterizations from SRMs.

How to cite: Grundner, A., Beucler, T., Gentine, P., Giorgetta, M. A., Iglesias-Suarez, F., and Eyring, V.: Data-Driven Cloud Cover Parameterizations, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-6306, https://doi.org/10.5194/egusphere-egu23-6306, 2023.

Earth system models are fundamental to understanding and projecting climate change, although there are considerable biases and uncertainties in their projections. A large contribution to this uncertainty stems from differences in the representation of clouds and convection occurring at scales smaller than the resolved model grid. These long-standing deficiencies in cloud parameterizations have motivated developments of computationally costly global high-resolution cloud resolving models, that can explicitly resolve clouds and convection. Deep learning can learn such explicitly resolved processes from cloud resolving models. While unconstrained neural networks often learn non-physical relationships that can lead to instabilities in climate simulations, causally-informed deep learning can mitigate this problem by identifying direct physical drivers of subgrid-scale processes. Both unconstrained and causally-informed neural networks are developed using a superparameterized climate model in which deep convection is explicitly resolved, and are coupled to the climate model. Prognostic climate simulations with causally-informed neural network parameterization are stable, accurately represent mean climate and variability of the original climate model, and clearly outperform its non-causal counterpart. Combining causal discovery and deep learning is a promising approach to improve data-driven parameterizations (informed by causally-consistent physical fields) for both their design and trustworthiness.

How to cite: Iglesias-Suarez, F., Eyring, V., Gentine, P., Beucler, T., Pritchard, M., Runge, J., and Solino-Fernandez, B.: The key role of causal discovery to improve data-driven parameterizations in climate models, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-6450, https://doi.org/10.5194/egusphere-egu23-6450, 2023.

To organise weather data as images, pixels represent coordinates and magnitude of pixels represents the state of the observed variable in a given time. Observed variables, such as air temperature, mean sea level pressure, wind components and others, may be collected into higher dimensional images or even into a motion structure. Codification of formers as a spatial and the latter as a spatio-temporal allows them to be processed using the deep learning methods, for instance autoencoders and autoencoder-like architectures. The objective of the original autoencoder is to reproduce the input image as much as possible, thus effectively equalising the input and output during the training. Then, an advantage of autoencoder can be utilised to calculate the deviations between (1) true states (effectively the inputs), which are derived by nature, and the (2) expected states, which are derived by means of statistical learning. Calculated deviations can then be interpreted to identify the extreme events, such as heatwaves, hot days or any other rare events (so-called anomalies). Additionally, by modelling deviations by statistical distributions, geographical areas with higher probabilities of anomalies can be deduced at the tails of the distribution. The capability of reproduction of the (original input) images is hence crucial in order to avoid addressing arbitrary noise as anomaly. We would like to run experiments to realise the effective architecture that give reasonable solutions, verify the benefits of implementing the variational autoencoder, realise the effect of selecting various statistical loss functions, and find out the effective architecture of the decoder part of the autoencoder.

How to cite: Fister, D., Pérez-Aracil, J., Peláez-Rodríguez, C., Drouard, M., G. Zaninelli, P., Barriopedro Cepero, D., García-Herrera, R., and Salcedo-Sanz, S.: Towards the effective autoencoder architecture to detect weather anomalies, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-7457, https://doi.org/10.5194/egusphere-egu23-7457, 2023.

The retrieval of atmospheric aerosol properties from satellite remote sensing is a complex and under-determined inverse problem. Traditional retrieval algorithms, based on radiative transfer models, must make approximations and assumptions to reach a unique solution or repeatedly use the expensive forward models to be able to quantify uncertainty. The recently introduced Invertible Neural Networks (INNs), a machine learning method based on Normalizing Flows, appear particularly suited for tackling inverse problems. They simultaneously model both the forward and the inverse branches of the problem, and their generative aspect allows them to efficiently provide non-parametric posterior distributions for the retrieved parameters, which can be used to quantify the retrieval uncertainty. So far INNs have successfully been applied to low-dimensional idealised inverse problems and even to some simpler scientific retrieval problems. Still, satellite aerosol retrievals present particular challenges, such as the high variability of the surface reflectance signal and the often comparatively small aerosol signal in the top-of-the-atmosphere (TOA) measurements.

In this study, we investigate the use of INNs for retrieving aerosol optical depth (AOD) and its uncertainty estimates at the pixel level from MODIS TOA reflectance measurements. The models are trained with custom synthetic datasets of TOA reflectance-AOD pairs made by combining the MODIS Dark Target algorithm’s atmospheric look-up tables and a MODIS surface reflectance product. The INNs are found to perform emulation and inversion of the look-up tables successfully. We initially train models adapted to different surface types by focusing our application on limited regional and seasonal contexts. The models are applied to real measurements from the MODIS sensor, and the generated AOD retrievals and posterior distributions are compared to the corresponding Dark Target and AERONET retrievals for evaluation and discussion.

How to cite: Pelucchi, P., Vicent, J., Johnson, J. E., Stier, P., and Camps-Valls, G.: Invertible neural networks for satellite retrievals of aerosol optical depth, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-7465, https://doi.org/10.5194/egusphere-egu23-7465, 2023.

The rapid development of deep learning approaches has conquered many fields, and precipitation prediction is one of them. Precipitation modeling remains a challenge for numerical weather prediction or climate models, and parameterization is required for low spatial resolution models, such as those used in climate change impact studies. Machine learning models have been shown to be capable of learning the relationships between other meteorological variables and precipitation. Such models are much less computationally intensive than explicit modeling of precipitation processes and are becoming more accurate than parametrization schemes.

Most existing applications focus either on precipitation extremes aggregated over a domain of interest or on average precipitation fields. Here, we are interested in spatial extremes and focus on the prediction of heavy precipitation events (>95th percentile) and extreme events (>99th percentile) over the European domain. Meteorological variables from ERA5 are used as input, and E-OBS data as target. Different architectures from the literature are compared in terms of predictive skill for average precipitation fields as well as for the occurrence of heavy or extreme precipitation events (threshold exceedance). U-Net architectures show higher skills than other variants of convolutional neural networks (CNN). We also show that a shallower U-Net architecture performs as well as the original network for this application, thus reducing the model complexity and, consequently, the computational resources. In addition, we analyze the number of inputs based on the importance of the predictors provided by a layer-wise relevance propagation procedure.

How to cite: Horton, P. and Otero, N.: Predicting spatial precipitation extremes with deep learning models. A comparison of existing model architectures., EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-7862, https://doi.org/10.5194/egusphere-egu23-7862, 2023.

Tropical cyclones (TCs) are one of the costliest and deadliest natural disasters due to the combination of their strong winds and induced storm surges and heavy precipitation, which can cause devastating floods. Unfortunately, due to its high spatio-temporal variability, complex underlying physical process, and lack of high-quality observations, precipitation is still one of the most challenging aspects of a TC to model. However, as precipitation is a key forcing variable for hydrological processes acting across multiple space-time scales, accurate precipitation input is crucial for reliable hydrological simulations and forecasts.

A popular source of precipitation data is the ERA5 reanalysis dataset, frequently used as input to hydrological models when studying floods. However, ERA5 systematically underestimates TC-induced precipitation compared to MSWEP, a multi-source observational dataset fusing gauge, satellite, and reanalysis-based data, currently one of the most accurate precipitation datasets. Moreover, the spatial distribution of TC-rainfall in ERA5 has large room for improvement.

Here, we present a precipitation correction scheme based on U-Net, a popular deep-learning architecture. Rather than only adjusting the per-pixel precipitation values at each timestep of a given TC, we explicitly design our model to also adjust the spatial distribution of the precipitation; to the best of our knowledge, we are the first to do so. The key novelty of our model is a custom-made loss function, based on the combination of the fractions skill score (FSS) and mean absolute error (MAE) metrics. We train and validate the model on 100k time steps (with an 80:20 train:test split) from global TC precipitation events. We show how a U-Net trained with our loss function can reduce the per-pixel MAE of ERA5 precipitation by nearly as much as other state-of-the-art methods, while surpassing them significantly in terms of improved spatial patterns of precipitation. Finally, we discuss how the outputs of our model can be used for future research.

How to cite: Ascenso, G., Ficchì, A., Cavicchia, L., Scoccimarro, E., Giuliani, M., and Castelletti, A.: Improving the spatial accuracy of extreme tropical cyclone rainfall in ERA5 using deep learning, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-8085, https://doi.org/10.5194/egusphere-egu23-8085, 2023.

With no end to anthropogenic greenhouse gas emissions in sight, policymakers are increasingly debating artificial mechanisms to cool the earth's climate. One such solution is stratospheric atmospheric injections (SAI), a method of solar geoengineering where particles are injected into the stratosphere in order to reflect the sun’s rays and lower global temperatures. Past volcanic events suggest that SAI can lead to fast substantial surface temperature reductions, and it is projected to be economically feasible. Research in simulation, however, suggests that SAI can lead to catastrophic side effects. It is also controversial among politicians and environmentalists because of the numerous challenges it poses geopolitically, environmentally, and for human health. Nevertheless, SAI is increasingly receiving attention from policymakers. In this research project, we use deep reinforcement learning to study if, and by how much, carefully engineered temporally and spatially varying injection strategies can mitigate catastrophic side effects of SAI. To do this, we are using the HadCM3 global circulation model to collect climate system data in response to artificial longitudinal aerosol injections. We then train a neural network emulator on this data, and use it to learn optimal injection strategies under a variety of objectives by alternating model updates with reinforcement learning. We release our dataset and code as a benchmark dataset to improve emulator creation for solar aerosol engineering modeling. 

How to cite: Agrawal, E. and Schroder de Witt, C.: Utilizing AI emulators to Model Stratospheric Aerosol Injections and their Effect on Climate, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-8496, https://doi.org/10.5194/egusphere-egu23-8496, 2023.

How to cite: Watson, P.: Machine learning applications for weather and climate need greater focus on extremes, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-8615, https://doi.org/10.5194/egusphere-egu23-8615, 2023.

Microscale RANS (Reynolds Averaged Navier Stokes) models are able to simulate the urban climate for entire large cities with a high spatial resolution of up to 5 m horizontally. They do this using data from geographic information systems (GIS) that must be specially processed to provide the models with information about the terrain, buildings, land use, and resolved vegetation. If high-performance computers, for example from research institutions, are not available for the simulations or are beyond the financial scope, the calculation on commercially available servers can take several weeks. The calculation of a reference initial state for a city is often followed by questions regarding adaptation measures due to climate change or the influence of smaller and larger future building developments on the urban climate. These changes lead locally to a change of the urban climate but are also influenced by the urban climate itself.

In order to save computational time and to comfortably give a quantitative fast initial assessment, we trained a neural network that predicts the simulation results of a RANS model (for example: air temperature at night and during the day, wind speed, cold air flow) and implemented this network in a GIS. The tool allows to calculate the impact of development projects on the urban climate in a fraction of the time required by a RANS simulation and comes close to the RANS model in terms of accuracy. It can also be used by people without in-depth knowledge of urban climate modeling and is therefore particularly suitable for use, for example, in specialized offices of administrative departments or by project developers.

How to cite: von Tils, R. and Wiemers, S.: An urban climate neural network screening tool, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-8661, https://doi.org/10.5194/egusphere-egu23-8661, 2023.

Natural gas is the primary fuel used in U.S. residences, especially during winter, when cold temperatures drive the heating demand. In this study, we use daily county-level gas consumption data to assess the spatial patterns of the relationships and sensitivities of gas consumption by U.S. households considering outdoor temperatures. Linear-plus-plateau functions are found to be the best fit for gas consumption and are applied to derive two key coefficients for each county: the heating temperature threshold (T_crit) below which residential heating starts and the rate of increase in gas consumption when the outdoor temperature drops by one degree (Slope). We then use interpretable machine learning models to evaluate the key building properties and socioeconomic factors related to the spatial patterns of T_crit and Slope based on a large database of individual household properties and population census data. We find that building age, employment rates, and household size are the main predictors of T_crit, whereas the share of gas as a heating fuel and household income are the main predictors of Slope. The latter result suggests inequalities across the U.S. with respect to gas consumption, with wealthy people living in well-insulated houses associated with low T_crit and Slope values. Finally, we estimate potential reductions in gas use in U.S. residences due to improvements in household insulation or a hypothetical behavioral change toward reduced consumption by adopting a 1°C lower T_crit than the current value and a reduced slope. These two scenarios would result in 25% lower gas consumption at the national scale, avoiding 1.24 million MtCO₂ of emissions per year. Most of these reductions occur in the Midwest and East Coast regions. The results from this study provide new quantitative information for targeting efforts to reduce household gas use and related CO₂ emissions in the U.S.

How to cite: Mittakola, R. T., Ciais, P., Schubert, J., Makowski, D., Zhou, C., Bazzi, H., Sun, T., Liu, Z., and Davis, S.: Drivers of Natural Gas Use in United States Buildings, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-8666, https://doi.org/10.5194/egusphere-egu23-8666, 2023.

In Brazil, the water system is essential for the electrical system and agribusiness. Understanding climate changes and predicting long-term hydrometeorological phenomena is vital for developing and maintaining these sectors in the country. This work aims to use data from the SIN system (National Interconnected System) in Brazil, from the main hydrological basins, as well as historical rainfall data, in complex networks and deep learning algorithms, to identify possible climate changes in Brazil and predict future hydrometeorological phenomena. Through the methodology developed in this work, the predictions generated showed satisfactory results, which allows identifying regions more sensitive to climate change and anticipating climate events. This work is expected to help the energy generation system in Brazil and the agronomy sector, the main sectors that drive the country's economy.

How to cite: Caseri, A. and A. Rodrigues, F.: Identification of sensitive regions to climate change and anticipation of climate events in Brazil, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-8921, https://doi.org/10.5194/egusphere-egu23-8921, 2023.

The biosphere displays high heterogeneity at landscape-scale. Vegetation modelers struggle to represent this variability in process-based models because global observations of micrometeorology and plant traits are not available at such fine granularity. However, remote sensing data is available: the Sentinel 2 satellites with a 10m resolution capture aspects of localized vegetation dynamics. The EarthNet challenge (EarthNet2021, [1]) aims at predicting satellite imagery conditioned on coarse-scale weather data. Multiple research groups approached this challenge with deep learning [2,3,4]. Here, we evaluate how well these satellite image models simulate the vegetation response to climate, where the vegetation status is approximated by the NDVI vegetation index.

Achieving the new vegetation-centric evaluation requires three steps. First, we update the original EarthNet2021 dataset to be suitable for vegetation modeling: EarthNet2021x includes improved georeferencing, a land cover map, and a more effective cloud mask. Second, we introduce the interpretable evaluation metric VegetationScore: the Nash Sutcliffe model efficiency (NSE) of NDVI predictions over clear-sky observations per vegetated pixel aggregated through normalization to dataset level. The ground truth NDVI time series achieves a VegetationScore of 1, the target period mean NDVI a VegetationScore of 0. Third, we assess the skill of two deep neural networks with the VegetationScore: ConvLSTM [2,3], which combines convolutions and recurrency, and EarthFormer [4], a Transformer adaptation for Earth science problems. 

Both models significantly outperform the persistence baseline. They do not display systematic biases and generally catch spatial patterns. Yet, both neural networks achieve a negative VegetationScore. Only in about 20% of vegetated pixels, the deep learning models do beat a hypothetical model predicting the true target period mean NDVI. This is partly because models largely underestimate the temporal variability. However, the target variability may partially be inflated by the noisy nature of the observed NDVI. Additionally, increasing uncertainty for longer lead times decreases scores: the mean RMSE in the first 25 days is 50% lower than between 75 and 100 days lead time. In general, consistent with the EarthNet2021 leaderboard, the EarthFormer outperforms the ConvLSTM. With EarthNet2021x, a more narrow perspective to the EarthNet challenge is introduced. Modeling localized vegetation response is a task that requires careful adjustments of off-the-shelf computer vision architectures for them to excel. The resulting specialized approaches can then be used to advance our understanding of the complex interactions between vegetation and climate.

 [1] Requena-Mesa, Benson, Reichstein, Runge and Denzler. EarthNet2021: A large-scale dataset and challenge for Earth surface forecasting as a guided video prediction task. CVPR Workshops, 2021.

 [2] Diaconu, Saha, Günnemann and Zhu. Understanding the Role of Weather Data for Earth Surface Forecasting Using a ConvLSTM-Based Model. CVPR Workshops, 2022.

 [3] Kladny, Milanta, Mraz, Hufkens and Stocker. Deep learning for satellite image forecasting of vegetation greenness. bioRxiv, 2022.

 [4] Gao, Shi, Wang, Zhu, Wang, Li and Yeung. Earthformer: Exploring Space-Time Transformers for Earth System Forecasting. NeurIPS, 2022.

How to cite: Benson, V., Requena-Mesa, C., Robin, C., Alonso, L., Carvalhais, N., and Reichstein, M.: Modeling landscape-scale vegetation response to climate: Synthesis of the EarthNet challenge, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-9337, https://doi.org/10.5194/egusphere-egu23-9337, 2023.

The rapidly growing number of low-cost environmental sensors and data from opportunistic sensors constantly advances the quality as well as the spatial and temporal resolution of weather and climate models. However, it also leads to the need for effective tools to ensure the quality of collected data.

Time series quality control (QC) from multiple spatial, irregularly distributed sensors is a challenging task, as it requires the simultaneous integration and analysis of observations from sparse neighboring sensors and consecutive time steps. Manual QC is very often time- and labour- expensive and requires expert knowledge, which introduces subjectivity and limits reproducibility. Therefore, automatic, accurate, and robust QC solutions are in high demand, where among them one can distinguish machine learning techniques. 

In this study, we present a novel approach for the quality control of time series data from multiple spatial, irregularly distributed sensors using graph neural networks (GNNs). Although we applied our method to commercial microwave link attenuation data collected from a network in Germany between April and October 2021, our solution aims to be generic with respect to the number and type of sensors, The proposed approach involves the use of an autoencoder architecture, where the GNN is used to model the spatial relationships between the sensors, allowing for the incorporation of contextual information in the quality control process. 

While our model shows promising results in initial tests, further research is needed to fully evaluate its effectiveness and to demonstrate its potential in a wider range of environmental applications. Eventually, our solution will allow us to further foster the observational basis of our understanding of the natural environment.

How to cite: Lasota, E., Polz, J., Chwala, C., Schmidt, L., Lünenschloß, P., Schäfer, D., and Bumberger, J.: Enhancing environmental sensor data quality control with graph neural networks, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-9434, https://doi.org/10.5194/egusphere-egu23-9434, 2023.

How to cite: Son, R., Carvalhais, N., Silva, L., Requena-Mesa, C., Weber, U., Gayler, V., Stacke, T., Schnur, R., Nabel, J., Winkler, A., and Zaehle, S.: Integration of a deep-learning-based fire model into a global land surface model, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-9810, https://doi.org/10.5194/egusphere-egu23-9810, 2023.

Forest health is affected by many interacting and correlated weather variables over multiple temporal scales. Climate change affects weather conditions and their dependencies. To better understand future forest health and status, an improved scientific  understanding of the complex relationships between weather conditions and forest mortality is required. Explainable AI (XAI) methods are increasingly used to understand and simulate physical processes in complex environments given enough data. In this work, an hourly weather generator (AWE-GEN) is used  to simulate 200,000 years of daily weather conditions representative of central Germany. It is capable of simulating low and high-frequency characteristics of weather variables and also captures the inter-annual variability of precipitation. These data are then used to drive an individual-based forest model (FORMIND) to simulate the dynamics of a beech, pine, and spruce forest. A variational autoencoder β-VAE is used to learn representations of the generated weather conditions, which include radiation, precipitation and temperature. We learn shared and specific variable latent representations using a decoder network which remains the same for all the weather variables. The representation learning is completely unsupervised. Using the output of the forest model, we identify single and compounding weather prototypes that are associated with extreme forest mortality. We find that the prototypes associated with extreme mortality are similar for pine and spruce forests and slightly different for beech forests. Furthermore, although the compounding weather prototypes represent a larger sample size (2.4%-3.5%) than the single prototypes (1.7%-2.2%), they are associated with higher levels of mortality on average. Overall, our research illustrates how deep learning frameworks can be used to identify weather patterns that are associated with extreme impacts.

How to cite: Anand, M., Bohn, F., Sweet, L., Camps-Valls, G., and Zscheischler, J.: Identifying compound weather prototypes of forest mortality with β-VAE, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-10219, https://doi.org/10.5194/egusphere-egu23-10219, 2023.

Hydrological models and machine learning models are widely used in streamflow simulation and data reconstruction. However, a global assessment of these models is still lacking and no synthesized catchment-scale streamflow product derived from multiple models is available over the globe. In this study, we comprehensively evaluated four conceptual hydrological models (GR2M, XAJ, SAC, Alpine) and four machine learning models (RF, GBDT, DNN, CNN) based on the selected 16,218 gauging stations worldwide, and then applied multi-model weighting ensemble (MWE) method to merge streamflow simulated from these models. Generally, the average performance of the machine learning model for all stations is better than that of the hydrological model, and with more stations having a quantified simulation accuracy (KGE>0.2); However, the hydrological model achieves a higher percentage of stations with a good simulation accuracy (KGE>0.6). Specifically, for the average accuracy during the validation period, there are 67% (27%) and 74% (21%) of stations showed a “quantified” (“good”) level for the hydrological models and machine learning models, respectively. The XAJ is the best-performing model of the four hydrological models, particularly in tropical and temperate zones. Among the machine learning models, the GBDT model shows better performance on the global scale. The MWE can effectively improve the simulation accuracy and perform much better than the traditional multi-model arithmetic ensemble (MAE), especially for the constrained least squares prediction combination method (CLS) with 82% (28%) of the stations having a “qualified” (“good”) accuracy. Furthermore, by exploring the influencing factors of the streamflow simulation, we found that both machine-learning models and hydrological models perform better in wetter areas.

How to cite: Zhang, J. and Liu, J.: Simulation and reconstruction of global monthly runoff based on hydrological models and machine learning models, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-10391, https://doi.org/10.5194/egusphere-egu23-10391, 2023.

Physical process-based numerical prediction models (NWPs) and radar-based probabilistic methods have been mainly used for short-term precipitation prediction. Recently, radar-based precipitation nowcasting models using advanced machine learning (ML) have been actively developed. Although the ML-based model shows outstanding performance in short-term rainfall prediction, it significantly decreases performance due to increased lead time. It has the limitation of being a black box model that does not consider the physical process of the atmosphere. To address these limitations, we aimed to develop a hybrid precipitation nowcasting model, which combines NWP and an advanced ML-based model via an ML-based ensemble method. The Weather Research and Forecasting (WRF) model was used as NWP to generate a physics-based rainfall forecast. In this study, we developed the ML-based precipitation nowcasting model with conditional Generative Adversarial Network (cGAN), which shows high performance in the image generation tasks. The radar reflectivity data, WRF hindcast meteorological outputs (e.g., temperature and wind speed), and static information of the target basin (e.g., DEM, Land cover) were used as input data of cGAN-based model to generate physics-informed rainfall prediction at the lead time up to 6 hours. The cGAN-based model was trained with the data for the summer season of 2014-2017. In addition, we proposed an ML-based blending method, i.e., XGBoost, that combines cGAN-based model results and WRF forecast results. To evaluate the hybrid model performance, we analyzed the performance of precipitation predictions on three heavy rain events in South Korea. The results confirmed that using the blending method to develop a hybrid model could provide an improved precipitation nowcasting approach.

Acknowledgements

 This work was supported by a grant from the National Research Foundation of Korea funded by the Ministry of Science, ICT & Future Planning (2020R1A2C2007670).

How to cite: Choi, S. and Kim, Y.: Developing hybrid precipitation nowcasting model with WRF and conditional GAN-based model, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-10431, https://doi.org/10.5194/egusphere-egu23-10431, 2023.

Extreme stratospheric events such as extremely weak vortex events and strong vortex events can influence weather in the troposphere from weeks to months and thus are important sources of predictability of tropospheric weather on subseasonal to seasonal (S2S) timescales. However, the predictability of weak vortex events is limited to 1-2 weeks in state-of-the-art forecasting systems, while strong vortex events are more predictable than weak vortex events. Longer predictability timescales of the stratospheric extreme events would benefit long-range surface weather prediction. Recent studies showed promising results in the use of machine learning for improving weather prediction. The goal of this study is to explore the potential of a machine learning approach in extending the predictability of stratospheric extreme events in S2S timescales. We use neural networks (NNs) to predict the monthly stratospheric polar vortex strength with lead times up to five months using the first five principal components (PCs) of the sea surface temperature (SST), mean sea level pressure (MSLP), Barents–Kara sea-ice concentration (BK-SIC), poleward heat flux at 100 hPa, and zonal wind at 50, 30, and 2 hPa as precursors. These physical variables are chosen as they are indicated as potential precursors for the stratospheric extremes in previous studies. The results show that the accuracy and Brier Skill Score decrease with longer lead times and the performance is similar between weak and strong vortex events. We then employ two different NN attribution methods to uncover feature importance (heat map) in the inputs for the NNs, which indicates the relevance of each input for NNs to make the prediction. The heat maps suggest that precursors from the lower stratosphere are important for the prediction of the stratospheric polar vortex strength with a lead time of one month while the precursors at the surface and the upper stratosphere become more important with lead times longer than one month. This result is overall consistent with the previous studies that subseasonal precursors to the stratospheric extreme events may come from the lower troposphere. Our study sheds light on the potential of explainable NNs in searching for opportunities for skillful prediction of stratospheric extreme events and, by extension, surface weather on S2S timescales.

How to cite: Wu, Z., Beucler, T., and Domeisen, D.: Extended-range predictability of stratospheric extreme events using explainable neural networks, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-10568, https://doi.org/10.5194/egusphere-egu23-10568, 2023.

How to cite: Buch, J., Williams, A. P., and Gentine, P.: Seasonal forecasts of wildfire frequency and burned area in the western United States using a stochastic machine learning fire model, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-11238, https://doi.org/10.5194/egusphere-egu23-11238, 2023.

  This study proposes a novel approach on estimation of fine dust concentration from raw video data recorded by surveillance cameras. At first, several regions of interest are defined from specific images extracted from videos in surveillance cameras installed at Chung-Ang University. Among them, sky fields are mainly considered to figure out changes in characteristics of each color. After converting RGB images into BGR images, a number of discrete pixels with brightness intensities in a blue channel is mainly analyzed by investigating any relationships with fine dust concentration measured from automatic monitoring stations near the campus. Here, different values of thresholds from 125 to 200 are considered to find optimal conditions from changes in values of each pixel in the blue channel. This study uses the Pearson correlation coefficient to calculate the correlation between the number of pixels with values over the selected threshold and observed data for fine dust concentration. As an example on one specific date, the coefficients reflect their positive correlations with a range from 0.57 to 0.89 for each threshold. It should be noted that this study is a novel attempt to suggest a new, simple, and efficient method for estimating fine dust concentration from surveillance cameras common in many areas around the world.

Keywords: Fine Dust Concentration, BGR Image, Surveillance Camera, Threshold, Correlation Analysis

Acknowledgment

  This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (No. NRF-2022R1A4A3032838) and this work was funded by the Korea Meteorological Administration Research and Development Program under Grant KMI2022-01910 and this work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (2020R1G1A1013624).

How to cite: Cha, H., Byun, J., Baik, J., and Jun, C.: Estimation of Fine Dust Concentration from BGR Images in Surveillance Cameras, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-11355, https://doi.org/10.5194/egusphere-egu23-11355, 2023.

Low-level jets are maximums in the vertical profile of the wind speed profile in the lowest levels of atmosphere. Low-level jets, when present, can make a significant impact on the wind energy. Wind conditions in low-level jets depart from traditional assumptions about wind profile and low-level jets can also influence the stability and turbulence that are important for wind energy applications.

In literature commonly an algorithm of identifying low-level jets is used to estimate frequency of low-level jets. The algorithm searches for maximum in the lowest levels of the atmosphere with a temperature inversion above the jet maximum. The algorithm is useful in identifying the presence of the low-level jets and estimating their frequency. However, low-level jets can be caused by a number of different mechanisms which leads to differences in low-level jet characteristics. Therefore, additional analysis is necessary to distinguish between different types of jets and characterize their properties. We aim to automate this process using Principal Component Analysis (PCA) to identify main patterns of wind speed and temperature. By analyzing diurnal and seasonal cycles of these patterns a better understanding about climatology of low-level jets in the region can be gained.

This study focuses on the central part of the Baltic Sea. Several recent studies have identified the presence of low-level jets near the coast of Kurzeme. Typically, maximums of low-level jets are located several hundred meters above the surface, while near the coast of Kurzeme maximums of low-level jets are usually within the lowest 100 meters of the atmosphere.

Data from UERRA reanalysis with 11 km horizontal resolution on 12 height levels in the lowest 500 meters of atmosphere was used. The algorithm that identifies low-level jets was applied to the data, to estimate frequency of low-level jets in each grid cell of the model. Jet events were grouped by the wind direction to identify main trajectories of low-level jets in the region. Several atmosphere cross-sections that low-level jets frequently flow through were chosen for further analysis.

Model data was interpolated to the chosen cross-sections and PCA was applied to the cross-section data of wind speed, geostrophic wind speed and temperature. Main patterns of these meteorological parameters, such as wind speed maximum, temperature inversion above the surface of the sea and temperature difference between sea and land were identified by the PCA. Differences of principal components between cross-sections and diurnal and seasonal patterns of principal components helped to gain better understanding of climatology, extent and mechanisms of low-level jets in the region.

How to cite: Pogumirskis, M., Sīle, T., and Bethers, U.: Identifying mechanisms of low-level jets near coast of Kurzeme using Principal Component Analysis, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-12137, https://doi.org/10.5194/egusphere-egu23-12137, 2023.

Explainable artificial intelligence (XAI) methods serve as a support for researchers to shed light onto the reasons behind the predictions made by deep neural networks (DNNs). XAI methods have already been successfully applied to climate science, revealing underlying physical mechanisms inherent in the studied data. However, the evaluation and validation of XAI performance is challenging as explanation methods often lack ground truth. As the number of XAI methods is growing, a comprehensive evaluation is necessary to enable well-founded XAI application in climate science.

In this work we introduce explanation evaluation in the context of climate research. We apply XAI evaluation to compare multiple explanation methods for a multi-layer percepton (MLP) and a convolutional neural network (CNN). Both MLP and CNN assign temperature maps to classes based on their decade. We assess the respective explanation methods using evaluation metrics measuring robustness, faithfulness, randomization, complexity and localization. Based on the results of a random baseline test we establish an explanation evaluation guideline for the climate community. We use this guideline to rank the performance in each property of similar sets of explanation methods for the MLP and CNN. Independent of the network type, we find that Integrated Gradients, Layer-wise relevance propagation and InputGradients exhibit a higher robustness, faithfulness and complexity compared to purely Gradient-based methods, while sacrificing reactivity to network parameters, i.e. low randomisation scores. The contrary holds for Gradient, SmoothGrad, NoiseGrad and FusionGrad. Another key observation is that explanations using input perturbations, such as SmoothGrad and Integrated Gradients, do not improve robustness and faithfulness, in contrast to theoretical claims. Our experiments highlight that XAI evaluation can be applied to different network tasks and offers more detailed information about different properties of explanation method than previous research. We demonstrate that using XAI evaluation helps to tackle the challenge of choosing an explanation method.

How to cite: Bommer, P., Kretschmer, M., Hedstroem, A., Bareeva, D., and Hoehne, M. M.-C.: Evaluation of explainable AI solutions in climate science, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-12528, https://doi.org/10.5194/egusphere-egu23-12528, 2023.

Compound heat waves and drought events draw our particular attention as they become more frequent. Co-occurring extreme events often exacerbate impacts on ecosystems and can induce a cascade of detrimental consequences. However, the research to understand these events is still in its infancy. DeepExtremes is a project funded by the European Space Agency (https://rsc4earth.de/project/deepextremes/) aiming at using deep learning to gain insight into Earth surface under extreme climate conditions. Specifically, the goal is to forecast and explain extreme, multi-hazard, and compound events. To this end, the project leverages the existing Earth observation archive to help us better understand and represent different types of hazards and their effects on society and vegetation. The project implementation involves a multi-stage process consisting of 1) global event detection; 2) intelligent subsampling and creation of mini-data-cubes; 3) forecasting methods development, interpretation, and testing; and 4) cloud deployment and upscaling. The data products will be made available to the community following the reproducibility and FAIR data principles. By effectively combining Earth system science with explainable AI, the project contributes knowledge to advancing the sustainable management of consequences of extreme events. This presentation will show the progress made so far and specifically introduce how to participate in the challenges about spatio-temporal extreme event prediction in DeepExtremes.

How to cite: Mora, K., Brandt, G., Benson, V., Brockmann, C., Camps-Valls, G., Fernández-Torres, M.-Á., Fincke, T., Fomferra, N., Gans, F., Gonzalez, M., Ji, C., Kraemer, G., Marco, E. S., Montero, D., Reichstein, M., Requena-Mesa, C., Valero, O. J. P., Weynants, M., Wieneke, S., and Mahecha, M. D.: DeepExtremes: Explainable Earth Surface Forecasting Under Extreme Climate Conditions, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-12657, https://doi.org/10.5194/egusphere-egu23-12657, 2023.

Berkeley Earth is premiering a new high resolution analysis of historical instrumental temperatures.

This builds on our existing work on climate reconstruction by adding a simple machine learning layer to our analysis.  This new approach extracts weather patterns from model, satellite, and reanalysis data, and then layers these weather patterns on top of instrumental observations and our existing interpolation methods to produce new high resolution historical temperature fields.  This has quadrupled our output resolution from the previous 1° x 1° lat-long to a new global 0.25° x 0.25° lat-long resolution.  However, this is not simply a downscaling effort.  Firstly, the use of weather patterns derived from physical models and observations increases the spatial realism of the reconstructed fields.  Secondly, observations from regions with high density measurement networks have been directly incorporated into the high resolution field, allowing dense observations to be more fully utilized.  

This new data product uses significantly more observational weather station data and produces higher resolution historical temperature fields than any comparable product, allowing for unprecedented insights into historical local and regional climate change.  In particular, the effect of geographic features such as mountains, coastlines, and ecosystem variations are resolved with a level of detail that was not previously possible.  At the same time, previously established techniques for bias corrections, noise reduction, and error analysis continued to be utilized.  The resulting global field initially spans 1850 to present and will be updated on an ongoing basis.  This project does not significantly change the global understanding of climate change, but helps to provide local detail that was often unresolved previously.  The initial data product focuses on monthly temperatures, though a proposal exists to also create a high resolution daily temperature data set using similar methods.

This talk will describe the construction of the new data set and its characteristics.  The techniques used in this project are accessible enough that they are likely to be useful for other types of instrumental analyses wishing to improve resolution or leverage basic information about weather patterns derived from models or other sources.

How to cite: Rohde, R. A. and Hausfather, Z.: New Berkeley Earth High Resolution Temperature Data Set, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-12889, https://doi.org/10.5194/egusphere-egu23-12889, 2023.

River floods are among the most devastating natural hazards, causing thousands of deaths and billions of euros in damages every year. Floods can result from a combination of compounding drivers such as heavy precipitation, snowmelt, and high antecedent soil moisture. These drivers and the processes they govern vary widely both between catchments and between flood events within a catchment, making a causal understanding of the underlying hydrological processes difficult.

Modern causal inference methods, such as the PCMCI framework, are able to identify drivers from complex time series through causal discovery and build causally aware statistical models. However, causal inference tailored to extreme events remains a challenge due to data length limitations. To overcome data limitations, here we bridge the gap between synthetic and real world data using 1,000 years of simulated weather to drive as state-of-the-art hydrological model (the mesoscale Hydrological Model, mHM) over a wide range of European catchments. From the simulated time series, we extract high runoff events, on which we evaluate the causal inference approach. We identify the minimum data necessary for obtaining robust causal models, evaluate metrics for model evaluation and comparison, and compare causal flood drivers across catchments. Ultimately, this work will help establish best practices in causal inference for flood research to identify meteorological and catchment specific flood drivers in a changing climate.

How to cite: Miersch, P., Jiang, S., Rakovec, O., and Zscheischler, J.: Identifying drivers of river floods using causal inference, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-12948, https://doi.org/10.5194/egusphere-egu23-12948, 2023.

Clouds are classified into types, classes, or regimes. The World Meteorological Organization distinguishes stratus and cumulus clouds and three altitude layers. Cloud types exhibit very different radiative properties and interact in numerous ways with aerosol particles in the atmosphere. However, it has proven difficult to define cloud regimes objectively and from remote sensing data, hindering the understanding we have of the processes and adjustments involved.

Building on the method we previously developed, we combine synoptic observations and passive satellite remote-sensing retrievals to constitute a database of cloud types and cloud properties to eventually train a cloud classification algorithm. The cloud type labels come from the global marine meteorological observations dataset (UK Met Office, 2006) which is comprised of near-global synoptic observations. This data record reports back information about cloud type and other meteorological quantities at the surface. The cloud classification model is built on different cloud-top and cloud optical properties (Level 2 products MOD06/MYD06 from the MODIS sensor) extracted temporally close to the observation time and on a 128km x 128km grid around the synoptic observation location. To make full use of the large quantity of remote sensing data available and to investigate the variety in cloud settings, a convolutional variational auto-encoder (VAE) is applied as a dimensionality reduction tool in a first step. Furthermore, such model architecture allows to account for spatial relationships while describing non-linear patterns in the input data. The cloud classification task is subsequently performed drawing on the constructed latent representation of the VAE. Associating information from underneath and above the cloud enables to build a robust model to classify cloud types. For the training we specify a study domain in the Atlantic ocean around the equator and evaluate the method globally. Further experiments and evaluation are done on simulation data produced by the ICON model.

How to cite: Lenhardt, J., Quaas, J., and Sejdinovic, D.: From MODIS cloud properties to cloud types using semi-supervised learning, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-13250, https://doi.org/10.5194/egusphere-egu23-13250, 2023.

Hybrid modeling describes the synergy between parametric models and machine learning [1]. Parts of a parametric equation are substituted by non-parametric machine learning models, which can then represent complex functions. These are inferred together with the parameters of the equation from the data. Hybrid modeling promises to describe complex relationships and to be scientifically interpretable. These promises, however, need to be taken with a grain of salt. With too flexible models, such as deep neural networks, the problem of equifinality arises: There is no identifiable optimal solution. Instead, many outcomes describe the data equally well, and we will obtain one of them by chance. Interpreting the result may lead to erroneous conclusions. Moreover, studies have shown that regularization techniques can introduce a bias on jointly estimated physical parameters [1].

We propose double machine learning (DML) to solve these problems [2]. DML is a theoretically well-founded technique for fitting semi-parametric models, i.e., models consisting of a parametric and a non-parametric component. DML is widely used for debiased treatment effect estimation in economics. We showcase its use for geosciences on two problems related to carbon dioxide fluxes: 

• Flux partitioning, which aims at separating the net carbon flux (NEE) into its main contributing gross fluxes, namely, RECO and GPP.
• Estimation of the temperature sensitivity parameter of ecosystem respiration Q10.

First, we show that in the case of synthetic data for Q10 estimation, we can consistently retrieve the true value of Q10 where the naive neural network approach fails. We further apply DML to the carbon flux partitioning problem and find that it is 1) able to retrieve the true fluxes of synthetic data, even in the presence of strong (and more realistic) heteroscedastic noise, 2) retrieves main gross carbon fluxes on real data consistent with established methods, and 3) allows us to causally interpret the retrieved GPP as the direct effect of the photosynthetically active radiation on NEE. This way, the DML approach can be seen as a causally interpretable, semi-parametric version of the established daytime methods. We also investigate the functional relationships inferred with DML and the drivers modulating the obtained light-use efficiency function. In conclusion, DML offers a solid framework to develop hybrid and semiparametric modeling and can be of widespread use in geosciences.

[1] Reichstein, Markus, et al. “Combining system modeling and machine learning into hybrid ecosystem modeling.” Knowledge-Guided Machine Learning (2022). https://doi.org/10.1201/9781003143376-14

[2] Chernozhukov, Victor, et al. “Double/debiased machine learning for treatment and structural parameters.” The Econometrics Journal, Volume 21, Issue 1, 1 (2018): C1–C68. https://doi.org/10.1111/ectj.12097

How to cite: Cohrs, K.-H., Varando, G., Reichstein, M., and Camps-Valls, G.: Double machine learning for geosciences, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-13462, https://doi.org/10.5194/egusphere-egu23-13462, 2023.

In recent years, both the intensity and extent of marine heatwaves have increased across the world. Anomalies in sea surface temperature have an effect on the health of marine ecosystems, which are crucial to the Earth's climate system. Marine Heatwaves' devastating impacts on aquatic life have been increasing steadily in recent years, harming aquatic ecosystems and causing a tremendous loss of marine life. Early warning systems and operational forecasting that can foresee such events can aid in designing effective and better mitigation techniques. Recent studies have shown that machine learning and deep learning-based approaches can be used for forecasting the occurrence of marine heatwaves up to a year in advance. However, these models are black box in nature and do not provide an understanding of the factors influencing MHWs. In this study, we used machine learning methods to forecast marine heatwaves. The developed models were tested across four historical Marine Heatwave events around the world. Explainable AI methods were then used to understand and analyze the relationships between the drivers of these events.

How to cite: Prasad, A. and Mazumder, S.: Towards explainable marine heatwaves forecasts, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-13622, https://doi.org/10.5194/egusphere-egu23-13622, 2023.

How to cite: Lovo, A., Herbert, C., and Bouchet, F.: Interpretable probabilistic forecast of extreme heat waves, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-14493, https://doi.org/10.5194/egusphere-egu23-14493, 2023.

Air pollution has become an integral part of modern life. The main source of air pollution can be considered combustion processes associated with energy-intensive corporate activities. Energy companies consume about one-third of the fuel produced and are a significant source of air pollution [1]. State and public air quality monitoring networks were created to monitor the situation. Public monitoring networks are cheaper and have more coverage than government ones. Although the state monitoring system shows more accurate data, an inexpensive network is sufficient to inform the public about the presence or absence of pollution (air quality). In order to inform the public, the idea arose to test the possibility of detecting types of pollution using data from cheap air quality monitoring sensors. In general, to use a cheap sensor for measurements, it must first be calibrated (corrected) by comparing its readings with a reference device. Various mathematical methods can be used for this. One of such method is neural network training, which has proven itself well for correcting PM particle readings due to relative humidity impact [2].

The idea of using a neural network to improve data quality is not new, but it is quite promising, as the authors showed in [3]. The main problem to implement this method is connected with a reliable dataset for training the network. For this, it is necessary to register sensor readings for relatively clean air and for artificially generated or known sources of pollution. Training the neural network on the basis of collected data can be used to determine (classify) types of air: with pollution (pollutant) or without. For this, an experiment was set up in the "ReLab" co-working space at the Taras Shevchenko National University of Kyiv. The sensors were placed in a closed box, in which airflow ventilation is provided. The ZPHS01B [4] sensor module was used for inbox measurements, as well as, calibrated sensors PMS7003 [5] and BME280 [6]. Additionally, IPS 7100 [7] and SPS30 [8] were added to enrich the database for ML training. A platform based on HiLink 7688 was used for data collecting, processing, and transmission.

Data was measured every two seconds, independently from each sensor. Before each experiment, the room was ventilated to avoid influence on the next series of experiments.

References

1. Zaporozhets A. Analysis of means for monitoring air pollution in the environment. Science-based technologies. 2017, Vol. 35, no3. 242-252. DOI: 10.18372/2310-5461.35.11844

2. Antonenko A, (2021) Correction of fine particle concentration readings depending on relative humidity, [Master's thesis, Taras Shevchenko National University of Kyiv], 35 pp.

3. Lee, J. Kang, S. Kim, Y. Im, S. Yoo , D. Lee, “Long-Term Evaluation and Calibration of Low-Cost Particulate Matter (PM) Sensor”, Sensors 2020, vol. 20, 3617, 24 pp., 2020.`

4. ZPHS01B Datasheet URL: https://pdf1.alldatasheet.com/datasheet-pdf/view/1303697/WINSEN/ZPHS01B.html

5. Plantower PMS7003 Datasheet URL: https://www.espruino.com/datasheets/PMS7003.pdf

6. Bosch 280 Datasheet URL: https://www.mouser.com/datasheet/2/783/BST-BME280-DS002-1509607.pdf

7. https://pierasystems.com/intelligent-particle-sensors/

8. https://sensirion.com/products/catalog/SPS30/

How to cite: Antonenko, A., Boretskij, V., and Zagaria, O.: Classification of Indoor Air Pollution Using Low-cost Sensors by Machine Learning, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-14856, https://doi.org/10.5194/egusphere-egu23-14856, 2023.

The current situation regarding food insecurity in the continent of Africa, and the Horn of Africa in particular, is at an unprecedented risk level triggered by continuous drought events, complicated interactions between food prices, crop yield, energy inflation and lack of humanitarian aid, along with disrupting conflicts and migration flows. The study of a food-secure environment is a complex, multivariate, multiscale, and non-linear problem difficult to understand with canonical data science methodologies. We propose an alternative approach to the food insecurity problem from a causal inference standpoint to discover the causal relations and evaluate the likelihood and potential consequences of specific interventions. In particular, we demonstrate the use of causal inference for understanding the impact of humanitarian interventions on food insecurity in Somalia. In the first stage of the problem, we apply different data transformations to the main drivers to achieve the highest degree of correlation with the interested variable. In the second stage, we infer causation from the main drivers and interested variables by applying different causal methods such as PCMCI or Granger causality. We analyze and harmonize different time series, per district of Somalia, of the global acute malnutrition (GAM) index, food market prices, crop production, conflict levels, drought and flood internal displacements, as well as climate indicators such as the NDVI index, precipitation or land surface temperature. Then, assuming a causal graph between the main drivers causing the food insecurity problem, we estimate the effect of increasing humanitarian interventions on the GAM index, considering the effects of a changing climate, migration flows, and conflict events. We show that causal estimation with modern methodologies allows us to quantify the impact of humanitarian aid on food insecurity.

References

[1] Runge, J., Bathiany, S., Bollt, E. et al. Inferring causation from time series in Earth system sciences. Nat Commun 10, 2553 (2019). https://doi.org/10.1038/s41467-019-10105-3

[2] Sazib Nazmus, Mladenova lliana E., Bolten John D., Assessing the Impact of ENSO on Agriculture Over Africa Using Earth Observation Data, Frontiers in Sustainable Food Systems, 2020, 10.3389/fsufs.2020.509914. https://www.frontiersin.org/article/10.3389/fsufs.2020.509914

[3] Checchi, F., Frison, S., Warsame, A. et al. Can we predict the burden of acute malnutrition in crisis-affected countries? Findings from Somalia and South Sudan. BMC Nutr 8, 92 (2022). https://doi.org/10.1186/s40795-022-00563-2

How to cite: Cerdà-Bautista, J., Tárraga, J. M., Varando, G., Arribas, A., Shepherd, T., and Camps-Valls, G.: Causal inference to study food insecurity in Africa, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-15000, https://doi.org/10.5194/egusphere-egu23-15000, 2023.

In this study, which is part of the M-VRE [https://mosaic-vre.org/about] project, we aim to improve a quality control (QC) system on arctic ocean temperature profile data using deep learning. For the training, validation, and evaluation of our algorithms, we are using the UDASH dataset [https://essd.copernicus.org/articles/10/1119/2018/]. In the classical QC setting, the ocean expert or "operator", applies a series of thresholding (classical) algorithms to identify, i.e. flag, erroneous data. In the next step, the operator visually inspects every data profile, where suspicious samples have been identified. The goal of this time-consuming visual QC is to find "false positives", i.e. flagged data that is actually good, because every sample/profile has not only a scientific value but also a monetary one. Finally, the operator turns all "false positive" data back to good. The crucial point here is that although these samples/profiles are above certain thresholds they are considered good by the ocean expert. These human expert decisions are extremely difficult, if not impossible, to map by classical algorithms. However, deep-learning neural networks have the potential to learn complex human behavior. Therefore, we have trained a deep learning system to "learn" exactly the expert behavior of finding "false positives" (identified by the classic thresholds), which can be turned back to good accordingly. The first results are promising. In a fully automated setting, deep learning improves the results and fewer data are flagged. In a subsequent visual QC setting, deep learning relieves the expert with a distinct workload reduction and gives the option to clearly increase the quality of the data.
Our long-term goal is to develop an arctic quality control system as a series of web services and Jupyter notebooks to apply automated and visual QC online, efficient, consistent, reproducible, and interactively.

How to cite: Chouai, M., Simon Reimers, F., and Mieruch-Schnülle, S.: Deep learning to support ocean data quality control, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-15185, https://doi.org/10.5194/egusphere-egu23-15185, 2023.

Natural disasters caused by cyclones, hail, landslides or floods are directly related to precipitation. Global climate models are an important tool to adapt to these hazards in a future climate. However, they operate on spatial and temporal discretizations that limit the ability to adequately reflect these fast evolving, highly localized phenomena which has led to the development of various downscaling approaches .

Conditional generative adversarial networks (cGAN) have recently been applied as a promising downscaling technique to improve the spatial resolution of climate data. The ability of GANs to generate ensembles of solutions from random perturbations can be used to account for the stochasticity of climate data and quantify uncertainties. 

We present a cGAN for not only downscaling the spatial, but simultaneously also the temporal dimension of precipitation data as a so-called video super resolution approach. 3D convolutional layers are exploited for extracting and generating temporally consistent  rain events with realistic fine-scale structure. We downscale coarsened gauge adjusted and climatology corrected precipitation data from Germany from a spatial resolution of 32 km to 2 km and a temporal resolution of 1 hr to 10 min, by applying a novel training routine using partly normalized and logarithmized data, allowing for improved extreme value statistics of the generated fields.

Exploiting the fully convolutional nature of our model we can generate downscaled maps for the whole of Germany in a single downscaling step at low latency. The evaluation of these maps using a spatial and temporal power spectrum analysis shows that the generated temporal and spatial structures are in high agreement with the reference. Visually, the generated temporally evolving and advecting rain events are hardly classifiable as artificial generated. The model also shows high skill regarding pixel-wise error and localization of high precipitation intensities, considering the FSS, CRPS, KS and RMSE. Due to the underdetermined downscaling problem a probabilistic cGAN approach yields additional information to deterministic models which we use for comparison. The method is also capable of preserving the climatology, e.g., expressed as the annual precipitation sum. Investigations of temporal aggregations of the downscaled fields revealed an interesting effect. We observe that structures generated in networks with convolutional layers are not placed completely at random, but can generate recurrent structures, which can also be discovered within other prominent DL downscaling models. Although they can be mitigated by adequate model selection, their occurrence remains an open research question.

We conclude that our proposed approach can extend the application of cGANs for downscaling to the time dimension and therefore is a promising candidate to supplement conventional downscaling methods due to the high performance and computational efficiency.

How to cite: Glawion, L., Polz, J., Fersch, B., Kunstmann, H., and Chwala, C.: Spatio-temporal downscaling of precipitation data using a conditional generative adversarial network, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-15286, https://doi.org/10.5194/egusphere-egu23-15286, 2023.

Climate variability and extremes are known to represent major causes for crop yield anomalies. They can lead to the reduction of crop productivity, which results in disruptions in food availability and nutritional quality, as well as in rising food prices. Extreme climates will become even more severe as global warming proceeds, challenging the achievement of food security. These extreme events, especially droughts and heat waves, are already evident in major food-production regions like the United States. Crops cultivated in this country such as corn and soybean are critical for both domestic use and international supply. Considering the sensitivity of crops to climate, here we present a dataset that couples remote sensing surface reflectances with climate variables (e.g. minimum and maximum temperature, precipitation, and vapor pressure) and extreme indicators. The dataset contains the crop yields of various commodities over the USA for nearly two decades. Given the advances and proven success of machine learning in numerous remote sensing tasks, our dataset constitutes a benchmark to advance the development of novel models for crop yield prediction, and to analyze the relationship between climate and crop yields for gaining scientific insights. Other potential use cases include extreme event detection and climate forecasting from satellite imagery. As a starting point, we evaluate the performance of several state-of-the-art machine and deep learning models to form a baseline for our benchmark dataset.

How to cite: Höhl, A., Ofori-Ampofo, S., Obadic, I., Fernández-Torres, M.-Á., Salih Kuzu, R., and Zhu, X.: USCC: A Benchmark Dataset for Crop Yield Prediction under Climate Extremes, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-15540, https://doi.org/10.5194/egusphere-egu23-15540, 2023.

Atmospheric conditions that are typical for lightning are commonly represented by proxies such as cloud top height, cloud ice flux, CAPE times precipitation, or the lightning potential index. While these proxies generally deliver reasonable results, they often need to be adapted for local conditions in order to perform well. This suggests that there is a need for more complex and holistic proxies. Recent research confirms that the use of machine learning (ML) approaches for describing lightning is promising.

In a previous study a deep learning model was trained on single spatiotemporal (30km x 30km x 1h) cells in the summer period of the years 2010--2018 and showed good results for the unseen test year 2019 within Austria. We now improve this model by using multiple neighboring vertical atmospheric columns to also address for horizontal moisture advection. Furthermore data of successive hours is used as input data to enable the model to capture the temporal development of atmospheric conditions such as the build-up and breakdown of convections.

In this work we focus on the summer months June to August and use data from parts of Central Europe. This spatial domain is thought to be representative for Continental Europe since it covers mountainous aswell as coastal regions. We take raw ERA5 parameters beyond the tropopause enriched with a small amount of meta data such as the day of the year and the hour of the day for training. The quality of the resulting paramaterized model is then evaluated on Continental Europe to examine the generalization ability.

Using parts of Central Europe to train the model, we evaluate its ability to generalize on unseen parts of Continental Europe using EUCLID data. Having a model that generalizes well is a building block for a retrospective analysis back into years where the structured recording of accurate lightning observations in a unified way was not established yet.

How to cite: Ehrensperger, G., Hell, T., Mayr, G. J., and Simon, T.: Evaluating the generalization ability of a deep learning model trained to detect cloud-to-ground lightning on raw ERA5 data, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-15817, https://doi.org/10.5194/egusphere-egu23-15817, 2023.

Atmospheric environments favorable for lightning and convection are commonly represented by proxies or parameterizations based on expert knowledge such as CAPE, wind shears, charge separation, or combinations thereof. Recent developments in the field of machine learning, high resolution reanalyses, and accurate lightning observations open possibilities for identifying tailored proxies without prior expert knowledge. To identify vertical profiles favorable for lightning, a deep neural network links ERA5 vertical profiles of cloud physics, mass field variables and wind to lightning location data from the Austrian Lightning Detection & Information System (ALDIS), which has been transformed to a binary target variable labelling the ERA5 cells as lightning and no lightning cells. The ERA5 parameters are taken on model levels beyond the tropopause forming an input layer of approx. 670 features. The data of 2010 - 2018 serve as training/validation. On independent test data, 2019, the deep network outperforms a reference with features based on meteorological expertise. Shapley values highlight the atmospheric processes learned by the network which identifies cloud ice and snow content in the upper and mid-troposphere as relevant features. As these patterns correspond to the separation of charge in thunderstorm cloud, the deep learning model can serve as physically meaningful description of lightning. 

How to cite: Hell, T., Ehrensperger, G., Mayr, G. J., and Simon, T.: Identifying Lightning Processes in ERA5 Soundings with Deep Learning, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-16098, https://doi.org/10.5194/egusphere-egu23-16098, 2023.

Unsupervised classification is becoming an increasingly common method to objectively identify coherent structures within both observed and modelled climate data. However, the user must choose the number of classes to fit in advance. Typically, a combination of statistical methods and expertise is used to choose the appropriate number of classes for a given study, however it may not be possible to identify a single ‘optimal’ number of classes. In this
work we present a heuristic method for determining the number of classes unambiguously for modelled data where more than one ensemble member is available. This method requires robustness in the class definition between simulated ensembles of the system of interest. For demonstration, we apply this to the clustering of Southern Ocean potential temperatures in a CMIP6 climate model, and compare with other common criteria such as Bayesian Information Criterion (BIC) and the Silhouette Score.

How to cite: Boland, E., Jones, D., and Atkinson, E.: A comparison of methods for determining the number of classes in unsupervised classification of climate models, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-16163, https://doi.org/10.5194/egusphere-egu23-16163, 2023.

Recent years have seen substantial performance-improvements of deep-learning-based
weather prediction models (DLWPs). These models cover a large range of temporal and
spatial resolutions—from nowcasting to seasonal forecasting and on scales ranging from
single to hundreds of kilometers. DLWPs also exhibit a wide variety of neural architec-
tures and training schemes, with no clear consensus on best practices. Focusing on the
short-to-mid-term forecasting ranges, we review several recent, best-performing models
with respect to critical design choices. We emphasize the importance of self-organizing
latent representations and inductive biases in DLWPs: While NWPs are designed to sim-
ulate resolvable physical processes and integrate unresolvable subgrid-scale processes by
approximate parameterizations, DLWPs allow the latent representation of both kinds of
dynamics. The purpose of this review is to facilitate targeted research developments and
understanding of how design choices influence performance of DLWPs. While there is
no single best model, we highlight promising avenues towards accurate spatio-temporal
modeling, probabilistic forecasts and computationally efficient training and infer

How to cite: Thümmel, J., Butz, M., and Goswami, B.: A review of deep learning for weather prediction, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-16186, https://doi.org/10.5194/egusphere-egu23-16186, 2023.

How to cite: Baghirov, Z., Kraft, B., Jung, M., Körner, M., and Reichstein, M.: Hybrid machine learning model of coupled carbon and water cycles, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-16443, https://doi.org/10.5194/egusphere-egu23-16443, 2023.

Seasonal Forecasts are critical tools for early-warning decision support systems, that can help reduce the related risk associated with hot or cold weather and other events that can strongly affect a multitude of socio-economic sectors. Recent advances in both statistical approaches and numerical modeling have improved the skill of Seasonal Forecasts. However, especially in mid-latitudes, they are still affected by large uncertainties that can limit their usefulness.

The MSCA-H2020 project ARTIST aims at improving our knowledge of climate predictability at the seasonal time-scale, focusing on the role of unexplored drivers, to finally enhance the performance of current prediction systems. This effort is meant to reduce uncertainties and make forecasts efficiently usable by regional meteorological services and private bodies. This study focuses on seasonal prediction of heat extremes in Europe, and here we present a first attempt to predict heat wave accumulated activity across different target seasons. An empirical seasonal forecast is designed based on Machine Learning techniques. A feature selection approach is used to detect the best subset of predictors among a variety of candidates, and then an assessment of the relative importance of each predictor is done, in different European regions for the four main seasons.

Results show that many observed teleconnections are caught by the data-driven approach, while a few features that show to be linked to the heat wave propensity of a season deserve a deeper understanding of the underpinning physical process.

How to cite: Materia, S., Jung, M., Donat, M. G., and Gomez-Gonzalez, C.: Data-driven seasonal forecasts of European heat waves, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-16449, https://doi.org/10.5194/egusphere-egu23-16449, 2023.

Discovering causal relationships from purely observational data is often not possible. In this case, combining observational and experimental data can allow for the identifiability of the underlying causal structure. In Earth Systems sciences, carrying out interventional experiments is often impossible for ethical and practical reasons. However, “natural interventions”, are often present in the data, and these represent regime changes caused by changes to exogenous drivers. In [3,4], the Invariant Causal Prediction (ICP) methodology was presented to identify the causes of a target variable of interest from a set of candidate causes. This methodology takes advantage of natural interventions, resulting in different cause variables distributions across different environments.  In [2] this methodology is implemented in a geoscience problem, namely identifying the causes of Pyrocumulunimbus (pyroCb), and storm clouds resulting from extreme wildfires. Although a set of plausible causes is produced, certain heuristic adaptations to the original ICP methodology were implemented to overcome some of the practical. limitations of ICP: a large number of hypothesis tests required and a failure to identify causes when these have a high degree of interdependence. In this work, we try to circumvent these difficulties by taking a different approach. We use a learning paradigm similar to that presented in [3] to learn causal representations invariant across different environments. Since we often don’t know exactly how to define the different environments best, we also propose to learn functions that describe their spatiotemporal extent. We apply the resulting algorithm to the pyroCb database in [1] and other Earth System sciences datasets to verify the plausibility of the causal representations found and the environments that describe the so-called natural interventions.. 

[1] Tazi et al. 2022. https://arxiv.org/abs/2211.13052

[2] Díaz et al. 2022 .https://arxiv.org/abs/2211.08883

[3] Arjovsky et al. 2019. https://arxiv.org/abs/1907.02893

[4] Peters et al.2016.  https://www.jstor.org/stable/4482904

[5] Heinze-Deml et al. 2018. https://doi.org/10.1515/jci-2017-0016

How to cite: Díaz, E., Varando, G., Iglesias-Suarez, F., Camps-Valls, G., Tazi, K., Lamb, K., and Watson-Parris, D.: Learning causal drivers of PyroCb, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-16846, https://doi.org/10.5194/egusphere-egu23-16846, 2023.

Climate change indices (CCI) defined by the expert team on climate change detection and indices (ETCCDI) profoundly contribute to understanding climate and its change. They are used to present climate change in an easy to understand and tangible way, thus facilitating climate communication. Many of the indices are peak over threshold indices needing daily and, if necessary, bias corrected data to be calculated from. We present a method to rapidly estimate specific CCI from monthly data instead of daily while also performing a simple bias correction as well as a localisation (downscaling). Therefore, we used the ERA5 Land data with a spatial resolution of 0.1° supplemented by a CMIP6 ssp5-8.5 climate projection to derive different regression functions which allow a rapid estimation by monthly data. Using a climate projection as a supplement in training the regression functions allows an application not only on historical periods but also on future periods such as those provided by climate projections. Nevertheless, the presented method can be adapted to any data set, allowing an even higher spatial resolution.

How to cite: Hasel, K., Bügelmayer-Blaschek, M., and Formayer, H.: A statistical approach on rapid estimations of climate change indices by monthly instead of daily data, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-17082, https://doi.org/10.5194/egusphere-egu23-17082, 2023.

Geoclimatic regions represent climatic forcing zones, which constitute important spatial entities that serve as a basis for a broad range of analyses in earth system sciences. The plethora of geospatial variables that are relevant for obtaining consistent clusters represent a high-dimensionality, especially when working with high-resolution gridded data, which may render the derivation of such regions complex. This is worsened by typical characteristics of geoclimatic data like multicollinearity, nonlinear effects and potentially complex interactions between features. We therefore present a nonparametric machine learning workflow, consisting of dimensionality reduction and clustering for deriving geospatial clusters of similar geoclimatic characteristics. We demonstrate the applicability of the proposed procedure using a comprehensive dataset featuring climatological and geomorphometric data from Austria, aggregated to the recent climatological normal from 1992 to 2021.
 
The modelling workflow consists of three major sequential steps: (1) linear dimensionality reduction using Principal Component Analysis, yielding a reduced, orthogonal sub-space, (2) nonlinear dimensionality reduction applied to the reduced sub-space using Uniform Manifold Approximation and Projection, and (3) clustering the learned manifold projection via Hierarchical Density-Based Spatial Clustering of Applications with Noise. The contribution of the input features to the cluster result is then assessed by means of permutation feature importance of random forest models. These are trained by treating the clustering result as a supervised classification problem. Results show the flexibility of the defined workflow and exhibit good agreement with both quantitatively derived and synoptically informed characterizations of geoclimatic regions from other studies. However, this flexibility does entail certain challenges with respect to hyperparameter settings, which require careful exploration and tuning. The proposed workflow may serve as a blueprint for deriving consistent geospatial clusters exhibiting similar geoclimatic attributes.

How to cite: Lehner, S., Enigl, K., and Schlögl, M.: Machine learning workflow for deriving regional geoclimatic clusters from high-dimensional data, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-17197, https://doi.org/10.5194/egusphere-egu23-17197, 2023.

Here we develop a statistical model emulating the surface temperature response to changes in emissions of short-lived climate forcers as simulated by an Earth system model. Short-lived climate forcers (SLCFs) are chemical components in the atmosphere that interact with radiation and have both an immediate effect on local air quality, and regional and global effects on the climate in terms of changes in temperature and precipitation distributions. The short atmospheric residence times of SLCFs lead to high atmospheric concentrations in emission regions and a highly variable radiative forcing pattern. Regional Temperature Potentials (RTPs) are metrics which quantify the impact of emission changes in a given region on the temperature or forcing response of another, accounting for spatial inhomogeneities in both forcing and the temperature response, while being easy to compare across models and to use in integrated assessment studies or policy briefs. We have developed a Gaussian-process emulator using output from the Norwegian Earth System Model (NorESM) to predict the temperature responses to regional emission changes in SLCFs (specifically back carbon, organic carbon, sulfur dioxide, and methane) and use this model to calculate regional RTPs and study the sensitivity of surface temperature in a certain region, e.g. the Arctic, to anthropogenic emission changes in key policy regions. The main challenge in developing the emulator was creating the training data set such that we included maximal SLCF variability in a realistic and policy relevant range compared to future emission scenarios, while also getting a significant temperature response. We also had to account for the confounding influence of greenhouse gases (GHG), which may not follow the same future emission trajectories as SLCFs and can overwhelm the more subtle temperature response that comes from the direct and indirect effects of SLCF emissions. The emulator can potentially provide accurate and customizable predictions for policy makers to proposed emission changes with minimized climate impact.

How to cite: Dewey, M., Hansson, H. C., and Ekman, A. M. L.: Emulating the regional temperature responses (RTPs) of short-lived climate forcers, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-17333, https://doi.org/10.5194/egusphere-egu23-17333, 2023.

In this work, we show the first achievement of inferring the electron temperature in ionospheric conditions from synthetic data using fixed-bias Langmuir probes operating in the electron saturation region. This was done using machine learning, as well as by altering the probe geometry. The electron temperature is inferred at the same rate as the currents are sampled by the probes. For inferring the electron temperature along with the electron density and the floating potential, a minimum number of three probes is required. Furthermore does one probe geometry need to be distinct from the other two, since otherwise the probe setup may be insensitive to temperature. This can be achieved by having either one shorter probe or a probe of a different geometry, e.g. two longer and a shorter cylindrical probe or two cylindrical probes and a spherical probe. We use synthetic plasma parameter data and calculate the synthetic collected probe currents to train a neural network (using TensorFlow) and verify the results with a test set as well as with data from the International Reference Ionosphere (IRI) model. A table with computed currents collected by a spherical probe by Laframboise was extended to calculate currents of the synthetic plasma parameters for high eta values (eta >25) to cover a large altitude range (100-500 km, within Earth's ionosphere). The extrapolated values were benchmarked with Particle-in-Cell simulations. Finally, we evaluate the robustness and errors of different probe setups that can be used to infer the electron temperature. As the inferred temperatures are compared to results from the International Reference Ionosphere model, we verify the validity of the inferred temperature in altitudes ranging from about 100-500 km. We show that electron temperature inference from different combinations of spherical and cylindrical probes - three cylindrical probes, three spherical probes, four cylindrical and a spherical probe - can be achieved. Even minor changes in the probe sizing enable the temperature inference and result in root mean square relative errors (RMSRE) between inferred and ground truth data of under 3%. With further optimizations, the RMSRE can even be decreased to under 1%. When limiting the temperature inference to 120-450 km altitude an RMSRE of under 0.7% is achieved for all probe setups. In future, the multi-needle Langmuir Probe (m-NLP) instrument dimensions can be adapted for higher temperature inference accuracy.

How to cite: Enengl, F., Marholm, S., Adhikari, S., Marchand, R., and Miloch, W. J.: Electron Temperature Inference from Fixed Bias Langmuir Probes Set-Ups in Ionospheric Conditions, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-542, https://doi.org/10.5194/egusphere-egu23-542, 2023.

Active-region emergence on the Sun shows a degree of clumpiness in both space and time. At a given time, multiple active regions can be seen in what is called active-region- or sunspot-group nests. This tendency also increases the potential to produce large flares and associated CMEs. In the literature, the nesting tendency of active regions is reported in the range of 30-50 per cent, but no statistically robust and ML-based approaches exist so far. Quantifying the nesting degree along an activity cycle and determining its spatial and temporal scales are important to investigate the processes that cause this phenomenon. 

In this study, we estimate the latitudinal and longitudinal extents of active region nesting using both continuum and magnetogram data, using SDO/HMI synoptic magnetograms and Kislovodsk Mountain Astronomical Station (KMAS) sunspot group data. We carry out kernel density estimation (Fig. 1) and unsupervised ML techniques (e.g., DBSCAN and Gaussian mixtures) in spatial and spatio-temporal domains. Our study reveals trends in the emergence characteristics of sunspot groups on the Sun.

Figure 1: Kernel density estimation with a Gaussian kernel on the time-longitude plane. The dot size indicates sunspot group areas in MSH. 

How to cite: Isik, E., Karapinar, N., and Cankurtaran, S. G.: Unsupervised learning of active-region nesting on the Sun, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-850, https://doi.org/10.5194/egusphere-egu23-850, 2023.

A PCA-NN model for the total electron content (TEC) for the midlatitudinal region (Iberian Peninsula) presented here uses the principal component analysis (PCA) to decompose TEC variations into different modes and to reconstruct/forecast amplitudes of these modes using neural networks (NN) with different sets of space weather parameters as predictors.

Feedforward, convolutional and recurrent NN algorithms are tested with different sets of predictors. The performance of the models is tested on 3.5 years of observational data obtained at the declined phase of the 24^th solar cycle, which allows us to estimate the models’ performance in relation to the solar activity level. The advantages and disadvantages of different NN algorithms are discussed.

How to cite: Morozova, A., Gafeira, R., Barata, T., and Barlyaeva, T.: Different types of PCA-NN model for TEC with space weather parameters as predictors: advantages and disadvantages of different NN algorithms, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-2719, https://doi.org/10.5194/egusphere-egu23-2719, 2023.

To understand the solar evolution and effects of solar eruptive events, the Sun is permanently observed by multiple satellite missions. The optically-thin emission of the solar plasma and the limited number of viewpoints make it challenging to reconstruct the geometry and structure of the solar atmosphere; however, this information is the missing link to understand the Sun as it is: a three-dimensional, evolving star. We present a method that enables a complete 3D representation of the uppermost solar layer observed in extreme ultraviolet (EUV) light. We use a deep learning approach for 3D scene representation that accounts for radiative transfer, to map the entire solar atmosphere from three simultaneous observations. We demonstrate that our approach provides unprecedented reconstructions of the solar poles, and directly enables height estimates of coronal structures, solar flux ropes, coronal hole profiles, and coronal mass ejections. We validate the approach using model-generated synthetic EUV images, finding that our method accurately captures the 3D geometry even from a limited number of viewpoints. We quantify uncertainties of our model using an ensemble approach that allows us to estimate the model performance in absence of a ground-truth. Our method enables a novel view of our closest star, and is a breakthrough technology for the efficient use of multi-instrument datasets, which paves the way for future cluster missions.

How to cite: Jarolim, R., Tremblay, B., Munoz-Jaramillo, A., Bintsi, K.-M., Jungbluth, A., Santos, M., Mason, J. P., Sundaresan, S., Downs, C., Caplan, R., and Vourlidas, A.: SuNeRF: AI enables 3D reconstruction of the solar EUV corona, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-2756, https://doi.org/10.5194/egusphere-egu23-2756, 2023.

Dynamic interactions between the solar wind and the magnetosphere give rise to dramatic auroral forms that have been instrumental in the ground-based study of magnetospheric dynamics. The general mechanism of aurora types and their large-scale patterns are well-known, but the morphology of small- to meso-scale auroral forms observed in all-sky imagers and their relation to magnetospheric dynamics  and the coupling of the magnetosphere to the upper atmosphere remain in question. Machine learning has the potential to provide answers to these questions, but most existing auroral image data lack the ground-truth labels required for supervised learning and conventional statistical analyses. To mitigate this issue, we propose a novel self-supervised semi-supervised algorithm to automatically label the THEMIS all-sky image database. Specifically, we adapt the self-supervised Simple framework for Contrastive Learning of Representations (SimCLR) algorithm to learn latent representations of THEMIS all-sky images. These representations are finetuned using a small set of manually labeled data from the Oslo Aurora THEMIS (OATH) dataset, after which semi-supervised classification is used to train a classifier, beginning by training on the manually labeled OATH dataset and gradually incorporating the classifier’s most confident predictions on unlabeled data into the training dataset as ground-truth. We demonstrate that (a) classifiers fit to the learned representations of the manually labeled images achieve state–of–the–art performance, improving the classification accuracy by almost 10% over the current benchmark on labeled data; and (b) our model’s learned representations naturally cluster into more clusters than manually assigned categories, suggesting that existing categorizations are coarse and may obscure important connections between auroral types and their drivers. Finally, we introduce AuroraClick, a citizen science project with the goal of manually annotating a large representative sample of THEMIS all-sky images for the validation of our current models and the training of future models.  

How to cite: Johnson, J., Ozturk, D., Connor, H., Hampton, D., Blandin, M., and Keesee, A.: Automatic Classification of THEMIS All-Sky Images via Self-Supervised Semi-Supervised Learning, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-2897, https://doi.org/10.5194/egusphere-egu23-2897, 2023.

Having precise knowledge of the near-Earth solar wind (SW) and the embedded interplanetary magnetic field (IMF) is of critical importance to space weather operation due to the usage of SW and IMF in almost all magnetospheric and ionospheric models. The most widely used data source, OMNI, propagates SW properties from Lagrangian point L1 to the Earth’s bow shock by estimating the propagation time of the SW. However, the time difference between OMNI timeshifted IMF and the best match-up of IMF can reach ˜15 min. Firstly, we aim to develop an improved statistical algorithm to contribute to the SW propagation delay problem of space weather prediction. The algorithm focuses on matching SW features around the L1 point and upstream of the bow shock by computing the variance, cross-correlation coefficient, the plateau-shaped magnitude index, and the non-dimensional measure of average error index between the measurements at the two locations. The obtained propagation times are then compared to OMNI. Factors that limit the OMNI accuracy are also examined. Secondly, the automatic algorithm allows us to generate large sets of input and target variables using multiple spacecraft pairs at L1 and near-Earth locations to train, validate, and test machine learning models to specify and forecast near-Earth SW conditions. Finally, we offer a machine learning (ML) approach to specify and predict the propagation time from L1 monitors to a given location upstream or at the bow shock and forecast near-Earth SW conditions with the gradient boosting and random forest prediction models in the form of an ensemble of decision trees.

How to cite: Tasnim, S., Zou, Y., Borries, C., Baumann, C., Walsh, B., Khanal, K., O'Brien, C., and Zhang, H.: Estimation and Prediction of Solar Wind Propagation from L1 Point to Earth’s Bow Shock, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-3379, https://doi.org/10.5194/egusphere-egu23-3379, 2023.

Abstract: Plasma-sheet bubbles play a major role in the process of magnetotail particle injections. They are defined as fast flows with reduced plasma density or pressure accompanied by magnetic field dipolarization. Typically, we can detect these bubbles from in-situ observations, but subjective uncertainty needs human verification. In this study, we combine three different methods including MINImally RandOm Convolutional KErnel Transform (MINIROCKET), 1D and 2D convolution neural network (CNN) to identify bubbles. The imbalanced training dataset consists of bubble and non-bubble events with a ratio of 1:40 from year 2007 to 2020. The results indicate that the accuracy of the all three models is around 99%, and the precision and recall rates of all three models are above 80% in both the validation and test datasets. The three methods are combined with the intersection set as the minimum set of predictions and the union set as the maximum set. The methods greatly reduce the number of false positives. To identify bubbles in the observations of year 2021, our neural network model is found to be comparably good to the traditional criterial and manual inspections. Using joint machine learning forecasting methods, we can easily and automatically identify bubbles without a priori knowledge like a domain expert.

Keywords: plasma-sheet bubble, multivariate time series classification, sample imbalanced, image identification

How to cite: Xuedong, F. and Jian, Y.: Plasma-Sheet Bubble Identification Using Muitivariate Time Series Classification, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-4069, https://doi.org/10.5194/egusphere-egu23-4069, 2023.

Laser altimeters create large amounts of data that often have to be preprocessed and checked before further use. The BepiColombo mission to Mercury is set to arrive in December 2025 and observations with the BepiColombo Laser Altimeter (BELA, (Benkhoff et al., 2010; Thomas et al., 2021)) will start during the following spring. These measurements are planned to be used to derive information about the tides of Mercury (Thor et al., 2020). Careful assessment, selection, and filtering on the raw data is needed to extract the small tidal signal. Until the BELA data becomes available artificial data and records from other missions have to be used to study the data selection strategy. We present our work on MESSENGER Laser Altimeter (MLA, (Cavanaugh et al., 2007)) using a convolutional neural network to sort observations on an orbit by orbit basis into different classes. The already existing neural network (Stenzel and Hilchenbach, 2021; Stenzel, Thor and Hilchenbach, 2021) is tuned and a new test data set is created.

How to cite: Stenzel, O., Maes, L., and Hilchenbach, M.: AI Assisted Data Selection of Laser Altimeter Observations, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-5254, https://doi.org/10.5194/egusphere-egu23-5254, 2023.

One of the main sources of solar wind disturbances are coronal holes which can be identified in extreme ultra-violet (EUV) images of the Sun. Previous research has shown the connection between coronal holes and an increase of the solar wind speed at Earth. The time lag between the appearance of coronal holes on the visible side of the Sun and its effects on Earth is 2-5 days. In this study, a machine learning model predicting the solar wind speed originating from coronal holes is proposed. It is based on the analysis of solar EUV images. A segmentation algorithm is applied to the images in order to identify coronal holes and derive their characteristics (e.g. area, location). We also present a new method to calculate the geoeffective coronal hole area: Instead of specifying in advance a sector of the solar surface in which the area is measured and a lag time between area measurement and the arrival of the solar wind, the specification of this sector and the corresponding delay are formulated as a mathematical optimization problem and included in the machine learning model. This approach facilitates an improvement of the prediction accuracy and also prolongs the prediction horizon, as the solar wind speed can be predicted up to approximately 5 days in advance of the disturbance. Several machine learning model architectures are explored. We also study how the time evolution can be included in the model.

How to cite: Collin, D., Bianco, S., Gallego, G., and Shprits, Y.: Forecasting solar wind speed by machine learning based on coronal hole characteristics, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-6968, https://doi.org/10.5194/egusphere-egu23-6968, 2023.

The detection and classification of landforms on planetary surfaces is a time-consuming task which deeply relies on expert knowledge. Such a process can be partially automated and optimized in a resource-efficient way using image processing algorithms. By classifying the surface into different landforms, such as volcanic craters, asteroid impacts, dunes, and more, several analyses can be performed, for instance the widely used crater counting age estimation method. In addition, by conducting these analyses, information about the characteristics and properties of a planet can be revealed. One of the major challenges for the implementation of these algorithms is to provide a generalized model. In many cases the generalization error tends to be very large and therefore a satisfactory accuracy on the test data set cannot be accomplished. This prevents reliable evaluation of new unseen data. In this work, a multi-class image segmentation algorithm is presented, which is based on a U-net convolutional neural network architecture. U-nets classify each pixel of a given input image and can thus produce segmentation masks for various landforms. Given that enough labeled data is available, such a classifier can replace manual detection and classification, thereby saving resources by providing a fast method for landform detection.

How to cite: Auer-Welsbach, F., Windisch, A., and Nodjoumi, G.: Landform detection on Mars using image segmentation with a u-net convolutional neural network architecture, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-7529, https://doi.org/10.5194/egusphere-egu23-7529, 2023.

Deep-learning methods are of interest for the analysis of imagery and digital elevation models from Mars orbiting satellites. They detect various atmosphere and surface characteristics. For instance, these include dust storms and craters [1,2]. We approach this topic by using the deep-learning-based crater detection algorithm DeepMars2 [3,4]. The algorithm is applied to two digital elevation models (DEMs) of the Mars surface. The DEMs are based on the satellite instruments MOLA/MGS (Mars Orbiter Laser Altimeter/Mars Global Surveyor) and HRSC/MEX (High Resolution Stereo Camera/Mars Express) and have different resolution. Crater detection statistics are compared between both DEMs.

[1] Alshehhi, R., Gebhardt, C. Detection of Martian dust storms using mask regional convolutional neural networks. Prog Earth Planet Sci 9, 4 (2022). https://doi.org/10.1186/s40645-021-00464-1

[2] R. Alshehhi and C. Gebhardt, "Automated Geological Landmarks Detection on Mars Using Deep Domain Adaptation From Lunar High-Resolution Satellite Images," in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 15, pp. 2274-2283, 2022, doi: 10.1109/JSTARS.2022.3156371.

[3] Lee, C. (2019). Automated crater detection on Mars using deep learning. Planetary and Space Science, 170, 16-28. https://doi.org/10.1016/j.pss.2019.03.008

[4] Lee, C. & Hogan, J. (2021). Automated crater detection with human level performance. Computers & Geosciences, 147, 104645. https://doi.org/10.1016/j.cageo.2020.104645

How to cite: AlRiyami, H., Gebhardt, C., and Lee, C.: Comparison study on the deep-learning-based detection of Mars craters, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-7761, https://doi.org/10.5194/egusphere-egu23-7761, 2023.

Machine learning models trained to reproduce space mission observations are precious resources to fill gaps of missing data in measurement time series or to perform data forecasting within a reasonable uncertainty degree. The latter option is of particular importance for future space missions that will not host instrumentation dedicated to interplanetary medium parameter monitoring. The future LISA mission for low-frequency gravitational wave detection, for instance, will benefit of particle detectors to measure the galactic cosmic-ray integral flux variations and magnetometers that will allow to monitor the passage of large scale magnetic structures through the three LISA spacecraft as part of a diagnostics subsystem. Unfortunately, no instruments dedicated to solar wind speed measurements will be present on board the spacecraft constellation. Moreover, LISA, scheduled to launch in 2035, will trail Earth on the ecliptic at 50 million km distance, far from the orbits of other space missions dedicated to the interplanetary medium monitoring.

Based on precious lessons learned with LISA Pathfinder, the ESA LISA precursor mission, about the correlation between galactic cosmic-ray flux short-term variations and solar wind speed increases, we built a machine learning ensemble model able to reconstruct the solar wind trend only on the basis of contemporaneous and preceding observations of galactic cosmic-ray flux variations. Details about the model creation and performance will be presented, together with a description of the underlying data set, weak predictors and training phase. Advantages and limitations will be discussed, showing that the model performance may be enhanced by providing interplanetary magnetic field intensity observations as additional input data, with the goal of providing the LISA mission with an effective solar wind speed predictive tool.

How to cite: Sabbatini, F. and Grimani, C.: Machine learning ensemble models for solar wind speed prediction, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-7941, https://doi.org/10.5194/egusphere-egu23-7941, 2023.

As is known, neural networks can universally approximate any complex functions. This ground truth naturally makes it a suitable candidate for solution representation of complex partial differential equation (PDE) governed. For planetary magnetic field modelling problem, spherical harmonic functions are most used as standard modelling method. Spherical harmonic method requires globally nearly uniformly distributed observations. Meanwhile this method has quite limited ability for conducting regional field modelling. Instead, neural networks have great potential to deal with global or regional modelling problems. In this work, we thoroughly investigate the representative ability of neural networks for magnetic field modelling problem at global and regional scale, and concentrate on a specific neural network, that is physics-informed neural networks (PINNs) for implementation. PINNs makes it easier to incorporate different kinds of informed physics within a uniform optimization framework. Through synthetic model tests and partial mathematical proof, we showcase the importance of employing natural boundary condition, Laplace equation constraint and Poisson equation constraint at suitable collocation points for a reasonable and accurate magnetic field representation and introduce the detailed scheme for implementation. Finally, we use newly released Juno mission measurements, and present a global PINNs model for Jupiter's magnetic field, and a regional PINNs model for Great Blue Spot (GBS) region. Comparison with spherical harmonic model has been conducted to evaluate the correctness and flexibility of PINNs models.

How to cite: Chen, L., Livermore, P., Wu, L., de Ridder, S., and Zhang, C.: Modelling Jupiter's global and regional magnetic fields using physics-informed neural networks, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-8430, https://doi.org/10.5194/egusphere-egu23-8430, 2023.

To accurately predict potential future impacts with the Earth, it is crucial to continuously examine the area around it for Near Earth Objects (NEOs) and particularly Near Earth Asteroids (NEAs). Large data sets of astronomical images must be analyzed in order to accomplish this task. NEARBY [1] offers such a processing and analysis platform based on Cloud computing. Despite the fact that this method is automated, the results are validated by human observers after potential asteroids have been identified from the raw data. It is crucial that the amount of candidate objects does not outweigh the available human resources. We believe we can maximize the advantages of having access to enormous amounts of data in the field of astronomy by combining artificial intelligence with the use of high-performance distributed processing infrastructures like Cloud-based solutions. This research is carried out as part of the CERES project which aims to design and put into practice a software solution that can classify objects found in astronomical images. The objective is to identify and recognize asteroids. We use machine learning techniques to develop an asteroid classification model in order to achieve this goal. It is essential to reduce the number of false negative findings. The major objective of the current paper is to assess how well deep CNNs perform when it comes to categorizing astronomical objects, particularly asteroids. We will compare the outcomes of several of the most well-known deep convolutional neural networks (CNNs), including InceptionV3, Xception, InceptionResNetV2, and ResNet152V2. These cutting-edge classification CNNs are used to investigate the best approach to this specific classification challenge, either through full-training or through fine-tuning.

Acknowledgment: This work was partially supported by a grant of the Romanian Ministry of Education and Research, CCCDI - UEFISCDI, project number PN-III-P2-2.1-PED-2019-0796, within PNCDI III. This research was partially supported by the project 38 PFE in the frame of the programme PDI-PFE-CDI 2021.

References:

1. Bacu, V., Sabou, A., Stefanut, T., Gorgan, D., Vaduvescu, O., NEARBY platform for detecting asteroids in astronomical images using cloud-based containerized applications, 2018 IEEE 14th International Conference on Intelligent Computer Communication and Processing (ICCP), pp. 371-376

How to cite: Bacu, V.: Software solution for detecting asteroids using machine learning techniques, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-8676, https://doi.org/10.5194/egusphere-egu23-8676, 2023.

To monitor the results of our instrument on a daily basis, we create a series of daily plots that are generated in an automated fashion.  In our case, we are creating two plot types for four spacecraft for five different species.  Unfortunately, due to circumstances beyond our control (primarily network and system issues), plots were failing and if not monitored daily, they were unavailable when finally needed.  

To solve this problem, we investigated using Computer Vision (OpenCV) to validate our generation of daily plots.  It was surprisingly easy and more advantageous than trying to either monitor it daily or more simplistic methods.  By using the cloud, we were able to improve throughput as well.  Future work would be to use Computer Vision to analyze the data within the plots for actual scientific study.

How to cite: Mukherjee, J.: Using Artificial Intelligence/Computer Vision for Automated Plot Validation, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-8946, https://doi.org/10.5194/egusphere-egu23-8946, 2023.

Coronal mass ejections (CMEs) are one of the most violent solar eruptions, which can burst out large amounts of magnetized plasma with speeds up to thousands of kilometers per second. When it reaches the Earth, a CME can cause geomagnetic storm, affecting aviation safety, satellite operations, communications systems and power facilities. Therefore, fast and accurate prediction of CME arrival time is crucial for avoiding severe damaging effects and reducing economic losses. The initial morphology and kinematics of a CME in the corona can be observed by the coronagraphs equipped on the Solar and Heliospheric Observatory (SOHO), so that the coronagraphs should be useful to predict the CME arrival times. In this study, convolutional neural network (CNN) is used to obtain the features of SOHO/LASCO coronagraph pictures related to the CME transit time, and establish a model capable of predicting the CME arrival time. The influence of different hyperparameters of CNN on the prediction results is studied. Further, we add a physical information constraint of the initial velocities of CME to the basic CNN outputs, and found that smaller prediction errors can be obtained. 

How to cite: Yang, Y., Shen, F., Li, Y., and Lin, R.: Predicting the 1 AU Arrival Time of Coronal Mass Ejections Based on Convolutional Neural Network, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-10654, https://doi.org/10.5194/egusphere-egu23-10654, 2023.

Solar eruptive events are complex phenomena, which most often include solar flares, filament eruptions, coronal mass ejections (CMEs), and CME-driven shock waves. CME-driven shocks in the corona and interplanetary space are considered to be the main producer of solar energetic particles (SEPs). A number of fundamental questions remain about how SEPs are produced. Current understanding points to CME-driven shocks and compressions in the solar corona.

A CME kinematics shows three phases - an initial rising phase (weakly accelerated motion), an impulsive phase and a residual propagation phase with constant or decreasing speed.

Despite significant amount of data available from ground-based (COSMO K-Cor, LOFAR) and remote instruments onboard of heliospheric space missions (SDO AIA, SOHO), processing of the data still requires noticeable effort. Most algorithms currently used in solar feature detection and tracking are known for their limited applicability and complexity of their processing chains, while usage of data-driven approaches for tracking of CME-related phenomena is currently limited due to insufficiency of training sets.

Recently (Stepanyuk et.al, J. Space Weather Space Clim. Vol 12, 20(2022)), we have demonstrated the method and the software(https://gitlab.com/iahelio/mosaiics/wavetrack) for smart characterization and tracking of solar eruptive features based on the a-trous wavelet decomposition technique, intensity rankings and a set of filtering techniques. In this work we use Wavetrack to generate training sets for data-driven feature extraction and characterization. We utilize U-Net, a fully convolutional network which training strategy relies on the strong use of data augmentation to use the available annotated samples more efficiently. U-NET can be trained end-to-end from a very limited set of images, while feature engineering allows to improve this approach even further by expanding available training sets.

Here we present pre-trained models and demonstrate data-driven characterization and tracking of solar eruptive features on a set of CME-events.

How to cite: Stepanyuk, O. and Kozarev, K.: Advanced Multi-Instrument and Multi-Wavelength Image Processing and Feature Tracking for Remote CME Characterization with Convolutional Neural Network, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-10705, https://doi.org/10.5194/egusphere-egu23-10705, 2023.

We present the analysis of a 2.06 Ga apatite crystal obtained from an ultramafic phoscorite rock from the Phalaborwa Complex (Limpopo Province, South Africa) [1]. A space-prototype laser ablation ionisation mass spectrometer (LIMS) [2,3] was used to study the chemical composition of the sample. Mass spectra were recorded from a sample area of 0.6x0.6 mm², with a spatial resolution of 30 μm and sub-micrometre depth resolution.

Apatite is a calcium phosphate mineral expressed by the chemical stoichiometric formula [Ca₅(PO₄)₃(F, Cl, OH)]. The halogen site, occupied by F, Cl, and OH, corresponds to an isomorpous series with fluor-, chlor- and hydroxyl-apatite end members, respectively. Apatite, being an accessory mineral in igneous and other rocks, commonly contains a range of other elements that do not fit well into the major rock forming minerals, such as rare earth elements (REE). These are suitable targets for investigating physical and chemical conditions in igneous rocks and the volatile evolution of magmas.

The analysis of the spectra recorded with our LIMS system for the abundances of the elements of interest at each location were performed in two steps. First, the abundances of each element across the sampled area were compiled in element maps. And second, an unsupervised machine learning algorithm based on clustering and network analysis was applied to the data set of analysed mass spectra to separate it into groups of distinct chemical composition. Subsequently, a more detailed analysis was conducted on each of the recovered groups to assign the corresponding mineral. In addition to the group of spectra belonging to apatite, which was assigned to fluorapatite, other minerals were identified, amongst others olivine. This method yields an unsupervised approach to identify different mineralogical entities present within a sample. This network analysis method was previously applied to a 1.88 Ga Gunflint sample (Ontario, Canada) to separate spectra recorded from the host (chert) from spectra containing signatures of organic matter from fossilized microbes [4].

Given that the data were recorded using a miniature mass spectrometer designed for space flight, this analysis demonstrates the analytical capabilities of our LIMS system that could be achieved in-situ on other planetary bodies in our Solar System, for example on the Moon or on Mars. The current performance of this miniature LIMS instrument to study the chemical composition of apatite is sufficiently high to measure volatiles (H, F, Cl) and nearly all relevant mineral and partially trace elements (Na, C, Mg, Si, S, K, Mn, Fe, Sr, Ba), including REE (La, Ce, Pr, Sm) which allows for a systematic quantitative analysis of their distribution.

[1] Tulej, M. et al., 2022, https://doi.org/10.3390/universe8080410.

[2] Riedo, A. et al., 2012, https://doi.org/10.1002/jms.3104.

[3] Tulej, M. et al., 2021, https://doi.org/10.3390/app11062562.

[4] Lukmanov, R.A. et al., 2022, https://doi.org/10.3389/frspt.2022.718943

How to cite: Gruchola, S., Tulej, M., Keresztes Schmidt, P., Lukmanov, R., Riedo, A., and Wurz, P.: Composition Analysis of an Apatite Crystal using a Space-Prototype Mass Spectrometric Instrument and Machine Learning for Unsupervised Mineralogical Phase Detection, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-11898, https://doi.org/10.5194/egusphere-egu23-11898, 2023.

The origin of cold materials identified by different criteria is unclear. They are highly suspected to be the erupted prominence. However, some cold materials defined by charge depletion exist in both solar wind and ICMEs. Recently, solar observations show failed prominence eruption in CMEs that it did not propagate into the interplanetary space. Besides, the related prominence eruptions of the earth-directed ICMEs at 1 au are difficult to identify before the launch of STEREO mission. This work uses Random Forest (RF) that is an interpretable classifier of supervised machine learning to study the distinct signatures of prominence cold materials (PCs) compared to quiet solar wind (SW) and ICMEs. 12 parameters measured by ACE at 1 au are used in this study, which are proton moments, magnetic field component Bz, He/H, He/O, Fe/O, mean charge of oxygen and carbon, C⁶⁺/C⁵, C⁶⁺/C⁴⁺, and O⁷⁺/O⁶⁺. According to the returned weights from RF classifier and the training accuracy from one black box classifier, the most important in situ signatures of PCs are obtained. Next, the trained RF classifier is used to check the category of the origin-unknown cold materials in ICMEs. The results show that most of the cold materials are from prominence, but 2 of them are possibly from quiet solar wind. The most distinct signatures of PCs are lower charges of C and O, proton temperature, and He/O. This work provides quantitative evidence for the charges of C and O being most effective solid criteria. Considering the obvious overlaps on key parameters between SW, ICMEs, and PCs, multi-parameter classifier of machine learning show an advantage in separating them than solid criteria.

How to cite: Meng, S., Yao, S., and Cheng, Z.: Key Signatures of Prominence Materials and Category of Unknown-origin Cold Materials identified by Machine Learning Classifier, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-12354, https://doi.org/10.5194/egusphere-egu23-12354, 2023.

Pits, or pit craters, are roughly circular depressions found in planetary surfaces which are generally formed through gravitational collapse. Pits will be primary targets for future space exploration and habitability for their presence on most rocky Solar System surfaces and their potential to be entrances to sub-surface cavities. This is particularly true on the Moon and Mars where future astronauts will also be exposed to high radiation dosages whilst on the surface. However, since pits are rarely found to have corresponding high-resolution elevation data, tools are required for approximating their depths in order to find those which are the ideal candidates for exploration and habitation.

We develop a tool that automatically calculates a pit’s apparent depth – the depth at the edge of its shadow - by measuring the shadow’s width as it appears in satellite imagery. The tool can produce a profile of the apparent depth along the entire length of the shadow, using just one cropped single- or multi-band image of a pit. Thus, allowing for the search for possible cave entrances to continue where altimetry or stereo image data is not available. Shadows are automatically extracted using k-means clustering with silhouette analysis for automatic cluster validation. We will present the results of testing the shadow extraction upon shadow-labelled Mars Reconnaissance Orbiter HiRISE imagery of Martian pits, as well as the findings of applying the tool to HiRISE images of Atypical Pit Craters (APCs) from the Mars Global Cave Candidate Catalog (MGC³) [1]. We will also present preliminary results of applying our tool to Lunar Reconnaissance Orbiter Narrow Angle Camera data taken of Lunar pits catalogued in the Lunar Pit Atlas [2].

[1] – Cushing et al. (2015). Atypical pit craters on Mars: New insights from THEMIS, CTX, and HiRISE observations, Journal of Geophysical Research: Planets, 120, 1023–1043

[2] – Wagner & Robinson (2021). Occurrence and Origin of Lunar Pits: Observations from a New Catalog, in 52nd Lunar and Planetary Science Conference, Lunar and Planetary Science Conference, p. 2530

How to cite: Le Corre, D., Mason, N., Bernard-Salas, J., Cox, N., and Mary, D.: Automatically Calculating Depths of Martian and Lunar Pits with Satellite Imagery, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-14927, https://doi.org/10.5194/egusphere-egu23-14927, 2023.

The BepiColombo mission will arrive at Mercury in 2025. It consists of two spacecraft, which both have a magnetometer on board. One of the science objectives of these instruments is to study the structure of Mercury’s magnetosphere and its dynamical interaction with the solar wind. To study this statistically, a large dataset of observations of the magnetopause (the magnetosphere’s outer boundary) is needed. However, identifying such magnetopause crossings in magnetic field data requires visual inspection by humans with expert knowledge and as such is a very time consuming process. We therefore design an algorithm to automatically detect the Hermean magnetopause in magnetometer time series data, making use of a convolutional neural network.

Since no BepiColombo data (in orbit) is available yet, we train the network on MESSENGER magnetometer data. However, we formulate the problem and design the architecture of the network in such a way that the algorithm should be easily transferable to BepiColombo magnetometer data, avoiding the possible impact of any instrumental particularities or orbital biases.

The goal is to have a neural network which is directly applicable to BepiColombo magnetometer data, as soon as the observations start and without any further training, thereby eliminating the necessity of manually creating a new dataset of BepiColombo magnetopause crossings.

How to cite: Maes, L., Fraenz, M., and Heyner, D.: Detecting the magnetopause of Mercury by neural network — using MESSENGER data to train for BepiColombo., EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-15160, https://doi.org/10.5194/egusphere-egu23-15160, 2023.

Aiming to unravel the astrobiology of Mars, the Perseverance mission came with a lot of unknowns. With the surface level knowledge that we have already known, The High Resolution Imaging Science Experiment (HiRISE) can already determine the observation or experimental sites through the images generated from the orbiter. Although the resolution is high, with the power of a 1-meter-size object determinator, we can always expect so much more from the ground-level observation.

The Mars Perseverance rover is equipped with a pair of Mastcam-Z set cameras that are equipped in a manner to simulate the human eye for depth determination in image processing. The instruments can process stereo colour images of the ground level. These images can be used to make detailed maps of the Mars surface scenery at ground level with high precision.

Building and analyzing these images can take days to process on Earth manually. But if we utilise machine learning tools and onsite computation, it might save a lot of time for the mission. The current model used in the Mars Perseverance is the AutoNav Mark 4 with a lot of tasks, including spacecraft positioning, in-flight orbit determination, target tracking, and ephemeris calculations. All those might be computationally expensive to process. Therefore, the aim of this research is to develop a simple algorithm to do object and slope determinations to feed into an autonomous path determination process. The data fed into the algorithm are panoramic images captured by the MastCam-Z mounted on Mars Perseverance.

How to cite: Swida, O. B., Foing, B., and Vleugels, C.: Mars Perseverance Panoramic Image for Self-Determination Mission Algorithm, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-16941, https://doi.org/10.5194/egusphere-egu23-16941, 2023.

Transformers in general have shown great promise in the sequence modeling. Recently proposed vision transformer (ViT) by Dosovitskiy et al. has shown optimal performance in image recogining [1]. Fourier Neural operator based token mixer transformers keeping ViT as backbone was proposed by Guibas et. al. has been used for predicting wind and precipitation on ERA5 dataset[2,3]. Following the previous work, we trained the Fourcastnet from scratch on the MERRA2 data set with 3 verticle levels (z450, z500, z550) and 11 variables (adding u, v, and temp). We trained on data from 2005 to 2015 and made predictions by providing the initial conditions from 2017. The prediction was made for 7 days in advance. For the first 24 hours model prediction, mean correlation was 0.998. Root mean squared error (RMSE) 6 hours prediction was 8.779 and for 24 hours was 19.581 on a scale range of -575.6 to 330.6. The model was further tested on 11 variables on the same training data to evaluate prediction of major events like Hurricane. Initial condition for category 5 Hurricane Sep 28, 2016 – Oct 10, 2016 was given to the model. The model was able to predict the hurricane for 18 hours. Further work will be done in order to tune to model and increase more environment variables from MERRA2 to make the prediction more robust and for a longer period.

References:
1. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold,
G., Gelly, S. and Uszkoreit, J., 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv
preprint arXiv:2010.11929.
2. Guibas, J., Mardani, M., Li, Z., Tao, A., Anandkumar, A. and Catanzaro, B., 2021, September. Efficient Token Mixing for
Transformers via Adaptive Fourier Neural Operators. In International Conference on Learning Representations
3. Pathak, J., Subramanian, S., Harrington, P., Raja, S., Chattopadhyay, A., Mardani, M., Kurth, T., Hall, D., Li, Z.,
Azizzadenesheli, K. and Hassanzadeh, P., 2022. Fourcastnet: A global data-driven high-resolution weather model using
adaptive fourier neural operators. arXiv preprint arXiv:2202.11214.

How to cite: Lee, T., Roy, S., Kumar, A., Ramachandran, R., and Nair, U.: Long-Term Forecasting of Environment variables of MERRA2 based on Transformers, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-2909, https://doi.org/10.5194/egusphere-egu23-2909, 2023.

Foundation Models (FM) are AI models that are designed to replace a task or an application specific model. These FM can be applied to many different downstream applications. These FM are trained using self supervised techniques and can be built on any type of sequence data. The use of self supervised learning removes the hurdle for developing a large labeled dataset for training. Most FM use transformer architecture utilizes the notion of self attention which allows the network to model the influence of distant data points to each other both in space and time. The FM models exhibit emergent properties that are induced from the data.

FM can be an important tool for science. The scale of these models results in better performance for different downstream applications and these applications show better accuracy over models built from scratch. FM drastically reduces the cost of entry to build different downstream applications both in time and effort. FM for selected science datasets such as optical satellite data, can accelerate applications ranging from data quality monitoring, feature detection and prediction. FM can make it easier to infuse AI into scientific research by removing the training data bottleneck and increasing the use of science data.

How to cite: Maskey, M., Ramachandran, R., Lee, T., and Ganti, R.: Foundation AI Models for Science, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-2944, https://doi.org/10.5194/egusphere-egu23-2944, 2023.

Due to climate change, we expect an exacerbation of fire in Europe and around the world, with major wildfire events extending to northern latitudes and boreal regions [1]. In this context, it is important to improve our capabilities to anticipate fire danger and understand its driving mechanisms at a global scale. As the earth is an interconnected system, large-scale processes can have an effect on the global climate and fire seasons. For example, extreme fires in Siberia have been linked to previous-year surface moisture conditions and anomalies in the Arctic Oscillation [2]. As part of the ESA-funded project SeasFire (https://seasfire.hua.gr), we gather and harmonize data related to seasonal fire drivers and develop deep learning models that are able to capture spatiotemporal associations with the goal to forecast burned area sizes on a seasonal scale, globally. We publish a global analysis-ready datacube for seasonal fire forecasting for the years 2001-2021 at a spatiotemporal resolution of 0.25 deg x 0.25 deg x 8 days [3]. The datacube includes a combination of variables describing the seasonal fire drivers , namely climate, vegetation, oceanic indices, human factors, land cover and the burned areas. We leverage the availability of big EO data and the advances in Deep Learning modeling [4, 5] to forecast global burned areas, capture the spatio-temporal interactions of the Earth System variables and identify potential teleconnections that determine wildfire regimes under the light of climate change. We present deep learning models that handle the Earth as a system, such as graph neural networks and transformer-based architectures. Applied on the prediction of wildfires at different temporal horizons we reveal that our deep learning models  skillfully predict burned area patterns. Exploring the explanation of the models, we reveal important spatio-temporal links.

Our approach, using AI to model the earth as a system and capture long spatio-temporal interactions, showcases the potential of an application-specific digital twin. The SeasFire datacube can be exploited as a baseline digital twin for modeling different natural hazards, including floods, heatwaves, and droughts. Thus, we will discuss insights and future directions for digital twins in anticipating climate extremes, inspired by our global wildfire prediction paradigm. 

[1] Wu, Chao, et al. "Historical and future global burned area with changing climate and human demography." One Earth 4.4 (2021): 517-530.

[2] Kim, Jin-Soo, et al. "Extensive fires in southeastern Siberian permafrost linked to preceding Arctic Oscillation." Science advances 6.2 (2020): eaax3308.

[3] Alonso, Lazaro, et al. Seasfire Cube: A Global Dataset for Seasonal Fire Modeling in the Earth System. Zenodo, 15 July 2022, p., doi:10.5281/zenodo.6834584.

[4] Kondylatos, Spyros et al. “Wildfire Danger Prediction and Understanding with Deep Learning.” Geophysical Research Letters, 2022.  doi: 10.1029/2022GL099368

[5] Prapas, Ioannis et al. “Deep Learning for Global Wildfire Forecasting.” NeurIPS 2022 workshop on Tackling Climate Change with Machine Learning, doi:  10.48550/arXiv.2211.00534

How to cite: Prapas, I., Karasante, I., Ahuja, A., Kondylatos, S., Panagiotou, E., Davalas, C., Alonso, L., Son, R., Dimitrios, M., Carvalhais, N., and Papoutsis, I.: Earth System Deep Learning towards a Global Digital Twin of Wildfires, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-5443, https://doi.org/10.5194/egusphere-egu23-5443, 2023.

Most of the DNNs are designed to predict a class, a segmentation map or detections, no matter it is interpolation or extrapolation. Then, a confidence score answers to the need of having interpretable outputs and it could help an AI4EO end-user to take a decision.

The first investigated use case was binary classification of small Sentinel-2 tiles containing ships or not (with 2 classes “tile containing ship” or “tile not containing ship”). The database gathered 16,947 small 140x140 tiles extracted from 37 Sentinel-2 products. The ground truth was generated using Danish AIS data and then checked by human-eye. It was divided into several datasets for training, validation, testing, and active learning.

The second investigated use case was the classification of 10 geophysical phenomena from Sentinel-1 wave mode [Wang et al, 2018]. The database gathered 30,032 images with a quite balanced repartition between the 10 classes.

Classification networks (VGG16) were trained on the training datasets of both use cases, reaching high performances (>95% accuracy). We added several Out Of Distribution (OOD) examples for the ship classification use case, and used the test database provided for the Ocean Features use cases. Models reach around 70% accuracy on these 2 harder datasets so that regressing confidence could have an interest, with many examples of wrong classifications.

The solution developed used the confidNet approach developed by Corbière et al. Without retraining the classification DNN, we added a second DNN, composed by several dense layers, taking latent space from the classification network as input, which objective was to estimate a confidence score, by trying to approach the True Class Probability. It proved to be easy to train when enough failure examples are available in the database.

The main objective of the confidNet is to find the “ID”/”OOD” boundary, qualifying which examples the classifier should be able to predict (interpolation), and those it should fail to predict (extrapolation). An important work was done to try to qualify the quality of the predictions of the confidNet (confidence score) to ensure that it didn’t just learn to map the subset of the dataset where the classifier fails, and the one where the classifier was right. It presented interesting properties of generalization and turned out to be less “dataset-dependent” than a classical DNN.

21 different network configurations were tested, making the size of the architecture vary from 4k to 2.5M parameters. It showed that many of these configurations could reach similar results, and that the number of layers was more decisive than the number of parameters of the intermediate feature maps.

The main results obtained in this study are the relevance to utilize the confidNet approach in AI4EO scenarios, the possibility to reduce the network in an on-boarding interest, and a first warranty that the confidNet approach can learn in a different way from classification networks, with interesting properties of generalization. This study demonstrates the possibility to associate confidence scores to the predictions of a DNN in a satisfying way.

How to cite: Dublé, N., De Vieilleville, F., Lagrange, A., and Le Saux, B.: Confidence estimation of DNN predictions for on-board applications, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-5746, https://doi.org/10.5194/egusphere-egu23-5746, 2023.

In recent years, Climate Change has been leading to an exacerbation of Extreme Weather Events (EWEs), such as storms and wildfires, raising major concerns in terms of their increase of their intensity, frequency and duration. Detecting and predicting EWEs is challenging due to the rare occurrence of these events and consequently the lack of related historical data. Additionally, gathering of data when the event manifests is not a straightforward process, due to the intrinsic difficulty of positioning and using acquisition systems. Advances in Machine Learning (ML) can provide cutting-edge modeling techniques to deal with EWE detection and prediction tasks, offering cost-effective and fast-computing solutions which are strongly required by policy makers for taking timely and informed actions in the presence of EWEs.

Solutions based on ML could, thus, support studies of such extreme events, providing scientists, policy makers and also the general public with powerful and innovative data-driven tools. However, from an infrastructural point of view, supporting such types of applications requires a wide set of integrated software components including data gathering and harmonisation pipelines, data pre-processing and augmentation modules, computing platforms for model training, results visualization tools, etc.

A Digital Twin for the analysis of extreme weather events, focusing on storms and wildfires, is being developed in the context of the EU-funded InterTwin project. The InterTwin project aims at defining a Digital Twin Engine for supporting scientific applications from different fields. In particular, for the EWEs, neural networks are being adopted as modeling tools capable of learning the underlying mapping between drivers and outcomes from past data and generalizing it to future projection data. This contribution will present the early concept behind the design of this machine learning-powered Digital Twin for EWE studies.

How to cite: Accarino, G., Elia, D., Donno, D., Immorlano, F., and Aloisio, G.: A machine learning-powered Digital Twin for extreme weather events analysis, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-6060, https://doi.org/10.5194/egusphere-egu23-6060, 2023.

Ever increasing computing capabilities and crave for high-resolution numerical weather prediction and climate information are specially interesting for the representation of Earth surfaces. Knowledge of accurate and up-to-date surface state for ecosystems such as forest, agriculture, lakes and cities strongly influence the skin temperatures, turbulent latent and sensible heat fluxes providing the lower boundary conditions for energy and moisture availability near the surface. A quick and automatic tool to assess the benefits of updating different surface fields, that makes use of a neural network regression model trained to simulate satellite observed surface skin temperatures, was developed. This tool was deployed to determine the accuracy of several global datasets for lakes, forest, and urban distributions. Comparison results will be shown. The neural network regression model has proven to be useful and easily adaptable to assess unforeseen impacts of ancillary datasets, also detecting erroneous regional areas over the globe, proving to be a valuable support to model development. 

How to cite: Choulga, M., Kimpson, T., Chantry, M., Balsamo, G., Boussetta, S., Dueben, P., and Palmer, T.: Deep Learning for Verification of Earth's surfaces, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-8777, https://doi.org/10.5194/egusphere-egu23-8777, 2023.

Environmental science is concerned with assessing the impacts of changing environmental conditions upon the state of the natural world. Environmental Digital Twins (EDT) are a new technology that enable environmental change scenarios for real systems to be modelled and their impacts visualised. They will be particularly effective with delivering understanding of these impacts on the natural environment to non-specialist stakeholders.

The UK Natural Environment Research Council (NERC) recently published its first digital strategy, which sets out a vision for digitally enabled environmental science for the next decade. This strategy places data and digital technologies at the heart of UK environmental science.

EDT have been made possible by the emergence of increasingly large, diverse, static data sources, networks of dynamic environmental data from sensor networks and time-variant process modelling. Once combined with visualisation capabilities these provide the basis of the digital twin technologies to enable the environmental scientists community to make a step-change in understanding of the environment. Components may be developed separately by a network but can be combined to improve understanding provided development follows agreed standards to facilitate data exchange and integration.

Replicating the behaviours of environmental systems is inevitably a multi-disciplinary activity. To enable this, an information management framework for Environmental digital twins (IMFe) is needed that establishes the components for effective information management within and across the EDT ecosystem. This must enable secure, resilient interoperability of data, and is a reference point to facilitate data use in line with security, legal, commercial, privacy and other relevant concerns. We present recommendations for developing an IMFe including the application of concepts such as an asset commons and balanced approach to standards to facilitate minimum interoperability requirements between twins while iteratively implementing an IMFe. Achieving this requires components to be developed that follow agreed standards to ensure that information can be trusted by the user, and that they are semantically interoperable so data can be shared. A digital Asset Register will be defined to provide access to and enable linking of such components.

This previously conceptual project has now been enhanced into the Pilot IMFe project aiming to define the architectures, technologies, standards and hardware infrastructure to develop a fully functioned environmental digital twin. During the project lifespan this will be tested with by construction of a pilot EDT for the Haig Fras Marine Conservation Zone (MCZ) that both enables testing of the proposed IMFe concepts and will provide a clear demonstration of the power of EDT to monitor and scenario test a complex environmental system for the benefit of stakeholders. 

How to cite: Buck, J., Kingdon, A., Siddorn, J., Blair, G., Kokkinaki, A., Blower, J., Fry, M., Marchant, B., Pepler, S., Watkins, J., and Byrne, J.: An Information Management Framework for Environmental Digital Twins (IMFe)  as a concept and pilot, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-10092, https://doi.org/10.5194/egusphere-egu23-10092, 2023.

An Earth System Digital Twin (ESDT) is a dynamic, interactive, digital replica of the state and temporal evolution of Earth systems. It integrates multiple models along with observational data, and connects them with analysis, AI, and visualization tools. Together, these enable users to explore the current state of the Earth system, predict future conditions, and run hypothetical scenarios to understand how the system would evolve under various assumptions. The NASA Advanced Information Systems Technology (AIST) program’s Integrated Digital Earth Analysis System (IDEAS) project partners with the Space for Climate Observatory (SCO) (https://www.spaceclimateobservatory.org/) FloodDAM Digital Twin effort led by CNES to establish an extensible open-source framework to develop digital twins of our physical environment for Earth Science with an initial focus on surface water hydrology in Earth’s rivers and lakes. The joint effort delivers an open-source system architecture with mechanisms for the outputs of one model to feed into others, for driving models with observation data, and for harmonizing observation data and model outputs for analysis. Water resource science is multidisciplinary in nature, and it not only assesses the impact from our changing climate using measurements and modeling, but it also offers opportunities for science-guided, data-driven decision support. The joint effort uses flood prediction and analysis as its primary use case. The work presents a multi-agency joint effort to define and develop a federated Earth System Digital Twin solution between NASA and CNES that powers advanced immersive science and custom user applications for scenario-based analysis.

How to cite: Huang, T. and the NASA AIST IDEAS and SCO FloodDAM Teams: Open-Source Framework For Earth System Digital Twins Applied to Surface Water Hydrology, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-10138, https://doi.org/10.5194/egusphere-egu23-10138, 2023.

Accurate weather predictions are essential for many aspects of social society. Providing a reliable high-resolution precipitation field is essential to capture the finer scale of heavy precipitation events,  which is normally poorly represented in the numerical models. Statistical downscaling is an appealing tool since it is computationally inexpensive. Thus, it has been widely used over the last three decades. In recent years, super-resolution with deep learning has been successfully applied to generate high-resolution from low-resolution images in the computer vision domain. This task is somewhat analogous to downscaling in the meteorological domain.

Inspired by this, we explore the use of deep neural networks with a super-resolution approach for statistical precipitation downscaling. We apply the Swin transformer architecture (SwinIR) as well as convolutional neural network (U-Net) with a Generative Adversarial Network (GAN) and a diffusion component for probabilistic downscaling. We use short-range forecasts from the Integrated Forecast System (IFS) on a regular spherical grid with Δx_IFS=0.1° and map to the high-resolution observation radar data RADKLIM (Δx_RK=0.01°). The neural networks are fed with nine static and dynamic predictors, similar to the study by Harris et al., 2022. All the models are comprehensively evaluated by grid point-level errors as well as error metrics for spatial variability and the generated probability distribution. Our results demonstrate that the Swin Transformer model can improve accuracy with lower computation cost compared to the U-Net architecture.  The GAN and diffusion models both further help the model to capture the strong spatial variability from the observed data.   Our results encourage further development of DNNs that can be potentially leveraged to downscale other challenging Earth system data, such as cloud cover or wind. 

How to cite: Gong, B., Ji, Y., Langguth, M., and Schultz, M.: Statistical downscaling of precipitation with deep neural networks, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-10488, https://doi.org/10.5194/egusphere-egu23-10488, 2023.

The representation of the atmospheric state at high spatial resolution is of particular relevance in various domains of Earth science. While global reanalysis datasets such as ERA5 provide comprehensive repositories of meteorological data, their spatial resolution (∆x≥25 km) is too coarse to capture relevant local features, mainly over complex terrain (e.g. cold pools in valleys, low-level jets, local heavy precipitation events).
Recently, various studies have started to apply deep neural networks adapted from computer vision to increase the spatial resolution of meteorological fields. Although these studies reveal great potential in the domain of statistical downscaling, intercomparison of the approaches is impeded due to a large variety of methods and deployed datasets. Comparisons to classical downscaling methods developed for decades in the meteorological community are also often underrepresented.

Inspired by the available benchmark datasets for various computer vision tasks and for weather forecasting (e.g. WeatherBench and WeatherBench Probability), our study aims to provide a benchmark dataset for statistical downscaling of meteorological fields. We choose the coarse-grained ERA5 reanalysis (∆x_ERA5≃30 km) and the fine-scaled COSMO-REA6 (∆x_CREA6≃6km) as input and target datasets. Both datasets enable the formulation of a real downscaling task: super-resolve the data and correct for model biases.
The benchmark dataset provides a collection of predictors and predictands for a couple of standard downscaling tasks. These comprise downscaling of the 2m temperature, the surface irradiance, the near-surface wind field and precipitation. Along with the dataset, benchmark deep neural networks, namely variants of U-Nets and GANs, will be provided. Well-chosen sets of evaluation metrics including baseline scores of the benchmarked deep neural networks are presented to enable comparison between different methods.
The envisioned benchmark dataset will provide a comprehensive basis for comparing neural network approaches on statistical downscaling of meteorological fields. This, in turn, is considered to enhance confidence and transparency in the application of deep learning methods on Earth system problems.

How to cite: Langguth, M., Gong, B., Ji, Y., Schultz, M. G., and Stein, O.: Towards a benchmark dataset for statistical downscaling of meteorological fields, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-11489, https://doi.org/10.5194/egusphere-egu23-11489, 2023.

Hurricanes, and more generally tropical cyclones, are among the most destructive natural hazards, and are arguably changing under climate change influences. Applying the power of AI to predict the extreme behavior of these events could be key to helping minimize hurricane damage. AI tools are a significant opportunity to: 1) identify non-linear relationships between changing hurricane-related characteristics and tropical storm intensification, and 2) anticipate responses to these changes. Another key part of this AI-based system is uncertainty quantification for decision-making processes. In this context, we present an improved ML hybrid model for predicting the development of extreme hurricane events, which includes effective information on spatio-temporal evolution variations of structural parameters extracted from IR satellite images. This approach, which combines Convolutional Neural Networks (CNNs) and a Random Forest (RF) classification framework, has been trained/tested with data from 1995 over the North Atlantic and NorthEast Pacific regions. Results from the CNN-RF model shows a performance of 80% or better considering lead-times of up to three days ahead (every 6 hours). With the proposed configuration, the overall precision has increased by at least 8%. This model could be yet further improved with the inclusion of new variables linked to environmental factors to be progressively explored. 

How to cite: Martinez Amaya, J., Radin, C., Nieves, V., Longépé, N., and Muñoz-Marí, J.: An AI hybrid predictive tool for extreme hurricane forecasting, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-12333, https://doi.org/10.5194/egusphere-egu23-12333, 2023.

Interferometric Synthetic Aperture Radar satellite measurements are an effective tool for monitoring ground motion with millimetric resolution over long periods of time. The Persistent Scatterer Pair method, developed in [1], is particularly useful for detecting differential displacements of buildings at multiple positions with few assumptions about the background environment. As a result, anomalous behaviours in building motion can be detected through PSP time series, which are commonly used to perform risk assessments in hazardous areas and diagnostic analyses after damage or collapse events. However, current autonomous early warning systems based on PSP-InSAR data are limited to detecting changes in linear trends and rely on sinusoidal and polynomial models [2]. This can be problematic if background signals exhibit more complex behaviours, as anomalous displacements may be difficult to identify. To address this issue, we propose an unsupervised anomaly detection method using Artificial Intelligence algorithms to identify potentially anomalous building motions based on PSP long time-series data.

To identify anomalous building motions, we applied two different AI algorithms based on Long Short-Term Memory Autoencoder inspired by [3] and a Graph Neural Network version of it. LSTM Autoencoder is an unsupervised representation learning framework that captures data representations by reconstructing the correct order of shuffled time series. Its encoder part is used to extract feature representations of a time series, while the decoder part is used to reconstruct the time series. By assuming that most stable samples exhibit similar temporal changes, this algorithm can be used for anomaly detection (as the reconstruction loss would be high for anomalous time series).

The data used in this study were provided by the European Ground Motion Service over a rectangular area surrounding the city of Rome and includes approximately 500.000 time-series aggregated over more than 80.000 buildings. The time period covered is from 2015 to 2020.

In our proposed approach, we first extract deep feature representations for each timestamp of a non-anomalous time series. The feature sequence is then shuffled and passed through an LSTM encoder-decoder network. By learning to reconstruct the feature sequence with the correct order, the network is able to recognize high-level representations of the time series. In the second step, the pre-trained network is used to reconstruct another time series. If the time series is non-anomalous, the correct order can be reconstructed with high confidence; otherwise, it is difficult to reconstruct the correct order. By selecting an appropriate threshold, anomalies can be detected with high reconstruction losses.

Overall, our proposed AI-based approach shows promising results for identifying anomalous building motions in PSP long time-series data. The use of unsupervised learning allows for more accurate statistical representations of the data and more reliable detection of anomalous behaviours. This approach has the potential to improve autonomous early warning systems for risk assessments and diagnostic analyses in dangerous areas.

This work is part of the RepreSent project funded by the European Space Agency (NO:4000137253/22/I-DT).

[1] https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=4779025
[2] https://www.mdpi.com/2072-4292/10/11/1816/pdf
[3] https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9307226

How to cite: Kuzu, R., Wang, Y., Dumitru, O., Bagaglini, L., Pasquali, G., Santarelli, F., Trillo, F., Saha, S., and Zhu, X. X.: An Unsupervised Anomaly Detection Problem in Urban InSAR-PSP Long Time-series, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-13106, https://doi.org/10.5194/egusphere-egu23-13106, 2023.

The notion of digital twin can be ambiguous because it can be defined in various ways. These last months have seen the emergence of many global digital twin initiatives. The challenge of these global digital twins is to create a qualified digital replica model of our planet, making it possible to monitor, simulate and anticipate natural phenomena and human activities. The target users are either scientists or decision makers. Through the digital twin, they have access to a digital representation of an environment using all available spatial and non-spatial data accompanied with a set of physical and statistical models to calculate projections, replay past events or simulate future ones.

Refining and evaluating the accuracy of these projections is a major challenge for digital twins. In addition to the knowledge of physical modeling, suitable data must also be available. Complementary to the global approach, the notion of local and dated digital twins appears then to be essential. Considering a digital representation of a restricted geographical area of interest (an urban area, watershed, coastline, etc.) allows to access to very high-resolution "fresh" data in 2D and 3D, in-situ data and small-mesh physical model. This user-centered and naturally thematic approach responds more finely and more pragmatically to the objectives presented. These local, dated and thematic digital twins are by essence ephemeral: a way to meet a specific need.

The challenge is therefore to setup a Digital Twin Factory (DTF). This DTF relies on a data lake, a high computing capacity via clouds and/or HPC and has thematic algorithms and methodologies able to generate registered and coherent layers of information in order to enrich a datacube from which physical indicators can be computed spatially. Thanks to its thematic, local and on-demand characteristics, the DTF can mitigate the need to have an universal model of metadata. This datacube allows to apply local physical and artificial intelligence models. The overarching architecture of the DTF will be presented. Specific examples on coastal, urban and risk topics will also be presented. These digital twins rely on a large number of expertises in both data and modeling involving various French (CNES, IGN, SHOM, IRD, CEA, INRAE, METEOFRANCE, CERFACS, BRGM, etc.) or international organizations (ESA, NASA, NOAA,…).

For coastal areas, the goal is to well describe the bathymetry topography continuum by taking into account the intertidal zones and the specialized dynamic models together with 3D coastal land cover characterisation. For urban areas, the ambition is first to automatically produce a qualified 3D map together with its additional layers of information: 3D objects and related semantics (land cover and land use) including temporal dynamic, thermal information. Then, for issues related to the management of natural risks (such as floods or fires) similar data layers can be used. Finally, new hypothesis can be injected in these digital replica and multiple scenarios can be applied to assess causal relationship between hypothesis and prediction. Very promising results will also be presented.

How to cite: Delvit, J.-M., Brunet, P.-M., Lassalle, P., Lallement, D., and Baillarin, S.: Towards a local, dated and thematic digital twins factory, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-13921, https://doi.org/10.5194/egusphere-egu23-13921, 2023.

How to cite: Drost, N., Kalverla, P., Schilperoort, B., Vreede, B., Alidoost, S., Verhoeven, S., Liu, Y., and Hut, R.: Digital twincubator eWaterCycle, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-14998, https://doi.org/10.5194/egusphere-egu23-14998, 2023.

Recent advances in High Performance Computing and Earth System Model resolution have enabled the Earth Science community to envision Digital Twins as an innovative approach to global environmental problems. This is also true of the Ocean Science community.

A Digital Twin of the Ocean (DTO) merges marine system models with observational data and machine learning analytics to produce a digital replica of the real ocean. In addition to natural phenomena, DTOs can include socio-economic factors (e.g. ocean-use, pollution). Thus, DTOs can be used to monitor the current ocean state, but also to simulate future ‘what-if’ scenarios for varying human interventions. Another benefit of DTOs is that they can be used by a variety of stakeholders: by scientists to understand the ocean, by policymakers to make well-informed decisions, and by citizens to improve ocean literacy. As such, DTOs are a powerful tool in future-proofing sustainable development. Moreover, they provide strong motivation to improve the marine data landscape and build an interoperable system with agreed upon formats and standards. DTOs are tailored to a specific ocean area or purpose, such that a DTO framework is needed to implement data connectivity and interoperability, ease of access, standards and to highlight gaps. The UN Ocean Decade Program DITTO aims to provide such a framework.  Specifically, DITTO advances worldwide collaboration between scientists, data and IT experts to develop a common understanding of DTOs, to establish best practices in their development, and to advance a digital framework for DTOs to empower ocean professionals from all sectors around the world to effectively create their own digital twins.

DTOs offer the technology for building a social-ecologically integrated ocean ecosystem with observation- and modelling networks that support sustainable ocean governance. 

How to cite: Kollert, J., Visbeck, M., and Brönner, U.: Digital Twins of the Ocean – Opportunities to Inform Sustainable Ocean Governance, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-15688, https://doi.org/10.5194/egusphere-egu23-15688, 2023.

Reservoir simulations often require statistical predictions to quantify production uncertainty or assess potential risks. Most existing uncertainty quantification procedures aim to decompose the input random field into independent random variables if the correlation scale is small compared to the domain size. In this work, we develop a K-means-based aggregation model, for efficiently estimating multiphase flow performance in multiple geological realizations. This approach performs a number of single-phase flow simulations and uses K-means clustering to select only a few representatives on which multiphase flow simulations are performed. In addition, an empirical model is then employed to describe the relationship between the single-phase solution and the multiphase solution using these representatives. Finally, the multiphase solution in all realizations can be easily predicted using empirical models. The method is applicable to both 2D and 3D synthetic models and has been shown to perform well in the trusted interval of productivity, and probability distribution as indicated by the cumulative density function. It is able to capture a large number of ensemble statistical realizations of Monte Carlo simulation results with significantly reduced computational cost.

How to cite: Liao, Q.: Clustering aggregation model for statistical forecasting of multiphase flow problems, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-973, https://doi.org/10.5194/egusphere-egu23-973, 2023.

During dry periods, river flow comprises baseflow, which typically generates from shallow aquifers. Understanding how such aquifers respond to climate events is key to managing environmental issues related to water supply or water quality. A typical indicator of groundwater response to climate events is the characteristic response time, which indicates the rate of depletion of shallow aquifers.

The traditional method to infer the characteristic response time analyzes the slope of the hydrograph recession curve. Such a method does not account for stormwater contribution in recession analysis, thereby assuming that the catchment is dry and the only contribution to discharge originates from groundwater. As a consequence, the recession analysis might underestimate the groundwater response time, owing to the presence of faster discharge components, i.e. surface runoff or interflow, in the falling limbs.

In this work, we propose an alternative methodology to calculate the characteristic response time, which is determined by analyzing the behavior of the baseflow time series in the frequency domain. The aquifer can be conceptualized as a low-pass filter, which smooths the high-fluctuating components in the recharge signal. Such behavior causes a cut-off frequency in the baseflow spectrum, which corresponds to the aquifer characteristic time. We applied this approach to several gauging stations in Germany, whose humid climate is ideal to compare the results with the classical recession analysis.

We observed that spectral analysis yields characteristic response times systematically larger than the ones calculated with recession analysis. On average there is a factor of two between the estimates provided by the two methods. Overall our study emphasizes careful consideration of the estimation of groundwater response times, especially in humid and sub-humid river basins.

How to cite: Di Dato, M., Houben, T., and Attinger, S.: Estimating groundwater response time in humid climate by using spectral analysis, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-3891, https://doi.org/10.5194/egusphere-egu23-3891, 2023.

Modelling hydrological processes in ungauged catchments is a major challenge in environmental sciences and engineering. An ungauged catchment is a catchment that lacks streamflow data suitable for traditional modelling methods. Predicting streamflow in ungauged catchments requires some form of extrapolation ("regionalisation") from other "similar" catchments, with variables of interest being flow "indices" or "signatures", such as quantiles of the flow duration curve, etc.

Another major question in hydrology is the estimation of model structure that reflects the hydrological processes relevant to the catchment of interest. This question is intimately tied to process representation. To paraphrase a common saying, all models are wrong, but some model mechanisms (process representations) might be useful. Our previous study contributed a Bayesian framework for the identification of individual model mechanisms from streamflow data.

In this study we extend the mechanism identification method to operate in ungauged basins based on regionalized flow indices. Candidate mechanisms and model structures are generated, and then the "dominant" (more a posterior probable) model mechanisms are identified using statistical hypothesis testing. As part of the derivation, it is assumed that the error in the regionalization of flow indices dominates the structural error of the hydrological model.

The proposed method is illustrated with real data and synthetic experiments based on 92 catchments from northern Spain, from which 16 catchments are treated as ungauged. We use 624 model structures from the flexible hydrological model framework FUSE. Flow indices are regionalised using random forest regression in principal component (PC) space; we select the first 4 leading indices in PC space. The case study set up includes an experiments using real data (where the true mechanisms are unknown) and a set of synthetic experiments with different error levels (where the “true” mechanisms are known).

Across the real and synthetic experiments, routing is usually among the most identifiable processes, whereas the least identifiable processes are percolation and unsaturated zone processes. The precision, i.e. the probability of making an identification (whether correct or not), remains stable at around 25%. In the synthetic experiments we can calculate the (conditional) reliability of the identification method, i.e. the probability that, when the method makes an identification, the true mechanism is identified. The conditional reliability varies from 60% to 95% depending on the magnitude of the combined regionalization and hydrological error. Our study contributes perspectives on hydrological mechanism identification under data-scarce conditions; we discus limitations and opportunities for improvement.

Prieto, C., N. Le Vine, D. Kavetski, F. Fenicia, A. Scheidegger, and C. Vitolo (2022) An Exploration of Bayesian Identification of Dominant Hydrological Mechanisms in Ungauged Catchments, Water Resources Research, 58(3), e2021WR030705, doi: https://doi.org/10.1029/2021WR030705.

How to cite: Prieto, C., Nataliya, L. V., Dmitri, K., Fabrizio, F., Andreas, S., and Claudia, V.: Towards identification of dominant hydrological mechanisms in ungauged catchments, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-3933, https://doi.org/10.5194/egusphere-egu23-3933, 2023.

Understanding the origin of errors in model predictions is a critical element in hydrologic model calibration and uncertainty estimation. While there exist a variety of plausible error sources, only one measure of the total residual error can be ascertained when the observed response is known. Here we show that collecting extra information a priori to characterise the data error before calibration can assist in improved model calibration and uncertainty estimation. A new model calibration strategy using the satellite metadata information is proposed as a means to inform the model prior, and subsequently to decompose data error from total residual error. This approach, referred to as Bayesian ecohydrological error model (BEEM), is first examined in a synthetic setting to establish its validity, and then applied to three real catchments across Australia. Results show that 1) BEEM is valid in a synthetic setting, as it can perfectly ascertain the true underlying error; 2) in real catchments the model error is reduced when utilizing the observation error variance as added error contributing to total error variance, while the magnitude of total residual error is more robust when utilizing metadata about the data quality proportionality as the basis for assigning total error variance ; 3) BEEM improves model calibration by estimating the model error appropriately and estimating the uncertainty interval more precisely. Overall, our work demonstrates a new approach to collect prior error information in satellite metadata and reveals the potential for fully utilizing metadata about error sources in uncertainty estimation.

How to cite: Zou, H., Marshall, L., and Sharma, A.: Characterising errors using satellite metadata for eco-hydrological modelling, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-5100, https://doi.org/10.5194/egusphere-egu23-5100, 2023.

Groundwater resources are heavily exploited to supply domestic, industrial and agricultural water consumption. Climate and societal changes and associated higher abstraction will alter the subsurface storage in terms of quantity and quality in currently unpredictable ways. In order to ensure sustainable groundwater management, we must evaluate the intrinsic and spatially variable vulnerability of aquifers in terms of water quality issues and the resilience of groundwater volumes to external perturbations such as severe droughts in connection with intensive irrigation. For this purpose, physically based numerical groundwater models are of great importance, especially on the regional scale. The equations applied in these models must be fed with the hydrogeological parameters: The transmissivity T and the storativity S.

Both parameters are typically obtained through time consuming and cost intensive hydrogeological in-situ tests or by laboratory analysis of core samples from point information (drillings and wells), resulting in parameters with limited transferability to regional settings. Instead, we propose to determine the parameters by spectral analysis of groundwater level fluctuations using (semi-)analytical solutions for the frequency domain. We developed a fully automatized workflow, taking groundwater level and recharge time series together with little information about the geometry of the aquifer to derive T and S as well as t_c (the characteristic response time). While the first two will be used for hydrogeological modelling, the latter can serve as an indication to assess the resilience of the groundwater system directly without additional modelling attempts. The methodology was tested with great success in simplified numerical environments and was applied to real groundwater time series in southern Germany. The response times and the storativities could be robustly estimated while the transmissivities inherit quantifiable uncertainties. Depending on the hydrogeological regime, the parameters represented effective and regional estimates.

How to cite: Houben, T., Di-Dato, M., Siebert, C., Kalbacher, T., Fischer, T., and Attinger, S.: Spectral analysis of groundwater level time series reveals hydrogeological parameters, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-5635, https://doi.org/10.5194/egusphere-egu23-5635, 2023.

As part of the Great-Lakes Runoff Inter-comparison Project (GRIP-GL; Mai et al., 2022), which aims at comparing the performances of different hydrologic models over the Great-Lakes when calibrating them using the same meteorological inputs and geophysical databases, the GEM-Hydro hydrologic model used at Environment and Climate Change Canada (ECCC) to perform operational hydrologic forecasts was calibrated using different strategies. Following the calibration work related to GRIP-GL, progress has been achieved with regard to improving the calibration of the GEM-Hydro model.

The work presented here focuses on improvements achieved with regard to calibrating the GEM-Hydro model, compared to the default version of the model and to the performances obtained during the GRIP-GL project. For various reasons explained, the GEM-Hydro calibration performed as part of GRIP-GL was suboptimal. The general calibration framework remains the same as in GRIP-GL, for example by using the MESH-SVS-Raven model to speed-up simulation times and transferring the calibrated parameters into GEM-Hydro afterwards, by relying on global calibrations for each of the 6 Great-Lakes subdomains, etc. However, several important changes have been made compared to the work performed in GRIP-GL, like a new approach to represent the effect of Tile Drains, changing the set of flow stations used for calibration, revising the objective function, etc.

The proposed calibration methodology updates significantly improve GEM-Hydro streamflow performance across the Great-Lakes domain and in addition also improve or maintain similar performance levels as the default version of the model, with respect to auxiliary variables and surface fluxes: snow, soil moisture, evapotranspiration, 2m air temperature and dew point. Indeed, the model relies on 40m atmospheric forcings for wind speed, temperature and humidity, and simulates its own 2m atmospheric variables. To achieve this, it was necessary to constrain some parameter interval values during calibration, in order to prevent the calibration algorithm to choose physically-irrelevant parameter values that could allow to improve streamflow performances while degrading other hydrologic variables, due to equifinality.

Reference:

Mai, J., Shen, H., Tolson, B. A., Gaborit, E., Arsenault, R., Craig, J. R., Fortin, V., Fry, L. M., Gauch, M., Klotz, D., Kratzert, F., O'Brien, N., Princz, D. G., Rasiya Koya, S., Roy, T., Seglenieks, F., Shrestha, N. K., Temgoua, A. G. T., Vionnet, V., and Waddell, J. W. (2022). The Great Lakes Runoff Intercomparison Project Phase 4: The Great Lakes (GRIP-GL). Hydrol. Earth Syst. Sci., 26, 3537–3572. Highlight paper. Accepted Jun 10, 2022.  https://doi.org/10.5194/hess-26-3537-2022

How to cite: Gaborit, E., Princz, D., Mai, J., Shen, H., Tolson, B., and Fortin, V.: On the elaboration of a robust calibration strategy for the large-scale GEM-Hydro model, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-6986, https://doi.org/10.5194/egusphere-egu23-6986, 2023.

Temporal sensitivity analyses can be used to detect dominant model parameters at different time steps (e.g. daily or monthly) providing insights on their temporal patterns and reflecting the temporal variability in dominant hydrological processes. However, hydrological processes do not only vary in time under different hydrometeorological conditions, but also the time scales of implemented processes are different. Here, the impact of different time scales (e.g. daily vs. monthly) on sensitivity patterns is investigated.

A temporal parameter sensitivity analysis is applied to three hydrological models (HBV, mHM and SWAT) for nine catchments in Germany. These catchments represent the variability of landscapes in Germany and are dominated by different runoff generation processes. In addition to discharge, further model fluxes and states such as evapotranspiration or soil moisture are used as target variables for the sensitivity analysis.

To analyse the impact of different time scales, two approaches are compared. In a first approach, daily simulated time series are used for the sensitivity analysis and aggregated then to monthly averaged sensitivities (Post-Agg). In a second approach, the simulated time series is first aggregated to a monthly time series and than used as input for the sensitivity analysis (Pre-Agg).

Our analysis shows that monthly averaged sensitivity patterns of different model outputs vary between Post- and Pre-Aggregation approach. Model parameters that are related to fast-reacting runoff processes, e.g. surface runoff or fast subsurface flow, are more sensitive when using daily time series for the sensitivity analysis (Post-Agg). In contrast, model parameters related processes with longer time scales such as snowmelt or evapotranspiration are more emphasized in monthly time series (Pre-Agg). These differences in the sensitivity results between Post-Agg and Pre-Agg are in particularly pronounced when using the integrated value of discharge as the target variable. Instead, the differences are smaller when applying the sensitivity analysis directly to represent model fluxes.

Moreover, our analysis shows changes in dominant parameters along a north-south gradient which can be explained by the physiographic characteristics of the catchments. The differences in the sensitivity results between the models can be related to the different model structures.

Based on our analysis, we recommend to either using model outputs of the major hydrological variables or different time scales for the sensitivity analysis to derive the maximum information from the diagnostic model analysis and to understand how model parameters describe hydrological systems.

How to cite: Guse, B., Herzog, A., Thober, S., Spieler, D., Melsen, L., Kiesel, J., Staudinger, M., Wagner, P., Loritz, R., Müller, S., Stölzle, M., Scholz, L., Berg, J., Pilz, T., Ehret, U., Düthmann, D., Houska, T., Pool, S., and Tarasova, L. and the other members of the DFG Scientific network IMPRO: Time-varying sensitivity analysis across different hydrological model structures, variables and time scales, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-8423, https://doi.org/10.5194/egusphere-egu23-8423, 2023.

Common in-situ methods like pumping tests, slug tests and laboratory analysis reveal aquifer parameters (that is the transmissivity and storativity) that are localized and specific to the measurement location. A need for regionally valid aquifer parameters arises when setting up regional scale physically based groundwater models. The models would help water resource managers to plan and predict the quality and quantity of groundwater resources, thus supports decision making as well as sustainable fresh water supply. A study from Houben et al. 2022 indicate that regional aquifer parameters can be obtained by analysing the frequency content of groundwater level time-series. Their work builds upon a semi-analytical solution for the groundwater head spectrum stochastically derived from the Boussinesq equation evoking the Dupuit assumptions. They found that the solution can be used to infer the transmissivity and storativity from groundwater level fluctuations and validated their hypothesis in simplified numerical environments of different complexity.

In this work, we extended the numerical experiments and applied the semi-analytical solution in homogeneous and heterogeneous 2D (x-y-plane) aquifers as well as in a complex numerical 2D (x-y-plane) model of the upper Danube catchment. We tested the hypothesis that certain locations can reveal regional aquifer parameters. In a homogeneous simulated model, the semi-analytical solution reveals effectively the model input parameters which serves as a proof-of-concept. In a heterogeneous numerical model, the obtained parameters show the complex interplay between zones of different permeability. The effects of high permeable zones can be observed on the low permeable zones which are further apart and vice versa. The obtained parameters were in the range of the model input parameters and followed the trend of the input parameters along the direction of flow. In the model of the upper Danube the obtained parameters were systematically larger than the input parameters. The shift in the obtained parameters was attributed to a violation of the assumptions of the semi-analytical solution. Thus, the complexity of model leads to a breakdown of the semi-analytical solution in some areas. Analyses on a sub-catchment scale revealed that when the assumptions of the analytical solution are met, the obtained parameters reflect the effective parameters.

How to cite: Javed, R. A., Houben, T., Kalbacher, T., and Attinger, S.: Investigating the spectral analysis of groundwater level fluctuations in a numerical model of the upper Danube catchment in Germany, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-10001, https://doi.org/10.5194/egusphere-egu23-10001, 2023.

How to cite: Kavianihamedani, H., Quinn, J., and Smith, J.: New Diagnostic Assessment of MCMC Algorithms Effectiveness, Efficiency, Reliability, and Controllability in Calibrating Hydrological Models, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-10326, https://doi.org/10.5194/egusphere-egu23-10326, 2023.

There are significant uncertainties associated with the estimates of model parameters in hydrological and environmental modeling. Such uncertainties could propagate within a modeling framework, leading to considerable deviation of the predicted value from its real value. Quantifying the uncertainties associated with model parameters could be computationally exhaustive and is still a daunting challenge to hydrological and environmental engineers. In this study, a series of Polynomial Chaos Expansion (PCE) methods, which have a significant advantage in computational efficiency, is developed to assess the propagation of parameter uncertainty. The proposed approaches were applied to two hydrological/environmental modeling case studies. The uncertainty quantification results will be compared with those from the traditional Monte Carlo simulation technique, to demonstrate the effectiveness and efficiency of the proposed approaches. This work will provide an efficient and reliable alternative to assess the impacts of the parameter uncertainties in hydrological and environmental modeling.

How to cite: Li, Z., Zhou, P., and Ghaith, M.: Uncertainty Quantification in Hydrological and Environmental Modeling based on Polynomial Chaos Expansion, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-10510, https://doi.org/10.5194/egusphere-egu23-10510, 2023.

Understanding how land surface models (LSMs) partition precipitation into evapotranspiration and runoff under changing climate is key to improved future hydrologic predictions. This sensitivity is rarely tuned in land models, as evidenced by prevalent biases in the sensitivity of simulated runoff to precipitation and temperature change compared to observational estimates. Here, using the Community Land Model (CLM5) over the Colorado River basin (CRB), we investigate what the informative model parameters for runoff sensitivities are and how their choices affect the sensitivities under changing temperature and precipitation. We focus on the headwater region of the CRB, motivated by inconsistent model estimates of runoff sensitivities in the region and the critical need to better understand runoff changes to address the ongoing water crises in the CRB. In each headwater basin, a set of informative parameters were identified through parameter perturbations using “one at a time” method within an adaptive surrogate-based model optimization scheme (ASMO). Results of perturbations highlight that different parameter sets with similar performance (with respect to water-year discharge) provide very different runoff sensitivities to temperature and precipitation during the 1951-2010 period. Additionally, both precipitation and temperature sensitivities of runoff show sensitivity to similar parameters across the region. The most sensitive parameters control the conductance-photosynthesis relationship, soil surface resistance for direct evaporation, the partitioning of runoff into the surface and the subsurface component, and soil hydraulic properties. We show how the importance of each parameter varies through the parameter space and derive parameter estimates by maximizing the “fit to observed sensitivities” within the ASMO scheme. Our results provide key insights regarding parameters optimization to improve long-term hydrologic sensitivities in LSMs.

How to cite: Pokhrel, Y., Elkouk, A., Luo, L., Payton, L., Livneh, B., and Cheng, Y.: Impact of Model Parameters on Runoff Sensitivities in the Community Land Model: A Study on the Upper Colorado River Basin, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-10644, https://doi.org/10.5194/egusphere-egu23-10644, 2023.

Bayesian inference has become the most popular approach to uncertainty assessment in vadose zone hydrological modeling. By combining prior information with observations and model predictions, it became popular among hydrologists as it enables them to infer parameter posterior distributions, verify model adequacy, and assess the model's predictive uncertainty. In particular, the posterior distribution is frequently the variable of interest for modelers as it describes the epistemic uncertainty of model parameters conditioned on measurements. Gradient-free Markov-Chain Monte Carlo (MCMC) ensemble samplers based on Differential Evolution (DE) or Affine Invariant (AI) strategies have been used to approximate the posterior distribution, which is frequently anisotropic and correlated in vadose zone-related problems. However, a rigorous benchmark of different MCMC algorithms to provide guidelines for their application in vadose zone hydrological model calibration is still missing. In this study, we elucidate the behavior of MCMC ensemble samplers by performing an in-depth comparison of four samplers that use AI moves or DE-based strategies to approximate the target density. Two Rosenbrock distributions, and one synthetic and one actual case study focusing on the inverse estimation of soil hydraulic parameters using HYDRUS-1D, are used to compare algorithms in different dimensions. The analysis reveals that AI-based samplers are immune to affine transformations of the target density, which instead double the autocorrelation time for DE-based samplers. This behavior is reiterated in the synthetic scenario, for which AI-based algorithms outperform DE-based strategies. However, this performance gain disappears when the number of soil parameters increases from 7 to 16, with both samplers exhibiting poor acceptance rates, which are not improved by increasing the number of chains from 50 to 200 or by mixing different strategies.

How to cite: Brunetti, G., Simunek, J., Wöhling, T., and Stumpp, C.: Pitfalls and Opportunities in the Use of Markov-Chain Monte Carlo Ensemble Samplers for Vadose Zone Model Calibration, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-11129, https://doi.org/10.5194/egusphere-egu23-11129, 2023.

In small agricultural catchments over Europe, intensive use of pesticides leads to widespread contamination of rivers and groundwater, largely due to hydraulic transfers of these reactive solutes from plots to rivers. These transfers must be better understood and described in the watershed in order to be able to propose best management practices adapted to the catchment and to reduce its contamination. The physically based model CATHY simulates interactions between surface and subsurface hydrology and reactive solute transport. However, the high sensitivity of pesticide transfers to spatially heterogeneous soil properties induces uncertainty that should be quantified and reduced. In situ data on pesticides in a catchment are usually rare and not continuous in time and space. Likewise, satellite imagery can provide spatial observations of hydrologic variables but not generally of pesticide fluxes and concentrations, and at limited scale and time frequency. The objective of this work is to combine these 3 types of information (model, in situ data, images) and their associated errors with data assimilation methods, in order to reduce pesticide and hydrological variable uncertainties. The sensitivity to spatial density and temporal frequency of the data will be evaluated, as well as the coupled data assimilation efficiency, i.e., the effect of assimilating hydrological data on pesticide-related variables. The methods will be developed using a Python package, and compared/evaluated on twin experiments using virtual data that are however generated over a real vineyard catchment, in Beaujolais, France, in order to ensure realism of the experiments, data, and associated errors.

How to cite: Lauvernet, C., Paniconi, C., Rouzies, E., Gatel, L., and Caisson, A.: Combining water and pesticide data with coupled surface/subsurface hydrological modeling to reduce its uncertainty., EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-13104, https://doi.org/10.5194/egusphere-egu23-13104, 2023.

Numerous methods exist to gain insight on a model performance. Sensitivity analysis (SA) tools provide information on how a model output depends on model parameters. It is widely argued that SA is an essential tool for assessing model uncertainty. Here, I review global SA using Variogram Analysis of Response Surfaces (VARS), variance-based methods (Sobol' indices) and polynomial chaos expansion. For the comparison, we use a set of denitrification models, which are needed to assess the fate of nitrate, a global challenge. For each of the models, we assess the uncertainty and reliability of predictions, and the use of SA tools in designing experiments to reduce model uncertainty.

How to cite: Carrera, J. and Petchamé, J.: A comparison of sensitivity analysis methods and their value for comparing denitrification models, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-13589, https://doi.org/10.5194/egusphere-egu23-13589, 2023.

Catchments in Saxony differ regarding their physiographic characteristics (topography, geomorphology, geology, land use, soils, etc.) and their climatic boundaries. Both factors influence the flow behavior and the water balance components of catchments. How sensitive the water balance of catchments responds to current and future changes in the climatic boundary conditions is difficult to predict for each catchment and is associated with significant uncertainties. In Saxony, the pronounced drought in groundwater and surface water from 2018 to 2020 led to considerable regional problems in water supply and quality.

Schwarze et al. (2017) already investigated trends of the observed discharge and variables derived by hydrograph separation (e.g. baseflow) in a sensitivity study. In this presentation, we show the results of an extension of this analysis with current observation data until 2020. The following research questions are investigated: (i) Are catchments in Saxony already responding to changing climatic conditions? (ii) Which regions show the most significant changes in discharge behavior relative to other water balance components? (iii) What are the factors and drivers of changes in the water balance in Saxonian Catchments?

The study is based only on observational data for precipitation, temperature, and discharge in the period of 1961 to 2020 in Saxony. Break point analysis, hydrograph separation, and sensitivity analysis of hydrological signatures are performed for different sets of climate periods to quantify changes and elasticity of the water balance components. As a result, a decreasing trend for the mean flow can be seen for almost all 88 investigated and undisturbed catchments in Saxony. This trend is more pronounced in the mountainous regions than in the lowland of Saxony. Despite the slight increase in the mean annual precipitation, the temperature rise of about one °C from 1991-2020 compared to 1961-1990 in all catchments leads to an increasing evapotranspiration, reduced discharge, and groundwater recharge.

References:

Schwarze, R., Wagner, M. and Röhm, P. (2017). Adaptation strategies to climate change - Analysis of the sensitivity of water balance variables of Saxon gauge catchments with respect to the increased temperature level from 1988 onwards compared to the reference state of 1961-1987. Ed.: Saxon State Office for Environment, Agriculture and Geology (LfULG), 2017.

How to cite: Schuetze, N., Hauffe, C., Pahner, S., Brandes, C., Lei, K., and Udo, M.: Sensitivity analysis of water balance components under climate change in Saxony, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-13981, https://doi.org/10.5194/egusphere-egu23-13981, 2023.

This abstract introduces a recipe for an adaptive general likelihood function and its application in the Bayesian epistemology of model parameters and structure uncertainty. The proposed methodology focuses on a special class of likelihood function, hereinafter mentioned as adaptive general likelihood function (AGL), which require a minimum priori assumptions/knowledge about the model residuals. The goal of the AGL is to characterize the model residuals independently from the inference framework in order to avoid incorrectly posterior estimation as a result of jointly inferencing of model and error model parameters. Mathematically, AGL is structured with a mixture of gaussian distributions joined with a first order autoregressive model, account for error model shape and autocorrelation respectively. To assess the AGL application, it is benchmarked with a formal likelihood function formulated by Schoups and Vrugt (2010) and evaluated for 24 Camels basins where the blended model has been deterministically applied with success (Chlumsky et al. 2022). Both approaches are compared with the residual’s empirical distributions using various statistical tests. The model used here is a blended hydrologic model introduced by Mai et al., (2021) which is a class of hydrologic models constructed by averaging (blending) various process options at the process flux level. This blending means calibration of the model functions to identify traditionally calibrated model process parameters as well as the weights utilized to average multiple process options. The model is deployed in the Raven hydrologic framework (Craig et al., 2020) and simultaneously both processes weights and parameters were calibrated deterministically for both high flows and low flows using PADDS algorithm (Asadzadeh and Tolson, 2013). This multi-objective calibration yields a suite of sample of calibrated blended models which is then utilized for error model development and testing. The tests results indicated a statistically comparable performance for both methods for t-distributed residuals highly skewed and long-tailed residual errors which are apparent in many hydrologic model residuals. Finally, to disjoin the epistemic Bayesian inference framework from the error model parameters, an epsilon-support vector regression (eps-SVR) is deterministically trained as a surrogate model to map the structural/parametric variability to residual error model parameters. The eps-SVR calibration performance metrics indicated high quality of surrogate for training set indicating promising performance.

How to cite: Arabzadeh, R., Romero-Cuellar, J., Chlumsky, R., Craig, J., and Tolson, B.: Adaptive Surrogate Likelihood Function for Blended Hydrologic Models, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-15689, https://doi.org/10.5194/egusphere-egu23-15689, 2023.

The applications of statistical learning theory (SLT) in hydrology have been either in the form of Support Vector Machines and other complexity regularized machine learning algorithms that learn and predict input-output patterns such as rainfall-runoff time series or of identifying optimal complexity of low order models such as k nearest neighbour models to predict hydrological time series such as streamflow. The regularization of model complexity offers a way to identify minimal complexity of a model to accurately predict a time series of interest. However such applications often assume that the modelled residual are independent of each other. This limits its application to conceptual hydrological models where residuals are often auto-correlated. This paper applies recent results of risk bounds for time series forecasting and SLT approaches to dynamical system identification to conceptual hydrological models, offering a means to identify optimal complexity of conceptual models and complexity regularised streamflow predictions based on it.

Basins from CAMELS data set are used to demonstrate the effect of regularizing the problem of hydrological model calibration on streamflow prediction over unseen data. SAC-SMA and SIXPAR (a lower order version of SACSMA) are used as model examples. Preliminary results show that prediction uncertainty bounds are narrower if regularization does not improve the performance of a calibrated model over unseen data. This effect is stronger in drier basins than in humid ones. Also, as expected, this effect is stronger when training data size is small and holds for both SACSMA and SIXPAR. 

How to cite: Pande, S. and Moayeri, M.: Complexity-based robust hydrologic prediction: extension of statistical learning theory to conceptual hydrological models, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-16039, https://doi.org/10.5194/egusphere-egu23-16039, 2023.

Due to the lack of accurate representation of hydrological processes and parameter measurements, physically-based hydrological models consist of many parameters requiring calibration to historical observations so that reliable hydrological inference can be obtained. With the increasing data availability from various sources (e.g., satellite remote sensing, climate model reanalysis), additional information on different water balance components (e.g., soil moisture, groundwater storage, etc.) are used to constrain and validate hydrological models, resulting in better model performance and parameter identifiability. However, given the emergence of multiple datasets for various water budget components, and their differences in temporal and spatial resolutions, the uncertainties in these datasets, when used together in driving and evaluating hydrological models, could introduce potential inconsistencies in water balance estimation and lead to a non-closure problem, which could result in potentially biased parameter and water balance component estimates in hydrological modelling.

This study addresses this issue by examining the impact of inconsistent water balance component data on model performance and exploring the importance of hydrologically consistent data for robust hydrological inference. The assessment is done using a Canadian Hydrologic-Land Surface Models named MESH in the Saskatchewan River basin, Canada over the period of 2002 to 2016. Seven precipitation datasets, seven evapotranspiration products, one source of water storage data – GRACE from three different centers using spherical harmonic and mass concentration approaches – and observed discharge data from hydrometric stations are selected as the input and evaluation data. A reference water balance dataset is developed to optimally combine all available data sources for each water balance component and to obtain water balance closure though a constrained Kalman filter data assimilation technique. The MESH model is rerun with this reference dataset and results are assessed and compared to different combinations of input and evaluation data. Preliminary results reveal great variations of model performance in the water balance components when using different combinations of input and evaluation data and results of using the reference dataset is expected to have less biased water balance component estimates. This study aims to highlight the necessity of using a set of hydrologically consistent data before any model runs and model evaluation.

How to cite: Wong, J. S., Yassin, F., and Famiglietti, J. S.: Does hydrologically consistent data improve model performance? The importance of closing the water balance of input and evaluation data, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-16924, https://doi.org/10.5194/egusphere-egu23-16924, 2023.

Soil moisture is an important parameter of the Earth system and plays a key role in understanding soil-atmosphere interactions through energy balance and the hydrological cycle. Information on its spatiotemporal variability is of crucial importance in several research topics and applications. Remote sensing, today, provides a very promising avenue towards obtaining information on the variability of soil moisture at varying spatial and temporal resolutions. Also, currently, a number of relevant operational products is available from different satellite sensors.

The objective of the present study has been to evaluate one such product specifically that from the SMOS satellite in a typical Mediterranean setting located in Greece. In particular, this study examines the agreement of the SMOS soil moisture product with collocated field measurements from the Prefecture of Larisa for calendar year of 2020 acquired from Neuropublic S.A.. The agreement between the two datasets was evaluated on the basis of several statistical measures. Also, the effect of topographical and geomorphological features, land use/cover and the relative satellite orbit type and the Radio Frequency Interference (RFI) was examined as part of our analysis.

To our knowledge, this study is one of the few providing an insight of the SMOS soil moisture product accuracy in a Greek setting. Findings of our study can provide important insights towards understanding the practical value of such products in agricultural and arid/semi-arid Mediterranean environments such that of Greece and also help efforts directed towards improving their retrieval accuracy.

Keywords: soil moisture; operational product; remote sensing; SMOS; validation; agriculture; Mediterranean setting

How to cite: Petsini, T. and Petropoulos, G. P.: Assessing the retrieval accuracy of SMOS soil moisture product in a Greek agricultural setting, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-1668, https://doi.org/10.5194/egusphere-egu23-1668, 2023.

The Environmental Mapping and Analysis Program (EnMAP), is a new spaceborne German hyperspectral satellite mission for monitoring and characterizing the Earth’s environment on a global scale. EnMAP mission supports the retrieval of high-quality and abundant detailed spectral information in VNIR and SWIR ranges within a large-scale area in wide temporal coverage and high spatial resolution. Taking advantage of high-quality data freely available to the scientific community great potential is revealed in a wide range of ecological and environmental applications, such as i.e. accurate and up-to-date LULC thematic maps.

The objectives of the present study are to explore the accuracy of EnMAP in land cover mapping over a heterogeneous landscape. As a case study is used a typical Mediterranean setting located in Greece. The methodology is based on the synergistic use of machine learning techniques and ENMAP imagery coupled with other ancillary data and was carried out in EnMAP Box-3, a toolbox designed within a GIS open-source software. Validation of the derived LULC maps has been carried out using the standard error matrix approach and also via comparisons versus existing LULC operational products.

To our knowledge, this research is one of the first to explore the advantages of the hyperspectral EnMAP satellite mission in the context of LULC mapping. Results of the present study are expected to provide valuable input for applications of LULC mapping and demonstrate the potential of hyperspectral EnMAP data for improved performance and the highest accuracy of LULC mapping.

KEYWORDS: EnMAP, Land cover, Land use, Hyperspectral remote sensing, Machine Learning

How to cite: Lekka, C., Detsikas, S. E., and Petropoulos, G. P.: Exploring the synergy of EnMAP hyperspectral imagery with Machine Learning for land use- land cover mapping in a Mediterranean setting, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-1761, https://doi.org/10.5194/egusphere-egu23-1761, 2023.

Information on Impervious Surface Areas (ISA) is required in various studies related to the urban environment. The continuous expansion of these surfaces is being noticed in large urban centers as a result of urbanization. The development of automated methodologies for mapping the ISas using remote sensing data has experienced a great growth in recent years.

The aim of the present study is the long-term mapping of ISA changes in Athens, Greece, from 1984 to 2022, exploiting the Landsat archive and contemporary methods of geospatial data processing, such as Machine Learning. The study implementation is also carried out in Google Earth Engine cloud platform and the final results obtained are presented in a WebGIS environment.

The results of the present study can contribute to a better understanding of the urban expansions dynamics and the key drivers linked to the urban sprawl that affect cities such as Athens. Furthermore, they can serve as a reference for further development of applications related to urban environments, using machine learning techniques combined with remote sensing data.

KEYWORDS: ISA, urban sprawl, Landsat, GEE, WebGIS, Greece

How to cite: Dermosinoglou, K. and Petropoulos, G. P.: Long term monitoring of the changes in Impervious Surface Areas in a Greek setting using Machine Learning and Remote Sensing data: the case of Athens Greece, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-1785, https://doi.org/10.5194/egusphere-egu23-1785, 2023.

Cloud penetrating Synthetic Aperture Radar (SAR) imagery has proven effective for tropical forest monitoring at national and pan-tropical scales. Current SAR-based disturbance detection methods rely on identifying decreased post-disturbance backscatter values as an indicator of forest disturbances. However, these methods suffer from a major shortcoming, as they show omission errors and delayed detections for some disturbance types (e.g., logging or fires). Here, post-disturbance debris or tree remnants result in stable SAR backscatter values similar to those of stable forest. Despite fairly stable backscatter values we hypothesize that different orientation and arrangement of tree remnants lead to an increased heterogeneity of adjacent disturbed pixels. Increased heterogeneity can be quantified by textural features. We assessed six Gray-Level Co-Occurrence Matrix (GLCM) textural features utilizing Sentinel-1 C-band SAR time series. We used a pixel-based probabilistic change detection algorithm to detect forest disturbances based on each GLCM feature and compared them against forest disturbances detected using only backscatter data. We further developed a method to combine both backscatter and GLCM features to detect forest disturbances. GLCM Sum Average (SAVG) performed best out of the tested GLCM features. Omission errors were reduced of up to 36% and the timeliness of detections was improved of up to 30 days by applying the combination method of backscatter and GLCM SAVG. Test sites characterized by large unfragmented disturbance patches (e.g., large-scale clearings, fires and mining) showed the greatest spatial and temporal improvement. A GLCM kernel size of 5 leads to the best trade-off of improving timeliness of detections and reducing omission errors while not introducing commission errors. The robustness of the developed method was verified for a variety of natural and human-induced forest disturbance types in the Amazon Biome. Our results show that combined SAR-based textural features and backscatter can overcome omission errors caused by post-disturbance tree remnants. Combining textural features and backscatter can support law enforcement activities by improving spatial and temporal accuracy of operational SAR-based disturbance monitoring and alerting systems.

How to cite: Balling, J., Herold, M., and Reiche, J.: The benefit of textural features for SAR-based tropical forest disturbance mapping, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-3434, https://doi.org/10.5194/egusphere-egu23-3434, 2023.

Obtaining Soil Moisture Content (SMC) over large scales is of key importance in several environmental and agricultural applications especially in the context of climate change and transition to digital farming. Remote sensing (RS) has a demonstrated capability in retrieving SMC over large areas with several operational products already available at different spatiotemporal resolutions. At the same time, cosmic-ray neutron sensing is a recently emerged approach in retrieving high temporal resolution SMC at intermediate spatial scales. The present study conducts an intercomparison between different RS-based soil moisture products, daily SMC retrievals from a cosmic-ray neutron sensor (CRNS) station and a network of in situ SoilNet wireless sensors installed at the Pinios Hydrologic Observatory ILTER site in central Greece for a time period of 2018-2019. The RS-based soil moisture products included herein are from NASA’s Soil Moisture Active Passive (SMAP) and Metop-A/B Advanced Scatterometer (ASCAT) satellite missions. The methodological workflow adopted includes standardized validation procedures employing a series of statistical measures to quantify the agreement between the different RS-based soil moisture products, CRNS-based SMC and the SoilNet ground truth data. Our study results contribute towards global efforts aiming at exploiting CRNS data in the context of soil moisture retrievals and their potential synergies with RS-based products. Furthermore, our findings provide valuable insights into assessing the capability of CRNS at retrieving more accurate SMC estimates at arid and semi-arid environments such as those found in the Mediterranean basin, while supporting also ongoing global validation efforts.

Keywords: Cosmic Ray Neutron Sensors; SMAP; ASCAT; SoilNet; Soil Moisture Content

How to cite: Detsikas, S. E., Petropoulos, G. P., Koutsias, N., Gasparatos, D., Pisinaras, V., Bogena, H., Wendland, F., Herrmann, F., and Panagopoulos, A.: A comparative study of SMAP and ASCAT satellite soil moisture products with cosmic-ray neutron sensing and in-situ data in a Mediterranean setting, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-5175, https://doi.org/10.5194/egusphere-egu23-5175, 2023.

The upper range limit of trees is the most conspicuous boundary on the Earth. However, the publicly available forest extent or forest cover datasets systematically underestimated sparse tree cover, which hindered our recognition of tree limit distribution and its drivers over cold and arid regions. Here, we built a three-step upscaling strategy, that integrates in situ measured vegetation types with spaceborne Light Detection and Ranging (LiDAR), microwave, and Landsat images in a Convolutional Neural Networks (CNN) classification algorithm, to develop a new map of the upper range limit of trees over the Three-River-Source National Park circa 2020 at 30 m resolution. The multi-satellite-based new products consider vertical structure information that could better detect sparse trees and better distinguish between the shrub, grass, and forest. Validation shows our result reveals high consistency with manual interpretations from Google Earth high-resolution images (R² = 0.97, slope = 0.99, ME = 18 m). Our proposed method provides a fast and effective tree limit mapping solution at the global scale.

How to cite: Xu, J., Wang, X., Lv, G., and Wang, T.: High-resolution map of the upper range limit of trees over the cold and arid region, a case study in the Three-River-Source National Park, Tibetan Plateau, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-6800, https://doi.org/10.5194/egusphere-egu23-6800, 2023.

The presence of clouds and aerosols in satellite imagery hamper their use to monitor, observe and analyze the Earth's surface. Multisensor fusion can alleviate this problem. The HISTARFM algorithm developed by Moreno-Martinez et al. (2020) can generate monthly gap-filled reflectance data at 30 m spatial resolution by blending Landsat (30 m pixel size every 16 days) and MODIS (500 m pixel size daily) data using a bias-aware Kalman filter. 

Cloud computing platforms such as Google Earth Engine (GEE) help us to efficiently process public data archives from different remote sensing data sources. Therefore, GEE allows us to adapt the HISTARFM algorithm to obtain gap-filled data at higher spatial resolution. To reduce the massive number of images involved in the process, the bias-aware Kalman filter blends the available and preprocessed HISTARFM monthly gap-filled reflectance (30 m pixel size every month) and Sentinel-2 (10 m pixel size at five days) data. The very high spatial gap-filled images provide reflectance information at feasible scales to obtain new products that improve decision-making activities in variable territories with complex topographies. Also, new derivative products (e.g. land cover maps, biophysical parameters, or phenological indicators) will provide the scientific community better understanding and monitoring of bio-geographical and ecoclimatic characteristics of the Earth.

Additionally, a reduction of the time resolution of the temporal series is manageable with this approach by linear interpolation producing five days of gap-filled reflectance Sentinel-2 data. The proposed approach shows promising preliminary results and provides gap-free reflectance Sentinel-2 images with their associated uncertainties. These results foster the development of improved near-real-time applications for crop and natural vegetation monitoring at continental scales.

How to cite: Izquierdo-Verdiguier, E., Moreno-Martínez, Á., Muñoz-Mari, J., Clinton, N., Vuolo, F., Atzberger, C., and Camps-Valls, G.: Enhanced and gap-free Sentinel-2 reflectance data at vast scales with GEE, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-8451, https://doi.org/10.5194/egusphere-egu23-8451, 2023.

Photovoltaic (PV) and wind are currently the highest growing renewable energies according to the annual World Energy Outlook 2022. However, in the case of solar PV in Europe, this growth is mainly driven by utility-scale installations. Distributed residential generation has many benefits, such as relieving the electrical grid and an increase of self-sufficiency. One challenge of this topic is to accurately estimate the rooftop PV potential of different regions, in order to best allocate economic resources and regulate accordingly. Multiple approaches have been proposed in the past, such as infering from proxy variables like population density, automatically analyzing residential 3D point clouds or automatically analyzing satellite images. The latter has gained popularity in the recent years given the increased availability of satellite imagery and the improvement of Computer Vision methods. However, in research, the analysis of satellite imagery is impeded by the lack of transparency, reproducibility, and standardization of methods. Studies are heterogeneous, target different types of potential with redundant efforts, and are mostly not open source or using private datasets for training. This makes it challenging for users of various backgrounds to find and use the existing approaches.

For these reasons, this paper proposes a conceptual framework that describes and categorizes the tasks that need to be considered when estimating PV potential, thus creating a clear framework along which the contents of this research report can be classified. Addidionally the open source workflow PASSION is introduced, which integrates the assessment of geographical, technical and economic potentials of regions under consideration along with the calculation of surface areas, orientations and slopes of individual rooftop sections. It also includes the detection of obstacles and existing PV installations. It is based on a novel two-look approach, in which three independent models are deployed in parallel for the identification of rooftops, sections and superstructures. The three models show a mean Intersection Over Union (IoU) between classes of 0.847, 0.753 and 0.462 respectively, and more importantly show consistent results in non-selected real life samples.

How to cite: Pueblas, R., Weinand, J., Kuckertz, P., and Stolten, D.: PASSION: a workflow for the estimation of rooftop photovoltaic potential from satellite imagery., EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-8825, https://doi.org/10.5194/egusphere-egu23-8825, 2023.

Distinguishing trees on agricultural land from forests is essential for a better understanding of the relationship between forests and human farming activities. However, it is difficult to separate them with remote sensing imagery since they share similar canopy cover, especially on the edge of the amazon rain forest, which has a much-complicated agriculture pattern. Except for annual crops and pasture, there are also lots of agroforestry applications and shifting cultivation, which integrates many tree systems. And those tree systems are not well separated from the forest in the existing land cover map. Recent techniques allow for the mapping of single trees outside of forests, now we take the next step by identifying those diverse tree-involved systems in agricultural land. Here we aim to generate a robust, cost-efficient method to distinguish trees within agricultural land from the forest. We started our exploration from Peruvian Amazon, where the competition for land has increased in the last decades, causing possible adverse effects on livelihoods and ecosystem services. Deep learning models, data sampling, and fine-tuning strategies are tested and optimized with PlanetScope satellite imagery. Our research target is to provide a tool for separating tree systems in farmland from the forest. It can also be used as a base map to explore the dynamic of agriculture transition and its impact on livelihoods and ecosystem services. 

How to cite: Yang, W., Ortiz Gonzalo, D., Tong, X., Pierre Johannes Gominski, D., Brandt, M., Kariryaa, A., Reiner, F., and Fensholt, R.: Separating tree systems in agricultural lands from forests using Deep learning, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-8916, https://doi.org/10.5194/egusphere-egu23-8916, 2023.

Due to climate change, understanding the changes in the water cycle has become a pressing issue. It is increasingly important to study prolonged periods of intense precipitation or dry spells to better manage water supply, infrastructure and agriculture. However, obtaining fine-scale precipitation data is challenging due to the intermittent nature of rain in time and space. Ground-based instruments could have mismatches between different regions due to spatial distribution, calibration, and complex topography. On the other hand, space-borne observations have uncertainties in their retrieval algorithms. This study proposes to deal directly with microwave images from space remote sensing, as this type of data makes it possible to study the evolution of the atmospheric water cycle on a global scale and with a temporal coverage of several decades by avoiding the uncertainties from retrieval methods. In recent years, convolutional neural networks have shown promising capabilities in identifying cyclones and weather fronts in large labelled climate datasets. However, these models required large labelled datasets for training and testing. The present study aims to test unsupervised segmentation approaches of microwave images, which are thus segmented into different classes. Instead of focusing only on one aspect, for example, precipitation, the obtained classes contain many physical properties. This is due to the fact that microwave brightness temperatures contain essential information relative to the atmospheric water cycle that can be used to derive many products such as rain intensity, water vapour, cloud fraction, and sea surface temperature. The unsupervised segmentation model consists of blocks of fully convolutional networks serving as feature extractors. Without labels, pseudo-targets from the feature extractors are used to train the model. The performance of the model in terms of intra-class and inter-class distances is compared with those of simpler models such as Kmeans. A major challenge in the unsupervised approach is validating and interpreting the resulting classes. Most of the obtained cluster patterns provide geographically coherent regions whose mode of variability of geophysical quantities can be highlighted. The presented study will then explore how the different classes computed by the unsupervised methods can be labelled and how the properties of the said classes change through time and space.

How to cite: Sambath, V., Viltard, N., Barthès, L., Martini, A., and Mallet, C.: Unsupervised Segmentation Of Microwave Brightness Temperatures To Study The Changes In The Water Cycle, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-13610, https://doi.org/10.5194/egusphere-egu23-13610, 2023.

The developments of global aquaculture ponds provide valuable socio-economic benefits in the Anthropocene epoch, also cause potential environmental and ecological impacts. However, the extent and trajectory of aquaculture ponds over the past 37 years remain unknown on a global scale. Our study maps the global distribution of aquaculture ponds over 9 periods (1984-1994, 1995-2000, and every 3 years from 2001 to 2021) based on a deep-learning method and Landsat observations. The total area of global aquaculture ponds expands from 10043.3 km² to 18779.70 km² and showed a slowing growth rate. Asia fishpond area accounts for up to 82% of the world's area. The extent of aquaculture ponds in Asia and South America have doubled in size since 1984. China, Vietnam, and Indonesia- the three countries with the largest fishpond area- exhibited the largest fishponds area at 2004-2006. Our study provides a critical basis for assessing spatial-temporal trajectory and potential influences of aquaculture ponds.

How to cite: Xu, Y. and Feng, L.: The Global Distribution and Trajectory of Aquaculture Ponds, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-14137, https://doi.org/10.5194/egusphere-egu23-14137, 2023.

Flooding is a natural disaster that has been increasing in recent years due to climate and land-use changes. Earth observations, such as Synthetic Aperture Radar (SAR) data, are valuable for assessing and mitigating the negative impacts of flooding. Cloud cover is highly correlated with flooding events, making SAR a preferable choice over optical data for flood mapping and monitoring.

Traditional methods for flood mapping and monitoring using SAR data, such as otsu and CVA, can be affected by noise, false detections due to shadows and occlusions, and geometric distortions. While automatic thresholding can be effective with these methods, manual adjustment of the threshold is often required to produce an accurate change map. 

Supervised deep learning methods using large amounts of labeled data could potentially improve the accuracy of flood mapping and monitoring. We have a large amount of earth observation data, but the availability of labeled data is limited and labeling data is time-consuming and requires domain expertise. On the other hand, Supervised model training on small datasets causes severe generalizability issues when inference is taken on a new site. 

To address these challenges, we propose a novel self-supervised method for mapping and monitoring floods on Sentinel-1 SAR time-series data. We propose a probabilistic model trained on unlabeled data using self-supervised techniques, such as reconstruction and contrastive learning. The model is trained to learn the spatiotemporal features of the area. It monitors the changes by comparing the latent feature distribution at each time stamp and generates change maps to reflect the changes in the area. 

We also propose a framework for flood monitoring that continuously monitor the area using time series data. This framework automatically detects the change point i.e. when the major change started reflecting on available SAR data. Our continuous monitoring framework combined with a better temporal resolution (better than Sentinel-1) can potentially detect flood events in an early stage, allowing for more time for evacuation planning. 

The model is evaluated on nine recent flood events from ‘Mekong’, ‘Somalia’, ‘Scotland’, ‘Australia’, ‘Bosnia’, ‘Germany’, ‘Spain’, ‘Bolivia’, and ‘Slovakia’ sites. We compared our results with traditional methods, and existing supervised and unsupervised methods. Our detailed evaluation indicates that our model is more accurate and generalizable to new sites. The model achieves an average Intersection Over Union (IoU) value of 70% and an F1 score of 81.14%, which are both higher than the scores of the previous best-performing method. Overall, our proposed model’s improvement range from 7-26% in terms of F1 and 8-31% in terms of IoU score. 

How to cite: Yadav, R., Nascetti, A., Azizpour, H., and Ban, Y.: Self-Supervised Contrastive Model for Flood Mapping and Monitoring on SAR Time-Series, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-14375, https://doi.org/10.5194/egusphere-egu23-14375, 2023.

Mapping of Land use and land cover (LULC) changes over time requires automated processes and has been investigated using various machine learning algorithms and, more recently, deep learning models for semantic classification. New applications of these models to different satellite data and areas are regularly published. However, studies on the transfer of these models to other data and study areas are rather scarce. In a previous study [1], we used multi-modal and –temporal Sentinel data for LULC classification using traditional and novel deep learning models. The data covered parts of the Amazon basin and was comprised of a twelve-month time series of radar imagery (Sentinel-1), combined with a singular multi-spectral image (Sentinel-2). All satellite images were captured throughout the year 2018. The label map (Collection 4) of the Amazon produced by the MapBiomas project [2] was used as training and test labels. Besides state-of-the-art models, we developed five variations of a deep learning model—DeepForest—which leverages on the multi-temporal and -modal aspect of the data. The best model variation (DF1c) reached an overall accuracy of 74.4% on the test data.

Currently we are investigating the transferability of these models to more recent data of the same region. The new dataset was processed in the same way as in the previous study. It comprises a Sentinel-1 time-series and a single Sentinel-2 images from 2020, with an updated version of the label map of the MapBiomas project (Collection 6). This posed some challenges, as the classification scheme changed and is not fully backwards compatible with the one used to train the DeepForest models. A test dataset was chosen in the state of Mato Grosso, as the satellite scenes cover most classes used in the classification scheme. However, this data exhibits some class imbalance, as two of the eleven classes are dominating the scene. All five DeepForest variations reached accuracies higher than 79% and thus generalize well on the major LULC classes. For comparison and to further improve our models, we currently retrain the models on the new, larger data set (114,376 training image tiles compared to 18,074). Preliminary results will be shown during the session.

References

• Cherif, E.; Hell, M.; Brandmeier, M. DeepForest: Novel Deep Learning Models for Land Use and Land Cover Classification Using Multi-Temporal and -Modal Sentinel Data of the Amazon Basin. Remote Sensing 2022, 14, 5000, doi:10.3390/rs14195000.
• MapBiomas Brasil; Available online: https://mapbiomas.org/en

How to cite: Hell, M., Brandmeier, M., and Nüchter, A.: Transfer Learning for LULC Classification on multi-modal data in the Amazon Basin, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-15020, https://doi.org/10.5194/egusphere-egu23-15020, 2023.

Riparian ecosystems are biodiversity hotspots and provide crucial services to human wellbeing. Currently, the knowledge of how riparian ecosystems respond to and in turn influence the variations of the environment remains considerably limited. As a first step toward filling the gap, this research aims to characterize the dynamics of riparian vegetation during the past several decades across multiple aquatic sites operated by the National Ecological Observatory Network (NEON) of the US. Specifically, it leverages high-resolution hyperspectral and lidar data collected by NEON’s airborne observational platform (AOP) surveys, the long-term records of satellite optical and radar imagery, and advanced data fusion and classification techniques to generate a time-series record of riparian vegetation on a seasonal-to-yearly basis. The maps derived will provide a new basis for understanding how riparian vegetation has changed across continental US, and for predicting how it is likely to change in the future. This work is sponsored by NSF’s Macrosystems Biology and NEON-Enabled Science (MSB-NES) Program (2021/9–2024/8), and the overarching goal of the project is to mechanistically link riparian vegetation dynamics to hydroclimate variations and assess the functional importance of riparian ecosystems to macrosystem fluxes of carbon and water.

How to cite: Jin, H. and Tai, X.: Spatiotemporal mapping of riparian vegetation through multi-sensor data fusion and deep learning techniques, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-17425, https://doi.org/10.5194/egusphere-egu23-17425, 2023.

Recent advancement in deep generative models (GANs) has brought the attention of many researchers to explore the feasibility of using realistic synthetic data as (i) a digital twin of the original dataset and (ii) a new approach to augment the original dataset. Previous works highlighted that GANs can replicate both esthetical and statistical characteristics of datasets, up to the point of being indistinguishable from real samples, even when being examined by domain experts. In addition, the weights learned during the unsupervised training of these generative models are useful to further extract specific features of interest from the given dataset. In geosciences, many computer vision tasks are related to semantic segmentation, from pore quantification to fossil characterization. In such a task, the labeling process becomes the main limiting point, being both time-consuming and requiring domain experts. Hence, in this study, we repurpose GANs to obtain self-labeled geological datasets for semantic segmentation to be readily applicable in geological machine learning workflows. In this work, we used trained style-based GANs of foraminifera specimens, ooids, and mudstones. Our experiments show that with one or a few labels, we can successfully generate self-labeled and synthetic datasets featuring the labels of interest. This achievement is pivotal in geosciences in exploring the idea of GANs for one-shot and few-shot segmentation and in minimizing the efforts of manual labeling for segmentation requiring domain experts. 

How to cite: Ferreira, I. and Koeshidayatullah, A.: Generating self-labeled geological datasets for semantic segmentation using pretrained GANs, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-888, https://doi.org/10.5194/egusphere-egu23-888, 2023.

Visual AI (artificial Intelligence) YOLOv5 algorithm was used in order to detect marine organisms from underwater images, and the test results showed an average high detection rate (>90%). As performance indicators of the AI model, both precision and recall showed very good performance, exceeding 0.95. So as to minimize the change in object detection performance according to the variation of underwater conditions, image correction was conducted, and more objects could be detected after image correction.

In order to determine which species the object detected in the video or image corresponds to, the performance was evaluated by AI learning classification model (YOLO-Classification), which is a deep learning algorithm (approximately 3% accuracy improved after image correction). We tried to identify the taxonomic species of organisms using deep learning, and although the number of target species was small, we achieved a classification accuracy of about 80% or more based on the data collected so far.

High-quality image DB data of the target species have to be established from a long-term perspective in order to accurately classify object (fish) species, and imaged taken from various angles of the target species must be collected simultaneously improve performance. 

As a prerequisite for measuring the size of an object detected in an image, MDE (Monocular Depth Estimation), a deep learning algorithm for estimating the depth of a mono camera image, was applied and the distance from a certain reference point was calculated with the MiDAS v3 algorithm. As a result of the MiDas v3 algorithm test, the excessive error has been reduced compared to before application and the distance measurement accuracy of up to 2m, which is longer than the guide stick length, has been obtained.

How to cite: Son, Y.-T., Jin, S.-Y., and Kang, T.-S.: Object detection and classification applying AI (computer vision) to underwater images, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-2203, https://doi.org/10.5194/egusphere-egu23-2203, 2023.

As a result of advancements in satellite technology, archives of remote sensing (RS) images are continuously growing, providing a valuable source of information for monitoring the Earth's surface. Researchers construct well-designed and ready-to-use datasets from the plethora of RS images for the broader community to make it easier to develop and compare novel algorithms, models, and architectures to further deepen our understanding of our planet from space. However, the descriptions of these datasets are often published in scientific papers as PDF files with several limitations:

• The target audience is typically domain experts familiar with scientific jargon;
• The work is required to adhere to a specific page limit;
• Once the document is published, it is difficult to update sections or to centralize discussions around it. 

To overcome these issues, here we introduce the concept of interactive dataset websites that aim at making the dataset and research based on it more accessible. With visual and interactive examples, users can see exactly how the data is structured and how the data can be used in different contexts. For example, when working with RS data, it is beneficial to get a quick overview of the geographical distribution. By providing more in-depth background information about data sources and product specifications, these websites can also help users understand the context in which the data was collected, how it might be relevant to their work, and how to avoid common pitfalls. Another important aspect of interactive dataset websites is the inclusion of example code for using, loading, and visualizing the data. Especially when working with RS images (e.g., multispectral, hyperspectral, synthetic aperture radar data, etc), it is often not trivial to visualize the data. Providing example code can be especially useful for researchers unfamiliar with the specific tools required to work with the data or to introduce to the community tools specifically written to make it easier to work with the dataset. Quick feedback can be vital, as it allows researchers to report problems or ask questions that the authors or community can address in an open and centralized manner. Creating these "living, ever-evolving documents" makes them an increasingly valuable resource for anyone working with the dataset, leading to more robust and reliable research.

It might seem daunting at first to create such an interactive dataset website, but due to recent open-source projects such as Executable Books (https://executablebooks.org/) and free hosting providers such as GitHub Pages (https://pages.github.com/), it has become relatively easy to produce and host such websites. The HTML content can be generated from Jupyter Notebooks, a tool that many researchers and data scientists are familiar with. To provide an example, in our talk we will showcase an interactive dataset website for the BigEarthNet-MM dataset, which you can find here: https://docs.kai-tub.tech/ben-docs/

How to cite: Clasen, K. N. and Demir, B.: Construction of Interactive Websites for Remote Sensing Datasets, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-3254, https://doi.org/10.5194/egusphere-egu23-3254, 2023.

Training datasets are a crucial component of any machine learning approach, with significant human effort spent creating and curating these for specific applications. However, a historical absence of standards has resulted in inconsistent and heterogeneous training datasets with limited discoverability and interoperability. Therefore, there is a need for best practices and guidelines for generating, structuring, describing, and curating training datasets.

The Open Geospatial Consortium (OGC) Testbed-18 initiative covered several topics related to geospatial data, focussing on issues around cataloguing and interoperability. Within Testbed-18, the Machine Learning Training Datasets task aimed to develop a foundation for future standardization of training datasets for Earth observation applications.

For this task, members from Pixalytics, FrontierSI, and Curtin University authored an Engineering Report that reviewed:
·       Examples of how training datasets have been used in Earth observation applications
·       The current best-practice methods for documenting training datasets
·       The various requirements for training dataset metadata
·       How the Findability, Accessibility, Interoperability, and Reuse (FAIR) principles apply to training datasets

The Engineering Report provides a foundation that OGC can leverage in creating the future standard for machine learning training data for Earth observation applications. The Engineering Report also provides a useful overview of the state of work and key considerations for anyone wishing to improve how they document their training datasets.

In our presentation, we discuss the key findings from the Engineering Report, including key metadata identified from Earth observation use cases, the current state of the art, thoughts on cataloguing and describing training data quality, and how the FAIR principles apply to training data. 

How to cite: Lavender, S., Adams, C., Ivánová, I., and Williams, K.: OGC Testbed-18 Machine Learning Training Datasets Task: Application of standards to Machine Learning training datasets, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-3493, https://doi.org/10.5194/egusphere-egu23-3493, 2023.

In the field of spatial machine learning, access to high-quality data sets is a crucial factor in the success of any analysis or modeling project, especially in subsurface hydrology. However, finding and utilizing such data sets can be a challenging and time-consuming process. This is where AwesomeGeodataTable comes in. AwesomeGeodataTable aims to establish a community-maintained searchable table of data sets that are easily usable as predictors for spatial machine learning starting with the focus on subsurface hydrology. With its user-friendly interface and currently small but growing number of data sets, AwesomeGeodataTable will make it easier for researchers and practitioners to find and use the data they need for their work. It brings the usability of existing data set collections to a next level through adding features for filtering and searching meta information on data sets. This talk will introduce attendees to the AwesomeGeodataTable project, its goals and features, and how they can get involved in maintaining and extending its database and expanding its features and user experience. Overall, AwesomeGeodataTable is a valuable resource for anyone working in the field of spatial machine learning, and we hope to see it become a widely used and respected resource in the community.

How to cite: Nölscher, M., Cooke, A.-K., Willkommen, S., Gomez, M., and Broda, S.: AwesomeGeodataTable - Towards a community-maintained searchable table for data sets easily usable as predictors for spatial machine learning, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-5394, https://doi.org/10.5194/egusphere-egu23-5394, 2023.

In the context of the project CORTEX, a study was carried out to build a method to generate synthetic images with associated labels for hyperspectral use cases. Such a method is interesting in the case where too few annotated data are available to train a deep neural network (DNN). The context of hyperspectral images is particularly suited for this problem since labeled datasets of hyperspectral images are scarce and generally of very small size.

Therefore, the first step of the project was to define an interesting hyperspectral use case to carry out the study. More concretely, generative models must be trained to achieve this objective. It means that a set of hyperspectral images and their associated ground truth are necessary to train the models. A dataset was created with PRISMA images associated with the IGN BD forest v2. The result is a segmentation dataset of 1268 images of size 256x256 pixels with 234 spectral bands. The associated ground truth includes 4 classes: not-forest, broad-leaved forest, coniferous forest and mixed forest. To correctly match the ground truth and the images, an important work was done about the improvement of the geolocalization of the PRISMA images by coregistering patches with Sentinel-2 images. We want to underline the interest of this database that remains from our knowledge one the few large scale HS database and is made available on the platform Zenodo.

Then, a segmentation model was trained with the dataset to assess its quality and the feasibility of the task of forest-type segmentation. Good results were obtained using a Unet-EfficientNet segmentation DNN. It showed that the dataset is coherent but the problem still difficult since the ‘mixed forest’ class remains challenging to identify.

Finally, an important research work was conducted to develop a Generative Adversarial Network method able to generate synthetic hyperspectral images. The state-of-the-art StyleGAN2 was modified to this purpose. An additional discriminator was added and tasked with the job of discriminating synthetic and real images in a reduced image space. Good results were obtained for the generation of 32-bands images, but the results worsen when increasing more the number of bands. The difficulty of the problem appears directly linked to the number of bands that we look to generate.

The final goal was to generate synthetic ground truth masks alongside the images and the method SemanticGAN was elected to address this problem. Since this method is based on StyleGAN2, the improvements of StyleGAN2 for HS images were included in the method. At the end, a modified version of SemanticGAN was proposed. The discriminator assessing the coherence between masks and images was modified to use an image of reduced dimension and a specific training strategy was introduced to help the convergence. The initial expectation was that the generation of masks would help stabilizing the generation of images, but the experiments showed the contrary. Early results are promising, but more research will be necessary to obtain couples of images and masks that could be used to train a DNN.

How to cite: De Vieilleville, F., Lagrange, A., Dublé, N., and Le Saux, B.: Towards generation of synthetic hyperspectral image datasets with GAN, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-7299, https://doi.org/10.5194/egusphere-egu23-7299, 2023.

The global availability of dense point-clouds provides the potential to better assess changes in our dynamic world, particularly environmental changes and natural hazards. A core step to make use of modern point-clouds is to have a reliable classification and identify features of importance for a successful classification. However,  the quality of classification is affected by both the classifier and the complexity of the features which describe the classes. To address the limitations of classification performance, we attempt to answer the question: To what extent can a classifier learn the separation into different classes based on the available features in a given training dataset?

We compare several measures of class separability to assess the descriptive value of each feature. A ranked list is generated that includes all individual features as well as all possible combinations within specific groups. Selecting high-ranked features based on their descriptive value allows us to summarize datasets without losing essential information about the individual classes. This is an important step in processing existing training data or in setting priorities for future data collection.

In our application experiments, we compare geometric and echo-based features of lidar point-clouds to obtain the most useful sets of features for separating ground and vegetation points into their respective classes. Different scenarios of suburban and natural areas are studied to collect various insights for different classification tasks. In addition, we group features based on various attributes such as acquisition or computational cost and evaluate the benefits of these efforts in terms of a possible better classification result.

How to cite: Hess, M., Rheinwalt, A., and Bookhagen, B.: Point-Cloud Class Separability: Identifying the Most Discriminative Features, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-12352, https://doi.org/10.5194/egusphere-egu23-12352, 2023.

The availability of large training datasets (TDS) has enabled much of the innovative use of Machine Learning (ML) and Artificial Intelligence (AI) in fields such as computer vision or language processing. In Earth Observation and geospatial science/applications, the availability of TDS has generally been limited and there are a number of specific geospatial challenges to consider (e.g. spatial reference systems, spatial/spectral/radiometric/temporal resolutions). Creating TDS for EO applications commonly involves labor intensive processes and the willingness to share such datasets has been very limited. While the current open accessibility of EO datasets is unprecedented, the availability of training and ground truth datasets has not improved much over the last years, and this is limiting the potential innovative impact that new ML/AI methodologies could have in the EO domain. Next to general availability and accessibility, further challenges need to be addressed in terms of making TDS interoperable and findable and lowing the barriers for non-geospatial experts.

In the response to these challenges, ESA has initiated development of the Earth Observation Training Data Lab (EOTDL). EOTDL is being developed on top of federated European cloud infrastructure and aims to address the EO community requirements for working with TDS in EO workflows, adopting FAIR data principles and following open science best-practices.

The specific capabilities that EOTDL will support include:

• Repository and Curation: host, import and maintain training datasets, ground truth data, pretrained models and benchmarks, providing versioning, tracking and provenance.
• Tooling: provide a set of integrated open-source tools compatible with the major ML/AI frameworks to create, analyze and optimize TDS and to support data ingestion, model training and inference operations.
• Feature engineering: Link with the main EO data archives and EO analytics platforms to support feature engineering and large-scale inference.
• Quality assurance: embed QA throughout the offered capabilities, also taking advantage of automated deterministic checks and defined levels of TDS maturity.

To achieve these goals, EOTDL is building on proven technologies, such as STAC (Spatio Temporal Asset Catalog) to support data cataloguing and discoverability, openEO and SentinelHub APIs for EO data access and feature engineering, GeoDB for vector geometry and attribute handing, and EoxHub to support interactive tooling. The EOTDL functionality will be exposed via web-based GUIs, python libraries and command line interfaces.  

A central objective is also the incentivization of community engagement to support quality assurance and encourage the contribution of datasets. For this award mechanisms are being established. The initial data population consists of around 100 datasets while intuitive data ingestion pipelines allow for continuous community contributions. Three defined product maturity levels are linked to QA procedures and support the trustworthiness of the data population. The development is coordinated with Radiant ML Hub to seek synergies rather than duplicate the offered capabilities.  

This presentation will showcase the current development status of EOTDL and discuss in detail some key aspects such as the data curation with STAC and the adopted quality assurance and feature engineering approaches. A set of use cases that establish new TDS creation tools and result in large scale datasets are presented as well.

How to cite: Griffiths, P., Pedro, J., Brandt, G., Meissl, S., Milcinski, G., and Moreno, L.: The Earth Observation Training Data Lab (EOTDL) - addressing training data related needs in the Earth Observation community., EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-14061, https://doi.org/10.5194/egusphere-egu23-14061, 2023.

The development of Artificial Intelligence (AI), especially Machine Learning (ML) technology, has injected new vitality into the geospatial domain. Training Data (TD) plays a fundamental role in geospatial AI/ML. They are key items for training, validating, and testing AI/ML models. At present, open access Training Datasets (TDS) are usually packaged into public or personal file repository, without a standardized method to express its metadata and data content, making it difficult to be found, accessed, interoperated, and reused.

Therefore, based on the Open Geospatial Consortium (OGC) standards baseline, the OGC Training Data Markup Language for AI (TrainingDML-AI) Standard Working Group (SWG) tried to develop the TD model and encoding methods to exchange and retrieve TD in the Web environment. The scope includes: how TD are prepared, how to specify different metadata used for different AI/ML tasks, how to differentiate the high-level TD information model and extended information models specific to various AI/ML applications. The work will describe the latest progress and status of the standard development.

The TrainingDML-AI conceptual model includes the most relevant entities of the TD covering from dataset to individual training samples and labels. It specifies how and into which parts of the TD should be decomposed and classified. The core concepts include: AI_TrainingDataset, which represents a collection of training samples; AI_TrainingData, which is an individual training sample in a TDS; AI_Task, which identifies what task the TDS is used for; AI_Label, which represents the label semantics for TD; AI_Labeling, which provides the provenance for the TD; AI_TDChangeset, which records TD changes between two TDS versions; DataQuality, which can be associated with the TDS to document its quality.

The TrainingDML-AI content model focuses on implementations with basic attributes defined for off-the-shelf deployment. Concepts related to the EO AI/ML applications are defined as additional elements. Six key components are highlighted:

• Training Dataset/Data. AI_AbstractTrainingDataset indicates the TDS, while each training sample is represented as AI_AbstractTrainingData. AI_EOTrainingDataset and AI_EOTrainingData are defined to convey attributes specific to EO domain.
• AI_EOTask is proposed by extending AI_AbstractTask to represent specific AI/ML tasks in the EO domain. The task type can refer to a particular type defined by an external category.
• Labels for each individual training sample can be represented using features, coverages, or semantic classes. The AI_AbstractLabel is extended to specify AI_SceneLabel, AI_ObjectLabel, and AI_PixelLabel respectively.
• AI_Labeling records basic provenance information on how to create the TDS. It includes the labeler and labeling procedure, which can be mapped to the agent and activity respectively in W3C PROV.
• DataQuality and QualityElements defined in the ISO 19157-1 are used to align with the existing efforts on geographic data quality.
• Change procedures of the TDS are documented in the AI_TDChangeset, which composes of changed training samples in the collection level.

Finally, use case scenarios and best practices are provided to illustrate intended use and benefits of TrainingDML-AI for EO AI/ML applications. Totally five different tasks are provided, covering scene classification, object detection, semantic segmentation, change detection and 3D model reconstruction. Some software implementations including pyTDML and LuojiaSet are also presented.

How to cite: Yue, P., Shangguan, B., and Ziebelin, D.: The OGC Training Data Markup Language for Artificial Intelligence (TrainingDML-AI) Standard, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-16998, https://doi.org/10.5194/egusphere-egu23-16998, 2023.

Machine Learning (ML) algorithms had successfully contributed in the creation of automated methods of recognizing patterns in high-dimensional data. Remote sensing data  covers  wide  geographical areas and could be used to solve the problem of the demand of various  in-situ data.  Lithologicall mapping using remotely sensed data  is one of the most challenging  applications of ML algorithms. In the framework of the “AI for Geoapplications” project , ML and especially Deep Learning (DL) methodologies are investigated  for  the identification and characterization of the lithology based on remote sensing data in various  pilot areas  in Greece.  In order to train and test the various ML algorithms, a dataset consisting of  30 ROIs selected  mainly  from low -vegetated areas,  that cover 2% of the total  area of Greece was created . For each  ROI 

• the corresponding shape file  with the lithological units
• the corresponding  Sentinel2 (10 bands)  and/or Aster (14 bands) images

are provided

The dataset is  being publicly  available in the cloud along with the necessary code for visualization and processing.

How to cite: Vernikos, I., Giannopoulos, G., Christopoulou, A., Begaj, A., Stefouli, M., Bratsolis, E., and Charou, E.: A dataset of Earth Observation Data for Lithological Mapping using Machine Learning, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-17570, https://doi.org/10.5194/egusphere-egu23-17570, 2023.

While deep machine learning approaches are getting pervasively used in remote sensing and modeling the earth, difficulties due to the size of satellite data are always pains for scientists in implementing such experiential software. We present a programming model for geospatial machine learning based on TorchGeo and PyTorch, which are getting the de fact standards in programming with PyTorch/Python.  TorchGeo is open-sourced and designed to make it simple for remote sensing experts to explore machine learning solutions. Our objective is to allow machine-learning programs using TorchGeo to scale leveraging proprietary high-performance computing (HPC) and multicloud HPC resources, from ones notebook. One of key technologies specifically needed in geospatial machine learning is the smart integration of peta-scale data services and data-distributed parallel frameworks.  We implement such a platform as a part of IBM Research Geospatial Discovery Network (GDN) and experiment segmentation tasks such as flood detection from satellite data to show its scalability.

How to cite: Tatsubori, M., Kimura, D., Moriyama, T., Simumba, N., and Ishikawa, T.: A Programming Model for Geospatial Machine-Learning with Scalability in Hybrid Multiclouds, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-3441, https://doi.org/10.5194/egusphere-egu23-3441, 2023.

Earth observation (EO) data are critical for monitoring the state of planet Earth and can be helpful for various real-world applications [1]. Although numerous benchmark datasets have been released, there is no unified platform for developing and fairly comparing deep learning models on EO data [2]. For deep learning methods, the backbone networks, hyper-parameters, and training details are influential factors while comparing the performances.. However, existing works usually neglect these details and even evaluate the performance with different training/validation/test dataset splits. This makes it difficult to fairly and reliably compare different algorithms. In this study, we introduce the EarthNets platform, an open deep-learning platform for remote sensing and Earth observation. The platform is based on PyTorch [3] and TorchData. There are about ten different libraries, covering different tasks in remote sensing. Among them, Dataset4EO is designed as a standard and easy-to-use data-loading library, which can be used alone or together with other high-level libraries like RSI-Classification (for image classification), RSI-Detection (for object detection), RSI-Segmentation (for semantic segmentation), and so on. Two factors are considered for the design of the EarthNets platform: the first one is the decoupling between dataset loading and high-level EO tasks. As there are more than 400 RS datasets with different data modalities, research domains, and download links, efficient preparation of analysis-ready data can largely accelerate the research for the whole community. The other factor is to bring advances in machine learning to EO by providing new deep-learning models. The EarthNets platform provides a fair and consistent evaluation of deep learning methods on remote sensing and Earth observation data [4]. It also helps bring together the remote sensing and a larger machine-learning community. The platform, dataset collections are publicly available at https://earthnets.github.io.

[1] Zhu, Xiao Xiang, et al. "Deep learning in remote sensing: A comprehensive review and list of resources." IEEE Geoscience and Remote Sensing Magazine 5.4 (2017): 8-36.

[2]Long, Yang, et al. "On creating benchmark dataset for aerial image interpretation: Reviews, guidances, and million-aid." IEEE Journal of selected topics in applied earth observations and remote sensing 14 (2021): 4205-4230.

[3] Paszke, Adam, et al. "Pytorch: An imperative style, high-performance deep learning library." Advances in neural information processing systems 32 (2019).

[4] Xiong, Zhitong, et al. "EarthNets: Empowering AI in Earth observation." arXiv preprint arXiv:2210.04936 (2022).

How to cite: Xiong, Z. and Zhu, X. X.: EarthNets: An Open Deep Learning Platform for Earth Observation, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-3501, https://doi.org/10.5194/egusphere-egu23-3501, 2023.

Every experiment starts with the data, which needs to be fine-tuned for the specific use-case. We call this "analysis ready data (ARD)". In some cases, for the sake of reusability and comparability, the specifications for ARD are well defined. In many other cases, however, the procedures are not yet mature enough to support standardisation. In Earth Observation (EO) field this is especially true, as the whole community is moving from (semi)manually analysing individual scenes, from the time there were any data barely available, to processing of time-series, now that Landsat and Sentinel made this possible. We are now even facing a problem where there is simply too much of data, with PBs of open and commercial imagery being readily available. With the data being distributed at different places (Copernicus Data Access Service for Sentinel, AWS for Landsat) the challenge is further magnified. Machine learning (ML) approach can address the challenge of shifting through data, but ML as well requires data to be pre-processed for purpose and made available at the place where ML is running. Therefore, it is essential to have facility, which can generate ARD data customised for the specific analysis' requirements.

Sentinel Hub (SH) is a satellite imagery processing service, which is capable of on-the-fly gridding, re-projection, re-scaling, mosaicking, compositing, orthorectification and other actions required, either for integration in web-applications, where pictures are mostly served, or in ML and similar analysis processes, where pixel values and statistics are essential. SH works with original satellite data files and does not require replication or pre-processing.  It uses cloud infrastructure and innovative methods to efficiently process and distribute data in a matter of seconds. Sentinel Hub gives access to a rich collection of satellite data including a full set of Sentinel satellites, Landsat collections, commercial VHR collections and other complimentary collections. It also provides an ability for users to onboard their own data in one of the standardised formats. Furthermore, the data located at different clouds, can be fused together in one single process, benefiting from the variability and volume of different sensors.

There are two main capabilities, which make SH especially fit for purpose of generating on-demand ARD data. First one is the support for user-provided processing scripts, which are a set of recipes on what should happen with the sensor data (band composites, indices, even simple neural networks combining available data). The second one is a set of processing orchestration options. There is a Process API for immediate, access to the pixel values. Statistical API is optimised for time-series analysis, which aggregates the data over specific area of interest and provides configurable statistics through time. And then there are asynchronous siblings of these services, which are fine-tuned for large scale processing - if one wants to prepare ML features for the entire continent or get time-series for millions of agriculture parcels.

We will present the technology behind the scenes, making the processing possible, as well as several use-cases, how one can efficiently make use the service in ML.

How to cite: Milcinski, G. and Kolaric, P.: Sentinel Hub - federated on-demand ARD generation, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-4160, https://doi.org/10.5194/egusphere-egu23-4160, 2023.

Understanding and quantifying the risk of the physical impacts of climate change and their subsequent consequences have crucial importance in the changing climate for both businesses and society more widely. Historically, modelling workflows to assess such impacts have been bespoke and constrained by the data they can consume, the compute infrastructure, the expertise required to run them and the specific ways they are configured. Here we present, a cloud-native modelling framework for running geospatial models in a flexible, scalable, configurable, user-friendly manner. This enables models (physical or ML/AI) to be rapidly onboarded and composed into workflows. These workflows can be flexible, dynamic and extendable, running as for historical events, or as forecast ensembles, with varying data inputs, or extended to model impact in the real world (e.g. for example to infrastructure and populations). The framework supports the streamlined training and deployment of AI models, which can be seamlessly integrated with physical models to create hybrid workflows. We demonstrate the application and features of the framework for the examples of flooding and wildfire.

How to cite: Edwards, B., Fraccaro, P., Stoyanov, N., Jones, A., Butt, J., Kuehnert, J., Taylor, A., and Garikipati, B.: A flexible, scalable, cloud-native framework for geospatial modelling, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-7233, https://doi.org/10.5194/egusphere-egu23-7233, 2023.

Fast response to natural catastrophe events is crucial in our fast changing world. Creating comprehensible solutions based on the Earth Observation (EO) and geospatial data is complex and requires combining multiple data sources and maintaining high level of configuration parameters.

In this talk we discuss the application of microservices architecture to tackle some of the issues inherent to building products based on EO and geospatial data. We will present how decomposing sophisticated algorithms into small services can help with continuous delivery, scaling and deployment of large, complex applications that can be reused for various products. This architecture enables reproducibility of analysis which is a crucial component for applying machine learning and automation into any EO based product. We will also address the additional complexity of creating a distributed system as well as high dependency on data consistency and availability.

How to cite: Sarna, K., Hiekkasaari, J., and Taajamo, J.: Microservice architecture to enable fast assessment of the NatCat events based on EO and geospatial data, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-11709, https://doi.org/10.5194/egusphere-egu23-11709, 2023.

In many parts of the world, reforestation is an ongoing activity, but due to the deforestation processes (e.g. the change in soil conditions, agricultural expansion, and infrastructure expansion such as urbanisation or road building), the success rate of replanting is far from sure; therefore, it is essential to:

• have a good idea of the pre-planting conditions at the location,
• monitor the growth,
• improve the growing conditions whenever possible, and
• adapt the site selection criteria

In the proposed method, it is not possible to change the site selections of already planted locations, but it is possible to monitor the selected location and check under which conditions the trees are growing best. 

Several data sources are identified to predict plant health and stress, first to establish a baseline and, from this baseline, project into the future (short and mid-term). We compute the main vegetation index (NDVI) from the high-resolution image data provided by Planet (through the NICFI Basemaps for Tropical Forest Monitoring program). The historical NDVI values are obtained from the Sentinel 2 (and potentially LandSat) data at lower resolutions. Environmental conditions are added to the stress index by extracting the relevant meteorological parameters from the ERA5 database (temperature and precipitation) to compute the drought indices (e.g. KBDI/SPI/SPIE) and water availability (AWC) with the dominant soil type, supplemented with supporting indices from the satellite data (e.g. NDWI/SAVI/EVI-2).

For reforestation projects, it is vital to monitor the impact of environmental parameters on plant health and stress, and to assist with the forest maintenance of the sites, we built time series models for temperature, precipitation, and various vegetation indices to create a baseline for site-specific growing conditions. Deep Learning (DL) models like semantic segmentation based on Convolutional Neural Network (CNN) can be built on top of it using transfer learning to extract the features from pre-trained models using large (global) datasets. The model can not only predict tree health but can also be used to predict growing conditions in the near future by flagging out potential dry periods before they happen.

The high-resolution remote sensed products are available in the (sub)tropical zone [30N - 30S], while the lower resolution products and the ERA5 data have a global cover. The test sites in this study are example sites, but the developed method can be applied to any reforestation monitoring project. The result of the analysis is a near-term growth indicator, which can be used to adjust the growing conditions of the site, as well as assist with the site selection for new reforestation projects (based on the established baseline and predictions).

The next step, after validation, is to create a dashboard where the user can select any location (within the data domain) and construct the baseline and prediction, based on available information.

How to cite: Van den Dool, H. G. J. and Bidwai, D.: Satellite data as a predictor for monitoring tree health and stress in reforestation projects, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-13481, https://doi.org/10.5194/egusphere-egu23-13481, 2023.

Key variable of earth observation (EO) systems is precipitation, as indicated by the wide spectrum of applications that is involved (e.g., water resources and early warning systems for flood/drought events). During the last decade, the EO community has put significant research efforts towards the development of satellite-based precipitation products (SPPs), however, their deployment in real-world applications has not yet reached the full potential, despite their ever-growing availability, spatiotemporal coverage and resolution. This may be associated with the reluctancy of end-users to employ SPPs, either worrying about uncertainty and biases inherited in SPPs or even due to the existence of multiple SPPs, whose performance fluctuates across the globe, and thus making it difficult to select the most appropriate SPP (some sort of a choice paradox). To address this issue, this work targets the development of an explainable machine learning approach capable of integrating multiple satellite-based precipitation (P) and soil moisture (SM) products into a single precipitation product. Hence, in principle, to create a new dataset that optimally combines the properties of each individual satellite dataset (used as predictors), better matching the ground-based observations (used as predictand, i.e., reference dataset). The proposed approach is showcased via a benchmark dataset consisted of 1009 cells/locations around the world (Europe, USA, Australia and India), highlighting its robustness as well as its application capability which are independent of specific climatic regimes and local peculiarities.

How to cite: Tsoukalas, I., Kossieris, P., Brocca, L., Barbetta, S., Mosaffa, H., and Makropoulos, C.: Can machine learning help us to create improved and trustworthy satellite-based precipitation products?, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-13852, https://doi.org/10.5194/egusphere-egu23-13852, 2023.

The key to drive innovation in EO science and applications, boosting geospatial mass adoption and in turn ‘geo-enabling’ companies, researchers and institutions, is moving away from complex, inefficient, and expensive workflows and making fundamental changes in ML practices. This is where geospatial MLOps, and platforms such as Picterra play a crucial role: cloud-native, shared platforms offer user friendly and efficient interfaces, smart toolkit and features paired with auto-scaling infrastructure and state of the art deep learning architecture. They allow to create and operate geospatial ML models at scale, enabling organizations to complete geospatial ML projects faster than ever before.

MLOps platforms systemize the process of building and training experimental machine learning models as well as translating them into production. This workflow efficiency empowers teams working with massive datasets, and allows organizations to leverage data analytics for decision-making and building better customer experiences.

Achieving productivity and speed requires streamlining and automating processes, as well as building reusable assets that can be managed closely for quality and risk. When significant model drift is detected, the ability to retrain and redeploy ML models in an automated fashion is crucial to ensure business continuity.

Shared platform, managed infrastructure, and integrable architecture results in streamlined pipelines and straightforward integration. This agility reduces the time to value and frees up time to serve more use cases, leading to increased value to the business. Companies implementing geospatial MLOps can speed up model training times, dramatically improve accuracy, and go from an idea to a live solution in just days – without increasing headcount or technical debt. Over time, they will also collect a library of strategic ML assets that will enable them to act on timely data - fast.

Using Picterra as a prime example of geospatial ML platform built with MLOps processes in its core, we will dive into how it facilitatesthe key steps of ML workflows incl: 

• Direct access to a diverse range of satellite imagery sources via the platform ie. Sentinel-1/2, Planetscope, open aerial imagery campaigns, ingesting WMS//XYZ server streams.
• Compatibility with any geospatial imagery sources (e.g. Optical, SAR, hyperspectral, thermal infrared, etc.) and possibility to connect to data cloud storage or directly upload via web interface, besides the above mentioned images servers.
• A unequalled MLOps interface to prototype the extraction of new information from imagery around any custom defined use case ie. biodiversity monitoring, crops mapping and classification, assets management and many more. Trained model are directly served and made available for inference at large scale.
• Extensive toolset on explainable & interpretable AI which is bringing robustness & efficiency in creating geospatial Machine Learning models for example dataset exploration 
• Fast turnaround time in creating and validating Machine Learning models to save time and resources, thanks to the auto-scaling infrastructure leveraging Kubernetes and an intuitive interface for fast prototyping.
• A unique set of advanced GIS pre/post-processing tools to manage imagery and the geospatial outputs extracted.
• A complete API interface and Python library to further integrate with existing workflows or softwares (e.g. ESRI ArcGIS, Safe FME, etc.)

How to cite: de Morsier, F. and Rebetez, J.: MLOps in practice: how to scale your geospatial practice with cloud-based shared MLOps platform, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-14353, https://doi.org/10.5194/egusphere-egu23-14353, 2023.

Analysing EO data is a complex process, and solutions often require custom tailored algorithms. On top of that, in the EO domain most problems come with an additional challenge: How can the solution be applied on a large scale?

Within the H2020 project Global Earth Monitor (GEM) we have updated and extended eo-learn with additional functionalities that allow for new approaches to scalable and cost-effective Earth Observation data processing. We have tied it with the Sentinel Hub’s unified main data interface (Process API), the Data Cube processing engine for constructing analysis-ready adjustable data cubes using Batch Process API, and, finally, the Statistical API and Batch Statistical API to streamline access to spatio-temporally aggregated satellite data.

As part of GEM processing framework, we have built eo-grow which facilitates extraction of valuable information from satellite imagery. eo-grow tackles the issues of scalability by enabling coordination of clusters to run the EO workflows over large areas using  Ray. At the same time the framework provides reproducibility and traceability of the experiments using schemed input configurations and their validation.

In eo-grow a workflow based solution is wrapped into a pipeline object, which takes care of parametrization, logging, storage, multi-processing, data management and more. The pipeline object is configured via a well-defined schema allowing straightforward experimentation and scaling up: going to larger area of interest, running on different time interval, or tweak of any other pipeline parameter becomes just a matter of updating (json) configuration, which additionally serve as record of the experiment.

eo-grow library has been publicly released on GitHub: https://github.com/sentinel-hub/eo-grow. The documentation available in the repository provides the overview of the eo-grow general structure, its core objects, and instructions on installation and using eo-grow with command line interface. Additional repository, https://github.com/sentinel-hub/eo-grow-examples showcases eo-grow on a few use-cases.

In the presentation we will introduce the framework and showcase its usability on concrete examples. We will illustrate how eo-grow is used in large-scale research experiments, explain its role in reproducibility and show how the no-code approach and code reuse facilitate the productionalization of the workflows.

How to cite: Batič, M., Lukšič, Ž., and Milcinski, G.: eo-grow - Earth Observation framework for scaled-up processing in Python, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-16802, https://doi.org/10.5194/egusphere-egu23-16802, 2023.

How to cite: Pardo Meza, R., Quiané-Ruiz, J.-A., Demir, B., and Markl, V.: Wayang AgoraEO Plugin: The Framework for Scalable EO Workflows, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-17046, https://doi.org/10.5194/egusphere-egu23-17046, 2023.

An LSTM-based distributed hydrologic model for an urban watershed of Korea was developed. The input of the model is the time series of the 10-minute radar-gauge composite rainfall data and 10-minute temperature data at the 239 model grid cells, and the output of the model is the 10-minute flow discharge at the watershed outlet. The Nash-Sutcliffe Efficiency (NSE) coefficients of the calibration period (2013-2016) and validation period (2017-2019) were 0.99 and 0.67, respectively. Normal events were better predicted than the extreme ones. Further in-depth analyses revealed that: (1) the model composes the watershed outlet flow discharge by linearly superimposing multiple time series created by each of the LSTM units. Unlike conventional hydrologic models, most of these time series greatly fluctuated in both positive and negative domain; (2) the runoff to rainfall ratio of each of the model grid cells does not reflect its counterpart parameters of the conceptual hydrologic models  revealing that the model simulates the watershed responses in a unique manner; (3) the model successfully reproduced the soil-moisture dependent runoff processes, which is an essential prerequisite of continuous hydrologic models; (4) Each of the LSTM units have different temporal sensitivity to a unit rainfall stimulus, and the LSTM units that is sensitive to rainfall input have greater output weight factors nearby the watershed outlet, and vice versa. This means that the model learned a mechanism to separately consider the hydrologic components with distinct response time such as direct runoff and the low frequency baseflow. 

Acknowledgement

This research was supported by the Basic Science Research Program (Grant Number: 2021R1A2C2003471) and the Basic Research Laboratory Program (Grant Number: 2022R1A4A3032838) through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT.

How to cite: Kim, D. and Lee, Y.: Machines simulate hydrologic processes using a simple structure but in a unique manner – a case study of predicting fine scale watershed response on a distributed framework, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-245, https://doi.org/10.5194/egusphere-egu23-245, 2023.

This study developed a distributed hydrologic model based on Long Short-Term Memory (LSTM) to predict flow discharge of Joongrang stream located in a highly urbanized area in Seoul, Korea. The model inputs are the time series of 10-minute radar-gauge composite precipitation data at 239 grid cells (1km²) in the watershed and the Normalized Difference Vegetation Index (NDVI) data derived from Landsat 8 images and the model output is the 10-minute flow discharge at the watershed outlet as output. The model was trained for the calibration period of 2013-2016 and was validated for the period of 2017-2019. The NSE value over the validation period corresponding to the optimal model architecture (256 LSTM hidden layers) with and without NDVI input data was 0.68 and 0.52, respectively, which suggests that the machine can learn dynamic processes of soil infiltration and plant interception from the remotely sensed information provided by satellite.

This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (No. NRF-2022R1A4A3032838). 

How to cite: Lee, J. and Kim, D.: Effectiveness of Satellite-based Vegetation Index for Simulating Watershed Response Using an LSTM-based model in a Distributed Framework, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-339, https://doi.org/10.5194/egusphere-egu23-339, 2023.

Natural Language Processing (NLP), empowered by the most recent developments in Deep Learning, demonstrates its potential effectiveness for handling texts. Urban water research  benefits from both subfields of NLP, namely, Natural Language Understanding (NLU) and Natural Language Generation (NLG). In this work, we present three recent studies that use NLP for: (1) automated processing and responding to registered customer complaint within Dutch water utilities, (2) automated collection of up-to-date water-related information from the Internet, (3) extraction of key information about chemical compounds and pathogen characteristics from scientific publications. These applications, using the latest NLP models and tools (e.g., Rasa, Spacy), take into account studies on both water quality and quantity for the water sector. According to our findings, NLU and rule-based text mining are effective in extracting information from unstructured texts. In addition, NLU and NLG can be integrated to build a human-computer interface, such as a value-based Chabot to understand and address the demands made by customers of water utilities.

How to cite: Vertommen, I., Tian, X., Pronk, T., Seshan, S., Paraskevopoulos, S., and Wols, B.: Exploring the Value of Natural Language Processing for Urban Water Research, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-1218, https://doi.org/10.5194/egusphere-egu23-1218, 2023.

Accurate flood frequency analysis is essential for developing effective flood management strategies and designing flood protection infrastructure, but it is challenging due to the complex, nonlinear hydrological system. In regional flood frequency analysis (RFFA), the flood quantiles at ungauged sites can be estimated by establishing a relationship between interdependent physio-meteorological variables and observed flood quantiles at gauge sites in the region. However, this regional approach implies a loss of information due to the prior aggregation of hydrological data at gauged locations and can be difficult for data-sparse regions due to limited data. In this study, we evaluated an alternate approach or path for RFFA in two case studies: a data-sparse region in India and a data-dense region in the USA. In this approach, daily streamflow is predicted first using a deep learning-based hydrological model, and then flood quantiles are estimated from the predicted daily streamflow using statistical methods. We compared the results obtained using this alternate approach to those from the traditional RFFA technique, which used the Random Forest (RF) and eXtreme Gradient Boosting (XGB) algorithms to model the nonlinear relationship between flood quantiles and relevant physio-meteorological predictor variables such as meteorological forcings, topography, land use, and soil properties. The results showed that the alternate approach produces more reliable results with the least mean absolute error and higher coefficient of determination in the data-sparse region. In the data-dense region, both traditional and alternate approaches produced comparable results. However, the alternate approach has the advantage of being flexible and providing the complete time series of daily flow at the ungauged location, which can be used to estimate other flow characteristics, develop flow duration curves, or estimate flood quantiles of any return period without creating a separate traditional RFFA model. This study shows that the alternate approach can provide accurate flood frequency estimates in data-sparse regions, offering a promising solution for flood management in these areas.

How to cite: Mangukiya, N. K. and Sharma, A.: Evaluating Machine Learning Approach for Regional Flood Frequency Analysis in Data-sparse Regions, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-1278, https://doi.org/10.5194/egusphere-egu23-1278, 2023.

Neural networks belong to the best available methods for numerous hydrological model challenges. However, although they have shown to outperform classical hydrological models in several applications there is still some doubt whether neural networks are, despite their excellent interpolation skills, capable to make predictions beyond the support of the training data. This study addresses this issue and proposes an approach to infer the ability of neural network to predict unusual, extreme system states. We show how we can use the concept of data surprise and model surprise in a complementary manner to assess which unusual events a neural network can predict, which it can predict but only with additionally data and which it cannot predict at all hinting toward the wrong model choice or towards an incomplete description of the data.

How to cite: Loritz, R. and Gupta, H.: Extrapo… what? Predictions beyond the support of the training data, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-1526, https://doi.org/10.5194/egusphere-egu23-1526, 2023.

Having a continuous and complete karst discharge data record is necessary to understand hydrological behaviors of the karst aquifer and manage karst water resources. However, caused by many problems such as equipment errors and failure of observation, lots of hydrological and research dataset contains missing spring discharge values, which becomes a main barrier for further environmental and hydrological modeling and studies. In this work, a novel approach that integrates deep learning algorithms and ensemble empirical mode decomposition (EEMD) is developed to reconstruct missing karst spring discharge values with the local precipitation. EEMD is firstly employed to decompose the precipitation data, extract useful features, and remove noises. The decomposed precipitation components are then fed as input data to various deep learning models for performance comparison, including convolutional neural network (CNN), long short-term memory (LSTM), and hybrid CNN-LSTM models to reconstruct the missing discharge values. Root mean squared error (RMSE) and Nash–Sutcliffe efficiency coefficient (NSE), are calculated to evaluate the reconstruction performance as metrics. The models are validated with the spring discharge and precipitation data collected at Barton Spring in Texas. The reconstruction performance of various deep learning models with and without EEMD are compared and evaluated. The main conclusions can be summarized as: 1) by using EEMD, the integrated deep models significantly improve reconstruction performance and outperform the simple deep models; 2) among three integrated models, the LSTM-EEMD model obtains the best reconstruction results among three deep learning algorithms; 3) For models with monthly data, the reconstruction performance decreases greatly with the increase of missing rate: the best reconstruction results are obtained when the missing rate is low. If the missing rate was 50%, the reconstruction results become notably poorer. For models with daily data, the reconstruction performance is less impacted by the missing rate and the models can obtain satisfactory reconstruction results when missing rates range from 10% to 50%.

How to cite: Zhou, R. and Zhang, Y.: Reconstruct karst spring discharge data with hybrid deep learning models and ensemble empirical mode decomposition method, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-2382, https://doi.org/10.5194/egusphere-egu23-2382, 2023.

Machine Learning and Deep Learning have been proving their potential for streamflow modelling in various studies. In particular, long short-term memory (LSTM) models showed exceptionally good results. However, machine learning models often are considered “black boxes” with limited interpretability. Explainable artificial intelligence (XAI) comprise methods that analyze the internal processes of the machine learning network and allow to have a glance in the “black box”. Most proposed XAI techniques are designed for the analysis of images, and there is currently only limited work on time series data available.

In our study, we applied various XAI algorithms including gradient-based methods (Saliency, InputXGradient, Integrated Gradient, GradientSHAP) but also perturbation-based methods (Feature Ablation, Feature Permutation) to compare their applicability for reasonable interpretation in the hydrological context. To our knowledge, only Integrated Gradient has been applied to a LSTM in hydrology so far. Gradient-based methods analyze the gradient of the output with respect to the input feature. Whereas perturbation-based methods gain information by altering or masking specific input features. The different methods were applied to a LSTM trained for the low-land Ems catchment in Germany, which has a major baseflow share of total streamflow.

We analyzed the results regarding their “timestep of influence”, which describes the amount of past days having importance for the prediction of streamflow at a particular day. All of the algorithms applied result in a comparable annual pattern, characterized by relatively small timesteps of influence in spring (wet season) and increasing timesteps of influence in summer and autumn (dry season). However, the range of the absolute days of attribution varies between the methods. In conclusion, all methods produces reasonable results and appear to be suitable for interpretation purposes.

Furthermore, we compare the results to ERA-5 reanalysis data and gained evidence that the LSTM recognizes soil water storage as the main driver for streamflow generation in the catchment: we found an inverse seasonality of soil moisture and timestep of influence.

How to cite: Ley, A., Bormann, H., and Casper, M.: Exploring different explainable artificial intelligence algorithms applied to a LSTM for streamflow modelling, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-3125, https://doi.org/10.5194/egusphere-egu23-3125, 2023.

How to cite: Thibaut, R., Ferré, T., Laloy, E., and Hermans, T.: Sequential optimization of temperature measurements to estimate groundwater-surface water interactions, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-4137, https://doi.org/10.5194/egusphere-egu23-4137, 2023.

Rainfall-runoff (RR) modeling remains a challenging task in the field of hydrology especially when it comes to regional scale hydrology. Recently, the Long Short-Term Memory (LSTM) - which is known for its ability to learn sequential and temporal relations - has been widely adopted in RR modeling. The Convolutional Neural Networks (CNN) have matured enough in computer vision tasks, and trials were conducted to use them in hydrological applications. Different combinations of CNN and LSTM have proved to work; however, questions remain about suitability of different model architectures, the input variables needed for the model and the interpretability of the learning process of the models for regional scale.

In this work we trained a sequential CNN-LSTM deep learning architecture to predict daily streamflow between 1980 and 2014, regionally and simultaneously, over 86 catchments from CAMELS dataset in the US. The model was forced using year-long spatially distributed (gridded) input with precipitation, maximum temperature and minimum temperature for each day, to predict one day streamflow. The model takes advantage of the CNN to encode the spatial patterns in the input tensor, and feed them to the LSTM for learning the temporal relations between them. The trained model was further fine-tuned to predict for 3 local sub-clusters of the 86 stations. This was made in order to test the significance of fine-tuning in the performance and model learning process. Also, to interpret the spatial patterns learning process, a perturbation was introduced in the gridded input data and the sensitivity of the model output to the perturbation was shown in spatial heat maps. Finally, to evaluate the performance of the model, different benchmark models were trained using -as possible- a similar training setup as for the CNN-LSTM model. These models are CNN without the LSTM part (regional model), LSTM without CNN part (regional model), simple single-layer ANN (regional model), and LSTM trained for individual stations (considered as state of the art). All of these benchmark models have been fined-tuned for the 3 clusters as well.

CNN-LSTM model, after being fine-tuned, performed well predicting daily streamflow over the test period with a median Nash-Sutcliffe efficiency (NSE) of 0.62 and 65% of the 86 stations with NSE > 0.6 outperforming all benchmark models that were trained regionally using the same training setup. The model also achieved a comparable performance as for the -state of the art- LSTM trained for individual stations. Fine-tuning improved the performance for all of the models during the test period. The CNN-LSTM model, was shown to be more sensitive to input perturbations near the stations in which the prediction is intended. This was even clearer for the fine-tuned model, indicating that the model is learning spatially relevant information from the input gridded data, and fine tuning is helping on guiding the model to focus more on the relevant input.  

This work shows the potential of CNN and LSTM for regional Rainfall-runoff modeling by capturing spatiotemporal patterns involved in RR process. The work, also, contributes toward more physically interpretable data-driven modeling paradigm.

How to cite: Mohammed, A. and Corzo, G.: Evaluation of regional Rainfall-Runoff modelling using convolutional long short-term memory:  CAMELS dataset in US as a case study., EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-4177, https://doi.org/10.5194/egusphere-egu23-4177, 2023.

Producing accurate hourly streamflow forecasts in large basins is difficult without a distributed model to represent both streamflow routing through the river network and the spatial heterogeneity of land and weather conditions. HydroForecast is a theory-guided deep learning flow forecasting product that consists of short-term (hourly predictions out to 10 days), seasonal (10 day predictions out to a year), and daily reanalysis models. This work focuses primarily on the short-term model which has award winning accuracy across a wide range of basins.

In this work, we discuss the implementation of a novel distributed flow forecasting capability of HydroForecast, which splits basins into smaller sub-basins and routes flows from each subbasin to the downstream forecast points of interest. The entire model is implemented as a deep neural network allowing end-to-end training of both sub-basin runoff prediction and flow routing. The model's routing component predicts a unit hydrograph of flow travel time at each river reach and timestep allowing us to inspect and interpret the learned river routing and to seamlessly incorporate any upstream gauge data. 

We compare the accuracy of this distributed model to our original flow forecasting model at selected sites and discuss future improvements that will be made to this model.

How to cite: Lambl, D., Elkurdy, M., Butcher, P., Read, L. K., and Sampson, A. K.: Improving Data-Driven Flow Forecasting in Large Basins using Machine Learning to Route Flows, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-4179, https://doi.org/10.5194/egusphere-egu23-4179, 2023.

How to cite: Magotra, B., Saharia, M., and Dhanya, C. T.: Improving Streamflow Predictions over Indian Catchments using Long Short Term Memory Networks, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-4801, https://doi.org/10.5194/egusphere-egu23-4801, 2023.

Process-based global hydrological models are an important tool for sustainable development and policy making in today’s water-scarce world. These models are able to inform national to regional scale water management with basin-scale accounting of water availability and demand and project the impacts of climate change and adaptation on water resources. However, the increasing need for better and higher resolution hydrological information is proving difficult for these state-of-the-art process-based models as the associated computational requirements are significant.

Recently, the deep-learning community has shown that neural networks (in particular the LSTM network) can provide hydrological information with an accuracy that rivals, if not exceeds, that of process-based hydrological models. Although the training of these neural networks takes time, prediction is fast compared to process-based simulations. Nevertheless, training is mostly done on historical observations and thus projections under climate change and adaptation are uncertain.

Inspired by the complementary strengths and weaknesses of the process-based and deep-learning approaches, we present DL-GLOBWB: a deep-learning surrogate of the state-of-the-art PCR-GLOBWB global hydrological model. DL-GLOBWB predicts all water-balance components from the process-based model, including human water demand and abstraction, with a nRSME of 0.05 (range between 0.0001 and 0.32). The DL-GLOBWB surrogate is orders of magnitudes faster than its process-based counterpart, especially as surrogates trained at low resolutions (e.g. 30 arc-minute) can effectively be downscaled to higher resolutions (e.g. 5 arc-minute).

In addition to introducing DL-GLOBWB, our presentation will explore future applications of this deep-learning surrogate, such as (1) improving model calibration and performance by comparing DL-GLOBWB outputs with ins-situ data and satellite observations; (2) training DL-GLOBWB on  future model projections to include global change; and (3) the implementation of DL-GLOBWB to dynamically, and at high resolution, visualize the impact of climate change and adaptation to stakeholders.

How to cite: Droppers, B., Leijnse, M., Bierkens, M. F. P., and Wanders, N.: Introducing DL-GLOBWB: a deep-learning surrogate of a process-based global hydrological model, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-4842, https://doi.org/10.5194/egusphere-egu23-4842, 2023.

IMERG is a global satellite-based precipitation dataset, produced by NASA. It has provided valuable rainfall information to facilitate the design or the operation of the disaster and risk management worldwide. In operation, NASA offers three types of IMERG Level 3 (L3) products, with different levels of trade-offs in terms of time latency and accuracy. These are Early run (4-hour latency), Late run (14-hour latency) and Final run(3.5-month latency). The final-run product integrates multi-sensor retrievals and provides the highest-quality precipitation estimates among three IMERG products. It however suffers from a long processing latency, which hinders its applicability to near real-time applications. In the past 10 years, deep learning techniques have made significant breakthroughs in various scientific fields, including short-term rainfall forecasting. Deep learning models have shown to have the potential to learn the complex variations in weather systems and to outperform the Numerical Weather Prediction (NWP) in terms of short lead-time predictability and the required computational resources for operation.

In this research, we would like to explore the potential of deep learning (DL) in generating high-quality satellite-based precipitation product with low latency. More specifically, we investigate if DL models can learn the difference between Final- and Early-run products, and thus predict a Final-run-like product using Early-run product as input. Low-latency yet high-quality IMERG precipitation product can be therefore obtained. Various DL techniques are being tested in this work, including Auto-Encoder(AE), ConvLSTM and Deep Generative model. IMERG data between 2018 and 2020 over a rectangular area centred in the UK is used for model training and testing, and ground rain gauge records will be used to evaluate the performance of the original and predicted products. This pilot includes both ocean and land regions, which enables the comparison of the model performance between two different surface conditions. Preliminary analysis suggests that given patterns do exist in the differences between Early- and Final-run products, and the capacity of the selected DL models to learn the differences will be further investigated. The proposed work is of great potential to improve the applicability of IMERG products in an operational context.

How to cite: Hung, H. T. and Wang, L.-P.: IMERG Run Deep: Can we produce a low-latency IMERG Final run product with a deep learning based prediction model?, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-4887, https://doi.org/10.5194/egusphere-egu23-4887, 2023.

How to cite: Janbain, I., Deloffre, J., Jardani, A., Vu, M. T., and Massei, N.: Use of Long-Short Term Memory network (LSTM) in the reconstruction of missing water level data in the Seine River., EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-4970, https://doi.org/10.5194/egusphere-egu23-4970, 2023.

The objective function plays an important role in the training process for deep learning models, since it largely determines the trained values of the model parameters and influences the model performance. In this study, we establish two application-orientated objective functions, namely high flow balance error (HFBE) and transformed mean absolute percentage error (MAPE*), for the forecasts of high flows and low flows, respectively, in the LSTM model. We examine the strength and weakness of these streamflow forecast models trained on HFBE, MAPE* and mean square error (MSE) based on multiple performance metrics. Furthermore, we propose the objective function-based ensemble model (OEM) framework that integrates the models trained on different objective functions, so as to take advantages of the trained models focusing on different aspects of streamflow and thus achieve a better overall performance. Our results in 273 catchments over USA show that the models trained on HFBE can alleviate underestimation in high flows existing in the models trained on MSE, and perform remarkably better for high flows. It is also found that the models trained on MAPE* outperform the other two models in low flow forecast, no matter what algorithm is used for the model establishment. By incorporating the three models trained on HFBE, MAPE* and MSE, respectively, our proposed OEM performs well in the forecasts of both high flows and low flows, and realistically capture the mean and the variability of the observational streamflow under different scenarios under a variety of hydrometeorological conditions. This study highlights the necessity of applying application-orientated objective functions for given projects and the great potential of the ensemble learning methods for multi-optimization in hydrological modeling.

How to cite: Wang, D.: The role of ensemble learning in multi-optimization for streamflow prediction, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-5044, https://doi.org/10.5194/egusphere-egu23-5044, 2023.

Machine learning models have recently gained popularity in hydrological modelling at the catchment scale, fuelled by the increasing availability of large-sample data sets and the increasing accessibility of deep learning frameworks, computing environments, and open-source tools. In particular, several large-sample studies at daily and monthly time scales across the globe showed successful applications of the LSTM architecture as a regional model learning of the hydrological behaviour at the catchment scale. Yet, a deeper understanding of how machine learning models close the water balance and how they deal with inter-catchment groundwater flows is needed to move towards better process understanding. We investigate the performance and behaviour of the LSTM architecture at a monthly time step on a large sample French data set coined CHAMEAU – following the CAMELS initiative. To provide additional information to the learning step of the LSTM, we use the parameter sets and fluxes from the conceptual GR2M model that has a dedicated formulation to deal with inter-catchment groundwater flows. We see this study as a contribution towards the development of hybrid hydrological models.

How to cite: Weaver, N., El-Ouahabi, T.-A., Hallouin, T., Bourgin, F., Perrin, C., and Andréassian, V.: How do machine learning models deal with inter-catchment groundwater flows?, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-5199, https://doi.org/10.5194/egusphere-egu23-5199, 2023.

The application of numerical models for flood and inundation modelling has become widespread in the past decades as a result of significant improvements in computational capabilities. Computational approaches to flood forecasting have significant benefits compared to empirical approaches which estimate statistical patterns of hydrological variables from observed data. However, there is still a significant computational cost associated with numerical flood modelling at high spatio-temporal resolutions. This limitation of numerical modelling has led to the development of statistical emulator models, machine learning (ML) models designed to learn the underlying generating process of the numerical model. The data-driven approach to ML involves relying entirely upon a set of training data to inform decisions about model selection and parameterisations. Deep learning models have leveraged data-driven learning methods with improvements in hardware and an increasing abundance of data to obtain breakthroughs in various fields such as computer vision, natural language processing and autonomous driving. In many scientific and engineering problems however, the cost of obtaining data is high and so there is a need for ML models that are able to generalise in the ‘small-data’ regime common to many complex problems. In this study, to overcome extrapolation and over-fitting issues of data-driven emulators, a Physics-Informed Neural Network model is adopted for the emulation of all two-dimensional hydrodynamic models which model fluid according the shallow water equations. This study introduces a novel approach to encoding the conservation of mass into a deep learning model, with additional terms included in the optimisation criterion, acting to regularise the model, avoid over-fitting and produce more physically consistent predictions by the emulator.

How to cite: Donnelly, J., Daneshkhah, A., and Abolfathi, S.: Physics-Informed Neural Networks for Statistical Emulation of Hydrodynamical Numerical Models, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-5445, https://doi.org/10.5194/egusphere-egu23-5445, 2023.

Streamflow predictions are a vital tool for detecting flood and drought events. Such predictions are even more critical to Sub-Saraharan African regions that are vulnerable to the increasing frequency and intensity of such events. These regions are sparsely gauged, with few available gauging stations that are often plagued with missing data due to various causes, such as harsh environmental conditions and constrained operational resources. 

This work presents a novel workflow for predicting streamflow in the presence of missing gauge observations. We leverage bias correction of the GEOGloWS ECMWF streamflow service (GESS) forecasts for missing data imputation and predict future streamflow using the state-of-the-art Temporal Fusion transformers at ten river gauging stations in the Benin Republic.

We show by simulating missingness in a testing period that GESS forecasts have a significant bias that results in poor imputation performance over the ten Beninese stations. Our findings suggest that overall bias correction by Elastic Net and Gaussian Process regression achieves superior performance relative to traditional imputation by established methods such as Random Forest, k-Nearest Neighbour, and GESS lookup. We also show that the Temporal Fusion Transformer yields high predictive skill and further provides explanations for predictions through the weights of its attention mechanism. The findings of this work provide a basis for integrating Global streamflow prediction model data and state-of-the-art machine learning models into operational early-warning decision-making systems (e.g., flood/ drought alerts) in resource-constrained countries vulnerable to drought and flooding due to extreme weather events.

How to cite: Mbuvha, R., Adounkpe, P. J. Y., Houngnibo, M. C. M., and Newlands, N.: A Novel Workflow for Streamflow Prediction in the Presence of Missing Gauge Observations, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-5736, https://doi.org/10.5194/egusphere-egu23-5736, 2023.

Decision-making related to groundwater management often relies on results from a deterministic groundwater model representing one ‘optimal’ solution. However, such a single deterministic model lacks representation of subsurface uncertainties. The simplicity of such a model is appealing, as typically only one is needed, but comes with the risk of overlooking critical scenarios and possible adverse environmental effects. Instead, we argue, that groundwater management should be based on a probabilistic model that incorporates the uncertainty of the subsurface structures to the extent that it is known. If such a probabilistic model exists, it is, in principle, simple to propagate the uncertainties of the model parameter using multiple numerical simulations, to allow a quantitative and probabilistic base for decision-makers. However, in practice, such an approach can become computationally intractable. Thus, there is a need for quantifying and propagating the uncertainty numerical simulations and presenting outcomes without losing the speed of the deterministic approach.

This presentation provides a probabilistic approach to the specific groundwater modelling task of determining well recharge areas that accounts for the geological uncertainty associated with the model using a deep neural network. The results of such a task are often part of an investigation for new abstraction well locations and should, therefore, present all possible outcomes to give informative decision support. We advocate for the use of a probabilistic approach over a deterministic one by comparing results and presenting examples, where probabilistic solutions are essential for proper decision support. To overcome the significant increase in computation time, we argue that this problem can be solved using a probabilistic neural network trained on examples of model outputs. We present a way of training such a network and show how it performs in terms of speed and accuracy. Ultimately, this presentation aims to contribute with a method for incorporating model uncertainty in groundwater modelling without compromising the speed of the deterministic models.

How to cite: Busk Dahl, M., Norvin Vilhelmsen, T., Bødker Madsen, R., and Mejer Hansen, T.: Moving away from deterministic solutions: A probabilistic machine learning approach to account for geological model uncertainty in groundwater modelling, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-6313, https://doi.org/10.5194/egusphere-egu23-6313, 2023.

Neural Ordinary Differential Equation (ODE) models have demonstrated high potential in providing accurate hydrologic predictions and process understanding for single catchments (Höge et al., 2022). Neural ODEs fuse a neural network model core with a mechanistic equation framework. This hybrid structure offers both traceability of model states and processes, like in conceptual hydrologic models, and the high flexibility of machine learning to learn and refine model interrelations. Aside of the functional dependence of internal processes on driving forces, like of evapotranspiration on temperature, Neural ODEs are also able to learn the effect of catchment-specific attributes, e.g. land cover types, on processes when being trained over multiple basins simultaneously.

We demonstrate the performance of a generic Neural ODE architecture in a hydrologic large-sample setup with respect to both predictive accuracy and process interpretability. Using several hundred catchments, we show the capability of Neural ODEs to learn the general interplay of catchment-specific attributes and hydrologic drivers in order to predict discharge in out-of-sample basins. Further, we show how functional relations learned (encoded) by the neural network can be translated (decoded) into an interpretable form, and how this can be used to foster understanding of processes and the hydrologic system.

Höge, M., Scheidegger, A., Baity-Jesi, M., Albert, C., & Fenicia, F.: Improving hydrologic models for predictions and process understanding using Neural ODEs. Hydrol. Earth Syst. Sci., 26, 5085-5102, https://hess.copernicus.org/articles/26/5085/2022/

How to cite: Höge, M., Scheidegger, A., Baity-Jesi, M., Albert, C., and Fenicia, F.: Neural ODE Models in Large-Sample Hydrology, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-6466, https://doi.org/10.5194/egusphere-egu23-6466, 2023.

How to cite: de FLEURY, M., Kergoat, L., Brandt, M., Fensholt, R., Kariryaa, A., Kovács, G. M., Horion, S., and Grippa, M.: Deep learning for mapping water bodies in the Sahel, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-7347, https://doi.org/10.5194/egusphere-egu23-7347, 2023.

Water managers need accurate rainfall forecasts for a wide spectrum of applications, ranging from water resources evaluation and allocation, to flood and drought predictions. In the past years, several frameworks based on Artificial Intelligence have been developed to improve the traditional Numerical Weather Prediction (NWP) forecasts, thanks to their ability of learning from past data, unravelling hidden relationships among variables and handle large amounts of inputs. Among these approaches, Long Short-Term Memory (LSTM) models emerged for their ability to predict sequence data, and have been successfully used for rainfall and flow forecasting, mainly with short lead-times. In this study, we explore three different multi-variate LSTM-based models, i.e. vanilla LSTM, stacked LSTM and bidirectional LSTM, to forecast daily precipitation for the upcoming 30 days in the area of Rhine Delta, the Netherlands. We use both local atmospheric and global climate variables from the ERA-5 reanalysis dataset to predict rainfall, and we introduce a fuzzy index for the models to account for seasonality effects. The framework is developed within the H2020 project CLImate INTelligence (CLINT), and its outcomes have the potential to improve forecasting precipitation deficit in the study area.

How to cite: Bertini, C., Corzo, G., van Andel, S. J., and Solomatine, D.: Sub-seasonal daily precipitation forecasting based on Long Short-Term Memory (LSTM) models, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-7828, https://doi.org/10.5194/egusphere-egu23-7828, 2023.

Terrestrial water storage (TWS) anomalies from Gravity Recovery and Climate Experiment (GRACE) and its follow on GRACE-FO satellite missions provide a unique opportunity to measure the impact of different climate extremes and human intervention on water use at regional and continental scales. However, temporal gaps within GRACE and GRACE-FO mission (GRACE: 20 months, between GRACE and GRACE-FO: 11 months and GRACE-FO: 2 months) pose difficulties in analyzing spatiotemporal variations in TWS. In this study, Convolutional Long Short-Term Memory Neural Networks (CNN-LSTM) model was developed to fill these gaps and reconstruct the TWS for the Indian subcontinent (April 2002-July 2022). Various meteorological and climatic variables, such as precipitation, temperature, run-off, evapotranspiration, and vegetation, have been integrated to predict GRACE TWS. The performance of the models was evaluated with the help of Pearson’s correlation coefficient (PR), Nash-Sutcliffe efficiency (NSE), and Normalised Root Mean Square Error (NRMSE). Results indicate that the CNN-LSTM model yielded a mean PR of 0.94 and 0.89, NSE of 0.87 and 0.8, and NRMSE of 0.075 and 0.101 on training and testing, respectively. Overall, the CNN-LSTM achieved good performance except in the northwestern region of India, which showed a relatively poor performance might be due to high anthropogenic activity and arid climatic conditions. Further reconstructed time series were used to study the Spatiotemporal variations of TWS over the Indian Subcontinent.

Keywords: GRACE; Deep Learning; TWSA; Indian subcontinent

How to cite: Moudgil, P. S. and Rao, G. S.: Filling Temporal Gaps within and between GRACE and GRACE-FO Terrestrial Water Storage Changes over Indian Sub-Continent using Deep Learning., EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-8218, https://doi.org/10.5194/egusphere-egu23-8218, 2023.