EES Data Lab – Spatiotemporal Data Models and Algorithms for Earth and Environmental Sciences

Scientists and engineers working in fields such as the environmental sciences, the oceans, climate, or earth sciences have access to massive amounts of geo-referenced data. These data allow monitoring and studying the behavior of objects or events of interest over time, making diagnoses and predictions, etc. These tasks assume the existence of good quality data and methods and tools to analyze the data with little effort. Currently, there are many tools help on managing, processing, and analyzing spatial data, but the same does not happen when one intends to work with spatial data that evolves over time. This project focuses on the development of models and tools for the processing of spatiotemporal (SPT) data, based on two case studies: environmental engineering and marine ecology. The focus will be on SPT data modeled as 2D and 3D geometries that can change position, shape, or size continuously over time (moving objects).

Resources

BurnedAreaUAV dataset – a new manually annotated dataset for burned area segmentation for the training and evaluation of video burned area segmentation models. This dataset is based on a video of a prescribed fire captured by a drone with an RGB-only sensor, in northern Portugal. The footage is characterized by periods where smoke and flames obstruct the area of interest, which makes the segmentation of the burned area more challenging.

  • The paper Burned area semantic segmentation: A novel dataset and evaluation using convolutional networks describes tools to support the benchmarking for testing and validating burned area segmentation models in a wildfire context. As such, in that paper, we propose a new manually annotated dataset for segmentation of forest fire burned area based on a video captured by a UAV to train and evaluate semantic segmentation models. We suggest specific temporal consistency metrics to validate burned area polygons generated by the models in successive frames of non-annotated data. We also explore deep learning-based techniques and establish baselines, including IoU values superior to 95% on the test set. The paper is available at https://www.sciencedirect.com/science/article/pii/S0924271623001831

    To cite the paper, please use:

    Ribeiro, T. F., Silva, F., Moreira, J., & Costa, R. L. D. C. (2023). Burned area semantic segmentation: A novel dataset and evaluation using convolutional networks. ISPRS Journal of Photogrammetry and Remote Sensing202, 565-580. https://doi.org/10.1016/j.isprsjprs.2023.07.002
  • The BurnedAreaUAV Dataset is available at https://zenodo.org/record/7944963.

    To cite the dataset, please use:

    Ribeiro, Tiago F. R., Silva, Fernando, Moreira, José, & Costa, Rogério Luís de C. (2023). BurnedAreaUAV Dataset (v1.1) [Data set]. In ISPRS Journal of Photogrammetry and Remote Sensing (1.1, Vol. 202, pp. 565–580). Zenodo. https://doi.org/10.5281/zenodo.7944963
  • Video comparing segmentation results and original data

Spatiotemporal Data Compression and Simulation – The continuous representation of spatiotemporal data commonly relies on using abstract data types, such as moving regions, to represent entities whose shape and position continuously change over time. In this work, we explore the capabilities of Conditional Variational Autoencoder (C-VAE) models to generate smooth and realistic representations of the spatiotemporal evolution of moving regions. We apply compression operations to sample from the dataset and use the C-VAE model and other commonly used interpolation algorithms to generate in-between region representations. To evaluate the performance of the methods, we compare their interpolation results with manually annotated data and regions generated by a U-Net model. We also assess the quality of generated data considering temporal consistency metrics.

  • To cite the paper, please use:

    Ribeiro, T.F.R., Silva, F., de C. Costa, R.L. (2023). Reconstructing Spatiotemporal Data with C-VAEs. In: Abelló, A., Vassiliadis, P., Romero, O., Wrembel, R. (eds) Advances in Databases and Information Systems. ADBIS 2023. Lecture Notes in Computer Science, vol 13985. Springer, Cham. https://doi.org/10.1007/978-3-031-42914-9_5
  • Video comparing interpolation results and original data

Best Poster Award in MIT Portugal Conference 2023From Fire to Data: Capturing Wildfire Dynamics with Semantic Segmentation & Spatiotemporal Reconstruction The work presented by at MIT Portugal 2023 Annual Conference received the Best Poster Award in the Climate Science & Climate Change category. More info in: https://mitportugal.mit.edu/

See the poster in https://mitportugal.mit.edu/poster-gallery/2023/student-posters/fire-data-capturing-wildfire-dynamics-semantic-segmentation-spatiotemporal-reconstruction

Underwater image improvement and reef segmentation – Image enhancement methods are used to improve the quality of underwater images, which are often degraded by the effects of scattering and absorption.

Contacts

For additional information, please send us an email to: 

rogerio.l.costa AT ipleiria DOT pt

Acknowledgements

This work is funded by FCT – Fundação para a Ciência e a Tecnologia, I.P.,.

Reference: MIT-EXPL/ACC/0057/2021

Funding: 14.961,00 € (Polytechnic of Leiria)