Overcoming the challenges of data integration in ecosystem studies with machine learning workflows: an example from the Santos project (Ocean and Coastal Research, 2024)
Integrating intricate environmental data within a unified analytical framework for extensive conservation andmonitoring initiatives encounters several challenges. These challenges encompass defining a conceptual modeloutlining cause-and-effect relationships, addressing dissimilarities in data source quantity and information content,grappling with missing or noisy data, fine-tuning model optimization, achieving accurate predictions, and tackling theissue of imbalanced observations across factors. In the context of the Santos project, dedicated to comprehendingthe spatio-temporal dynamics of benthic, pelagic, and physical systems for the facilitation of conservation andmonitoring programs, the application of machine learning's random forest (RF) technique for modeling univariatedata offers notable advantages. This approach adeptly handles non-linearity, covariation, and interactive effectsamong predictors. For modeling multivariate data sets, a hybrid strategy combining a self-organizing map (SOM)and RF is harnessed to effectively tackle the challenges. Addressing missing values, the bagging imputationtechnique demonstrated superior performance compared to other methods. Both machine learning techniquesdiscussed herein exhibit resilience against the impact of noisy data, yet the identification of noisy data remainsfeasible based on model outputs. In scenarios of imbalanced data sets, we investigate the correlation betweenthe RF model's overall statistics and those of individual classes. The joint interpretation of these statistics aids incomprehending model limitations and facilitates discussions on the environmental mechanisms shaping observedpatterns. We propose two analytical workflows that not only enable the exploration and enhancement of modelaccuracy but also facilitate the investigation of potential cause-and-effect relationships inherent in the data.Furthermore, these workflows lay the foundation for implementing long-term learning algorithms, a pivotalincrement for monitoring initiatives. Notably, these workflows, alongside the discussed analytical challenges,can be seamlessly implemented within iMESc, an open-source application.