Modeling health outcomes of air pollution in the Middle East by using support vector machines and neural networks – Scientific Reports

This research employed information sourced from “the Institute of Health Metrics and Evaluation GBD” websites (1990–2019) focusing on eight Middle East Arab States: Syrian Arab Republic, Iraq, Kuwait, United Arab Emirates, Iran, Lebanon, Saudi Arabia, and Qatar.

The research employs the aforementioned dataset, covering the years 1990–2019 for Middle Eastern countries, consisting of 240 observations (eight countries with 30 values each). In applying machine learning techniques, 80% of the data (1990–2013) was used for model training, while the remaining 20% (2013–2019) was reserved for model testing. Forecasting plots were created to visually represent the predicted values generated by the models. The Mean Absolute Percentage Error (MAPE) was utilized for model validation. Both traditional statistical methods and machine learning algorithms, including ARIMA, Exponential Smoothing (ES), Support Vector Machines (SVM), and Neural Networks (NN), were employed in constructing these forecasting models.

ARIMA model

The ARIMA modeling approach with Box Jenkin Methodology offers the conventional framework for time series analysis. Its strengths lie in flexibility and automatic model selection through functions like auto.arima() in R and the interpretability of its components. The steps for ARIMA model are illustrated in Fig. 1.

The ARIMA is denoted as “ARIMA(p, d, q)”. The expression is as follows:

where, \(d\): the integrated value,\(p\): the order of autoregressive process, \(\mu\): mean,\(q\): the order of moving average process,\(\phi\): the autoregressive operator, \({y}_{t}\): the time series, \(\varepsilon\): the random error, \({y}_{t}^{\prime}\): the first derivative of time series, \(\theta\):the moving average operator.

Exponential smoothing method

The Single Exponential Smoothing (ES) method was utilized to generate forecasts for a time series dataset, selected due to the absence of any seasonal patterns and the lack of a significant upward or downward trend in the data. This approach provides a way to estimate the current level of the time series using a single smoothing parameter, α, which determines the weight assigned to the most recent observation. The value of α varies between 0 and 1. The method uses exponentially decreasing weights and also incorporates weights moving average; the calculation can be explained with the help of the following formula.

\(t\) the time, st: c: the quantity of interest, yt: the target statistic, α: the smoothing factor, st − 1: the previous smoothed statistic. In the course of our analysis, we chose Holt’s Exponential Smoothing (ES) method because in the observed data samples no periodic relations were identified. The goal of employing Holt’s Exponential Smoothing in our analysis is to capture and understand the fundamental patterns in the data, ultimately leading to more precise predictions and better-informed decision-making:the time, st: the smoothed statistic, and α: the smoothing factor, st − 1: the previous smoothed statistic However, there is conflict in the data; therefore, in the current analysis, we opted for Holt’s Exponential Smoothing (ES) method since there was no evidence of a seasonality present in the data. The objective to apply Holt’s Exponential Smoothing as the method of our analysis will allow to unearth the underlying characteristics of the patterns and generate more accurate forecasts and effective decisions. The formula is expressed as: \({s}_{1}={x}_{1}\) and \({b}_{1}={x}_{1}-{x}_{0}\) for \(t>1\),

here, \(t\):the time, \(\alpha\): the smoothing factor, \({b}_{t}\): the trend best estimate at time \(t\), \({s}_{t-1}\): the previous smoothed statistic, \({s}_{t}\): the smoothed statistic, \(\beta\): the trend smoothing factor; \(0

Support Vector Machine Regression

The ordinary Support Vector Machine (SVM) Regression is a popular technique in machine learning best used on time series data. SVMs are particularly good at working with large and complex data and have following characteristics; Time series data analysis. One of the significant advantages is that they do not tend to overfit because they are not influenced by outliers: they work with support vectors—the most significant points. This characteristic renders SVMs flexible and able to generalize if given new data, this boost up the accuracy in different applications.

SVR was implemented using a linear kernel which is defined as \(({\text{K}}\left({\text{x}}, {\text{x}}^{\prime}\right)= {\text{x}}^{{\rm T}}{\text{x}}^{\prime}).\) This choice of kernel allows for effective modeling of linear relationships within the data. Alternative kernels such as polynomial or radial basis function (RBF) could be employed for complex data. The regularization parameter (C) was set to 1.0 that helps balance the trade-off between minimizing training error and maintaining model simplicity to prevent overfitting. The ε-insensitive loss function was applied to ignore errors within a margin ε. It focuses only on deviations that exceed this threshold. The SVR objective function is expressed to minimize the succeeding objective function:

subject to the constraints:

where \({\upxi }_{i}\) are slack variables, w represents the weight vector, \((\upphi \left({\text{x}}_{\text{i}}\right))\) denotes the kernel-transformed feature space, and b is the bias term. Predictions are obtained using the function:

where \(({{\alpha }}_{\text{i}})\) are the Lagrange multipliers from the training phase and \((\text{K}\left({\text{x}}_{\text{i}},\text{x}\right))\) is the kernel function applied to the test input \((\text{x}).\) Predictions were made on the test set to assess the model’s performance, and a forward-looking forecasting loop was incorporated. This loop generates predictions for future years based on the last three observations in the processed data. The Fig. 2 presented the structure of SVM model.

Artificial neural networks

Artificial neural networks is a sequence of algorithms inspired by the cognitive processes of the human brain. They are at the lead of this predictive revolution. These networks showcase an amazing ability to approximate all functions without requiring extensive knowledge about the data’s particulars. ANNs are extensively applied in time series forecasting. Neural networks play a crucial role in shaping the future of predictions and form the cornerstone of deep learning, a subfield of machine learning inspired by the intricate structure of the human brain. These networks as shown in figure three, take data through them, and are capable of independent training to detect particular patterns. This unique ability allows them to predict the output for the new similar data sets, emulate the brain’s multiple steps to analyze the information. The methodology adopted in the present study can be described in the following manner:

Data preprocessing

The first resolute within the undertaken methodology is the preliminary analysis of the time series to ensure their compatibility with the input data for the neural network model. Next to this, the data is first ordered chronologically to preserve the sequential structure which is required when undertaking time series analysis. This alters the range of the target variable in order that it may be normally distributed. This normalization process is performed through reducing the mean to zero and standard deviation to one of the data it is used to improve the stability and the performance of the neural network while training.

Dataset construction

They are expressed in the form of data for training a neural network based on the sequences of historical observation expanded in time. This involves a creation of tuples each of which has the past time steps as an input together with the value that has to be predicted on the next time step as an output. It is favorable to the model to use a sequence-based way to analyze the data, as temporal dependencies and temporal patterns can be learned using the data. Either the value of ‘t’ or the length of the input sequence is a dire parameter that determines the extent of the temporal information that the models can learn.

Neural network architecture

Specifically, with the view of operationalizing the proposed forecasting model, a feedforward neural network has been used. The network includes multiple layers: The network can contain multiple levels, such as the system and the subjects filling posts in it.

A typical structure of a Neural Network (NN) design comprises of an input layer, hidden layers and an output layer. The input layer receives the input data and each of the neurons in this layer is in some way related to the feature present in the data. The latent layers are the layers that come in between the input and output layers and these neurons work on the inputs with applying weights, and sigmoid or tanh to put non-linearity in the model. All the results and decisions are given in the output layer and each neuron of the output layer contributes to the special possibility. While in forward propagation the network is in a position to make certain predictions and Sugimura, while using backpropagation, the weights of the network are in a position to be modified depending on the errors. The structure of the NN can be basic and have only the one hidden layer, or can be multilayer which consists from many layers in what is called Deep Learning Network with numerous layers or can have the adjusted layers as in convolutional or Recurrent NNs. Likewise, the structure of NN that Fig. 3 depicted was about organizing relation.

Model training and evaluation

Neural network learning occurs on a sample of the data referred as the training set while the performance is evaluated on another sample known as the testing set. To measure the accuracy of the created model, Performance metrics named Mean Absolute Percentage Error (MAPE) is used. MAPE computes the level of prediction accuracy in proportional levels.

Forecasting and visualization

Having derived the model, it is then applied to make forecast for subsequent periods. In this method, the subsequent values are predicted from the data set and this is done repeatedly. These forecasts are linked with historical data to make visualization of the performance of the model possible. To illustrate the actual data and forecasted values, plots are always created. This in turn allows consideration of the predictive accuracy of the model.

It is a flexible and hierarchical structure that enables the training of the neural network for the patterns which exist in the time series data for robust predictions. Its ability to learn different and more accurate relationships within the data makes it more reliable for forecasting in the time series analysis. This has been made clear by the following representation of the Neural Network model;

where, \({y}_{t-i}={\left({y}_{t-1},{y}_{t-2},\dots ,{y}_{t-8}\right)}{\prime}\) comprises a vector that includes past values of the series, and f represents ANN including 2 hidden nodes. The error system {ϵt} is presumed to exhibit homoscedasticity. ϵT + 1 is a stochastic draw from the error distribution at time T + 1, then \({y}_{T+1}^{*}=f\left({y}_{T+1}\right)+{\epsilon }_{T+1}^{*}\) represents a potential realization from the predicted distribution. for \({y}_{T+1}\).

\({y}_{T+2}\) is the value of the series at time T + 2. f is a function that describes the relationship between the past values and the future values of the series. \({y}_{T+1}^{*}\)​ is a modified or transformed value of the series at time T + 1. \({\epsilon }_{T+1}^{*}\)​ is an error term at time T + 1. This method allows a scholar to simulate upcoming sample paths iteratively. Through the repetitive simulation of these paths the researcher acquires an understanding of the distribution for all forthcoming values leveraging the information provided by the fitted ANN.