A data analytics-driven approach to backorder prediction using federated machine learning in industrial supply chains

In this section, the step-by-step working of the proposed model is presented, and a pictorial overview is depicted in Fig. 4.

A pictorial representation of the proposed federated machine learning-based backorder prediction system for enhancing supply chain efficiency.

Federated machine learning for backorder prediction

Federated Learning (FL) is a jointly decentralized privacy-preserving system designed to address the challenges of data privacy and security²⁶. In the context of backorder prediction, it is helpful when the company has multi-stakeholder collaborations, and each stakeholder maintains their own data. The proposed approach preserves data confidentiality and privacy while lowering the transfer costs generally associated with centralizing data for training. In this work, we propose a federated learning-based ML model that distributes backorder data among multiple clients. Each client trains its own Multi-Layer Perceptron (MLP) model on local data. To address data imbalance at the client level, we apply SMOTE oversampling individually on each client node. More formally, let there be \(N\) different backorder data holders \(\{{H}_{1}, {H}_{2}, {H}_{3}, \cdots , {H}_{n}\},\) each possessing its own local dataset \(\{{D}_{1}, {D}_{2}, {D}_{3}, \cdots , {D}_{n}\}.\) In a traditional machine learning setup, a centralized model is trained on the union of all datasets:

In contrast, our proposed federated learning approach enables each client to collaboratively train a shared model \({M}_{FED}\) without sharing raw data. Instead, only model updates are communicated:

where \({M}_{FED}\) is the federated learning model.

Data oversampling and under sampling

The data on backorders is distributed across different devices. Where, in the context of the supply chain system, it can be the data of different stakeholders. Now, data imbalance is one of the major issues hindering performance. Data oversampling and undersampling are two of the popular techniques to address the class imbalance problem. However, excessive oversampling will lead to overfitting, and excessive under-sampling can eliminate essential information from the dataset⁵⁶. Hence, in this research, we first undersampled the majority class samples and then oversampled the minority class. Under-sampling is done using the near-miss method⁵⁷. In this method, the majority class instances (i.e., non-backorder) are selected based on their average distance to their nearest neighbors from the minority class. For instance, let \({x}_{i}\) be a sample from the majority class, and let \({\mathcal{N}}_{k}({x}_{i})\) represent its \(k\) nearest neighbors from the minority class. The average distance is given by:

The majority of class instances with the smallest \({d}_{avg}\) values are retained for training. Similarly, one of the most prominent data oversampling approaches is the Synthetic Minority Oversampling Technique (SMOTE)⁵⁸. In this method, the data of a class having a smaller number of samples is oversampled. It employs the concept of k-nearest neighbors to generate synthetic samples in the direction of nearest neighbors. This method perfectly compels the decision boundaries of a class having a smaller number of samples to become more general. Mathematically, for a given minority class \({x}_{i}\), one of its k-nearest neighbors \({x}_{nn}\) is randomly selected. A new synthetic sample \({x}_{new}\) is generated as follows:

where \(\lambda \in [\text{0,1}]\) is a random number. The number of synthetic samples generated depends on the oversampling ratio, which is determined by the number of nearest neighbors \(k\) and the desired balance between classes. This strategy broadens the decision boundary of the minority class and enhances the model’s ability to discriminate between backorder and non-backorder classes.

Proposed decentralized single-layer neural networks

The straightforward form of artificial neural networks is referred to as ANNs. These neural networks are known as computational models that are designed to mimic the human mind to perform complex operations, including both classification tasks and regression tasks. When they are trained on large data sources, they are able to learn relationships among input and output attributes. Similar to the human brain, where neurons are interconnected, neural networks consist of hidden units, also known as neurons, that are linked together through parameters called weights. The nodes at the input layer receive input from the environment, and nodes in the middle layers can take input from nodes present in preceding layers. Machines are programmed to function similarly to artificial neural networks combined with human brain cells. The interruption of one or more ANN hidden units or neurons does not prevent the underlying model from producing a result, and this property gives the model a fault-tolerance property.

One of the most important characteristics of ANNs is to learn complex patterns from data without explicitly defined rules. ANNs and their advanced variants, such as multi-layer neural networks, recurrent neural networks, and convolutional neural networks, are widely employed in a variety of applications. In this research, an SLP (Single-layer neural) network is proposed, comprising only one input and one output layer. The total number of hidden units or neurons in the input layer of SLP is equal to the total number of features present in the backorder dataset.

More precisely, the SLP takes an input vector of backorder dataset features comprising “sku”, “national_inv”, “lead_time”, “in_transit_qty”, “forecast_x_month,” “sales_x_month”, “min_bank”, “potential_issue”, “pieces_past_due”, “perf_x_months_avg”, “local_bo_qty”, “ × 17- × 22”, “deck risk”, “oe constraint”, “ppap risk”, “stop auto buy”, and “rev stop” represented as \(x=\left({x}_{i1},{x}_{i2},{x}_{i3},\dots \dots .{x}_{in}\right)\).

Every \({x}_{in}\) is given as input to each hidden unit present in the input layer. In the following step, the outcome of these units from 1 to \(n\) is calculated by employing the following equation:

In the above Eq. (5) \({u}_{1}\)…\({u}_{n}\) is the total number of units that are present at the input layer of the ANN, where \({w}_{i}\) are the weights associated with linking input neurons and output neurons. More precisely, each input feature of the backorder dataset, denoted as \({x}_{i}\) is multiplied by its corresponding weights. These weights represent the significance of each connection between input and output neurons. Within a Single-Layer Perceptron (SLP), the weighted inputs are summed, and if the result equals zero, an additional parameter called bias is introduced to shift the output and prevent it from being zero. This process is illustrated in Eq. (5). Moreover, the proposed SLP starts learning by updating weights to their optimal values, and this procedure is repeatedly performed, followed by computing the model error. This method is repeated multiple times or for a defined number of iterations until a model has the lowest possible error.

Proposed decentralized multi-layer neural networks

A Multi-Layer Neural Network is one of the prominent examples of artificial neural networks, also referred to as a fully connected neural network and a Multi-Layer Perceptron. It is made up of numerous layers of artificial neurons, with each neuron in a particular layer connected to all neurons in the next, enabling sophisticated, nonlinear data processing. The difference between single-layer neural networks and multi-layer neural networks is the number of hidden layers present between the input and output layers. On each layer, the number of hidden units is known as neurons having non-linear activation functions.

In this paper, the proposed MLP has a total of three layers with hidden units of 8, 4, and 2. The non-linear activation function on each layer is ReLu. The ReLu performs better than the logistic sigmoid function and hyperbolic “tanh” activation. While making the algorithm converge faster towards optimum weights, the ReLU also addresses the problem of vanishing gradients. The following Eq. (6) is the mathematical form of ReLu⁵⁹.

More precisely, MLP takes an input vector of backorder features represented as \(x=({x}_{i1},{x}_{i2},{x}_{i3},\dots \dots .{x}_{in})\). The output of every neuron is given in Eq. (7):

In the above Eq. (7), \({w}_{i}\) is the weight used to link two neurons, \({a}_{i}\) is the input, \(b\) is the bias term and \(\varphi\) denotes the activation function, i.e., ReLu on hidden layers. On the last layer, the sigmoid activation function is applied to predict whether either product “went on backorder” or not. During training, the weight optimizer “Adam” is employed to update the weights of layers with an input batch size of 8. The mathematical formulas for updating weights and biases are given below:

In the above Eqs. (8–9), the terms \(W,x, B,l, n,t,m, \lambda , \text{and} C\) are known as weights, learning rate, bias, total number of layers, training examples, updating step, momentum, regularization parameter, and loss function. The loss function used to train neural networks is binary cross-entropy, as given below:

In the above Eq. (10), \(P\left({y}_{i}\right)\) is the probability of class where \(N\) denotes the total number of samples in the dataset \({{{D=\{y}_{i},{x}_{i1},\dots \dots {x}_{ip}\}}_{i=1}}^{n}\).

Broadcasting of neural network weights

The server model weights are initialized with the Glorot uniform method to initiate a federated learning-based training setup. The weights of the server model are updated in later stages after each local client is trained. Then, after initialization, the local clients are trained in their separate datasets of backorders. After training, these clients’ weights are first averaged out (i.e., weighted aggregation strategy), and then later the average weights are employed to update the server model weights. The updated server model is then again broadcast to different clients to start the next round of federated learning setup. In this way, the data at different client levels is not shared with the model at the server end. Communication between the server and clients is only for weights. These weights are the weights of single or multi-layer neural networks. Furthermore, the neural network design, including the number of layers, weight optimizer, hidden units at each layer, activation functions at each layer, loss functions, and learning rates, remains consistent between clients and servers.

This is due to the reason that weights of the client’s model are copied to different hidden units linking the nodes of layers of architecture present in the server. Mathematically, suppose the weights of a server model is \({w}_{0}\). A collection of client sets, denoted as client \(i\in \{\text{1,2},\dots N\}\) optimizes their weights independently on their own local dataset \({D}_{i}\). At the end of every round \(t\) when clients end up with their local training, it generates an updated set of weights \({w}_{i}^{
(11)