Unlearning in Distributed Budget Support Vector Machine

Removing the contribution of a part of the training data from a trained model is a task that has recently gained a lot of attention. A naïve approach is to retrain the model from scratch after excluding the unwanted training data, but the main research interest is in unlearning techniques that operate on the trained model with less computational cost than full retraining. Additionally, data contributors may no longer be available for a retraining process, for example in a distributed federated learning environment. Existing methods for unlearning in Support Vector Machine, based on forgetting samples using decremental methods, require active collaboration of all participants, and general unlearning methods that are based on gradient descent are not applicable in the SVM case since the cost function is not differentiable. We propose here a training scheme for Budget Support Vector Machines that has unlearning capabilities once the model has been trained. We benchmark the proposed unlearning method using a variety of datasets and unlearning scenarios in a federated distributed training setup and show its advantages in Accuracy, Efficiency and Effectiveness, with respect to the Sharded, Isolated, Sliced, and Aggregated approach, which consists of training weak local learners at each participant and obtaining the final decision by a voting mechanism.

Unlearning in Distributed Budget Support Vector Machine Articles