Training Machine Learning models at the Edge: A Survey

arXiv:2403.02619 · cs.LG, cs.DC · Submitted 2024-03-05 · Read on arXiv

cs.LG, cs.DC

Submitted: 2024-03-05

Updated: 2024-10-11

Comments: 30 pages, 7 figures

DOI: 10.1145/3848509

Code: https://github.com/OpenMined/KotlinSyft

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Machine Learning — 3× in this paper
• Federated Learning — 1× in this paper · explained in 17 episodes
• Machine Learning (ML) — 1× in this paper · explained in 2 episodes

Sources

• Artificial Intelligence Index Report 2023
• On-device Training: A First Overview on Existing Systems
• Edge Intelligence: Architectures, Challenges, and Applications
• Federated Learning in Mobile Edge Networks: A Comprehensive Survey
• Knowledge Distillation in Federated Edge Learning: A Survey
• A Survey on Edge Computing Systems and Tools
• Federated Learning with Noisy User Feedback
• FATE-LLM: A Industrial Grade Federated Learning Framework for Large Language Models
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
• AdaSplit: Adaptive Trade-offs for Resource-constrained Distributed Deep Learning
• On Handling Catastrophic Forgetting for Incremental Learning of Human Physical Activity on the Edge
• Inexact-ADMM Based Federated Meta-Learning for Fast and Continual Edge Learning
• Projected Latent Distillation for Data-Agnostic Consolidation in Distributed Continual Learning
• On First-Order Meta-Learning Algorithms
• Meta-Learning in Neural Networks: A Survey
• p-Meta: Towards On-device Deep Model Adaptation
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• The Forward-Forward Algorithm: Some Preliminary Investigations
• CCFC: Bridging Federated Clustering and Contrastive Learning

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks