Decentralized Training

Decentralized training brings together distributed compute resources to train machine learning models without relying on a single centralized cluster. The team explores federated learning, gossip-based gradient sharing, and coordination protocols for heterogeneous nodes.

Getting Started

• Federated Learning: The Grand Tour — by Peter Kairouz et al. Overview of federated learning concepts, challenges, and open problems.
• Distributed Training of Deep Learning Models: A Taxonomic Perspective — by Matthias Langer et al. A survey of data and model parallelism strategies.

Key Resources

Reserved for resources submitted by the team lead.

Tools & Repositories

Reserved for tools, repositories, datasets, workspaces, and operational references submitted by the team lead.

Related Curriculum

• AI/ML: Math for ML — foundational math for understanding training algorithms
• AI/ML: Model Lifecycle and Deployment — training workflows and deployment patterns
• Protocol Engineering: Communication Patterns — gossip protocols and peer-to-peer coordination