Distributed Machine Learning with Python

Table of Contents

1. Splitting Input Data
2. Parameter Server and All-Reduce
3. Building a Data Parallel Training and Serving Pipeline
4. Bottlenecks and Solutions
5. Splitting the Model
6. Pipeline Input and Layer Split
7. Implementing Model Parallel Training and Serving Workflows
8. Achieving Higher Throughput and Lower Latency
9. A Hybrid of Data and Model Parallelism
10. Federated Learning and Edge Devices
11. Elastic Model Training and Serving
12. Advanced Techniques for Further Speed-Ups