Scaling Production-Ready AI Workloads
A Systems Engineering Guide to Measuring and Improving Model Training and Inference With GPUsBy Kenji Watanuki
Length14h 43m
About this audiobook
Your model works. It is also three times slower and five times more expensive than it needs to be, and nobody on the team can say why.
Most AI performance problems are not solved by trying random optimisations until something improves. They are solved by building a model of the workload, measuring where time and memory actually go, and then applying the right fix to the right bottleneck. This book gives you that method. You will learn to construct a roofline for a real workload, classify whether it is compute-bound, memory-bound, or communication-bound, and only then reach for a fix. That discipline separates teams that ship reliable, cost-effective systems from teams that keep guessing.
Starting from GPU architecture and profiling fundamentals, the book moves through the full stack of production AI systems. You will profile kernels and input pipelines, adopt reduced precision without silently losing accuracy, fuse operators, and choose intelligently between data, tensor, pipeline, and sharded-optimiser parallelism once communication starts dominating. On the serving side, you will implement continuous batching, manage the key-value cache, apply quantisation and distillation with measured trade-offs, and size capacity against real latency objectives. Every technique is presented with the measurement that tells you whether it worked.
What you will learn:
• Build a performance model that predicts whether a workload is compute-, memory-, or communication-bound before you change a line of code
• Profile GPU kernels and data pipelines to find the real bottleneck instead of the assumed one
• Keep accelerators fed with input pipelines that do not stall on I/O or preprocessing
• Apply mixed precision and numerical techniques without degrading model quality
• Optimise at the kernel level through operator fusion and memory-aware implementation
• Choose between data, tensor, pipeline, and sharded-optimiser parallelism as communication begins to dominate
• Measure and improve interconnect and scaling efficiency across multi-GPU and multi-node training
• Design inference serving with continuous batching, key-value cache management, and quantisation or distillation trade-offs
• Size capacity and cost against explicit latency and service objectives, then operate the system in production
For machine learning engineers, platform and systems engineers, and technical leads running training or inference at scale, this is the systems engineering guide that turns performance work from folklore into a repeatable engineering practice.
Audiobook details
GenreTechnology
Length14 hrs 43 mins
Narrated byListen with 1,000+ voices
FormateBook with Audio
Publish dateSep 27, 2026
LanguageEnglish
Table of contents
1Scaling Production-Ready AI Workloads
2Foreword
3Preface
4About This Book
5Chapter 1: A Performance Model for AI Systems
Show all chaptersShow less
61.1 Latency, Throughput, and the Shape of the Curve
71.2 Arithmetic Intensity and the Memory Wall
81.3 Building a Roofline for a Workload
91.4 Classifying the Bottleneck
101.5 Amdahl, Gustafson, and the Cost of a Fix
111.6 A Worked End-to-End Performance Audit
12Chapter 2: GPU Architecture for Systems Engineers
132.1 Streaming Multiprocessors and the Execution Model
142.2 The Memory Hierarchy and Its Costs
152.3 Tensor Cores and Matrix Units
162.4 Latency Hiding and the Occupancy Trade
172.5 Hardware Classes and What They Imply
18Chapter 3: Measure Before You Optimise
193.1 What to Measure and Why
203.2 Profilers and Kernel Timelines
213.3 Hardware Counters and Derived Ratios
223.4 Benchmark Hygiene and Variance
233.5 Building a Repeatable Microbenchmark
24Chapter 4: Data Pipelines That Keep Accelerators Fed
254.1 Diagnosing a Starved Accelerator
264.2 Sharded Storage and Read Patterns
274.3 Prefetch Depth and Worker Sizing
284.4 The Real Cost of Augmentation
294.5 Caching, Determinism, and Reproducibility
30Chapter 5: Mixed Precision and Numerics
315.1 Reduced-Precision Formats
325.2 Loss Scaling and Gradient Underflow
335.3 Accumulation Order and Numerical Error
345.4 Verifying Accuracy Didn’t Move
35Chapter 6: Kernel-Level Optimisation
366.1 Operator Fusion and Traffic Reduction
376.2 Coalescing, Tiling, and Shared Memory
386.3 Writing and Reading Custom Kernels
396.4 Compiler Backends and Graph Capture
40Chapter 7: Distributed Training Strategies
417.1 Data Parallelism and Gradient Reduction
427.2 Tensor Parallelism
437.3 Pipeline Parallelism and Bubble Cost
447.4 Sharded Optimizers and State Memory
457.5 Composing Strategies
46Chapter 8: Interconnect and Scaling Efficiency
478.1 Collective Communication Patterns
488.2 Fabric Topology and Bandwidth Tiers
498.3 Overlapping Communication with Compute
508.4 Measuring Scaling Efficiency
The Honey BusMeredith May9h 31m4.4 (16.6K)$25 · $0.00
Collected Writings of Nikola TeslaThomas Commerford Martin, Nikola Tesla19h 42m4.4 (2.8K)$1.99 · $0.00
Operation PedestalMax Hastings12h 29m4.4 (5.2K)$29 · $0.00
When the Heavens Went on SaleAshlee Vance18h 20m4.4 (3K)$40 · $0.00
Everybody Has a Podcast (Except You)Justin McElroy, Travis McElroy, Griffin McElroy5h 9m4.3 (2.3K)$24 · $0.00
Hands of TimeRebecca Struthers8h 8m4.4 (1.3K)$26 · $0.00
Never Lost AgainBill Kilday10h 1m4.4 (805)$29 · $0.00
How the Internet HappenedBrian McCullough13h 28m4.3 (2.7K)$23 · $0.00
Collected Writings of Nikola TeslaNikola Tesla, Thomas Commerford Martin21h 50m4.3 (1.7K)$1 · $0.00
Just AspireAjai Chowdhry9h 13m4.4 (260)$29 · $0.00
The Little Book of AliensAdam Frank8h 20m4.3 (1.3K)$26 · $0.00
The inventions, researches and writings of Nikola Tesla (Annotated)Thomas Commerford Martin16h 59m4.3 (1.3K)$2 · $0.00
KargilV.P. Malik15h 18m4.3 (2K)$24 · $0.00
Experiments with Alternating CurrentsNikola Tesla9h 16m4.3 (874)$1.99 · $0.00
Fire on the HorizonTom Shroder, John Konrad8h 23m4.3 (858)$26 · $0.00
The Smell of Kerosene (Annotated)National Aeronautics and Space Administration, Donald L. Mallick, Peter W. Merlin11h 32m4.3 (523)$2 · $0.00
The Boy Who Harnessed the WindWilliam Kamkwamba, Bryan Mealer10h 5m4.2 (78.9K)$29 · $0.00
Across the Airless WildsEarl Swift10h 8m4.3 (1.3K)$29 · $0.00
Console WarsBlake J. Harris20h 41m4.2 (13.7K)$46 · $0.00
Improvised Munitions HandbookU.S. Department of Defense4h 49m4.3 (1.6K)$1.99 · $0.00
You may also like
High-Performance Machine Learning in C++Lionel Eriksen7h 37m$5 · $0.00
The AI Advantage for LawyersMiles Draper3h 52m4.8 (10)$17
Artificial Intelligence for Natural Language ProcessingTheo Riven3h 33m$5
Building and Scaling Machine Learning SystemsJonas Riedel15h 6m$85
Concurrent Python with asyncioLena Iqbal7h 43m$5
ITIL® 4 Create, Deliver and Support (CDS)Claire Agutter3h 36m4.6 (23)$40 · $0.00
Software Structure by DesignRohan Dasgupta14h 44m4.8 (10)$75
ITIL® 4 Essentials: Your essential guide for the ITIL 4 Foundation exam and beyond, second editionClaire Agutter4h4.2 (50)$40 · $0.00
HorizonsJames Poskett15h 18m3.7 (472)$33 · $0.00
Target Date FundsPenelope I. Westwood4h 32m$4 · $0.00
Airfoils, Lift and Drag IllustratedPatrick Zwolinska11h 22m$50
Statistical Methods for Engineers and ScientistsPriya Ramanathan15h 5m$95
How to Profit from Google AdSenseAnthony Ekanem37m$4 · $0.00
AP Computer Science Principles Exam Study GuideMarie Peterson8h 32m$10
Invisible IncomeSara Melo1h 34m$1.80 · $0.00
From Zero To AutomaticSara Melo1h 25m$1.80 · $0.00
Self Publishing: Optimize Your Entire Publishing Business With AI Assistants (How to Rise to the Top in Book Sales Through Creative Disruption)Peter Felts1h 8m$3.50 · $0.00
Algorithms in Java, Coded and TestedBianca Zielinski7h 25m$5 · $0.00
Machine Learning in the CloudLionel Jansen8h 21m$5 · $0.00
Build a 2.5D RPG with Godot 4 and C#Gregory Whitlock7h 8m$5 · $0.00