Efficiently Scaling Machine Learning Systems Across Heterogeneous Resources