Proceedings of Machine Learning and Systems 4 (MLSys 2022)

---

• Matchmaker: Data Drift Mitigation in Machine Learning for Large-Scale Systems Ankur Mallick, Kevin Hsieh, Behnaz Arzani, Gauri Joshi
• Hydrozoa: Dynamic Hybrid-Parallel DNN Training on Serverless Containers Runsheng Guo, Victor Guo, Antonio Kim, Josh Hildred, Khuzaima Daudjee
• SRIFTY: Swift and Thrifty Distributed Neural Network Training on the Cloud Liang Luo, Peter West, Pratyush Patel, Arvind Krishnamurthy, Luis Ceze
• TyXe: Pyro-based Bayesian neural nets for Pytorch Hippolyt Ritter, Theofanis Karaletsos
• A Tale of Two Models: Constructing Evasive Attacks on Edge Models Wei Hao, Aahil Awatramani, Jiayang Hu, Chengzhi Mao, Pin-Chun Chen, Eyal Cidon, Asaf Cidon, Junfeng Yang
• Sequential Aggregation and Rematerialization: Distributed Full-batch Training of Graph Neural Networks on Large Graphs Hesham Mostafa
• Random Offset Block Embedding (ROBE) for compressed embedding tables in deep learning recommendation systems Aditya Desai, Li Chou, Anshumali Shrivastava
• Bolt: Bridging the Gap between Auto-tuners and Hardware-native Performance Jiarong Xing, Leyuan Wang, Shang Zhang, Jack Chen, Ang Chen, Yibo Zhu
• NURD: Negative-Unlabeled Learning for Online Datacenter Straggler Prediction Yi Ding, Avinash Rao, Hyebin Song, Rebecca Willett, Henry (Hank) Hoffmann
• TAGLETS: A System for Automatic Semi-Supervised Learning with Auxiliary Data Wasu Piriyakulkij, Cristina Menghini, Ross Briden, Nihal Vivekanand Nayak, Jeffrey Zhu, Elaheh Raisi, Stephen Bach
• Collapsible Linear Blocks for Super-Efficient Super Resolution Kartikeya Bhardwaj, Milos Milosavljevic, Liam O'Neil, Dibakar Gope, Ramon Matas, Alex Chalfin, Naveen Suda, Lingchuan Meng, Danny Loh
• Pathways: Asynchronous Distributed Dataflow for ML Paul Barham, Aakanksha Chowdhery, Jeff Dean, Sanjay Ghemawat, Steven Hand, Daniel Hurt, Michael Isard, Hyeontaek Lim, Ruoming Pang, Sudip Roy, Brennan Saeta, Parker Schuh, Ryan Sepassi, Laurent Shafey , Chandu Thekkath, Yonghui Wu
• Randomness in Neural Network Training: Characterizing the Impact of Tooling Donglin Zhuang, Xingyao Zhang, Shuaiwen Song, Sara Hooker
• Sustainable AI: Environmental Implications, Challenges and Opportunities Carole-Jean Wu, Ramya Raghavendra, Udit Gupta, Bilge Acun, Newsha Ardalani, Kiwan Maeng, Gloria Chang, Fiona Aga, Jinshi Huang, Charles Bai, Michael Gschwind, Anurag Gupta, Myle Ott, Anastasia Melnikov, Salvatore Candido, David Brooks, Geeta Chauhan, Benjamin Lee, Hsien-Hsin Lee, Bugra Akyildiz, Maximilian Balandat, Joe Spisak, Ravi Jain, Mike Rabbat, Kim Hazelwood
• URSABench: A System for Comprehensive Benchmarking of Bayesian Deep Neural Network Models and Inference methods Meet Vadera, Jinyang Li, Adam Cobb, Brian Jalaian, Tarek Abdelzaher, Benjamin Marlin
• Towards the Co-design of Neural Networks and Accelerators Yanqi Zhou, Xuanyi Dong, Tianjian Meng, Mingxing Tan, Berkin Akin, Daiyi Peng, Amir Yazdanbakhsh, Da Huang, Ravi Narayanaswami, James Laudon
• QuadraLib: A Performant Quadratic Neural Network Library for Architecture Optimization and Design Exploration Zirui Xu, Fuxun Yu, Jinjun Xiong, Xiang Chen
• Revelio: ML-Generated Debugging Queries for Finding Root Causes in Distributed Systems Pradeep Dogga, Karthik Narasimhan, Anirudh Sivaraman, Shiv Saini, George Varghese, Ravi Netravali
• BNS-GCN: Efficient Full-Graph Training of Graph Convolutional Networks with Partition-Parallelism and Random Boundary Node Sampling Cheng Wan, Youjie Li, Ang Li, Nam Sung Kim, Yingyan Lin
• LightSecAgg: a Lightweight and Versatile Design for Secure Aggregation in Federated Learning Jinhyun So, Chaoyang He, Chien-Sheng Yang, Songze Li, Qian Yu, Ramy E. Ali, Basak Guler, Salman Avestimehr
• Learning Compressed Embeddings for On-Device Inference Niketan Pansare, Jay Katukuri, Aditya Arora, Frank Cipollone, Riyaaz Shaik, Noyan Tokgozoglu, Chandru Venkataraman
• Gyro Dropout: Maximizing Ensemble Effect in Neural Network Training Junyeol Lee, Hyeongju Kim, Hyungjun Oh, Jaemin Kim, Hongseok Jeung, Yung-Kyun Noh, Jiwon Seo
• A Transferable Approach for Partitioning Machine Learning Models on Multi-Chip-Modules Xinfeng Xie, Prakash Prabhu, Ulysse Beaugnon, Phitchaya Phothilimthana, Sudip Roy, Azalia Mirhoseini, Eugene Brevdo, James Laudon, Yanqi Zhou
• On the Utility of Gradient Compression in Distributed Training Systems Saurabh Agarwal, Hongyi Wang, Shivaram Venkataraman, Dimitris Papailiopoulos
• REX: Revisiting Budgeted Training with an Improved Schedule John Chen, Cameron Wolfe, Tasos Kyrillidis
• VirtualFlow: Decoupling Deep Learning Models from the Underlying Hardware Andrew Or, Haoyu Zhang, Michael None Freedman
• torch.fx: Practical Program Capture and Transformation for Deep Learning in Python James Reed, Zachary DeVito, Horace He, Ansley Ussery, Jason Ansel
• AccMPEG: Optimizing Video Encoding for Accurate Video Analytics Kuntai Du, Qizheng Zhang, Anton Arapin, Haodong Wang, Zhengxu Xia, Junchen Jiang
• QuClassi: A Hybrid Deep Neural Network Architecture based on Quantum State Fidelity Samuel A. Stein, Betis Baheri, Daniel Chen, Ying Mao, Qiang Guan, Ang Li, Shuai Xu, Caiwen Ding
• Graphiler: Optimizing Graph Neural Networks with Message Passing Data Flow Graph Zhiqiang Xie, Minjie Wang, Zihao Ye, Zheng Zhang, Rui Fan
• MLPerf Mobile Inference Benchmark: An Industry-Standard Open-Source Machine Learning Benchmark for On-Device AI Vijay Janapa Reddi, David Kanter, Peter Mattson, Jared Duke, Thai Nguyen, Ramesh Chukka, Ken Shiring, Koan-Sin Tan, Mark Charlebois, William Chou, Mostafa El-Khamy, Jungwook Hong, Tom St John, Cindy Trinh, Michael Buch, Mark Mazumder, Relja Markovic, Thomas Atta, Fatih Cakir, Masoud Charkhabi, Xiaodong Chen, Cheng-Ming Chiang, Dave Dexter, Terry Heo, Guenther Schmuelling, Maryam Shabani, Dylan Zika
• PAPAYA: Practical, Private, and Scalable Federated Learning Dzmitry Huba, John Nguyen, Kshitiz Malik, Ruiyu Zhu, Mike Rabbat, Ashkan Yousefpour, Carole-Jean Wu, Hongyuan Zhan, Pavel Ustinov, Harish Srinivas, Kaikai Wang, Anthony Shoumikhin, Jesik Min, Mani Malek
• GPU Semiring Primitives for Sparse Neighborhood Methods Corey J. Nolet, Divye Gala, Edward Raff, Joe Eaton, Brad Rees, John Zedlewski, Tim Oates
• Accelerating Training and Inference of Graph Neural Networks with Fast Sampling and Pipelining Tim Kaler, Nickolas Stathas, Anne Ouyang, Alexandros-Stavros Iliopoulos, Tao Schardl, Charles E. Leiserson, Jie Chen
• The CoRa Tensor Compiler: Compilation for Ragged Tensors with Minimal Padding Pratik Fegade, Tianqi Chen, Phillip Gibbons, Todd Mowry
• HALOS: Hashing Large Output Space for Cheap Inference Zichang Liu, Zhaozhuo Xu, Alan Ji, Junyan Zhang, Jonathan Li, Beidi Chen, Anshumali Shrivastava
• dPRO: A Generic Performance Diagnosis and Optimization Toolkit for Expediting Distributed DNN Training Hanpeng Hu, Chenyu Jiang, Yuchen Zhong, Yanghua Peng, Chuan Wu, Yibo Zhu, Haibin Lin, Chuanxiong Guo
• Understanding GNN Computational Graph: A Coordinated Computation, IO, and Memory Perspective Hengrui Zhang, Zhongming Yu, Guohao Dai, Guyue Huang, Yufei Ding, Yuan Xie, Yu Wang
• Bit-serial Weight Pools: Compression and Arbitrary Precision Execution of Neural Networks on Resource Constrained Processors Shurui Li, Puneet Gupta
• Efficient Strong Scaling Through Burst Parallel Training Seo Jin Park, Joshua Fried, Sunghyun Kim, Mohammad Alizadeh, Adam Belay
• SLA-Driven ML INFERENCE FRAMEWORK FOR CLOUDS WITH HETEROGENEOUS ACCELERATORS Junguk Cho, Diman Zad Tootaghaj, Lianjie Cao, Puneet Sharma
• FROTE: Feedback Rule-Driven Oversampling for Editing Models Oznur Alkan, Dennis Wei, Massimiliano Mattetti, Rahul Nair, Elizabeth Daly, Diptikalyan Saha
• ML-EXray: Visibility into ML Deployment on the Edge Hang Qiu, Ioanna Vavelidou, Jian Li, Evgenya Pergament, Pete Warden, Sandeep Chinchali, Zain Asgar, Sachin Katti
• TorchSparse: Efficient Point Cloud Inference Engine Haotian Tang, Zhijian Liu, Xiuyu Li, Yujun Lin, Song Han
• Improving Model Training with Multi-fidelity Hyperparameter Evaluation Yimin Huang, Yujun Li, Hanrong Ye, Zhenguo Li, Zhihua Zhang
• Plumber: Diagnosing and Removing Performance Bottlenecks in Machine Learning Data Pipelines Michael Kuchnik, Ana Klimovic, Jiri Simsa, Virginia Smith, George Amvrosiadis
• mmSampler: Efficient Frame Sampler for Multimodal Video Retrieval Zhiming Hu, Ning Ye, Iqbal Mohomed
• ULPPACK: Fast Sub-8-bit Matrix Multiply on Commodity SIMD Hardware Jaeyeon Won, Jeyeon Si, Sam Son, Tae Jun Ham, Jae W. Lee
• Apollo: Automatic Partition-based Operator Fusion through Layer by Layer Optimization Jie Zhao, Xiong Gao, Ruijie Xia, Zhaochuang Zhang, Deshi Chen, Lei Chen, Renwei Zhang, Zhen Geng, Bin Cheng, Xuefeng Jin
• Synthesizing Optimal Parallelism Placement and Reduction Strategies on Hierarchical Systems for Deep Learning Ningning Xie, Tamara Norman, Dominik Grewe, Dimitrios Vytiniotis
• DietCode: Automatic Optimization for Dynamic Tensor Programs Bojian Zheng, Ziheng Jiang, Cody Hao Yu, Haichen Shen, Joshua Fromm, Yizhi Liu, Yida Wang, Luis Ceze, Tianqi Chen, Gennady Pekhimenko