Proceedings of Machine Learning and Systems 6 (MLSys 2024)

---

• Punica: Multi-Tenant LoRA Serving Lequn Chen, Zihao Ye, Yongji Wu, Danyang Zhuo, Luis Ceze, Arvind Krishnamurthy
• ACROBAT: Optimizing Auto-batching of Dynamic Deep Learning at Compile Time Pratik Fegade, Tianqi Chen, Phillip Gibbons, Todd Mowry
• HeteroSwitch: Characterizing and Taming System-Induced Data Heterogeneity in Federated Learning Gyudong Kim, Mehdi Ghasemi, Soroush Heidari, Seungryong Kim, Young Geun Kim, Sarma Vrudhula, Carole-Jean Wu
• JIT-Q: Just-in-time Quantization with Processing-In-Memory for Efficient ML Training Mohamed Ibrahim, Shaizeen Aga, Ada Li, Suchita Pati, Mahzabeen Islam
• Schrodinger's FP Training Neural Networks with Dynamic Floating-Point Containers Milos Nikolic, Enrique Torres Sanchez, Jiahui Wang, Ali Hadi Zadeh, Mostafa Mahmoud, Ameer Abdelhadi, Kareem Ibrahim, Andreas Moshovos
• Lancet: Accelerating Mixture-of-Experts Training via Whole Graph Computation-Communication Overlapping Chenyu Jiang, Ye Tian, Zhen Jia, Shuai Zheng, Chuan Wu, Yida Wang
• AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, Song Han
• DiffusionPipe: Training Large Diffusion Models with Efficient Pipelines Ye Tian, Zhen Jia, Ziyue Luo, Yida Wang, Chuan Wu
• Keyformer: KV Cache reduction through key tokens selection for Efficient Generative Inference Muhammad Adnan, Akhil Arunkumar, Gaurav Jain, Prashant Nair, Ilya Soloveychik, Purushotham Kamath
• Accelerating ReLU for MPC-Based Private Inference with a Communication-Efficient Sign Estimation Kiwan Maeng, G. Edward Suh
• FlashDecoding++: Faster Large Language Model Inference with Asynchronization, Flat GEMM Optimization, and Heuristics Ke Hong, Guohao Dai, Jiaming Xu, Qiuli Mao, Xiuhong Li, Jun Liu, kangdi chen, Yuhan Dong, Yu Wang
• HeteGen: Efficient Heterogeneous Parallel Inference for Large Language Models on Resource-Constrained Devices ZHAO XUANLEI, Bin Jia, Haotian Zhou, Ziming Liu, Shenggan Cheng, Yang You
• CloudEval-YAML: A Practical Benchmark for Cloud Configuration Generation Yifei Xu, Yuning Chen, Xumiao Zhang, Xianshang Lin, Pan Hu, Yunfei Ma, Songwu Lu, Wan Du, Zhuoqing Mao, Ennan Zhai, Dennis Cai
• Atom: Low-Bit Quantization for Efficient and Accurate LLM Serving Yilong Zhao, Chien-Yu Lin, Kan Zhu, Zihao Ye, Lequn Chen, Size Zheng, Luis Ceze, Arvind Krishnamurthy, Tianqi Chen, Baris Kasikci
• ACCURATE LOW-DEGREE POLYNOMIAL APPROXIMATION OF NON-POLYNOMIAL OPERATORS FOR FAST PRIVATE INFERENCE IN HOMOMORPHIC ENCRYPTION Jingtian Dang, Jianming Tong, Anupam Golder, Cong "Callie" Hao, Arijit Raychowdhury, Tushar Krishna
• SiDA: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Models Zhixu Du, Shiyu Li, Yuhao Wu, Xiangyu Jiang, Jingwei Sun, Qilin Zheng, Yongkai Wu, Ang Li, Hai Li, Yiran Chen
• Does Compressing Activations Help Model Parallel Training? Song Bian, Dacheng Li, Hongyi Wang, Eric Xing, Shivaram Venkataraman
• Distributed Matrix-Based Sampling for Graph Neural Network Training Alok Tripathy, Katherine Yelick, Aydin Buluc
• Disaggregated Multi-Tower: Topology-aware Modeling Technique for Efficient Large Scale Recommendation Liang Luo, Buyun Zhang, Michael Tsang, Yinbin Ma, Ching-Hsiang Chu, Yuxin Chen, Shen Li, Yuchen Hao, Yanli Zhao, Guna Lakshminarayanan, Ellie Wen, Jongsoo Park, Dheevatsa Mudigere, Maxim Naumov
• VQPy: An Object-Oriented Approach to Modern Video Analytics Shan Yu, Zhenting Zhu, Yu Chen, Hanchen Xu, Pengzhan Zhao, Yang Wang, Arthi Padmanabhan, Hugo Latapie, Harry Xu
• SLoRA: Scalable Serving of Thousands of LoRA Adapters Ying Sheng, Shiyi Cao, Dacheng Li, Coleman Hooper, Nicholas Lee, Shuo Yang, Christopher Chou, Banghua Zhu, Lianmin Zheng, Kurt Keutzer, Joseph Gonzalez, Ion Stoica
• L-GreCo: Layerwise-adaptive Gradient Compression For Efficient Data-parallel Deep Learning Ilia Markov, Kaveh Alim, Elias Frantar, Dan Alistarh
• Prompt Cache: Modular Attention Reuse for Low-Latency Inference In Gim, Guojun Chen, Seung-seob Lee, Nikhil Sarda, Anurag Khandelwal, Lin Zhong
• Fine-Tuning Language Models Using Formal Methods Feedback: A Use Case in Autonomous Systems Yunhao Yang, Neel P. Bhatt, Tyler Ingebrand, William Ward, Steven Carr, Atlas Wang, Ufuk Topcu
• VIDUR: A LARGE-SCALE SIMULATION FRAMEWORK FOR LLM INFERENCE Amey Agrawal, Nitin Kedia, Jayashree Mohan, Ashish Panwar, Nipun Kwatra, Bhargav Gulavani, Ramachandran Ramjee, Alexey Tumanov
• Torch2Chip: An End-to-end Customizable Deep Neural Network Compression and Deployment Toolkit for Prototype Hardware Accelerator Design Jian Meng, Yuan Liao, Anupreetham Anupreetham, Ahmed Hasssan, Shixing Yu, Han-sok Suh, Xiaofeng Hu, Jae-sun Seo
• Q-Hitter: A Better Token Oracle for Efficient LLM Inference via Sparse-Quantized KV Cache Zhenyu Zhang, Shiwei Liu, Runjin Chen, Bhavya Kailkhura, Beidi Chen, Atlas Wang
• FedTrans: Efficient Federated Learning via Multi-Model Transformation Yuxuan Zhu, Jiachen Liu, Mosharaf Chowdhury, Fan Lai
• LIFL: A Lightweight, Event-driven Serverless Platform for Federated Learning Shixiong Qi, K. K. Ramakrishnan, Myungjin Lee
• Proteus: Preserving Model Confidentiality during Graph Optimizations Yubo Gao, Maryam Haghifam, Christina Giannoula, Renbo Tu, Gennady Pekhimenko, Nandita Vijaykumar
• QMoE: Sub-1-Bit Compression of Trillion Parameter Models Elias Frantar, Dan Alistarh
• vMCU: Coordinated Memory Management and Kernel Optimization for DNN Inference on MCUs Size Zheng, Renze Chen, Meng Li, Zihao Ye, Luis Ceze, Yun Liang
• UniDM: A Unified Framework for Data Manipulation with Large Language Models Yichen Qian, Yongyi He, Rong Zhu, Jintao Huang, Zhijian Ma, Haibin Wang, Yaohua Wang, Xiuyu Sun, Defu Lian, Bolin Ding, Jingren Zhou
• Efficient Post-training Quantization with FP8 Formats Haihao Shen, Naveen Mellempudi, Xin He, Qun Gao, Chang Wang, Mengni Wang
• COMET: Neural Cost Model Explanation Framework Isha Chaudhary, Alex Renda, Charith Mendis, Gagandeep Singh
• On Latency Predictors for Neural Architecture Search Yash Akhauri, Mohamed Abdelfattah
• FLASH: Fast Model Adaptation in ML-Centric Cloud Platforms Haoran Qiu, Weichao Mao, Archit Patke, Shengkun Cui, Chen Wang, Hubertus Franke, Zbigniew Kalbarczyk, Tamer Basar, Ravi Iyer