Schedule

Lab 0

• LookNN Neural network with no multiplication
• Channel Permutations for N:M Sparsity

Lab 1

• DepthShrinker A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks
• APQ Joint Search for Network Architecture, Pruning and Quantization Policy

Lab 2 (Lab 1 due)

• Device Placement Optimization with Reinforcement Learning
• EfficientNetV2 Smaller Models and Faster Training
• FBNet Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search

• DARTS Differentiable Architecture Search
• Efficient Neural Architecture Search via Parameter Sharing

• FracBNN Accurate and FPGA-Efficient Binary Neural Networks with Fractional Activations

Lab 3 (Lab 2 due)

• In-datacenter performance analysis of a tensor processing unit

• On the efficacy of knowledge distillation
• TinyTL Reduce Memory, Not Parameters for Efficient On-Device Learning

• Deep Gradient Compression Reducing the Communication Bandwidth for Distributed Training

Lab 4 (Lab 3 due)

• AdaComp Adaptive Residual Gradient Compression for Data-Parallel Distributed Training

• SWANN Small-World Architecture for Fast Convergence of Neural Networks
• On-Device Training Under 256KB Memory

Final Project

(Lab 4 due)

• P3 Distributed Deep Graph Learning at Scale
• BGL GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessing

• GAN Compression Efficient Architectures for Interactive Conditional GANs
• Ekya Continuous Learning of Video Analytics Models on Edge Compute Servers

• Accelerating Transformer-based Deep Learning Models on FPGAs using Column Balanced Block Pruning
• LightSeq2 accelerated training for transformer-based models on GPUs

• PipeTransformer Automated Elastic Pipelining for Distributed Training of Large-scale Models
• Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM

Project presentation poster due