Prediction Is All MoE Needs: Expert Load Distribution Goes from Fluctuating to Stabilizing

This project involves analyzing the load distribution of Mixture-of-Experts (MoE) models and training predictive models based on the collected data.

Directory Structure

• data/
  Contains the collected dataset, which records the load distribution across MoE experts.

• res/
  Stores the trained models and analysis results.

• modeling_lstm_train.py
  Defines and trains the LSTM-based model for predicting expert loads.

• paint_*.ipynb
  A series of Jupyter notebooks used to run various prediction algorithms and visualize the results.

Getting Started

To reproduce the results or run your own experiments, please refer to the individual scripts and notebooks described above.