pjlab-sys4nlp/llama-moe

⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (EMNLP 2024)

LLM & foundation modelsPython Apache-2.0 60created 2023-07-24

Stars

1,005

1-day growth

7-day growth

—

30-day growth

—

Star history

Tracked since 2026-11-13

#continual-pre-training#expert-partition#llama#llm#mixture-of-experts#moe

Top contributors

Not collected yet (contributors job).

pjlab-sys4nlp/llama-moe · AI Repo Stats