AviSoori1x/makeMoE · AI Repo StatsAVAviSoori1x/makeMoE
From scratch implementation of a sparse mixture of experts language model inspired by Andrej Karpathy's makemore :)
LLM & foundation modelsJupyter Notebook MIT 96created 2024-01-22 #deep-learning#large-language-models#llm#mixture-of-experts#neural-networks#pytorch#pytorch-implementation