Liuhong99/Sophia · AI Repo StatsLILiuhong99/Sophia
The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”
LLM & foundation modelsPython MIT 56created 2023-05-23 #deep-learning#large-language-models#optimizer