feifeibear/long-context-attention · AI Repo StatsFEfeifeibear/long-context-attention
USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference
Inference & local LLMPython Apache-2.0 86created 2024-03-27 #attention-is-all-you-need#deepspeed-ulysses#llm-inference#llm-training#pytorch#ring-attention