Models that understand video: video question answering, summarization, action recognition.
Ranked by total GitHub stars, refreshed every day.
| # | Repo |
|---|---|
| 1 | Active (pushed in the last 90 days)bradautomates/claude-video Give Claude the ability to watch any video. /watch downloads, extracts frames, transcribes, hands it all to Claude. 21KPython |
| 2 | Slowing down (last push 3–12 months ago)facebookresearch/SlowFast PySlowFast: video understanding codebase from FAIR for reproducing state-of-the-art video models. 7.4KPython |
| 3 | Slowing down (last push 3–12 months ago)open-mmlab/mmaction2 OpenMMLab's Next Generation Video Understanding Toolbox and Benchmark 5.2KPython |
| 4 | Active (pushed in the last 90 days)facebookresearch/pytorchvideo A deep learning library for video understanding research. 3.6KPython |
| 5 | Slowing down (last push 3–12 months ago)OpenGVLab/InternVideo [ECCV2024] Video Foundation Models & Data for Multimodal Understanding 2.4KPython |
| 6 | Active (pushed in the last 90 days)zai-org/GLM-V GLM-4.6V/4.5V/4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning 2.4KPython |
| 7 | Inactive (no push for over a year, or archived)mit-han-lab/temporal-shift-module [ICCV 2019] TSM: Temporal Shift Module for Efficient Video Understanding 2.2KPython |
| 8 | Slowing down (last push 3–12 months ago)HKUDS/VideoAgent [EMNLP2026] "VideoAgent: All-in-One Agentic Framework for Video Understanding and Editing, and Remaking" 2KPython |
| 9 | Inactive (no push for over a year, or archived)open-mmlab/mmaction An open-source toolbox for action understanding based on PyTorch 1.9KPython |
| 10 | Inactive (no push for over a year, or archived)facebookresearch/TimeSformer(archived) The official pytorch implementation of our paper "Is Space-Time Attention All You Need for Video Understanding?" 1.9KPython |
| 11 | Inactive (no push for over a year, or archived)MCG-NJU/VideoMAE [NeurIPS 2022 Spotlight] VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training 1.8KPython |
| 12 | |
| 13 | Inactive (no push for over a year, or archived)PaddlePaddle/PaddleVideo Awesome video understanding toolkits based on PaddlePaddle. It supports video data annotation tools, lightweight RGB and skeleton based action recognition model, practical applications for video tagging and sport action detection. 1.7KPython |
| 14 | Inactive (no push for over a year, or archived)yjxiong/temporal-segment-networks Code & Models for Temporal Segment Networks (TSN) in ECCV 2016 1.6KPython |
| 15 | Slowing down (last push 3–12 months ago)bytedance/Lance A 3B-active-parameter native unified multimodal model for image and video understanding, generation, and editing. 1.4KPython |
| 16 | Inactive (no push for over a year, or archived)DAMO-NLP-SG/VideoLLaMA3 Frontier Multimodal Foundation Models for Image and Video Understanding 1.2KJupyter Notebook |
| 17 | Inactive (no push for over a year, or archived)OpenGVLab/VideoMamba [ECCV2024] VideoMamba: State Space Model for Efficient Video Understanding 1.1KPython |
| 18 | Inactive (no push for over a year, or archived)yjxiong/tsn-pytorch Temporal Segment Networks (TSN) in PyTorch 1.1KPython |
| 19 | Active (pushed in the last 90 days)nghorbani/amass Data preparation and loader for AMASS 1KJupyter Notebook |
| 20 | Inactive (no push for over a year, or archived)HHTseng/video-classification Tutorial for video classification/ action recognition using 3D CNN/ CNN+RNN on UCF101 970Jupyter Notebook |