diff --git a/README.md b/README.md index 1714554..6d1e897 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ Official implementation of **Video2LoRA** --- -### [Install](#install)  •  [Checkpoints](#checkpoints)  •  [Inference](#inference)  •  [Data Format](#data-format)  •  [Training](#train) +### [Quick Start](#-quick-start)  •  [Install](#install)  •  [Checkpoints](#checkpoints)  •  [Inference](#inference)  •  [Data Format](#data-format)  •  [Training](#train)

Animated Video2LoRA method diagram @@ -25,6 +25,19 @@ Official implementation of **Video2LoRA** Video2LoRA trains a hypernetwork that converts a video into LoRA weights for a frozen vision-language model. The generated adapter lets the model answer later text prompts without feeding the video tokens again. +## 🚀 Quick Start + +Try zero-visual-token inference instantly in your browser via our self-contained interactive tutorial notebook: + +[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/video2lora/Video2LoRA/blob/demo/demo.ipynb) + +This interactive notebook guides you through: +1. Downloading the 2.2B SmolVLM2 hypernetwork checkpoint. +2. Linking and resolving the project's qualitative videos. +3. Parametric internalization (extracting visual features and generating custom adapter weights). +4. Running comparative Q&A inference (comparing the base vision-language model against the Video2LoRA adapter with zero visual tokens). +5. Visualizing the comparison using a custom dashboard layout. + ## Install Install `uv`. diff --git a/demo.ipynb b/demo.ipynb index 36f9100..58b5127 100644 --- a/demo.ipynb +++ b/demo.ipynb @@ -98,7 +98,7 @@ "\n", "# Install necessary packages\n", "!pip install torch torchvision torchaudio\n", - "!pip install transformers accelerate peft huggingface_hub decord av opencv-python matplotlib einops jaxtyping" + "!pip install transformers accelerate peft huggingface_hub decord av opencv-python matplotlib einops jaxtyping num2words" ] }, {