apunkt/doc-to-lora
1
0
Fork 0
mirror of https://github.com/SakanaAI/doc-to-lora.git synced 2026-07-23 17:01:04 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

Configure Step 8 to loop through and display all 6 qualitative examples

Browse source
This commit is contained in:
Sarvesh 2026-06-09 15:51:45 +05:30
parent 2ba3c1bd40
commit 327abd9e38
1 changed files with 23 additions and 15 deletions
Download patch file Download diff file Expand all files Collapse all files

38
demo.ipynb
View file

@ -12,11 +12,11 @@
"\n",
"All model architectures, helper functions, and dependencies are defined directly within this notebook so it can run independently of the main repository codebase.\n",
"\n",
"### 🔗 Project Resources\n",
"- [**🌐 Project Website**](https://video2lora.github.io/)\n",
"- [**📄 arXiv Paper**](https://arxiv.org/abs/2606.04351)\n",
"- [**🤗 Hugging Face Checkpoints**](https://huggingface.co/MananSuri27/Video2LoRA-SmolVLM-ckpts)\n",
"- [**💻 GitHub Repository**](https://github.com/MananSuri27/video2lora)"
"### \ud83d\udd17 Project Resources\n",
"- [**\ud83c\udf10 Project Website**](https://video2lora.github.io/)\n",
"- [**\ud83d\udcc4 arXiv Paper**](https://arxiv.org/abs/2606.04351)\n",
"- [**\ud83e\udd17 Hugging Face Checkpoints**](https://huggingface.co/MananSuri27/Video2LoRA-SmolVLM-ckpts)\n",
"- [**\ud83d\udcbb GitHub Repository**](https://github.com/MananSuri27/video2lora)"
]
},
{
@ -601,15 +601,23 @@
" \"\"\"\n",
" display(HTML(html_content))\n",
"\n",
"# Display the comparison dashboard dynamically for our completed example!\n",
"display_comparison(\n",
" video_path=examples[1][\"video_path\"],\n",
" question_prompt=examples[1][\"prompt\"],\n",
" ground_truth=examples[1][\"target_text\"],\n",
" base_model_output=base_prediction,\n",
" video2lora_output=video2lora_prediction,\n",
" dataset_name=examples[1][\"dataset\"]\n",
")"
"# Loop through all 6 qualitative examples, run internalization + inference, and display the dashboards\n",
"print(\"Processing and running qualitative comparison inference for all 6 examples...\")\n",
"for idx, example in enumerate(examples):\n",
" print(f\"\\n--- Running Example {idx+1}/{len(examples)}: {example['id']} ---\")\n",
" # 1. Internalize\n",
" loras = run_internalization(example, model, raw_model, processor, train_args, device)\n",
" # 2. Run inference\n",
" base_pred, v2l_pred = run_inference(example, loras, model, raw_model, processor, tokenizer, train_args, device)\n",
" # 3. Render dashboard\n",
" display_comparison(\n",
" video_path=example[\"video_path\"],\n",
" question_prompt=example[\"prompt\"],\n",
" ground_truth=example[\"target_text\"],\n",
" base_model_output=base_pred,\n",
" video2lora_output=v2l_pred,\n",
" dataset_name=example[\"dataset\"]\n",
" )"
]
}
],
@ -634,4 +642,4 @@
},
"nbformat": 4,
"nbformat_minor": 2
}
}