Meet VideoScore2: AI Film Critic That Thinks Before It Scores

The age of AI cinema is here. Text-to-video models generate everything from breathtaking landscapes to quirky short films, pushing the boundaries of digital creativity. But this creative explosion has exposed a fundamental challenge: with thousands of AI-generated videos, how do we separate the masterpieces from digital noise?

Until now, AI evaluators have acted like a simplistic ratings aggregator—giving a single, opaque score without any explanation. It’s like getting a thumbs-up or thumbs-down with no review. This isn’t helpful for developers who need to understand why a video failed or succeeded. What the field really needs is an expert critic — an AI that doesn't just give a rating but shows its work.

That’s why a global research team from top universities and AI organizations like 2077AI, M-A-P, UIUC and Abaka AI developed VideoScore2. It’s a revolutionary evaluation framework designed with a simple, powerful principle: think before you score.

As the first framework to combine multi-dimensional scoring with detailed rationales, VideoScore2 immediately sets itself apart from prior work.

VideoScore2 is the first framework to combine multi-dimensional scoring with detailed rationales, setting it apart from existing video reward models.

🎬 Inside the Critic's Mind: 3 Pillars of a Great AI Video

Unlike other evaluators, VideoScore2 doesn't just look at one thing. It assesses a video's quality across three distinct, crucial dimensions, much like a human critic would.

• 🎨 Visual Quality: Is the film technically proficient? This covers everything from resolution and clarity to the smoothness of motion and the absence of weird visual artifacts.

• ✍️ Text Alignment: Does it stick to the script? The AI must faithfully render all the subjects, actions, and styles requested in the text prompt.

• 🧠 Physical Consistency: Does the world make sense? The video must adhere to common sense and the basic laws of physics. Gravity should work, and objects shouldn't magically morph or disappear without reason.

Crucially, VideoScore2 doesn't just output three numbers. It first generates a detailed, chain-of-thought rationale, explaining its assessment for each dimension before concluding with a final score. This transparency is a game-changer for AI development.

Education of a Critic: How We Trained VideoScore2

Creating such a sophisticated AI critic required an equally sophisticated education. We developed a two-stage training process, akin to sending our model to an elite film school with a world-class curriculum.

Part 1: The Dataset

The foundation of this education is VideoFeedback2, a massive dataset of over 27,000 AI-generated videos. This "film library" was built with diversity at its core. Prompts were sourced from a wide range of real-world and manually designed scenarios to test everything from basic actions to complex camera movements.

The VideoFeedback2 dataset was built from a diverse mix of prompt sources to ensure broad coverage.

To create a realistic gradient of quality, videos were generated by over 20 different text-to-video models, from early open-source projects to state-of-the-art systems like Sora and Kling.

Videos were generated by over 20 different text-to-video models, creating a fine-grained quality spectrum from "Poor" to "Perfect."

Each video in this library was then watched and annotated by human experts who provided not just scores, but detailed reasoning traces — the exact "why" behind their ratings.

Examples from the VideoFeedback2 dataset, showcasing the range of quality from "Good" to "Perfect/Modern" tiers used to train VideoScore2.

The final dataset contains a natural distribution of scores, with most videos falling into the nuanced mid-range, providing the perfect material for training a discerning critic.

The distribution of human scores in our dataset captures the full spectrum of quality, providing a realistic and challenging curriculum for AI.

Part 2: Training Method

With the curriculum in place, the training began.

• First, through Supervised Fine-Tuning (SFT), the model studied the entire VideoFeedback2 library, learning to imitate the patterns of human experts' scores and reasoning.

• Then, it entered a "masterclass" to sharpen its analytical robustness. We employed Group Relative Policy Optimization (GRPO), an advanced Reinforcement Learning (RL) technique, to hone its judgment, allowing it to generalize its knowledge and align even more closely with nuanced human preferences, making it a truly robust critic.

🌟 Results: Putting VideoScore2 to the Test

After its training, it was time for the final exam. We tested VideoScore2 against a suite of existing benchmarks, and it passed with flying colors. It not only achieved state-of-the-art accuracy on our internal benchmark but, more importantly, demonstrated superior generalization across four out-of-domain benchmarks. This proves that VideoScore2 didn't just cram for its own exam; it truly learned the principles of what makes a good video.

🎥 From Critic to Director: What's Next?

Perhaps the most powerful aspect of VideoScore2 is that its journey doesn't end as a critic. It can now become a director.

By serving as a highly reliable reward model, VideoScore2 can provide the nuanced feedback needed to train the next generation of text-to-video models. We've already shown that using it to automatically select the "best-of-N" generated videos significantly improves output quality. This creates a powerful feedback loop where better evaluation leads to better generation.

This entire project is a testament to the power of open, collaborative research. By releasing VideoScore2 and its dataset, we are providing the entire AI community with a transparent, powerful tool to accelerate progress and build more aligned, creative, and controllable generative models