Have you ever wondered what happens behind the scenes of a product review video? For merchants on video commerce platforms, it's a goldmine of potential sales—if they can quickly connect the products in the video to their online store. But imagine having to manually identify and link every single product across thousands of videos. It's a slow, tedious, and often inaccurate process that costs valuable time and hurts a business's bottom line.

That was the challenge facing Firework, a leading video commerce platform. They needed a way to transform this manual struggle into an automated, efficient workflow. To make that vision a reality, they partnered with Tokyo Techies to develop a game-changing AI solution.

Within just one month, Tokyo Techies delivered an innovative service that uses Large Language Models (LLMs) and computer vision. This powerful combination efficiently identifies and categorizes a wide range of products embedded in videos. The result? A new service that processes a single video in under 20 seconds at a remarkably low cost of less than $0.10 per video.

This collaboration has had a substantial impact, leading to a significant improvement in Firework's operational efficiency and a boost in merchant conversion rates.

The Process: A Glimpse Behind the Curtain

So, how does it all work? The process is a seamless blend of advanced technologies:
First, the system recognizes the highly commercialized products in a video.
Then, it compares these products to a vast database to find the most similar matches.
Concurrently, the system analyzes the video's content to gather supplemental information from reviewers, such as their sentiment (positive, neutral, or negative).

Overcoming Key Challenges

Developing this solution wasn't without its hurdles. Tokyo Techies had to overcome two main challenges to ensure the service was both fast and accurate.

Challenge 1: The Need for Speed
Comparing every product in a video with every product in a massive database is a huge time sink. For millions of videos, this could add up to an enormous amount of time and resources.

The Solution: We engineered a more efficient system that acts like a smart librarian. Imagine a librarian who, instead of searching every single book, can first use a title fragment or category to pinpoint the exact section. Our AI does the same, analyzing metadata from the video—like the words spoken or text on screen—our AI gets a strong hint about the product's identity. This intelligently narrows the search to fewer than 50 relevant items. To further boost performance, we also implemented a concurrent processing strategy, allowing the system to perform product recognition, analysis, and matching simultaneously, which ultimately reduced the total processing time by up to 10 times.

Challenge 2: Precision Matching
Many products look very similar but are actually distinct. Think about subtle differences in fashion items like material or color, or small variations in electronics like size or function. Conversely, a single product might have many different versions.

The Solution: The team focused on a comprehensive analysis of each product from every angle. This meticulous approach ensures that even the smallest details—both in the video and in the database—are accounted for. By using both maximum and average scores during the matching process, the system can select the correct products with high accuracy, even when faced with nuanced differences.

Processing a video now takes under 20 seconds at a cost of less than $0.10 per video. Firework has seen substantial improvements in operational efficiency and merchant conversion rates as a direct result of this collaboration.

Ready to accelerate your business with AI?Our team specializes in creating custom AI-powered solutions that solve your most pressing business challenges. From enhancing efficiency to boosting conversion rates, we can help.