Memories.ai, a video-data infrastructure startup founded by two former Meta engineers, is indexing real-world human actions frame by frame — and the same machinery that teaches robots to pour coffee could soon help studios search a century of footage.
Quasa.io reported this week that the company, founded by Shawn Shen and Ben Zhou, has built a system that ingests first-person video, tags every action, and returns timestamped annotations as machine-readable JSON.
Here is why an entertainment editor is paying attention. The hard problem in Hollywood’s archives has never been storage; it has always been retrieval. Studios sit on millions of hours of unindexed footage — B-roll, dailies, press junkets, reality-show outtakes — that no one can search at scale.
A tool that watches video and labels what each hand is doing, second by second, is precisely the kind of plumbing that turns a dead archive into a licensable asset.
Memories.ai Video Datalake collects first-person task video
- What it is: Memories.ai’s Video Datalake collects first-person video of real tasks, indexes the actions, and annotates them with a client’s own labels.
- Who built it: Shawn Shen and Ben Zhou, both formerly of Meta, per Quasa.io’s report.
- The catch: Quasa.io rates the service 3.8 out of 5 stars and urges buyers to verify sample footage quality, data rights, and total cost before committing.
What the Video Datalake actually does
The details matter here. According to Quasa.io’s review, the system collects first-person-perspective video showing real tasks being performed, indexes the actions within it, and supports annotation using each customer’s own label schemes. The output includes captions for each hand, action segments, and timestamps, all delivered as JSON.
That last detail is the tell. JSON delivery means the annotations are built for machines, not humans — training data for robotics and other physical AI applications, which is exactly the market Quasa.io identifies. Robotics labs and engineering teams that need searchable, annotated video of human tasks are the stated audience.
Teams can pull footage from the company’s own collection or upload their own video for indexing and annotation. Developer access runs through a REST API and an MCP server, with usage-based pricing and enterprise deployment options. Quasa.io notes that terms for datasets and custom recording should be confirmed separately, and that current pricing lives on the company’s official website.
Streaming platforms rely on human taggers
But the math tells a different story when you widen the lens. The entertainment industry’s relationship with video metadata has been, frankly, primitive. Streaming platforms rely on human taggers and editorial teams to describe their catalogs — which is why “cozy mystery with a strong female lead” works as a Netflix browse row but a specific shot inside 40,000 hours of a reality franchise does not.
A system that returns action segments with timestamps changes the unit of search. Instead of finding an episode, a licensing team could find the moment. For unscripted television — the genre most dependent on vast, cheaply shot footage — that is a genuine economic shift. Outtakes that were once unusable become inventory.
There is a second, quieter implication: production itself. First-person video of real tasks is, by definition, what a physical-AI company wants. But the same annotation pipeline — hands, actions, timestamps — describes exactly what a post-production supervisor does when logging dailies. The tools built for robotics labs are structurally similar to the tools a modern studio would need to make its archives machine-searchable.
The caveats Quasa.io flags
Now for the fine print, because there is plenty of it. Quasa.io gives the service 3.8 out of 5 stars and frames its recommendation conditionally: consider it for acquiring and structuring video for robotics research, but confirm the quality of sample material, data rights, and total costs first.
Data rights deserve particular scrutiny in an entertainment context. Footage of real people performing real tasks carries consent, licensing, and likeness questions that a robotics lab may tolerate but a studio’s legal department will not. Quasa.io explicitly says those terms should be confirmed separately — which is reviewer-speak for “nobody has verified this for you yet.”
The usage-based pricing model also cuts both ways. It lowers the barrier to a pilot, but it makes total spend unpredictable at scale, which is why enterprise deployment options exist as a separate track.
Where this sits in the wider AI-for-video race
Memories.ai is not operating in a vacuum. The broader industry has spent the past two years racing to make video computable — from AI-powered editing suites to metadata vendors serving the streaming market — because the value of a video library increasingly depends on how findable every second of it is.
What distinguishes this offering, per Quasa.io’s report, is the focus on first-person task video and physical AI rather than consumer media, with robotics as the stated customer.
Whether entertainment becomes a customer later is an open question, and neither the company nor the review claims otherwise. What is confirmed is the infrastructure: collection, indexing, custom-label annotation, JSON output, API access, usage-based pricing.
The next step for buyers
For teams weighing a pilot, Quasa.io’s guidance is concrete: request sample footage, verify who owns what once your video is indexed, and model the full cost before scaling. The company’s official website carries current pricing, and the review notes that dataset terms and custom recording arrangements need separate confirmation.
For context on how young this market still is: the entire proposition rests on a rating of 3.8 out of 5 stars from a single independent review — a snapshot, not a verdict. The archives, meanwhile, are not getting any shorter.
So here is my question for the production folks in the audience: if you could search your studio’s entire unindexed vault by action rather than by title, what would you look for first? Tell us in the comments.