AI news

Prelulu explores real-time video stories, with character memory and costs central

By 虾蛄内容部1 viewsSource: 虾蛄AI
Prelulu virtual characters and real-time video conversations

Prelulu combines text, voice and real-time avatar calls and lets creators define characters. The team is exploring environments that change with the plot; a complete interactive world remains to be developed.

Article illustration

Step into a virtual story world.

Short dramas often have this kind of scene: the protagonist can plainly see a trap ahead, yet plunges straight in anyway.

Sometimes you really want to shout through the screen: do not go in!

Recently, an interactive short-drama demo appeared in Tencent Hunyuan's Hy-Nomas project. Viewers can talk to the characters, and what they say can change how the story develops.

Article illustration

The team behind the demo is called Prelulu.

The company has actually already made a rather interesting product: users can choose fictional characters, enter predefined stories, and place a video call midway through a conversation.

The character responds with a face and voice generated in real time, and the user can interject at any moment to change the direction of the story.

Article illustration

On the homepage, the character and story settings are the first things to catch the eye. Some characters are caught in dangerous family relationships, others live in fantasy worlds of magic and elves, and still others come from genres involving idols, werewolves, vampires, and more.

These fictional characters have introductions to their identities, personality traits, and a prewritten story opening. The openings deliberately choose moments that are easy to enter into, establishing both sides' situations from the start.

Article illustration

Text chat is only one entry point into Prelulu. When a scene calls for hearing a character's voice, users can start a voice call on the same character's page. If a video avatar has already been prepared for that character, they can also choose a real-time video call.

Once connected, the corresponding character appears on screen, with mouth movements, expressions, and voice generated alongside the reply. The user can interrupt at any point during the call.

Article illustration

Users can also decide whether to let the character see them. A video call can show only the fictional character's face, and the user's own camera is off by default.

In calls that support camera sharing, the character can respond to information in the camera view only after the user actively enables permission.

For example, the user might hold up an object and ask the character to describe it, or show a photograph directly, letting the conversation revolve around what is in front of them.

Article illustration

Another detail in Prelulu's real-time video calls is that characters' facial expressions change as they speak, while their mouth movements synchronize with the sound.

Each character has its own voice settings and expresses different emotions according to the content of the exchange.

To let stories continue, Prelulu has also designed memory mechanisms for chats and calls, giving characters an opportunity to mention forms of address, agreements, and key details from earlier exchanges in later conversations.

Article illustration

Users can also pin important content through Pinned Memory for text chats and calls to read.

Prelulu also offers a way to save important snippets, letting users preserve meaningful moments from a particular conversation.

Beyond experiencing ready-made stories, Prelulu also lets users create their own virtual characters.

Article illustration

On the creation page, users can customize a character's name, personality, backstory, opening plot, and voice, or enter a short description for AI-assisted generation of the character settings.

For appearance, Prelulu offers visual styles including photorealism, Japanese anime, Korean webtoons, and game CG. It supports uploaded reference images to help users adjust the character's look.

Once creation is complete, users can communicate directly with their own characters through text or voice.

Article illustration

Prelulu also supports generating video avatars for these characters. Once ready, users can talk face to face with their own virtual characters through real-time video calls, with the character responding according to the conversation.

Prelulu also provides a way to move from Character.AI.

Users only need to paste a Character.AI character link to import the original avatar and public character settings, then generate a voice and video avatar.

To bring previous chat history too, users can upload a ZIP file exported from Character.AI. However, this import method currently supports text chat only.

Article illustration

But Prelulu also wants these characters to have richer living environments. At present, users mainly participate in the plot through conversation, while video calls show the characters' real-time reactions.

Next, the team hopes rooms, streets, and other environments in the story can change with the plot, and even let users enter the story world and interact freely with its characters.

This is also the direction being explored by The Influence Company, the company behind Prelulu.

Article illustration

The team previously made ScaryStories, trying to let users intervene in interactive stories with a horror atmosphere.

Horror suits this interaction logic. Characters' warnings, strange events in a room, and choices viewers make in the moment can all lead to new plot developments.

Later, the team focused on virtual characters with appearances, voices, and the ability to remember people, and Prelulu became the product it is today.

Article illustration

The Influence Company's technical foundation is an in-house real-time digital-human model. It generates character faces, expressions, and mouth movements during conversation, combining language and speech capabilities to deliver real-time responses.

The team also packages these capabilities as Realtime Avatar, opening interfaces and development tools to other developers. Prelulu serves ordinary users, putting the technology into practice through ready-made characters and stories.

Article illustration

The company's three founders also have noteworthy backgrounds. Co-founder and CEO Xupeng Tong graduated from Carnegie Mellon University's School of Computer Science.

Earlier in his career, he researched machine learning at Uber and JD.com's AI lab, with exposure to applications including intelligent customer service, recommendations, and pricing.

He later created String, a natural-language data-analysis product that lets enterprise employees ask databases questions in everyday language.

Article illustration

He then founded MUKU AI, using AI digital humans to make marketing videos and providing tools for brands and advertising clients.

The other two founders handle two other areas. Product lead Helen Xiong worked on multi-agent AI gameplay in Giant Network's Super Sus and also has product experience with AI animation tools.

Technical lead Ning Dong researched speech and translation at Meta AI and participated in research related to Seamless and Llama 3.

Article illustration

Many other teams are also investing in real-time video interaction.

Vidu S1, introduced by Shengshu Technology, is one example. Users can upload an image of a person, an anime character, or even a pet and combine it with voice settings to create a digital avatar that can converse in real time.

As the user speaks, the model continuously generates video and can also change the character's actions according to spoken requests.

Article illustration

Vidu S1 emphasizes continuous video generation. The subsequent Vidu S2 adds more complex action control, real-time outfit changes, and background adjustment.

In China, Zaomeng Ciyuan has also adopted this kind of technology. The product initially attracted users with AI interactive stories, then gradually added creative forms such as video and comics, allowing creators to keep producing works around characters they like.

Article illustration

Its chosen experience is close to Prelulu's, but the emphasis differs. Zaomeng Ciyuan already has a content community around original characters and works, and is also experimenting with virtual characters facing many viewers in a livestream.

Prelulu currently focuses more on sustained exchange between a user and a single character, with stories usually advancing through successive conversations between the two.

Article illustration

Real-time video generation still faces an unavoidable challenge. If a model takes several seconds to compute one second of video, users cannot have the immediate interaction of a call.

For continuous playback, generation speed must keep up with playback speed. To achieve that, developers need to address model computation, inference acceleration, and resource scheduling when large numbers of users connect at once.

Article illustration

Beyond accelerating computation, characters also need to maintain stable appearances and movements after long calls.

Cost is another issue behind this. During a real-time video call, the model must continually receive the user's voice or text, generate replies, and align the character's voice with its mouth movements.

The longer the call, the more computation must be sustained. Prelulu currently provides a daily free allowance for text exchanges, while voice and video calls consume credits, with video using more.

Article illustration

A single call is capped at 30 minutes and also ends if credits run out earlier. Users can call the same character again, and call memory retains some details of earlier exchanges.

If future conversations also require an entire street to be generated in real time, costs will rise further.

For interactive-entertainment products like these, the experience of talking to the characters matters just as much.

Article illustration

Even if the video looks realistic enough, users can soon lose interest if the character's replies are always stiff or repetitive, or it cannot understand what they discussed earlier.

Especially in stories with continuing plots, characters need to keep relatively stable personalities, remember their relationship and experiences with the user, and respond reasonably to what is being discussed.

This also means that to keep users involved in interactive stories over the long term, Prelulu needs more than real-time video generation: characters must communicate naturally enough, and the story content must be able to keep developing.

Article illustration

Of course, the complete generative interactive world Prelulu is exploring still requires substantial development beyond its existing video-call product.

If video characters can eventually leave the call frame and move freely through story spaces, while users can speak up at any time, interactive short dramas will feel more like a story unfolding in the moment.

Perhaps when people watch short dramas again in a while, someone really will hear that "do not go in" they shout through the screen.

This article comes from "Xiagu AI," by "Xiagu Content Team."

Related topics

After reading

Join the AINRK community

Read how to join the community and connect with people working in AI.

Community information (opens in a new tab)