A China-based startup, Sand AI, has released an openly licensed video-generating AI model that’s garnered praise from entrepreneurs like Microsoft Research Asia founding director Kai-Fu Lee. But Sand AI appears to be censoring images that might raise the ire of Chinese regulators from the hosted version of the model, according to TechCrunch’s testing.
Earlier this week, Sand AI announced Magi-1, a model that generates videos by “autoregressively” predicting sequences of frames. The company claims the model can generate high-quality, controllable footage that captures physics more accurately than rival open models.
Magi-1 is too impractical to run on most consumer hardware. It’s 24 billion parameters in size, and requires between four and eight Nvidia H100 GPUs to run. (Parameters are the internal variables models use to make predictions.) For many users — this reporter included — Sand AI’s platform is the only place they can test drive Magi-1.
The platform needs a “prompt” image to kick off video generation. Not all prompts are permissible, TechCrunch quickly discovered. Sand AI blocks image uploads of Xi Jinping, Tiananmen Square and Tank Man, the Taiwanese flag, and insignias supporting Hong Kong liberation. Filtering appears to be happening at the image level; renaming image files didn’t skirt the blocking.
Sand AI isn’t the only Chinese startup preventing uploads of politically sensitive images to its video generation tool. Hailuo AI, Shanghai-based MiniMax’s generative media platform, blocks photos of Xi Jinping as well. But Sand AI’s filtering appears to be particularly aggressive; Hailuo allows images of Tiananmen Square.
As Wired explained in a piece from January, models in China are required to follow stringent information controls. A 2023 law forbids models from generating content that “damages the unity of the country and social harmony” — that is, counters the government’s historical and political narratives. To comply, Chinese startups often censor their models, either through prompt-level filters or fine-tuning.
Interestingly, while Chinese models tend to block political speech, they often have fewer filters than their American counterparts for pornographic content. 404 recently reported that a number of video generators released by Chinese companies lack basic guardrails that prevent people from generating nonconsensual nudity.
Source: https://techcrunch.com/
