内容审核来了一个开源决策模型：Musubi 的 PolicyLM-1.7B 用 50 毫秒把英文政策套到每条消息上，政策改了不用重新训练
How AI decision models could change content moderation
精选评分 76
导读
随着决策模型这一品类在全行业铺开，Musubi 给出了一个新用法：做内容审核。周二它发布了面向实时审核的轻量决策模型 PolicyLM-1.7B，并以开源权重释出。核心思路是把一份用平实英文写就的内容政策，在 50 毫秒内套用到消息上。其定位也很明确：成本与速度接近当下支撑多数社交平台审核的 AI 分类器系统，但因为具备现代 LLM 的灵活性，无需专门训练就能应用复杂政策；而更重要的是政策变更时不需要重新训练，人类制定政策的人可以想改多少次就改多少次。Musubi 联合创始人兼首席 AI 官 Filip Jankovic 的看法是，这给平台管理者一条主动为内容打标签的路：「产品团队只想更好地理解自己平台上正在发生什么，尤其是在内容量指数级增长的现在——能把这些以可扩展、可定制的方式都打上标签，极其有用。」文章同时把这一品类讲清：自 9 月 TypeSafe AI 的 Jev 发布、随后 OpenAI 与Amazon推出竞争性的决策模型之后，决策模型成了 AI 圈的热门话题；它不输出文本而是输出结果概率，在审核这一场景里输出的是二元判断（内容属于该类别还是不属于）。把输出限制在一组预定选项内，使决策模型能比 LLM 更快更便宜地运行，同时保住 transformer 架构的灵活性。一个早期用例是管住 AI 智能体的越轨行为——那么把同样的技术用于人类的越轨行为也就顺理成章。值得注意的是，Jankovic 说自己对决策模型的兴趣早于 Jev，可追溯到 2024 年一个叫 GLiNER（通用命名实体识别模型）的项目，那里面用了许多相同技术。公司并不忌讳与 Jev 相比较，反而乐见这波关注替内容审核引来注意：「如果 Jev 引起了你的注意，PolicyLM-1.7B 就是同一类模型，专为内容审核训练，你可以自己跑起来。」
推荐理由
决策模型这一新品类少见的落地应用，开源权重 + 50 毫秒 + 「政策改了不用重训」三条都是工程上可直接评估的口径。
主体与标签
Musubi#模型发布#开源#评测/基准
正文
As decision models spread across the industry, a company called Musubi has a new idea for how to put them to work: moderating content. On Tuesday, Musubi announced a lightweight decision model made for real-time moderation called PolicyLM-1.7B , released with open weights.
The idea is to take a content policy written in plain English and apply it to messages in under 50 milliseconds. Musubi’s model is designed to be similar in cost and speed to the AI classifier systems that power moderation on most social platforms — but because it has the flexibility of a modern LLM, it can apply complex policies without special training. Even more important, the model won’t need new training when the policy changes, allowing for human policy-setters to iterate as much as they need.
As Musubi co-founder and chief AI officer Filip Jankovic sees it, it gives platform managers a way to label content proactively.
“Product teams just want a better understanding of what’s happening on their platform, especially as the amount of content is exponentially increasing,” Jankovic says. “Being able to label all of that in a very scalable, customizable way is extremely useful.”
Decision models have become a hot topic in the AI world since the release of TypeSafe AI’s Jev in September , which was shortly followed by competing decision models from OpenAI and Amazon . Instead of outputting text, a decision model outputs outcome probabilities, though in this case the model outputs a binary judgement: Either the content is in the category or it isn’t. By limiting the model’s output to a set of predetermined choices, decision models are able to run faster and cheaper than large language models, while still maintaining the flexibility of the transformer architecture.
One early use case is reining in misbehavior by AI agents — so it’s only natural to apply the same technology to human misbehavior.
Notably, Jankovic says his interest in decision models predates Jev, tracing it back to a 2024 project called GLiNER (Generalist Model for Named Entity Recognition) that deployed many of the same techniques.
Still, Musubi isn’t wary of the comparison. If anything, the company is eager to use the new interest in decision models to shine a light on content moderation. “If Jev caught your eye, PolicyLM-1.7B is the same kind of model, trained specifically for content moderation, that you can run yourself,” the product announcement reads.
When you purchase through links in our articles, we may earn a small commission . This doesn’t affect our editorial independence.
Russell Brandom has been covering the tech industry since 2012, with a focus on platform policy and emerging technologies. He previously worked at The Verge and Rest of World, and has written for Wired, The Awl and MIT’s Technology Review. He can be reached at [email protected] or on Signal at 412-401-5489.