OpenAI 发布无监督预训练 + Transformer 的语言理解系统（GPT-1）：在一组多样的语言任务上拿到当时最好的结果，并放出了这套可扩展、任务无关的系统
Improving language understanding with unsupervised learning
评分 52
导读
OpenAI 宣布在一组多样的语言任务上取得了当时最优（state-of-the-art）的结果，用的是一套可扩展、任务无关的系统，并一并将其开源。方法是把两个已有想法结合到一起：Transformer 架构与无监督预训练。作者认为这为「把监督学习方法与无监督预训练搭配起来确实很有效」提供了有说服力的例证——这一思路此前已有多人探索，他们希望这一结果能推动人们在更大、更多样的数据上继续应用这一想法。
推荐理由
GPT 系列的起点，无监督预训练这一范式的第一次系统验证。
主体与标签
OpenAI#模型发布#论文/研究
正文
We’ve obtained state-of-the-art results on a suite of diverse language tasks with a scalable, task-agnostic system, which we’re also releasing. Our approach is a combination of two existing ideas: transformers and unsupervised pre-training. These results provide a convincing example that pairing supervised learning methods with unsupervised pre-training works very well; this is an idea that many have explored in the past, and we hope our result motivates further research into applying this idea on larger and more diverse datasets.