行业内容

OpenAI Scholars 2018: Meet our Scholars

Our first class of OpenAI Scholars is underway, and you can now follow along as this group of experienced software developers becomes machine learning practitioners.

OpenAI Five Benchmark

The OpenAI Five Benchmark match is now over!

Glow: Better reversible generative models

We introduce Glow, a reversible generative model which uses invertible 1x1 convolutions. It extends previous work on reversible generative models and simplifies the architecture. Our model can generate realistic high resolution images, supports efficient sampling, and discovers features that can be used to manipulate attributes of data. We’re releasing code for the model and an online visualization tool so people can explore and build on these results.

Learning Montezuma’s Revenge from a single demonstration

We’ve trained an agent to achieve a high score of 74,500 on Montezuma’s Revenge from a single human demonstration, better than any previously published result. Our algorithm is simple: the agent plays a sequence of games starting from carefully chosen states from the demonstration, and learns from them by optimizing the game score using PPO, the same reinforcement learning algorithm that underpins OpenAI Five.

OpenAI Five

Our team of five neural networks, OpenAI Five, has started to defeat amateur human teams at Dota 2.

Retro Contest: Results

The first run of our Retro Contest—exploring the development of algorithms that can generalize from previous experience—is now complete.

Learning policy representations in multiagent systems

OpenAI 发布无监督预训练 + Transformer 的语言理解系统（GPT-1）：在一组多样的语言任务上拿到当时最好的结果，并放出了这套可扩展、任务无关的系统

Improving language understanding with unsupervised learning

OpenAI 宣布在一组多样的语言任务上取得了当时最优（state-of-the-art）的结果，用的是一套可扩展、任务无关的系统，并一并将其开源。方法是把两个已有想法结合到一起：Transformer 架构与无监督预训练。作者认为这为「把监督学习方法与无监督预训练搭配起来确实很有效」提供了有说服力的例证——这一思路此前已有多人探索，他们希望这一结果能推动人们在更大、更多样的数据上继续应用这一想法。

模型#OpenAI

推荐理由：GPT 系列的起点，无监督预训练这一范式的第一次系统验证。

GamePad: A learning environment for theorem proving

OpenAI Fellows Fall 2018

We’re now accepting applications for the next cohort of OpenAI Fellows, a program which offers a compensated 6-month apprenticeship in AI research at OpenAI.

Gym Retro

We’re releasing the full version of Gym Retro, a platform for reinforcement learning research on games. This brings our publicly-released game count from around 70 Atari games and 30 Sega games to over 1,000 games across a variety of backing emulators. We’re also releasing the tool we use to add new games to the platform.

OpenAI 分析：2012 年以来最大 AI 训练运行的算力以每 3.4 个月翻倍的速度指数增长，累计超过 30 万倍——作为对照，摩尔定律的翻倍周期是 2 年

AI and compute

OpenAI 发布一篇分析，指出自 2012 年以来，最大规模 AI 训练运行所使用的算力呈指数增长，翻倍时间为 3.4 个月（作为对比，摩尔定律的翻倍周期约为 2 年）。自 2012 年起这一指标已增长了 30 万倍以上（若按 2 年翻倍，同期只有 7 倍增长）。作者认为算力的改进一直是 AI 进展的关键组成部分，只要这一趋势延续，就值得为远超当下能力的系统所带来的影响提前做准备。

论文#OpenAI

推荐理由：这是被反复引用的算力增长曲线，作为行业趋势的量化锚点仍有分量。

AI safety via debate

We’re proposing an AI safety technique which trains agents to debate topics with one another, using a human to judge who wins.

Evolved Policy Gradients

We’re releasing an experimental metalearning approach called Evolved Policy Gradients, a method that evolves the loss function of learning agents, which can enable fast training on novel tasks. Agents trained with EPG can succeed at basic tasks at test time that were outside their training regime, like learning to navigate to an object on a different side of the room from where it was placed during training.

Gotta Learn Fast: A new benchmark for generalization in RL

Retro Contest

We’re launching a transfer learning contest that measures a reinforcement learning algorithm’s ability to generalize from previous experience.

Variance reduction for policy gradient with action-dependent factorized baselines

Improving GANs using optimal transport

Report from the OpenAI hackathon

On March 3rd, we hosted our first hackathon with 100 members of the artificial intelligence community.

On first-order meta-learning algorithms

OpenAI 提出一个简单可扩展的元学习算法 Reptile：反复采样任务、在其上做随机梯度下降，再把初始参数朝该任务学到的最终参数方向更新。它是 Shortest Descent 算法在元学习场景下的应用，数学上类似一阶 MAML，只需对 SGD 或 Adam 这类优化器的黑盒访问，计算效率与表现相近

Reptile: A scalable meta-learning algorithm

OpenAI 提出了 Reptile——一个简单可扩展的元学习算法。做法是反复采样一个任务、在其上做随机梯度下降，然后把初始参数朝着在该任务上学到的最终参数方向更新。Reptile 相当于把 Shortest Descent 算法应用到元学习场景，数学上与一阶 MAML（MAML 的一个版本）相近，但只需要对某个优化器（如 SGD 或 Adam）的黑盒访问，计算效率和性能都与之相当。

论文#OpenAI

OpenAI Scholars

We’re providing 6–10 stipends and mentorship to individuals from underrepresented groups to study deep learning full-time for 3 months and open-source a project.

Some considerations on learning to explore via meta-reinforcement learning

Multi-Goal Reinforcement Learning: Challenging robotics environments and request for research