OpenAI 提出一个简单可扩展的元学习算法 Reptile：反复采样任务、在其上做随机梯度下降，再把初始参数朝该任务学到的最终参数方向更新。它是 Shortest Descent 算法在元学习场景下的应用，数学上类似一阶 MAML，只需对 SGD 或 Adam 这类优化器的黑盒访问，计算效率与表现相近
Reptile: A scalable meta-learning algorithm
评分 42
导读
OpenAI 提出了 Reptile——一个简单可扩展的元学习算法。做法是反复采样一个任务、在其上做随机梯度下降，然后把初始参数朝着在该任务上学到的最终参数方向更新。Reptile 相当于把 Shortest Descent 算法应用到元学习场景，数学上与一阶 MAML（MAML 的一个版本）相近，但只需要对某个优化器（如 SGD 或 Adam）的黑盒访问，计算效率和性能都与之相当。
主体与标签
OpenAI#论文/研究#数据/训练
正文
We’ve developed a simple meta-learning algorithm called Reptile which works by repeatedly sampling a task, performing stochastic gradient descent on it, and updating the initial parameters towards the final parameters learned on that task. Reptile is the application of the Shortest Descent algorithm to the meta-learning setting, and is mathematically similar to first-order MAML (which is a version of the well-known MAML algorithm) that only needs black-box access to an optimizer such as SGD or Adam, with similar computational efficiency and performance.