Paul Christiano

Learning Complex Goals with Iterated Amplification

OpenAI proposes iterated amplification, an AI safety technique for specifying complex goals beyond human scale by decomposing tasks into simpler sub-tasks rather than relying on labeled data or reward functions. The approach avoids the need for explicit reward engineering by having humans demonstrate task decomposition hierarchically. At publication, experiments were limited to simple toy algorithmic domains, but the authors argue it could be a scalable alignment approach. The paper is presented in preliminary form to solicit early community engagement.