PoEM: Predicting RL Outcomes from Existing Policies

AI 导读

PoEM框架通过现有策略预测新奖励函数的RL结果，减少重复训练成本。

PoEM predicts RL outcomes on new rewards using existing policies, avoiding redundant training.

重点速览

• 新奖励可分解为现有奖励的线性组合时，策略可直接合成 Linear reward combinations enable policy synthesis
• 非线性奖励场景下log-policy仍呈现低秩子空间特性 Log-policies form low-rank subspaces even non-linearly
• 仅需样本奖励/策略输出即可估计组合权重系数 Weights estimated from sample reward/policy outputs
• 无需额外RL训练即可近似目标策略 Target policy approximated without RL training
• 验证覆盖文本/图像多模态合成与真实奖励 Validated across synthetic/real rewards in multiple modalities

一句话：PoEM通过复用已有模型显著降低RL训练成本。 / PoEM reduces RL training costs by reusing existing models.

推荐理由 该方法为多奖励场景提供高效策略预测方案，具实际应用价值。