onPanda: Efficient Annotation of On-Policy Alignment Data for LLMs and Agents via Token-Level Correction

AI Digest

onPanda 是一种通过 token 级修正高效标注对齐数据的工具，显著降低人工标注成本并保留模型分布特性。

onPanda enables efficient annotation of on-policy alignment data via token-level correction, reducing annotation time and preserving model sampling distributions.

Key points

• 通过定位并修正首个不恰当 token 实现交互式标注 Interactive annotation via locating and correcting first inappropriate token
• 实验显示标注效率比手动编辑提升 52% 52% time reduction compared to manual post-editing
• 生成数据保留模型采样分布，适合构建 on-policy 数据集 Generated data preserves model sampling distribution
• 记录的 token 级修正提供细粒度监督信号 Token-level corrections provide fine-grained supervision
• 支持连接外部工具进行环境轨迹标注 Supports external tool integration for environment annotation

Takeaway: onPanda 为对齐数据标注提供了低成本、高质量的解决方案。 / onPanda offers a low-cost, high-quality solution for alignment data annotation.

Why it matters 该工具可显著提升对齐数据标注效率，对研究者和开发者具有实用价值。