Before Alignment
After Alignment
We develop a meta-agent collaboration system: ALICE, that generates high-quality data through multi-turn interactions and feedback. It produces data with traces from agent strategies like ReAct and Reflexion, which are scarce but offer potential for aligning advanced LLMs.
An Overview of the ALICE Workflow. Fine-Tunable models are labeled with the fire symbols, and frozen models are labeled with the snow symbols. For components with different colors, purple refers to structured arguments or configurations in the game engine, orange refers to textual prompts or examples, blue refers to different LLMs, red refers to textual data generated from the model or provided by users, and green refers to that the program successfully executes.
