In Their Own Words: Reasoning Traces Tailored for Small Models Make Them Better Reasoners

• paper / code / datasets / models

0. RSD Testrun

• The core mechanism of RSD is demonstrated in rsd_testrun.ipynb. Run it to see if RSD works correctly in your environment.
• But before that, install dependencies using requirements.txt. Also set HF_READ_TOKEN, HF_WRITE_TOKEN, OPENAI_API_KEY, WANDB_API_KEY in your .env file for the test run and subsequent code runs.

1. RSD Full Trace Generation

• Create a {dataset_name}.db file using dataset_manager.ipynb. It'll create a dataset file with 'question', 'answer', and 'trace' columns. The question and answer pairs are from the s1K dataset. The trace column is empty at this point.
• Run rsd_datagen.py. Set args accordingly. This will populate the trace column if the sampling gets the right answer.

2. RSD UPFT Trace Generation

• Run rsd_upft.py. Set args accordingly. This will populate the rest of the trace column with the trace prefix.

3. SFT

• Upload the dataset in .parquet format to Hugging Face using dataset_manager.ipynb. SFT code takes a remote HF dataset as an arg.
• Run sft.py. Set args accordingly.

4. Evaluation

• Run eval.py. Set args accordingly.