KeyError: 'synthetic_math'

[liyc-ai]

Hi, thanks for your work.

I am running the data_preprocessing scripts in your code. However, in stage3, I encounterred an issue: KeyError: 'synthetic_math'. I tried to identify the root cause and found in the following code:

subsets = ["amc_aime"]  # TODO

rm_stat = {}

for subset in subsets:
    rm_stat[subset] = {}

It seems that 'synthetic_math' was not added in rm_stat, right? But I am not sure whether I can directly add it into subsets. Could you please help me fix this issue? Thanks!

[liyc-ai]

I further found that in data_preprocessing/stage1_filter.py, only synthetic_math is included:

subsets = ["synthetic_math"]
for subset in subsets:
    rm_stat[subset] = {}