Sentence Ordering

EconLogicQA is a benchmark designed to test the sequential reasoning skills of large language models (LLMs) in economics, business, and supply chain management. It diverges from typical benchmarks by requiring models to understand and sequence multiple interconnected events, capturing complex economic logics. The benchmark includes multi-event scenarios and a thorough suite of evaluations to assess proficiency in economic contexts.

It includes 10 data sets that consists of both raw data set and encoded data set where it is encoded through BERT-Sort Encoder with MLM initialization of . In each data set folder, there are original files and encoded data sets with 4 different MLMs. For instance, bank/bank.csv is the original file for raw data set and bank/bank.csv_bs__roberta.csv is encoded raw data set with BERT-Sort Encoder which is initiated with RoBERTa MLM. Both raw and encoded data sets have been used to evaluate the proposed approach in 5 AutoML platforms.