With the continuous evolution and expanding applications of Large Language Models (LLMs), there has been a noticeable surge in the size of the emerging models. It is not solely the growth in model size, primarily measured by the number of parameters, but also the subsequent escalation in computational demands, hardware and software prerequisites for training, all culminating in a substantial financial investment as well. In this paper, we present novel techniques like supervision, parallelization, and scoring functions to get better results out of chains of smaller language models, rather than relying solely on scaling up model size. Firstly, we propose an approach to quantify the performance of a Smaller Language Models (SLM) by introducing a corresponding supervisor model that incrementally corrects the encountered errors. Secondly, we propose an approach to utilize two smaller language models (in a network) performing the same task and retrieving the best relevant output from the two, ensuring peak performance for a specific task. Experimental evaluations establish the quantitative accuracy improvements on financial reasoning and arithmetic calculation tasks from utilizing techniques like supervisor models (in a network of model scenario), threshold scoring and parallel processing over a baseline study.
References
[1]
Dodge, J., Ilharco, G., Schwartz, R., Farhadi, A., Hajishirzi, H. and Smith, N. (2020) Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping. https://arxiv.org/abs/2002.06305
[2]
Wei. J., Wang, X.Z., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. and Zhou, D. (2023) Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. https://proceedings.neurips.cc/paper_files/paper/2022/hash/9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html
[3]
LearnPrompting (2023) Chain of Thought Prompting. https://learnprompting.org/docs/intermediate/chain_of_thought
[4]
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C. and Schulman, J. (2021) Training Verifiers to Solve Math Word Problems. https://arxiv.org/abs/2110.14168
[5]
Fu, Y., Peng, H., Ou, L.T., Sabharwal, A. and Khot, T. (2023) Specializing Smaller Language Models towards Multi-Step Reasoning. https://arxiv.org/abs/2301.12726
[6]
An, S.N., Ma, Z.X., Lin, Z.Q., Zheng, N.N., Lou, J.-G. and Chen, W.Z. (2023) Learning from Mistakes Makes LLM Better Reasoner. https://arxiv.org/abs/2310.20689
[7]
Srivastava, A., Rastogi, A., Rao, A., Md Shoeb, A.A., Abid, A., Fisch, A. and Brown, A.R., et al. (2023) Beyond the Imitation Game: Quantifying and Extrapolating the Capabilities of Language Models. https://arxiv.org/abs/2206.04615
[8]
Liu, T.D. and Low, B.K.H. (2023) Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Tasks. https://arxiv.org/abs/2305.14201
[9]
Chen, L.J., Zaharia, M. and Zou, J. (2023) FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance. https://arxiv.org/abs/2305.05176
[10]
Chen, Z.Y., Chen, W.H., Smiley, C., Shah, S., Borova, I., Langdon, D., Moussa, R., Beane, M., Huang, T.-H., Routledge, B. and Wang, W.Y. (2022) FinQA: A Dataset of Numerical Reasoning over Financial Data. https://arxiv.org/abs/2109.00122
[11]
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., et al. (2023) Llama 2: Open Foundation and Fine-Tuned Chat Models. https://arxiv.org/abs/2307.09288
[12]
Dettmers, T., Pagnoni, A., Holtzman, A. and Zettlemoyer, L. (2023) QLoRA: Efficient Finetuning of Quantized LLMs.
[13]
Ligntning.ai (2023) Finetuning LLMs with LoRA and QLoRA: Insights from Hundreds of Experiments. https://lightning.ai/pages/community/lora-insights/