ChatGPT Passes CPA Exam on Second Attempt

An improved and better-trained version of ChatGPT retook a simulated CPA test after an earlier failure and passed, similar to the roughly half of human applicants who fail on their first attempt. But does this trial suggest that ChatGPT will soon compete for accounting positions?

“This calls into question the 'competitive advantage' of the human accountant relative to the machine. To our knowledge, for the first time, AI has performed as well as most human accountants on a real-world accounting task. This raises important questions of how machine and accountant will work together in the future,” the study's authors said.

The worry around artificial intelligence (AI) is that it will eventually replace human labor. There are some tasks that a machine simply cannot perform, but there are others that will adapt to take advantage of AI's efficiency. The question is, do accounting jobs fall into that category? That may be the case, according to the most recent test, but only after certain AI system adjustments.

How ChatGPT Did It. Although ChatGPT has showed its abilities in a variety of applications, the researchers aimed to improve its effectiveness in student examinations like the CPA exam.

The chatbot was changed by the researchers using “chain of thought” prompting and a “10-shot” scenario, where it was taught with 10 examples. These techniques helped the AI become more knowledgeable about the topic and keep important concepts.

The upgrade from the initial version, ChatGPT 4.0, was the one that passed the CPA exam. Newer ChatGPT models, which have stronger training and reasoning capabilities, outscored older models on average by 16.5%, according to the study.

The most recent Version 4.0 averaged a score of 85.1 out of 100 and successfully completed all four sections of the exam, in contrast to the prior Version 3.5's average score of merely 53.1. The chatbot received an 87.5 in the part that rated highest, auditing and attestation (AUD).

Version 3.5 of ChatGPT took the CPA Exam in early May under the direction of trade publication Accounting Today. The chatbot got a pitiful 35% in financial accounting and reporting (FAR), its weakest category, and just 48% in business environment and concepts (BEC), its strongest category.

The researchers' test results show that ChatGPT's most recent iterations, which have the most sophisticated reasoning capabilities yet can handle virtually every evaluation. This finding could have significant ramifications for important areas, like accounting and auditing.

ChatGPT has demonstrated its talents in several well-known exams, such as the Wharton MBA exam, the bar exam, and many AP exams.