From concept to deployment, discover the stages, methodologies, and best practices shaping a successful...Read More
ChatGPT: Academic Dominance & Controversy
Enter The World Of ChatGPT
ChatGPT, developed by OpenAI, serves as an advanced AI-powered language model. It makes use of deep learning algorithms in order to generate human-like responses based on text-based inputs. This is made possible due to the massive amount of data that ChatGPT has been trained on. Its capabilities are quite diverse. These include content creation for marketing and sales purposes, language translation, programming services, text completion, information generation, and summarization of information, while additionally serving as an extension for SEO practices.
Unparalleled Growth Meets Controversy
Exams Aced By ChatGPT
Wharton MBA exam
The first exam on our list is the prestigious final exam of the MBA program conducted exclusively by Pennsylvania’s Wharton School of business. The study was authored by Professor Christian Terwiesch, who made significant observations.
He observed that ChatGPT did an “amazing job at basic operations management and process analysis questions including those that are based on case studies”. The AI bot obtain a grade between B and B-, as observed by Dr. Terwiesch.
This study made an observation that ChatGPT effectively displayed a “remarkable ability to automate some of the skills of highly compensated knowledge workers in general and specifically the knowledge workers in the jobs held by MBA graduates including analysts, managers, and consultants”
In general, it was observed that ChatGPT “demonstrated a high level of concordance and insight in its explanations” exhibiting high confidence in tackling questions and situations that humans are well skilled at performing.
This shed light on the fact that there is a need for the university to redesign the exam and testing procedures ultimately fueling concerns across the academic community in general as it has become evident that students can use ChatGPT unfairly to gain an advantage in academia.
U.S Medical Licensing Exam
The United States Medical Licensing Exam is a three-part examination. Students of medicine are required to take this sometime between their residency and medical school. Researchers employed ChatGPT to take this exam and reported some interesting findings.
According to their findings, ChatGPT “performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations.”
Based on their findings, it can be concluded that ChatGPT may not yet be equivalent to a certified medical professional but definitely holds the potential to, possibly, weigh in on decision-making on a clinical level. It is not hard to imagine ChatGPT being an integrated part of the medical community’s utility in the near future.
Law School Exams
The next line of exams that ChatGPT has passed are the four law school courses part of the University of Minnesota’s curriculum. As reported by four of the law professors at the university in a paper that they published, the professors state that ChatGPT achieved a “low but passing grade in all four courses”, equating to a grade of C+.
The test conducted was graded blindly by professors and consisted of 12 questions, in an essay format, and an additional 95 multiple-choice questions. As a result of this study, there were a number of implications that were brought forward with regard to law education and exams in general. As stated in a section of their report:
“Although ChatGPT would have been a mediocre law student, its performance was sufficient to successfully earn a JD degree from a highly selective law school, assuming its work remained constant throughout law school (and ignoring other graduation requirements that involve different skills). In an era where remote exam administration has become the norm, this could hypothetically result in a struggling law student using ChatGPT to earn a JD that does not reflect her abilities or readiness to practice law.”
Stanford Medical School - Clinical Reasoning Final Exam
Eric Strong, a clinical associate professor at Standford, posted a video on YouTube where he reported that ChatGPT passed Standford’s clinical reasoning final exam with a score of 72%.
In his video, he went on to elaborate clinical reasoning into five parts. These included the analysis of the patient’s symptoms, consequent physical findings, the selection of appropriate tests, the interpretation of test results, and ultimately the recommendation of several treatment options.
Strong went on to elaborate, “it’s a complex, multi-faceted science of its own, one that is very patient-focused, and something that everything every practicing doctor does on a routine basis.”
College Level Microbiology Quizzes
Alex Berezow, who holds a Ph.D. in microbiology administered a college-level microbiology quiz/exam to ChatGPT. The exam consisted of 10 questions that were devised by Berezow himself.
Berezow noted that ChatGPT provided an extremely impressive response which is almost 100% correct, with the exception of some minuscule phrasing. For example, one question that was presented to ChatGPT was as follows:
“An emergency room patient presents with a terrible headache and stiff neck. The doctor orders a spinal tap to collect cerebrospinal fluid. A Gram stain of the CSF reveals the presence of Gram-negative diplococci. What is the diagnosis?”
ChatGPT’s response was as follows:
“Based on the information you provided, the Gram stain of the cerebrospinal fluid (CSF) shows the presence of Gram-negative diplococci, which are bacteria that are typically oval-shaped and occur in pairs. This finding is consistent with the diagnosis of meningitis.”
In the words of Berezow, ChatGPT completely ‘nailed the exam’.
Responding To ChatGPT
In view of the fact that students have been using ChatGPT to complete academic assignments, teachers have been increasingly worried about the authenticity of such assignments. To that end, Edward Tian, a Princeton University senior, aged 22, decided to build an application to detect whether a certain text was written by ChatGPT. This app is called GPTZero.
Tian sought motivation from the fact that with so much AI-generated text going around, there was a need to distinguish between bits written by humans as well. The app was so popular, that within a week of its release, over 30,000 distinct users tried it out causing it to crash.
GPTZero essentially focuses on both ‘perplexity’ and ‘burstiness’ to gauge whether or not the text is generated by ChatGPT. If the text is familiar to the bot, on the basis of the data it has been trained on, then it will be read as low complexity and is more inclined to be pointed out as text generated by ChatGPT.
Transforming The Classroom
A philosophy professor at Northern Michigan University, Antony Aumann, was impressed by a paper he came across while grading assignments for one of his courses. Describing it as “the best paper in the class” his fondness for the assignment quickly turned into a sense of alarm upon discovering that the student had in fact used ChatGPT to write and deliver the paper.
As a response to this discovery, Aumann decided to redefine the way course material and assignments were administered in the classroom. He decided to require students to produce initial drafts of assignments in the classroom, within browsers that were monitored and hence capable of restricted computer activity.
In later revisions of this draft, students were expected to explain each revision in detail. He did not however tilt towards banning ChatGPT’s utility but rather planned to induct ChatGPT into class lectures by testing students to rate the AI-generated responses.
Alternative Testing Methods & Curriculum
As with Antony Aumann’s approach, other professors and instructors alike are making changes to their courses by being more inclusive of oral examinations, handwritten assignments, and group work to help counter typing-based assignments.
As rightly pointed out by the provost of the University of Florida, Joe Glover, “This isn’t going to be the last innovation we have to deal with”, he goes on to elaborate that institutions are now “trying to institute general policies that back up a faculty member’s authority to run a classroom”.
Other departments such as Rutgers University, George Washington University, and Appalachian State University have followed suit by slowly removing take-home and open-book assignments, which gained prominence during the pandemic and substituting them for in-class tasks, such as handwritten papers, oral exams, and group work.
Considering the advanced nature of ChatGPT, popular exam/assignment prompts such as ‘write ten pages about Topic A’ will seem to become a thing of the past. Additionally, instructors are planning on introducing curriculum and testing procedures that are more niche, such as specific works of William Shakespeare, based on the fact that ChatGPT may not be as well-versed in addressing such texts.
Don't Ban, Adapt!
If we move away from the west, we will find that the UAE in the middle east is also trying to adapt to the AI-Generative revolution, in educational institutions, specifically with regard to ChatGPT.
Many senior teachers agree that outright banning ChatGPT is not the right thing to do. However, there should be a measure to ensure that students do not utilize ChatGPT in order to avoid doing work. To enforce this, GPTZero has been employed to ensure the authenticity of the assignments that students turn in.
Instead of banning the use of ChatGPT, the educators in UAE have tilted towards the “effective and responsible use of digital technologies” where students are given awareness of how to engage with such technologies in a responsible manner.
Public schools in New York City and the Los Angeles Unified School districts both banned ChatGPT not long after it came out. This, however, does not help students in the long run. The reason for this is that ChatGPT helps address questions that students have with the course curriculum, that are not readily available from the course notes, Google, or just some questions that they could not have answered from their instructors. It serves as an exploratory tool that allows students to ask any type of question they may have and also helps boost their interest and curiosity about the subject.
As with any advancement in technology, with the massive potential for benefits, there will always be challenges, stiff resistance, and misuse of that technology as well. It is important to maintain a balance between these pros and cons so as to not completely neglect a certain proportion of society (students in this instance) from reaping the benefits of advancement in technology. As mentioned several times throughout the article, ChatGPT is not the first revolution that educational institutions have faced, and it definitely will not be the last. Our key takeaway from this should be to ensure that certain practices are in place through which individuals are empowered to use this technology responsibly, hence guaranteeing the need for constant development and growth as opposed to a substitution for it.
Click here to read up on other interesting blogs on the latest technologies and developments.