Who Writes Better Essays: College Students or GPT-3? | Pepper Content

Who Writes Better Essays: College Students or GPT-3?

Team Pepper
Team Pepper
Posted on 4/04/215 min read
Who Writes Better Essays: College Students or GPT-3?

Nell Watson, a Futurist and AI, Technology Speaker, who speaks about upcoming trends for businesses or organizations, says that in the future, which is not so distant, machines will be making business decisions such as devising strategies, choosing employees, forming companies, and so on.

Artificial intelligence (AI) is no more an odyssey that we can experience only through cinema; it is very much a part of our real lives now. It may not be found in the shape of sentient machines with general artificial intelligence, as we see in movies, but AI is now all around us. Well, did you know that Netflix’s recommendations engine is powered by AI and is worth $1 billion a year?

Travel and navigation, smartphone apps, smart homes, driverless cars, security and surveillance, social media feed, personalized recommendations in online advertising, proactive healthcare management, disease mapping, smart assistants like SIRI, Cortana, Alexa; everywhere it’s AI at work. Isn’t AI making our lives convenient then? Indeed it is, but we all understand that with our increasing dependence on AI, we are heading for a future in which machines will outperform humans in many jobs, a possibility that is echoing across a variety of industries. 

Nell Watson,  a Futurist and AI, Technology Speaker, who speaks about upcoming trends for businesses or organizations, says that in the future, which is not so distant, machines will be making business decisions such as devising strategies, choosing employees, forming companies, and so on. Other experts agree that job automation is the most immediate risk of AI. 

Those involved in creative fields like music, art, and literature often think that their work is considerably more secure from robots. After all, AI is all about calculative power, superlative memory, and high-speed decision-making; it lacks imagination and ingenuity, which is innately human. But can’t AI be trained to ‘learn’ the rules of being creative? Can AI be a creative music composer or a writer? 

Hard to say, but the writing efficiency of Generative Pre-trained Transformer 3 (GPT-3), an AI-powered text generator, is indicative that content creation done by AI can be creative enough to fool people into believing that it is written by humans. Last year, The Guardian, one of the most revered British newspapers, published an Op-Ed titled, ‘A robot wrote this entire article. Are you scared yet, human?’ 

Source 

As per the news piece, GPT-3 produced not just one but eight different versions of the essay, and each essay was “unique, interesting and advanced a different argument”. The Guardian‘s article is very much descriptive of the writing potential of GPT-3. Although many criticized the Op-Ed as “yet another GPT-3 Hype“, at least it confirmed that AI could write (nearly) as well as humans. Maybe it is not efficient or creative enough to write a best-selling novel or a highly engaging blog yet. But is GPT-3’s writing skill on a par or even better than that of college students and content writers who write for SEO rankings? Before discussing this in detail, let’s first try to understand what GPT-3 is and what it can do. 

What is GPT-3? 

Introduced in May 2020, GPT-3 accelerated the hype and excitement in the world about AI. It even got referred to as “one of the most interesting and important AI systems ever produced.” Created by OpenAI, a San Francisco-based artificial intelligence firm, co-founded by Elon Musk, GPT-3 is a language model that uses deep machine learning to generate human-like text that has a language structure. 

GPT-3 is not the first-of-its-kind; similar types of language models are already in existence. Microsoft’s Turing NLG was the largest language model before the release of GPT-3. The capacity of these kinds of language models is defined in terms of ‘parameters’. Simply put, ‘more parameters’ means more data has been used to train the model. The Turing NLG parameter capacity is 17 billion parameters, which is less than a tenth of GPT-3’s parameter capacity, which is 175 billion.  

Source

What can GPT- 3 do? 

GPT-3 can create anything that has a language structure., which means it can write an essay, a blog, a news article, answer a question, summarize long texts, do translation and much more. It is interesting to learn how GPT-3 generates text. This AI uses a pre-trained algorithm for generating text. It has already been fed with a massive volume, around 570 GB, of textual information. Its algorithmic structure takes one piece of language (the prompt) as an input and then runs a training analysis on its vast body of pre-fed multiple datasets and then predicts the most useful piece of language for a reader.  

Following are the multiple datasets used to train the GPT-3 model:

Source 

So basically, GPT-3 is a language prediction model that has access to a huge amount of resources, which makes it more efficient in understanding how languages work and are structured. 

GTP-3 vs College Students: An Essay-writing Competition

There are many uploads on the internet that showcase how adept GPT-3 is at content creation, including the article published by The Guardian. You can read the Op-Ed online here.

However, to test the efficacy of GPT-3 as a writer, in comparison to college students, a company called EduRef conducted an essay-writing competition between GPT-3 and a group of recent college graduates and students. The students were asked to write essays on American history, research methods, creative writing, and law, based on writing instructions created by a group of professors. The same instructions were fed to the GPT-3 as prompt. The test papers were anonymized and given to the panel to test whether AI could get better grades than human students. 

  • GPT-3 could score the highest grade of B- (B minus) in the test. It wrote a history essay on the American state of emergency. The human rivals also scored more or less similar grades, ranging from C+ to B. 
  • During the test of writing a legal assignment, GPT-3 performed well as only one in three students could get a grade higher than the AI. 
  • In the research methodology paper on COVID-19 vaccine effectiveness, GPT-3 scored a C, while students received Bs or Ds.
  • Creative writing was the only paper in which GPT-3 failed with student writers scoring grades ranging from A to D+.

Based on the test result analysis, overall, it can be concluded that GPT-3’s technical skills are more refined than its creative skills. Its content showcases an impressive understanding of grammar, syntax and word frequency. It lacks in craftsmanship, as it fails to demonstrate strong narratives in creative writing tasks. As per EduRef project manager Sam Larson, who is an academic himself, the low craftsmanship of GPT-3 in this area could be because of how GPT-3 pulls information:

Source

When students were revealed that the articles were AI-written, they seemed rather more interested in the AI’s capacity to provide underhand aid to them!

This competitive experiment also highlighted another important aspect, while real students took an average of three days to complete the assignment, GPT-3 spent between 3 to 20 minutes generating content for each task. 

Although Mr. Larson was impressed with the performance of GPT-3, he did emphasize that AI-generated content needs editors. Even while publishing the GPT-3-written article, The Guardian confirmed that it did edit the essays, and asserted that “Editing GPT-3’s Op-Ed was no different to editing a human Op-Ed. We cut lines and paragraphs and rearranged the order of them in some places. Overall, it took less time to edit than many human Op-Eds.” 

The Assessment

Who writes better essays: College students or GPT-3? This is a subjective question as not all human writers are of equal capacity. As the EduRef study revealed, some students did beat the AI model, but GPT-3 outperformed some students. So, it is a fair conclusion that GPT-3 is an efficient text-generating AI, and undeniably, its content creation capacities are way ahead of previously existing language models. But considering the fact that GPT-3 is an early glimpse of the rapid AI evolution, in all probability, its succeeding versions will show higher degrees of sophistication. 

So, without getting tangled in deep discussions like who writes better or will GPT 3 take over writing jobs, it is important to accept that just like AI is revolutionizing other industries, it will also have an impact on content marketing. To stay on top, humans need to team up with AI like GPT-3. 

Take a cue of The Guardian‘s article. Here is a screenshot from the editor’s note.

Someone (a human) in The Guardian came up with the idea of getting an article written by GPT-3, maybe another human came up with that sensational topic and the awe-inspiring intro. And then GPT-3 wrote the essay, editors edited that. And the final output triggered discussions across social media and helped news organization generate a lot of impressions and ad revenue. 

Final words

The quality of GPT-3 generated content gets better after it has been fine-tuned by expert human writers or editors. In the same way, performances of content writers or content marketing campaigns can be made even more impressive by using the AI of GPT-3. For instance, consider pepper.ai. It is a GPR 3-based virtual content assistant that can be used by content professionals, college students or by anyone to generate short-scale content such as website headlines/copy, brand/product descriptions, tweet ideas, social media post captions, blog ideas, etc. 

The idea behind the tool is that content writing is itself a difficult job, and content ideation often takes much of a writer’s time; so this GPT-3 content creating tool tries to help writers with their ideation needs. Professional writers in content marketing, SEO or digital agencies can use this AI tool to get multiple content ideas for Google ads, Facebook ads, tweets, blogs, articles, etc., all in a matter of just one click. The tool is also efficient in writing short content pieces such as e-commerce product descriptions and SEO meta descriptions. 

Latest from peppercontent, right in your inbox.

From our latest podcasts and articles, to the detailed guides and extensive resources. Follow the evolution of content.

By subscribing, you agree to Peppercontent’s Term of Use  and Privacy Policy.

Similar Posts