Yesterday’s post juxtaposed two essays, one written by a person and one by ChatGPT, an AI program. I asked which was which.
When I tried to write this follow-up, YouTube decided I was a robot and wouldn’t let me add links to the post. The post then disappeared. The next take froze and wouldn’t accept my CAPTCHA responses. Now it was sure I was a robot. Then ChatGPT erased the additional essays that I was going to discuss. I finally used two different browsers, one for Chat GPT and YouTube links and another to write the post. So here is #2, take 3.
ChatGPT wrote a convincing essay about a non-existent oil painting. One might say it was brimming with “alternative facts”. John Wright (1934-2017) was a jazz pianist from Chicago. His debut album, “South Side Soul” was released in 1960 with Wendell Roberts on bass and Walter McCants on drums.
When I changed the prompt to call it a “composition” instead of a “piece”, ChatGPT took minutes to begin writing, then spit out the same essay with a different conclusion, using new adjectives in the final paragraph to describe the fictional painting.
I tried again, calling it a “jazz composition”. ChatGPT continued to insist it was a 1991 work and now highlighted the virtuosity of the saxophonist, discussing his “quick runs” and arpeggios. Since the work is for piano, bass, and drums, I wondered where the AI found sax, so I asked. ChatGPT told me there were many recordings of this piece with different saxophonists, so I needed to be more specific. When I gave it specifics, it spit this back:
Thank you for providing additional information. The 1960 recording of “South Side Soul” features John Wright on piano, Frank Derrick on drums, and Frank Walton on trumpet. The saxophonist on this recording is Harold Ousley. Ousley was a talented tenor saxophonist who played with a number of notable jazz musicians over the course of his career, including Lionel Hampton, Count Basie, and Dizzy Gillespie. His soulful, bluesy style of playing is a perfect fit for the “La Salle St After Hours” composition, and his solo on the recording is a highlight of the performance.
As you may note, the only fact on this which is accurate and relevant is that John Wright plays the piano; but I told it that. The others are real musicians but did not play on this record. These actually are “alternative facts”, in that they are factual but have nothing to do with the question asked. It would be like the student who has some knowledge of the general topic but none about the question asked, so fills an essay with stuff that is true but irrelevant. Ousley’s “soulful, bluesy style of playing” is certainly not “a highlight of the performance”.
While I’ve read multiple essays warning about ChatGPT being used by students to do their homework, I would give the bot an ‘F’ on every one of its attempts. If I knew nothing of the work, I may have been convinced. Knowing the work, I’d know the student cheated. Not knowing the work would have necessitated about a minute of Googling to know the essays on painting were lies. Another couple of minutes would have been required to know those which called it jazz were no better.
The other essay, about the pianist playing late into the night after the club had closed, was written by a student from an exam prompt. The students were given two prompts and the piece was played twice in succession. They had to place the piece in a musical, cultural, and historical context in one essay and write a personal reaction in the second essay. It had to be written while the piece was playing and handed in a couple of minutes after the music stopped, so the bot and the human were working in about the same timeframe, kind of like John Henry and the steam drill.
It may not have been fair that I used the student’s second essay and not the first, which may have borne some minor stylistic resemblance to the ChatGPT essay; but I think this illustrates perhaps the most important issue. What is “intelligence”? To me, it is more about the ability to learn than it is about the accumulated knowledge. We don’t call an encyclopedia “intelligent” though it contains a lot of information. At the same time, we don’t belittle a child’s intelligence because they have not yet accumulated a vast store of knowledge. We note a child’s intelligence via the capacity to learn. Learning arises from not knowing. If we don’t know and we are aware of that, we can learn. This AI bot is stupid. Why? Because it makes shit up. If you hide your lack of knowledge by making shit up, you don’t learn. As a programmer, I learned GIGO – Garbage In, Garbage Out. What came out of ChatGPT was Garbage.
Thanks for playing. As your prize for playing our game, here is the title cut from “South Side Soul”.
Half-fast Cycling Club 