Add a pinch of salt to creative claims for AI

Sometimes someone or even somethings -- as in the case of generative AI -- will confidently assert something to be true. Don't suspend your disbelief. Investigate and demand the basis for that claim. 

Creative  lies from AI

Having investigated the output of both Chat GPT and Bard, I have concluded that you always have to fact-check whatever generative AI tells you because it is prone to what is called hallucinations. 

In fact, as related in The perils of prompt engineering, Bard  does not hesitate to fabricate facts and will double-down on its assertions with even more hallucinational as it spins out an even more elaborate web of AI-generated reality. 

While I remain baffled about what motivates AI to generate false information when it should have access to all the data necessary to report accurately,I can't say that humans have a monopoly on data-backed truth. They may not fabricate editions out of thin air, but they are quite prone to fabricate conclusions on the flimsiest of pretexts.

To profess is human 

Unlike the AI thought process, I can understand the human motivation in this case.  Human want to make a name for himself (really it is most often a he) about an area and knows that saying something surprising will get more attention than reporting what people expect. 

Knowing that some kind of apparently scientific approach is needed to justify the claim,the person just draws on a handful of students, gives them the task, and draws conclusions that turn into a major claim based on those very limited results. 

I've read a lot of Dan Ariely books and really enjoy his insights into human responses and triggers. But I always take them with a grain or two of salt. 

That's because the majority of his experiments involve a tiny, not terribly diverse segment of the population, namely the undergraduates at the university who can be induced to participate. Often that means that his conclusions are based on just two dozen people or so. 

The small sample size doesn't guarantee that the conclusions are incorrect. However, assuming that it is conclusive proof is the classic logical fallacy of hasty generalization. 

What he does with whichever group of students happen to pass by that today is not necessarily replicable by the population at large. It's kind of like just polling your siblings or neighbors to conclude that everyone's favorite flavor of ice cream is chocolate. That may be true, but we'd really need a bigger sample size to be sure.

Claiming AI is more creative than humans

Yet who can fault Ariely for doing this when everyone else does it? Indeed, that is precisely what  Erik Guzik, Assistant Clinical Professor of Management at University of Montana did for is article in The Conversation entitled  "AI scores in the top percentile of creative thinking." 

What does a person who just reads that title think was involved in making such a claim for AI creativity?  It sounds like thousands of creative people were compared to the AI to determine that its output was comparable to the top 1%.

But that's not what happened at all. Instead, this major claim is based on nothing more than 24 students whose output was deemed to be not all that creative relative to the output of ChatGPT(after 4 iterations, it sounds like) based on the assessment of the Torrance Tests of Creative Thinking, or TTCT.

We get very little shared about what the process for the students was and how they scored. The only clue we get about the creative prowess of ChatGPT is the professor's assertion:

Consider the following prompt offered to GPT-4: “Suppose all children became giants for one day out of the week. What would happen?” The ideas generated by GPT-4 touched on culture, economics, psychology, politics, interpersonal communication, transportation, recreation and much more – many surprising and unique in terms of the novel connections generated.

This combination of novelty and utility is difficult to pull off, as most scientists, artists, writers, musicians, poets, chefs, founders, engineers and academics can attest.

Yet AI seemed to be doing it – and doing it well. 

Notice that the professor is merely telling us that ChatGPT did well. He fails to show us the creativity by sharing the actual output and letting us see a contrast between it and even one of the 24 students who may well have all been math or engineering majors who don't considers themselves creative writers at all.

 

My own test

This is not a strict replication because I had to use ChatGPT3.5 instead of 4, but it should still show what most people would actually get when putting exactly that prompt to the generative AI. Here's its response enclosed in rows of stars:

*************************************************************************

If all children were to magically transform into giants for a single day each week, the world would undoubtedly witness a fascinating array of changes and events:

1. Physical Transformations: The sudden change in size would lead to various practical adjustments. Everyday items like furniture, clothing, and even vehicles might need to be redesigned or adapted to accommodate the giant children.

2. Playful Activities: Children would engage in unique and imaginative activities that capitalize on their newfound size. Parks and playgrounds would transform into vast landscapes for epic games, obstacle courses, and adventures.

3. Exploration: With their enhanced height, children could explore their surroundings from a different perspective, possibly leading to new discoveries or insights about their environment.

4. Education and Learning: Schools might introduce special days to leverage the giants' size advantage for educational purposes. Science classes, for instance, could involve experiments that demonstrate the effects of size on various phenomena.

5. Social Interactions: The dynamics of social interactions might shift as children grapple with their changing sizes. Friendships could take on new dimensions, and children might learn valuable lessons about empathy and acceptance.

6. Impact on Economy: The demand for goods and services tailored to giant children could create unique economic opportunities. Businesses might offer oversized toys, clothing, and other products on the designated giant day.

7. Environmental Impact: The increased scale of activities could lead to concerns about environmental effects. Authorities might need to manage waste, traffic, and other logistics associated with giant-sized activities.

8. Scientific Exploration: Scientists and researchers would be intrigued by the temporary transformation and could use the opportunity to study the physiological, psychological, and ecological impacts of such a phenomenon.

9. Creativity and Art: Artists and creators would likely draw inspiration from the whimsical idea, resulting in a surge of giant-themed literature, artwork, movies, and more.

10. Sense of Wonder: The experience of becoming giants would foster a sense of wonder and imagination among children, encouraging them to think beyond conventional boundaries and consider new possibilities.

Overall, a day when all children become giants would be a remarkable and unforgettable event that would influence various aspects of society, culture, and individual experiences. It would provide a unique chance for children and adults alike to see the world from an entirely different perspective and explore the imaginative and transformative power of change.

************************************************************************

Color me not impressed. This is just an analysis of various points of view one can take on such a scenario presented in a fairly analytical way. It's not at all what I consider creative output. What would be creative is to construct a story in which some -- but not all -- of the possibilities delineated would be manifested. But simply listing them as possible considerations is analytical rather than creative thinking.  

Visit WriteWayPro.weebly.com  Like and follow on Facebook and on LinkedIn

Related: Why you always need the original source  A matter of degree Bard vs Bard,