Add a pinch of salt to creative claims for AI

Sometimes someone or even somethings -- as in the case of generative AI -- will confidently assert something to be true. Don't suspend your disbelief. Investigate and demand the basis for that claim. 

Creative  lies from AI

Having investigated the output of both Chat GPT and Bard, I have concluded that you always have to fact-check whatever generative AI tells you because it is prone to what is called hallucinations. 

In fact, as related in The perils of prompt engineering, Bard  does not hesitate to fabricate facts and will double-down on its assertions with even more hallucinational as it spins out an even more elaborate web of AI-generated reality. 

While I remain baffled about what motivates AI to generate false information when it should have access to all the data necessary to report accurately,I can't say that humans have a monopoly on data-backed truth. They may not fabricate editions out of thin air, but they are quite prone to fabricate conclusions on the flimsiest of pretexts.

To profess is human 

Unlike the AI thought process, I can understand the human motivation in this case.  Human want to make a name for himself (really it is most often a he) about an area and knows that saying something surprising will get more attention than reporting what people expect. 

Knowing that some kind of apparently scientific approach is needed to justify the claim,the person just draws on a handful of students, gives them the task, and draws conclusions that turn into a major claim based on those very limited results. 

I've read a lot of Dan Ariely books and really enjoy his insights into human responses and triggers. But I always take them with a grain or two of salt. 

That's because the majority of his experiments involve a tiny, not terribly diverse segment of the population, namely the undergraduates at the university who can be induced to participate. Often that means that his conclusions are based on just two dozen people or so. 

The small sample size doesn't guarantee that the conclusions are incorrect. However, assuming that it is conclusive proof is the classic logical fallacy of hasty generalization. 

What he does with whichever group of students happen to pass by that today is not necessarily replicable by the population at large. It's kind of like just polling your siblings or neighbors to conclude that everyone's favorite flavor of ice cream is chocolate. That may be true, but we'd really need a bigger sample size to be sure.

Claiming AI is more creative than humans

Yet who can fault Ariely for doing this when everyone else does it? Indeed, that is precisely what  Erik Guzik, Assistant Clinical Professor of Management at University of Montana did for is article in The Conversation entitled  "AI scores in the top percentile of creative thinking." 

What does a person who just reads that title think was involved in making such a claim for AI creativity?  It sounds like thousands of creative people were compared to the AI to determine that its output was comparable to the top 1%.

But that's not what happened at all. Instead, this major claim is based on nothing more than 24 students whose output was deemed to be not all that creative relative to the output of ChatGPT(after 4 iterations, it sounds like) based on the assessment of the Torrance Tests of Creative Thinking, or TTCT.

We get very little shared about what the process for the students was and how they scored. The only clue we get about the creative prowess of ChatGPT is the professor's assertion:

Consider the following prompt offered to GPT-4: “Suppose all children became giants for one day out of the week. What would happen?” The ideas generated by GPT-4 touched on culture, economics, psychology, politics, interpersonal communication, transportation, recreation and much more – many surprising and unique in terms of the novel connections generated.

This combination of novelty and utility is difficult to pull off, as most scientists, artists, writers, musicians, poets, chefs, founders, engineers and academics can attest.

Yet AI seemed to be doing it – and doing it well. 

Notice that the professor is merely telling us that ChatGPT did well. He fails to show us the creativity by sharing the actual output and letting us see a contrast between it and even one of the 24 students who may well have all been math or engineering majors who don't considers themselves creative writers at all.

 

My own test

This is not a strict replication because I had to use ChatGPT3.5 instead of 4, but it should still show what most people would actually get when putting exactly that prompt to the generative AI. Here's its response enclosed in rows of stars:

*************************************************************************

If all children were to magically transform into giants for a single day each week, the world would undoubtedly witness a fascinating array of changes and events:

1. Physical Transformations: The sudden change in size would lead to various practical adjustments. Everyday items like furniture, clothing, and even vehicles might need to be redesigned or adapted to accommodate the giant children.

2. Playful Activities: Children would engage in unique and imaginative activities that capitalize on their newfound size. Parks and playgrounds would transform into vast landscapes for epic games, obstacle courses, and adventures.

3. Exploration: With their enhanced height, children could explore their surroundings from a different perspective, possibly leading to new discoveries or insights about their environment.

4. Education and Learning: Schools might introduce special days to leverage the giants' size advantage for educational purposes. Science classes, for instance, could involve experiments that demonstrate the effects of size on various phenomena.

5. Social Interactions: The dynamics of social interactions might shift as children grapple with their changing sizes. Friendships could take on new dimensions, and children might learn valuable lessons about empathy and acceptance.

6. Impact on Economy: The demand for goods and services tailored to giant children could create unique economic opportunities. Businesses might offer oversized toys, clothing, and other products on the designated giant day.

7. Environmental Impact: The increased scale of activities could lead to concerns about environmental effects. Authorities might need to manage waste, traffic, and other logistics associated with giant-sized activities.

8. Scientific Exploration: Scientists and researchers would be intrigued by the temporary transformation and could use the opportunity to study the physiological, psychological, and ecological impacts of such a phenomenon.

9. Creativity and Art: Artists and creators would likely draw inspiration from the whimsical idea, resulting in a surge of giant-themed literature, artwork, movies, and more.

10. Sense of Wonder: The experience of becoming giants would foster a sense of wonder and imagination among children, encouraging them to think beyond conventional boundaries and consider new possibilities.

Overall, a day when all children become giants would be a remarkable and unforgettable event that would influence various aspects of society, culture, and individual experiences. It would provide a unique chance for children and adults alike to see the world from an entirely different perspective and explore the imaginative and transformative power of change.

************************************************************************

Color me not impressed. This is just an analysis of various points of view one can take on such a scenario presented in a fairly analytical way. It's not at all what I consider creative output. What would be creative is to construct a story in which some -- but not all -- of the possibilities delineated would be manifested. But simply listing them as possible considerations is analytical rather than creative thinking.  

Visit WriteWayPro.weebly.com  Like and follow on Facebook and on LinkedIn

Related: Why you always need the original source  A matter of degree Bard vs Bard, 

 

Isaac Asimov on Creativity

Clearly Isaac Asimov lived before "synergy" (now displaced by "collaboration") was  buzz word. In 2014 MIT Technology Review ran "Isaac Asimov Asks, 'How Do People Get New Ideas?' and cited Asimov's 1959 essay on creativity. 

from https://c2.staticflickr.com/8/7380/12625238314_6794bf272c_b.jpg

Like Woz, quoted in Susan Cain's Quiet and here, he does believe "isolation is required" to achieve creativity. His description of a creative mind also corresponds to how introverts operate: "His mind is shuffling his information at all times, even when he is not conscious of it."

What's interesting, though is his describing the intrusion of others as not being a problem due to distraction but to introducing self-consciousness that would impede progress: "For every new good idea you have, there are a hundred, ten thousand foolish ones, which you naturally do not care to display."

However, Asimov doesn't go so far as to say that you should shut yourself off from society altogether. He explains that interacting with others has other benefits for the mind

No two people exactly duplicate each other’s mental stores of items. One person may know A and not B, another may know B and not A, and either knowing A and B, both may get the idea—though not necessarily at once or even soon.
Furthermore, the information may not only be of individual items A and B, but even of combinations such as A-B, which in themselves are not significant. However, if one person mentions the unusual combination of A-B and another the unusual combination A-C, it may well be that the combination A-B-C, which neither has thought of separately, may yield an answer.

Yet that doesn't mean that he considers the group dynamics to lead directly to new creative insights. Instead they "educate the participants in facts and fact-combinations, in theories and vagrant thoughts." For the group to work in that way, he warns that the group has to not be at all censorious. He explains that even one person can poison the atmosphere in which all creative expression is unimpeded:

If a single individual present is unsympathetic to the foolishness that would be bound to go on at such a session, the others would freeze. The unsympathetic individual may be a gold mine of information, but the harm he does will more than compensate for that. It seems necessary to me, then, that all people at a session be willing to sound foolish and listen to others sound foolish.
If a single individual present has a much greater reputation than the others, or is more articulate, or has a distinctly more commanding personality, he may well take over the conference and reduce the rest to little more than passive obedience. The individual may himself be extremely useful, but he might as well be put to work solo, for he is neutralizing the rest.

He also offers advice on capping the number of group members. Any more than five, he believes would be counter productive because of "the tension of waiting to speak, which can be very frustrating." But even more important than that is the question of expectation. In other words, one's official job should not be to what today is called "ideate."

The way he puts it is this: "The great ideas of the ages have come from people who weren’t paid to have great ideas, but were paid to be teachers or patent clerks or petty officials, or were not paid at all." The ideas just came while they pursuing other things, which, he feels is important to remove a sense of obligation: "To feel guilty because one has not earned one’s salary because one has not had a great idea is the surest way, it seems to me, of making it certain that no great idea will come in the next time either."

Perspectives on creativity

Update: Jonah Lehrer admitted he fabricated Dylan quotes for his book, see http://mediadecoder.blogs.nytimes.com/2012/07/30/jonah-lehrer-resigns-from-new-yorker-after-making-up-dylan-quotes-for-his-book/?smid=tw-share

I recently finished reading Jonah Lehrer's Imagine: How Creativity Works. (New York: Houghton Mifflin 2012). The book includes numerous anecdotes that are presented as proof that creativity does register on the brain (that's our scientific part) and that people get inspired from other people. OK, I simplified a bit but really just a bit. While he does give a nod to people who get their "best ideas" in the shower or on solitary walks or in lonely and melancholy contemplation, the thrust of the book is that creativity is largely collaborative, something that is quite the opposite of the argument Susan Cain makes.


In a letter to Helen Keller, Mark Twain asserted:

The kernel, the soul — let us go further and say the substance, the bulk, the actual and valuable material of allhuman utterances — is plagiarism. For substantially all ideas are second-hand, consciously and unconsciously drawn from a million outside sources, and daily use by the garnerer with a pride and satisfaction born of the superstition that he originated them; whereas there is not a rag of originality about them anywhere except the little discoloration they get from his mental and moral calibre and his temperament, and which is revealed in characteristics of phrasing.

Lehrer doesn't quote the letter, though he maintain a similar stance: "The most creative ideas, it turns out, don't occur when we're alone. Rather, they emerge from out social circles, from collections of acquaintances who inspire novel thoughts. Sometimes the most important people in life are the people we barely know" (204.)

However, some people who thrive on working alone maintain that creativity stems from individuals working through things on their own. Several posts back I quoted from Quiet: The Power of Introverts in a World That Can't Stop Talking. Cain suggests that solitude is necessary for great achievement. She quotes the following from Steve Wozniak's memoir iWoz (pp. 73-74):

Most inventors and engineers I’ve met are like me – they’re shy and they live in their heads. They’re almost like artists. In fact, the very best of them are artists. And artists work best alone where they can control an invention’s design without a lot of other people designing it for marketing or some other committee. I don’t believe anything really revolutionary has been invented by committee. If you’re that rare engineer who’s an inventor and also an artist, I’m going to give you some advice that might be hard to take. That advice is: Work alone. You’re going to be best able to design revolutionary products and features if you’re working on your own. Not on a committee. Not on a team.

Lehrer also references Steve Wozniak's memoir, but he spins his inventiveness completely differently, saying "the innovations of the first Apple computers depended entirely on this Homebrew culture" (p. 197). That is the "horizontal interactions" that took place in the  club made up of like-minded engineers who swapped ideas in "friendly collaborations."

Cain had mentioned the club, but said that Wozniak saw the creativity itself that only works independently. Lehrer uses the exact same example to try to prove the opposite. That the two writers diverge in this way is important to note because some articles lumped them together because they both pointed to the fact that brainstorming doesn't work. However, Cain would say that is because one head is better than several, while Lehrer maintains a group does enhance creativity. His explanation for the failure of brainstorming is that the ban on criticizing any ideas is what makes it ineffective (160-161).

He then elaborates:

 the reason criticism leads to more new ideas is that it encourages us to fully engag with the work of others. We think about their concepts because we want to improve them; it's the imperfection that leads us to really listen. (And isn't that the point of a group? If we're not here to make one another better , then why are we here?) (161)

Cain's answer to that is that groups themselves inhibit creativity and simply allow the loudest person to assume leadership and direction. In fact, she did say so in her depiction of the failure of group work in classrooms. The introverts are utterly silenced and the outgoing kids just take over. There is no possibility of thoughtful criticism because the first one to assert something confidently gets everyone to follow suit. Now, I'm not saying it always is that way, but many group dynamics do have a leader and followers rather than equal contributions from all.

Lehrer touches on one of the problems inherent in collaboration; the fact that people tend to gravitate to friends. That is what Brian Uzzi, a sociologist, points out in identifying why so many Broadway shows flopped in the 1920s: "'the shows were too full of repeat relationships, and that stifled creativity" (142).The ideal mixture for success is made up a group of people who are not too familiar with each other, so that they have different ideas and will not just second everything their pals suggest.Being too comfortable, as one is when surrounded by friends, does not result in the best work.


Side note: what's fit to print and what is not has definitely been redefined. In my review of Orwell's Down and Out, I noted that the book includes an analysis of swear words in which not a single one other than "bloody" is spelled out. In contrast, Lehrer's book includes a number of quotes that include swear words that are not allowed to be pronounced on television. In fact, most books written in the 21st century seem to include them, whereas really old books, like Twain's letter quoted above, didn't even finish out mild swear words: Just before he ends off, Twain writes:

Oh, dam—
But you finish it, dear, I am running short of vocabulary today.

Related posts: http://uncommoncontent.blogspot.com/2012/05/perspectives-on-introversion-this-is.html
http://uncommoncontent.blogspot.com/2012/04/great-introvert.html