The perils of prompt engineering

You may have noticed a few people who formerly called themselves writers or content creators now calling themselves "prompt engineers." This title gives a veneer of a more scientific credentials to what really amounts to playing around with Chat GPT or Bard or some of the lesser-known generative AI tools out there to elicit the response you seek.

At present, though, there is absolutely no licensing or testing to "prove" oneself as a prompt engineer. It's a self-declared title that can't really be disproven until we come to a stage where Microsoft, Google, or OpenAI decide to make it an official credential. 

You get to call the shots and take credit for when the generative AI delivers. Sounds ideal, right? 

It might have been -- if only you really could wholly rely on the AI to be accurate. Unfortunately, you can't. You have to fact-check everything it assert, no matter how confident it may sound about it. 

Here Be Hallucinations

Why is that? Generative AI has learned to fabricate facts to back the narrative it's constructing. 

This is what has come to be called "hallucinations," though I really don't like the term because it makes it sound like what humans experience from external forces acting on their minds. No one seems to be giving generative AI the equivalent of mind-altering drug to prompt it to come up with this --sometimes rather elaborate -- hallucinations. 

Bard Gets Creative 

 Bard got incredibly expansive in one hallucination, positing an edition of Ben Horowitz's book The Hard Thing About Hard Things that doesn't exist. I asked it if the book ever references Built to Last, an extremely popular and influential business book from the late 20th century, when I noted some striking similarities and even the same title  for a chapter at the end of both books.   

Bard answered:

The passages where Ben Horowitz mentions "Built to Last" explicitly in his book "The Hard Thing about Hard Things" are on the following pages:
Chapter 6: The Flywheel, page 125
Chapter 10: Core Ideology, page 197

I countered that the pages and chapter titles didn't correspond to the copy of the book I had in front of me, and so it compounded the original hallucination as follows:  and it answered:

The passages where Ben Horowitz mentions "Built to Last" explicitly in his book "The Hard Thing about Hard Things" are on the following pages:
Chapter 6: The Flywheel, page 125
Chapter 10: Core Ideology, page 197

Say What?

I countered that the pages and chapter titles didn't correspond to the copy of the book I had in front of me, and so it compounded the original hallucination as follows:

You are correct. The pages I provided correspond to the 2015 edition of the book. The 2014 edition has different page numbers. The passages where Ben Horowitz mentions "Built to Last" explicitly in the 2014 edition are on the following pages:
Chapter 6: The Flywheel, page 107
Chapter 10: Core Ideology, page 177

It then went on to explain that that the chapter titles were changed:

You are correct. The chapter names in the 2014 edition of "The Hard Thing about Hard Things" are different from the chapter names in the 2015 edition. The chapter that I referred to as "The Flywheel" is actually called "Concerning the Going Concerns" in the 2014 edition. The chapter that I referred to as "Core Ideology" is actually called "The Ideology of Success" in the 2014 edition.
I apologize for the confusion. I have updated my previous response to reflect the correct chapter names.

It sounds pretty credible until you look for the 2015 edition of this book and find it never existed. Bard simply created this out of thin air and went on to support the fiction with additional details.

This is very dangerous for anyone who is not by nature as skeptical as I am. Most people would likely accept that if Bard can give an account of an edition of a book as detailed as this, its must really exist. In fact, though, you'd have to not be taken in by the confidence of the assertion and do your own due-diligence.

I'd say that would be the real work of the prompt engineer today: not just coming up with the request for information but rigorously questioning the results and doing your own research to be certain that they're not the product of hallucinations.

Related: 

An A/B Test of Generative AI

Did Open AI Open Pandora's Box?

Bard vs Bard: why you can't just accept the first AI prompt response

What the Little Prince teaches us about marketing

To market effectively, you have to see thing through your customer's eyes, not the other way around. 

 Glasses are not one-size-fits-all 

Picture this scenario: you have difficulty seeing things clearly, so you go to an optometrist.  You expect to have your eyes checked and to read off charts with different lenses.                                                                                                                                                                                                                                                                                 

But that doesn't happen. Instead, the optometrist just takes the glasses she is wearing off her own face and tells you: "This make everything clear for me. They should work for you, too."                                                                                        

Say what? What optometrist assumes that anyone who comes in needs the same exact prescription she has for herself? 

That it's completely irrational to project that about eyeglasses is something most people would acknowledge. Yet they ail to see that do just that in business all the time. 

t.

Home vs investment

I was thinking abut this week because of a realtor's failure to grasp what I wanted to have in an apartment. She  was stuck in her mindset of what believed made a property a good investment.

 She kept sending me suggestions with apartments that didn't have the features I wanted, claiming the included parking spots made them desirable. I explained multiple times that parking was not a major priority for me but other features were. 

Even after I explained this, she still tried to talk me into an apartment that clearly lacked what I wanted, saying it was a good buy. Perhaps it was a good value, but I want a home to live in with the features that are important to me.

From real estate to theoretical pills

This kind of disconnect is very common in all kinds of businesses in which the sellers fail to see things from the customer's eyes and project their own tastes and values on others. It  doesn't occur to them that other people have their own calculus and that features and benefits they go on and on about may not matter to them at all.   

  

In chapter 23 of The Little Prince, we get the perfect illustration of the misalignment between the value proposition of a product and what the customer actually wants in the  interaction between the prince and a merchant he meets on his travels:

.

"This was a merchant who sold pills that had been invented to quench thirst. You need only swallow one pill a week, and you would feel no need for anything to drink.

"'Why are you selling those?' asked the little prince.

"'Because they save a tremendous amount of time,' said the merchant. "'Computations have been made by experts. With these pills, you save fifty-three minutes in every week.'"

"'And what do I do with those fifty-three minutes?'"

"'Anything you like..."

"'As for me,' said the little prince to himself, 'if I had fifty-three minutes to spend as I liked, I should walk at my leisure toward a spring of fresh water.'”

Think about it: are you selling someone on the assumption that they'd rather save time than enjoy their water and the walk it takes to get it?  Are you pushing your eyeglass prescription on someone without determining what type of corrective lenses she needs for her vision?

 You may be doing it in other ways like by promising AI automation that summarizes every book for you. But what if your potential customer enjoys reading through hundreds of pages of novels to appreciate the  narrative style and dialog? Not everyone dismisses lengthy prose with TL:DR. 

Assuming that everyone shares your priorities when pushing something is not the way to go to show customers you are about them and what they want. You have to drop the projection and really listen.

Related: 

Marketing in Uncommon Times

Tech Overload in the Bathroom

A Matter of Degree

Casting the Hero of Your Story