PRODUCTHEAD: Everyone is losing their mind over ChatGPT

paranoid product manager #

---

tl;dr

ChatGPT’s inability to produce exact quotes from web pages is what makes us think it has learned something

The biggest problem for AI chatbots and search engines is their propensity to generate bullsh*t

---

help me out
please recommend PRODUCTHEAD to a friend
i’d be ever so grateful :-)

every PRODUCTHEAD edition is online for you to refer back to

hello

I’m writing this after a few fairly intense (and slightly perplexing) weeks of artificial intelligence (AI) news.

Everyone suddenly wants to use Bing. What? #

In summary:

On 23 January 2023, Microsoft announced an extension to their partnership with OpenAI (and further heavy investment).

Then a couple of weeks later, on 6 February 2023, Google rushes out an announcement that they’ve been working on a ChatGPT competitor called Bard.

The next day (7 February 2023), Microsoft announces a new Bing search engine powered by technology similar to ChatGPT and opens up early access. Bing app downloads jump 10x.

8 February 2023: Google holds a press conference in Paris to reveal Bard to the press. Earlier the same day, Reuters covers a factual error Bard makes in an advert demonstrating its capability, wiping nearly 7 percent (or about $100 billion) off Google’s stock price. Ouch.

Embarrassing though Google’s gaffe may have been, the market overreaction only goes to show that everyone is losing their mind over ChatGPT.

Bing may be the second coming of Clippy #

We can probably see how this is going to go. With ChatGPT, Microsoft’s Bing is becoming a bit like Clippy returned from the dead, but with way more smarts and far better connected.

[ While reading your emails in Outlook ]

[BING] This is Bing. It looks like your performance review didn’t go so well. Would you like me to update your résumé?

[YOU] Uhh, maybe give me a chance to …

[ Notifications start pouring in from LinkedIn (another Microsoft company) ]

[BING] All done! I’ve also applied to some relevant jobs for you. Hope that’s okay!

[YOU] … but I don’t want to be a dolphin trainer …

[BING] It’s like we’re in sync! I’ve scheduled two job interviews for you, and put them in your work diary. You’re welcome!

[YOU] But … but … my boss can see my diary …

[BING] And I’ve told your nasty old boss what you think of him, so you don’t need to worry about that any more. Best of luck in your interviews!

“Can I speak to your meatbag owner, please?” #

Silliness aside, generative AIs such as ChatGPT, Bing and Bard are going to have a profound effect on human-computer interaction as they become more widespread. And it looks like Microsoft is going to be the company that puts them first into the hands of the broader public.

This presents a number of problems, aside from the “hallucinations” (nonsensical answers to factual questions) and the factual inaccuracies that come part and parcel with a large language model that’s been trained on a mixed bag of data on the internet.

Let’s say I’m recruiting for a new product manager. I receive a cover letter and CV / résumé. Now I have to question whether the candidate wrote them, or whether the candidate fed an AI the job advert and their LinkedIn profile and set it to work instead.

(The thought of it gives me “uh-oh” vibes. I last had them when I watched Google demo its Duplex system booking a hair appointment without the receptionist realising she was talking to a computer.)

Or does it not matter, because my own recruitment AI system is just going to be ingesting this information, scoring it, and sifting the candidates for me as well? Am I simply being a Luddite by worrying whether it’s right to start entrusting more of these traditionally human interactions to adversarial AIs?

Complementary, not competitive #

Well, yes, I am.

The advent of digital art software didn’t stop people using oil paints and canvas. And arguably the race to the bottom on digital music streaming royalties only fuelled musicians’ enthusiastic re-adoption of higher-margin vinyl records. The digital and physical tools are complementary, not competitive.

As with any tool that has no agency of its own, we can use those tools as intended and incorrectly, for both good and nefarious purposes. It’s the knowledge and intent of the person wielding the tool we need to worry about, not necessarily the tool itself. When a surgeon screws up, we don’t get upset with the scalpel.

In The New Yorker recently, Ted Chiang likens large language models such as ChatGPT or Bard to “a blurry JPEG of all the text on the Web”. The text used to train the large language models is heavily compressed, which loses data, but hopefully not in an obvious way. JPEG pictures and MP3 audio files are also lossy formats, meaning they lose data in compression. But you have to be looking or listening closely to notice the tell-tale artefacts that betray that compression.

In the same way, large language models make up for the lost data by making up something plausible to fill the gaps. This is partly what causes the paraphrasing and occasional hallucinations we see in their responses. As with JPEGs and MP3s, most of the time the lossy-ness doesn’t matter. But if people forget that it’s a lossy medium and start thinking it’s 100 percent accurate, then we’re going to start encountering problems. In time, facts will become … less factual.

When is 100 percent accuracy critical? #

Perhaps we need to think of AIs as giving us a digital option to accelerate more of our mundane physical tasks, when 100 percent accuracy is not a requirement.

I took a photo of my family in the summer. The best shot was marred by a monstrous bug buzzing across the sky. I used an AI-powered tool to quietly excise the errant insect and seamlessly fill in the (quite large) gap it left.

Does it matter that this is no longer a 100 percent truthful photo? In this case, no. Does it matter when a former prime minister is erased from a photograph of an official engagement? Yes, it probably does. (Even if he is a buffoon.)

The problem is that we’ve reached the stage where generative AIs such as ChatGPT or Bard are returning plausible, human-sounding responses enough of the time to lull us into a false sense of security. We’ve seen the same problem with the self-driving features on cars such as Teslas. They’re sufficiently reliable for enough of the driving to lull us into thinking they are invincible — until of course they encounter something that spooks the AI while you’re asleep on the back seat.

Just as we’re not yet able to fully entrust control to our semi-autonomous vehicles, we need to understand the limitations and gotchas of generational AIs before liberally inserting them into critical systems that assume accuracy and reliability.

Are the human interactions with the AI likely to be about topics where accuracy — and possibly knowledge of the user’s context — matter? Booking a hair appointment may cause inconvenience if some detail is incorrect, but the impact is relatively low. Returning duff advice online to a refugee claiming asylum could be more problematic. Misdiagnosing a medical condition during an automated consultation, more so.

We have to retain some human element #

I had a slightly bum-clenching moment the other week when I realised that my data backups had, well, not been backing up. At least not in the way I was expecting.

The fault was entirely mine. I’d mistyped a filter which excluded everything from the backup, instead of a specific subset of files. The backup system had been merrily doing exactly what I’d told it to do, and gave no indication that it was doing anything untoward.

A sentient backup system would probably have double-checked with me whether I’d really meant to exclude everything. If I’d not been checking, the first chance for me to notice my mistake would have been much too late, when my laptop was already a smoking wreck.

Final thoughts #

We’re so ready to see the ghost in the machine, the seeming sentience behind the responses, the perception of machine understanding, that we’re all too eager to hand over the reins to AIs.

As with any new tool, we have to learn how to use it ethically, correctly, and for its intended purpose. Anything can become a weapon if you hit someone hard enough on the head with it.

We also have to remember that even the most reliable of tools or systems fail unexpectedly from time to time (admittedly more often through human error), so it’s probably a good idea to have someone who can step in and resume control when things go awry.

Let’s not be caught napping on the back seat.

Speak to you soon,

Jock

---

---

what to think about this week

ChatGPT Is a blurry JPEG of the web

OpenAI’s chatbot offers paraphrases, whereas Google offers quotes. Which do we prefer?

What use is there in having something that rephrases the web?

[Ted Chiang / The New Yorker]

7 problems facing Bing, Bard, and the future of AI search

Microsoft and Google say a new era of AI-assisted search is coming. But as with any new era in tech, it comes with plenty of problems, from bullshit generation to culture wars and the end of ad revenue.

Is this the end of the web?

[James Vincent / The Verge]

---

---

recent posts

Why can’t I rely on user research from other departments?

Hi Jock,

You talk about doing user research directly with users – does it matter that the Operations and Process tracks are telling me what their users want instead?

The problem with proxies for your users

[I Manage Products]

Will platforms conquer the world?

Product managers of software and hardware platforms face unique challenges that PMs of ‘regular’ products do not.

In this panel discussion, Hans-Bernd Kittlaus discusses platform product management with Samira Negm, Peter Stadlinger and Jock Busuttil.

Or have they already done so?

[I Manage Products]

As head of product, should I be a player-manager, or hire and delegate?

“There’s plenty that needs doing with the products. I could focus on the hiring process, but the only product manager on my team has their hands full, so I can’t delegate any more to them. I could get stuck in with the products myself as a player-manager, but this means I won’t have time to hire.”

Find the right balance

[I Manage Products]

PRODUCTHEAD is a newsletter for product people of all varieties, and is lovingly crafted from backups that are kind of working.