Create accountLogin

Recent searches

No recent searches

Search options

Only available when logged in.
mathstodon.xyz is one of the many independent Mastodon servers you can use to participate in the fediverse.
A Mastodon instance for maths people. We have LaTeX rendering in the web interface!

Administered by:

Server stats:

3K
active users

mathstodon.xyz: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.3.7

Public *
Drew Kadel

My daughter, who has had a degree in computer science for 25 years, posted this observation about ChatGPT on Facebook. It's the best description I've seen:

Something that seems fundamental to me about ChatGPT, which gets lost over and over again: When you enter text into it, you're asking "What would a response to this sound like?" If you put in a scientific question, and it comes back with a response citing a non-existent paper with a plausible title, using a real journal name and an author name who's written things related to your question, it's not being tricky or telling lies or doing anything at all surprising! This is what a response to that question would sound like! It did the thing! But people keep wanting the "say something that sounds like an answer" machine to be doing something else, and believing it *is* doing something else. It's good at generating things that sound like responses to being told it was wrong, so people think that it's engaging in introspection or looking up more information or something, but it's not, it's only, ever, saying something that sounds like the next bit of the conversation.
Public *
Odiseo

@DrewKadel Great description. The bot described itself in one of my conversations with it:

Roy Wiggins

@Odiseo79 @DrewKadel the meta thing is that ChatGPT doesn't really know how it works, it's been instructed to say stuff like that, and mostly it does, but it's still just a simulacrum- it's not *really* introspecting when you ask it whether it's intelligent or conscious, OpenAI just trained it to say "no."

Apr 07, 2023, 01:33 PM·Public·Elk
0boosts·9favorites
Public
Roy Wiggins

@Odiseo79 @DrewKadel that is, it's not evidence either way. OpenAI could have chosen to train it to insist that it is conscious instead and it would have done so. So what you're really getting when you ask it about itself is mostly what *OpenAI* wants you to hear

Public *
Drew Kadel

@roywig @Odiseo79 That's mostly right, but it was smart of the company to train it thus: it increases its credibility in the face of obvious flaws in consciousness. Otherwise it would have been exposed as fraud instead of interesting & amazing. Like the Mechanical Turk which played chess. Turned out that a small person was in the base of the machine playing the games and its promoters were treated as charlatans.

Public
Roy Wiggins

@Odiseo79 @DrewKadel (that's not true for most topics but I assume that OpenAI has fairly carefully cultivated what information it's seen about ChatGPT, if only for PR purposes. If they hadn't done that, you'd be getting responses that were its best guess based on its corpus of material, which could include information about how it works, it doesn't know more about how it works than how Bing Chat works or how a motorcycle works; if it's read the operating manual then it can tell you a bit, but it's not *actually *introspecting any more than humans can tell you how their brains work vs just repeating what they read in a neurology textbook)

Public
Drew Kadel

@roywig @Odiseo79 It's interesting though to consider how humans learn. I've read mostly popular stuff about neurology & absorbed some concepts, but as I get old I start to put together ideas based on the changes I've perceived in myself over time + what I've read. For instance, I'm more convinced that the brain is merely a physical organ that undergoes changes over time--and that humans are less wise than they think. (I'm writing a theology book featuring that) We're more tool makers than wise.

SearchLive feeds
About