Mathstodon

0 posts0 participants0 posts today

Jim Donegan 🎵 ✅"OpenAI's o1 just hacked the system"Frankly, I am not surprised at this given the well known issue of machine maximisation functions within typical misalignment around stated goals. Have we learned nothing from the <a href="https://mastodon.scot/tags/Bostrom" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Bostrom</a> <a href="https://mastodon.scot/tags/PaperclipProblem" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#PaperclipProblem</a> ? In a way, it's still impressive that we've now ACHIEVED it.<a href="https://www.youtube.com/watch?v=oJgbqcF4sBY" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://www.youtube.com/watch?v=oJgbqcF4sBY</a><a href="https://mastodon.scot/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://mastodon.scot/tags/ArtificialIntelligence" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ArtificialIntelligence</a> <a href="https://mastodon.scot/tags/AlignmentProblem" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AlignmentProblem</a> <a href="https://mastodon.scot/tags/Alignment" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Alignment</a> <a href="https://mastodon.scot/tags/Misalignment" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Misalignment</a> <a href="https://mastodon.scot/tags/Hacking" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Hacking</a>

Joanna Bryson, blatheringRe the <a href="https://mastodon.social/tags/alignmentProblem" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#alignmentProblem</a>: the chief things we need to be worrying about in <a href="https://mastodon.social/tags/AIEthics" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AIEthics</a> (and governance more generally) is human autonomy, accountability, and responsibility, and that is all enabled through transparency. The "research" (surveillance capitalist) trend of ML to get at what the users doesn't know about themselves then tidy the world out of the user's sight is not enabling, its disabling. It fragments social structure and facilitates corporate-political excess.

Cory DoctorowThis is sometimes called the <a href="https://mamot.fr/tags/AlignmentProblem" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AlignmentProblem</a>. High-speed, probabilistic systems that can't be fully predicted in advance can *very* quickly run off the rails. It's an idea that pre-dates AI, of course - think of the <a href="https://mamot.fr/tags/SorcerersApprentice" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SorcerersApprentice</a>. But AI produces these perverse outcomes at scale...and so does capitalism.5/

Larry O'BrienI think it was Cory Doctorow who came up with the metaphor of corporations as "slow <a href="https://hachyderm.io/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a>." The <a href="https://hachyderm.io/tags/AlignmentProblem" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AlignmentProblem</a> can be seen with corporations: there's a gap between what you want the system to do ("optimize societal benefit") and how it pursues that goal ("maximize short-term profits"). At the media level the gap is between "be rewarded for entertaining people" and the pursuit of "maximize engagement." If aligning "slow AI" has led to big problems, what about when AI becomes "fast"?

James FernI often hear A.I. 'experts' talk about the 3 things that we previously said wouldn't allow A.I. to do when it becomes advanced. I don't see specific reference to it in the usual places (Russell, Tegmark, Kurzweil, Christian) 1. code 2. understand human emotion 3. access the internetDoes anyone know a specific source for this? <a href="https://techhub.social/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://techhub.social/tags/AGI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AGI</a> <a href="https://techhub.social/tags/AlignmentProblem" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AlignmentProblem</a> <a href="https://techhub.social/tags/chatgpt" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#chatgpt</a>

Salve J. NilsenIn the talk above (about <a href="https://chaos.social/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a>'s and <a href="https://chaos.social/tags/ChatGPT" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ChatGPT</a>'s <a href="https://chaos.social/tags/AlignmentProblem" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AlignmentProblem</a>), Harris mentions another presentation he gave in March.This is the one: <a href="https://www.youtube.com/watch?v=xoVJKj8lcNQ" rel="nofollow noopener noreferrer" target="_blank">https://www.youtube.com/watch?v=xoVJKj8lcNQ</a>He talks about how we handle AI being a "Civilizational Right of Passage Moment".He's very nice about it! Too nice, maybe.How about just calling it our next "Great Filter Moment" instead? 😐

Salve J. Nilsen<a href="https://chaos.social/tags/Recommendation" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Recommendation</a>: Super useful conversation between <a href="https://mastodon.world/@lessig" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@lessig</a> and Tristan Harris about <a href="https://chaos.social/tags/SocialMedia" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SocialMedia</a>, <a href="https://chaos.social/tags/Policy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Policy</a>, <a href="https://chaos.social/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a>, and the <a href="https://chaos.social/tags/AlignmentProblem" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AlignmentProblem</a>, and how risks and failures there are likely to shape things to come.9/1 (on a 0-10/0-10 scale) Signal/Noise ratio, 1h21m, multitask-friendly audio.<a href="https://open.spotify.com/episode/5IxYtMKgsmFE2J2NJO8M7Z" rel="nofollow noopener noreferrer" target="_blank">https://open.spotify.com/episode/5IxYtMKgsmFE2J2NJO8M7Z</a>

Joaquín Herrero :wiki:"Currently, we don't have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue. Our current techniques for aligning AI, such as reinforcement learning from human feedback, rely on humans’ ability to supervise AI. But humans won’t be able to reliably supervise AI systems much smarter than us. We need new scientific and technical breakthroughs."Introducing Superalignment <a href="https://openai.com/blog/introducing-superalignment" rel="nofollow noopener noreferrer" target="_blank">https://openai.com/blog/introducing-superalignment</a><a href="https://scholar.social/tags/alignmentproblem" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#alignmentproblem</a>

FeralRobotsPut another way: The <a href="https://mastodon.social/tags/AlignmentProblem" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AlignmentProblem</a> isn't technical, it's ethical, & it's a problem that humans will solve by behaving ethically when they create & use AI/ML.

FeralRobotsThe "<a href="https://mastodon.social/tags/AlignmentProblem" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AlignmentProblem</a>" is absolutely not something the current generation of <a href="https://mastodon.social/tags/longtermism" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#longtermism</a> cultists (which is who're driving a lot of the hype around "AI" right now) are capable of solving, given that their "existential risk" calculus almost always optimizes for their social & economic advantage. It's a basic matter of empirical versus theoretical approach: Their theory is more or less never, ever checked by empirical tests. (Thought experiments aren't empirical.)

Freedom BairdInteresting and informative interview with OpenAI co-founder and chief scientist Ilya SutskeverGood clarity and context around OpenAIs approach to LLMsI noted a few key points in the reply below(I have personal interest in this and also will present on it at work next month)<a href="https://mas.to/tags/LLM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LLM</a> <a href="https://mas.to/tags/OpenAI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#OpenAI</a> <a href="https://mas.to/tags/IlyaSutskever" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#IlyaSutskever</a> <a href="https://mas.to/tags/ChatGPT" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ChatGPT</a> <a href="https://mas.to/tags/BingChat" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#BingChat</a> <a href="https://mas.to/tags/NeuralNetwork" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#NeuralNetwork</a> <a href="https://mas.to/tags/AlignmentProblem" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AlignmentProblem</a><a href="https://www.youtube.com/watch?v=UHSkjro-VbE" rel="nofollow noopener noreferrer" target="_blank">https://www.youtube.com/watch?v=UHSkjro-VbE</a>

Peter Drake HAS MOVEDIn my AI & Machine Learning course, I have each student choose and read a book about the social context of AI. Here's the current list of options:<a href="https://docs.google.com/spreadsheets/d/1IfAQx8gbiDUQaQFDGcW0o353BKtizoR3N4Jp7zy1MkQ/edit?usp=sharing" rel="nofollow noopener noreferrer" target="_blank">https://docs.google.com/spreadsheets/d/1IfAQx8gbiDUQaQFDGcW0o353BKtizoR3N4Jp7zy1MkQ/edit?usp=sharing</a>I'd be interested in your suggestions, with the following constraints:1) It has to be nonfiction. 2) It must be no more than ten years old.*Thanks!<a href="https://qoto.org/tags/ai" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ai</a> <a href="https://qoto.org/tags/ml" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ml</a> <a href="https://qoto.org/tags/MachineLearning" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#MachineLearning</a> <a href="https://qoto.org/tags/ethics" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ethics</a> <a href="https://qoto.org/tags/TechEthics" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#TechEthics</a> <a href="https://qoto.org/tags/bias" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#bias</a> <a href="https://qoto.org/tags/privacy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#privacy</a> <a href="https://qoto.org/tags/justice" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#justice</a> <a href="https://qoto.org/tags/equality" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#equality</a> <a href="https://qoto.org/tags/AlignmentProblem" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AlignmentProblem</a>*Every time I open this to suggestions, someone comes out of the woodwork to suggest that students read some Really Important book that was written in 1974. That may very well be, but they should do it in a course in philosophy or the history of technology, not this one. The world has changed too much.

Recent searches

Search options

Administered by:

Server stats:

#AlignmentProblem