Artwork

Innhold levert av BlueDot Impact. Alt podcastinnhold, inkludert episoder, grafikk og podcastbeskrivelser, lastes opp og leveres direkte av BlueDot Impact eller deres podcastplattformpartner. Hvis du tror at noen bruker det opphavsrettsbeskyttede verket ditt uten din tillatelse, kan du følge prosessen skissert her https://no.player.fm/legal.
Player FM - Podcast-app
Gå frakoblet med Player FM -appen!

Low-Stakes Alignment

13:56
 
Del
 

Fetch error

Hmmm there seems to be a problem fetching this series right now. Last successful fetch was on January 02, 2025 12:05 (20d ago)

What now? This series will be checked again in the next hour. If you believe it should be working, please verify the publisher's feed link below is valid and includes actual episode links. You can contact support to request the feed be immediately fetched.

Manage episode 424087978 series 3498845
Innhold levert av BlueDot Impact. Alt podcastinnhold, inkludert episoder, grafikk og podcastbeskrivelser, lastes opp og leveres direkte av BlueDot Impact eller deres podcastplattformpartner. Hvis du tror at noen bruker det opphavsrettsbeskyttede verket ditt uten din tillatelse, kan du følge prosessen skissert her https://no.player.fm/legal.

Right now I’m working on finding a good objective to optimize with ML, rather than trying to make sure our models are robustly optimizing that objective. (This is roughly “outer alignment.”) That’s pretty vague, and it’s not obvious whether “find a good objective” is a meaningful goal rather than being inherently confused or sweeping key distinctions under the rug. So I like to focus on a more precise special case of alignment: solve alignment when decisions are “low stakes.” I think this case effectively isolates the problem of “find a good objective” from the problem of ensuring robustness and is precise enough to focus on productively. In this post I’ll describe what I mean by the low-stakes setting, why I think it isolates this subproblem, why I want to isolate this subproblem, and why I think that it’s valuable to work on crisp subproblems.

Source:

https://www.alignmentforum.org/posts/TPan9sQFuPP6jgEJo/low-stakes-alignment

Narrated for AI Safety Fundamentals by TYPE III AUDIO.

---

A podcast by BlueDot Impact.
Learn more on the AI Safety Fundamentals website.

  continue reading

Kapitler

1. Low-Stakes Alignment (00:00:00)

2. 1. What is the low-stakes setting? (00:01:07)

3. 2. Why do low stakes require only outer alignment? (00:01:49)

4. 3. Why focus on this subproblem first? (00:03:26)

5. 4. Is the low-stakes setting actually scary? (00:05:10)

6. 5. Why focus on "low stakes" rather than "outer alignment"? (00:06:09)

7. 6. More formal definition of low-stakes (00:07:23)

8. 7. More formal argument that outer alignment is sufficient (00:08:55)

9. 8. Why expect SGD to work online even for neural networks? (00:11:38)

85 episoder

Artwork
iconDel
 

Fetch error

Hmmm there seems to be a problem fetching this series right now. Last successful fetch was on January 02, 2025 12:05 (20d ago)

What now? This series will be checked again in the next hour. If you believe it should be working, please verify the publisher's feed link below is valid and includes actual episode links. You can contact support to request the feed be immediately fetched.

Manage episode 424087978 series 3498845
Innhold levert av BlueDot Impact. Alt podcastinnhold, inkludert episoder, grafikk og podcastbeskrivelser, lastes opp og leveres direkte av BlueDot Impact eller deres podcastplattformpartner. Hvis du tror at noen bruker det opphavsrettsbeskyttede verket ditt uten din tillatelse, kan du følge prosessen skissert her https://no.player.fm/legal.

Right now I’m working on finding a good objective to optimize with ML, rather than trying to make sure our models are robustly optimizing that objective. (This is roughly “outer alignment.”) That’s pretty vague, and it’s not obvious whether “find a good objective” is a meaningful goal rather than being inherently confused or sweeping key distinctions under the rug. So I like to focus on a more precise special case of alignment: solve alignment when decisions are “low stakes.” I think this case effectively isolates the problem of “find a good objective” from the problem of ensuring robustness and is precise enough to focus on productively. In this post I’ll describe what I mean by the low-stakes setting, why I think it isolates this subproblem, why I want to isolate this subproblem, and why I think that it’s valuable to work on crisp subproblems.

Source:

https://www.alignmentforum.org/posts/TPan9sQFuPP6jgEJo/low-stakes-alignment

Narrated for AI Safety Fundamentals by TYPE III AUDIO.

---

A podcast by BlueDot Impact.
Learn more on the AI Safety Fundamentals website.

  continue reading

Kapitler

1. Low-Stakes Alignment (00:00:00)

2. 1. What is the low-stakes setting? (00:01:07)

3. 2. Why do low stakes require only outer alignment? (00:01:49)

4. 3. Why focus on this subproblem first? (00:03:26)

5. 4. Is the low-stakes setting actually scary? (00:05:10)

6. 5. Why focus on "low stakes" rather than "outer alignment"? (00:06:09)

7. 6. More formal definition of low-stakes (00:07:23)

8. 7. More formal argument that outer alignment is sufficient (00:08:55)

9. 8. Why expect SGD to work online even for neural networks? (00:11:38)

85 episoder

Alle episoder

×
 
Loading …

Velkommen til Player FM!

Player FM scanner netter for høykvalitets podcaster som du kan nyte nå. Det er den beste podcastappen og fungerer på Android, iPhone og internett. Registrer deg for å synkronisere abonnement på flere enheter.

 

Hurtigreferanseguide

Copyright 2025 | Sitemap | Personvern | Vilkår for bruk | | opphavsrett
Lytt til dette showet mens du utforsker
Spill