LW - MATS Winter 2023-24 Retrospective by Rocket

The Nonlinear Library: LessWrong

Link to original article
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: MATS Winter 2023-24 Retrospective, published by Rocket on May 11, 2024 on LessWrong. Co-Authors: @Rocket, @Ryan Kidd, @LauraVaughan, @McKennaFitzgerald, @Christian Smith, @Juan Gil, @Henry Sleight The ML Alignment & Theory Scholars program (MATS) is an education and research mentorship program for researchers entering the field of AI safety. This winter, we held the fifth iteration of the MATS program, in which 63 scholars received mentorship from 20 research mentors. In this post, we motivate and explain the elements of the program, evaluate our impact, and identify areas for improving future programs. Summary Key details about the Winter Program: The four main changes we made after our Summer program were: Reducing our scholar stipend from $40/h to $30/h based on alumni feedback; Transitioning Scholar Support to Research Management; Using the full Lighthaven campus for office space as well as housing; Replacing Alignment 201 with AI Strategy Discussions. Educational attainment of MATS scholars: 48% of scholars were pursuing a bachelor's degree, master's degree, or PhD; 17% of scholars had a master's degree as their highest level of education; 10% of scholars had a PhD. If not for MATS, scholars might have spent their counterfactual winters on the following pursuits (multiple responses allowed): Conducting independent alignment research without mentor (24%); Working at a non-alignment tech company (21%); Conducting independent alignment research with a mentor (13%); Taking classes (13%). Key takeaways from scholar impact evaluation: Scholars are highly likely to recommend MATS to a friend or colleague (average likelihood is 9.2/10 and NPS is +74). Scholars rated the mentorship they received highly (average rating is 8.1/10). For 38% of scholars, mentorship was the most valuable element of MATS. Scholars are likely to recommend Research Management to future scholars (average likelihood is 7.9/10 and NPS is +23). The median scholar valued Research Management at $1000. The median scholar reported accomplishing 10% more at MATS because of Research Management and gaining 10 productive hours. Mentors are highly likely to recommend MATS to other researchers (average likelihood is 8.2/10 and NPS is +37). Mentors are likely to recommend Research Management (average likelihood is 7.7/10 and NPS is +7). The median mentor valued Research Management at $3000. The median mentor reported accomplishing 10% more because of Research Management and gaining 4 productive hours. The most common benefits of mentoring were "helping new researchers," "gaining mentorship experience," "advancing AI safety, generally," and "advancing my particular projects." Mentors improved their mentorship abilities by 18%, on average. The median scholar made 5 professional connections and found 5 potential future collaborators during MATS. The average scholar self-assessed their improvement on the depth of their technical skills by +1.53/10, their breadth of knowledge by +1.93/10, their research taste by +1.35/10, and their theory of change construction by +1.25/10. According to mentors, of the 56 scholars evaluated, 77% could achieve a "First-author paper at top conference," 41% could receive a "Job offer from AI lab safety team," and 16% could "Found a new AI safety research org." Mentors were enthusiastic for scholars to continue their research, rating the average scholar 8.1/10, on a scale where 10 represented "Very strongly believe scholar should receive support to continue research." Scholars completed two milestone assignments, a research plan and a presentation. Research plans were graded by MATS alumni; the median score was 76/100. Presentations received crowdsourced evaluations; the median score was 86/100. 52% of presentations featured interpretability research, representing a significant proport...