The Risks
We're concerned about a wide variety of risks: loss of control, AI-enabled pandemics, AI-related great power conflict, societal-scale cyberattacks, informational warfare, gradual disempowerment, and more. (Note: some of these might be more challenging for us to find mentors for)
Our Promise
Other programs in AI safety are primarily designed to accelerate your career as fast as possible. Whilst we believe that participating in this fellowship will increase your chance of breaking into the field, this objective is merely secondary for us. Our promise to you is that we will do our best to equip you to make a difference, as difficult as this is given the extreme level of uncertainty around AGI.
Program Details
10 Week Structure
In-Person Days
Weekly Schedule
What We Offer
A Typical Day
Schedule for our in-person days (Saturday: 10am-6pm)
Mandatory Day
RequiredDiscussion Prep
30 minDiscussion
2 hoursSpeaker
1 hourLunch
1 hourMentorship Session
30 minProject Work
3 hoursOptional Secondary Day
OptionalReadings
1 hourSpeaker
1 hourSelf-organised Activities
1 hourLunch
1 hourProject Work
4 hours
Additional Project Work: Remaining hours completed outside the 10-6pm
schedule
Note: Socials, dinners, and some mentorship sessions occur outside scheduled
hours
Is this for you?
An honest assessment to help you decide if this fellowship aligns with your goals and circumstances.
This may NOT be for you if:
Consider whether these factors apply to your situation
- You're looking for a high-prestige program or you'd prefer a program where advancing your career is the primary focus.
- We unfortunately can't offer stipends or accommodation for this program.
- You believe predicting future technology is pointless. If the course of future technology is so hard to predict that there's no point in even trying.
- You'd get frustrated with epistemological discussions. The fellowship spends significant time delving into various epistemological frames people have tried to apply to understand AGI; you'd much rather discussions purely focus on the concrete.
- You want only coding or only non-technical work. You'd prefer a program where you spend all your time doing heads-down coding instead of taking time to understand the strategic landscape. Or, conversely, if detailed technical discussions make your eyes glaze over.
- You don't know the basics of ML. We believe that some technical knowledge is important even for governance fellows and you'll likely struggle during discussions if you don't understand basic concepts like vector spaces, pre-training vs. post-training, the distinction between test and training sets, or what gradient descent is. That said, we might be willing to make an exception if you can convince us that you're willing to work hard to get up to speed before the start of the fellowship.
- You don't know core AI safety concepts. We'd be willing to accept candidates who haven't completed any AI safety program before—indeed this described one of our top fellows from the original fellowship—but we expect you to already be up to speed with core concepts like instrumental convergence, the orthogonality thesis, inner/outer alignment and reward hacking. This is less of a hard requirement for candidates with deep knowledge of a particular threat vector (bio, cyber, social influence, etc.).
On the other hand:
Reasons you should still consider applying
- Don't self-select out. We won't judge you for applying even if you aren't a perfect fit. Research shows many qualified candidates don't apply unless they meet every single criterion. In reality, it's rare that any applicant is the 'ideal candidate,' and if you're on the fence, we'd rather see your application than have you self-select out.
- Diverse strengths welcome: We're looking for a wide array of talents—we actively encourage entrepreneurial thinkers and governance-focused individuals, not just technical researchers.
- Open to pivots: We'll consider candidates who have already decided on how they wish to contribute to AI safety long-term, however this program will best fit candidates who are open to pivoting. We're more likely to accept candidates who have already settled on a path if they're proposing to do something novel.
- Already in AI safety? We're open to candidates who are already working in AI safety, however, we'd want to know why you think participating in this fellowship might allow you to substantially increase your impact. We think this is much more likely to be the case if you're considering pivoting in some way.
- Exploring options: We're open to candidates who are still deciding between pursuing AI safety and other options (indeed considering multiple options is prudent); however we're looking for more than just a vague interest.
- Flexibility: We're aware that the university year will resume towards the end of the fellowship. We're willing to offer a degree of flexibility to accommodate university schedules and senior candidates' commitments.
- Adjacent areas: Even though it is outside the primary focus of the program, we might be open to exceptional candidates interested in adjacent areas like AI welfare, economics of transformative AI, Better Futures, etc. Finding a mentor for these areas may be challenging though.
Alumni Outcomes
It's standard for programs to describe alumni outcomes, so we'll share a bit—though this is certainly not counterfactual impact and AI safety is much more competitive these days.
This is the third iteration of this fellowship and the second one in-person. In the first in-person iteration we accepted five fellows. Three of them are currently working in AI safety. Another co-founded an existential risk organisation that still exists today, but he's working on other things.
Whilst the second iteration being online made it subjectively much less effective than the first, several participants from that iteration are now working professionally in AI safety.