Mark Xu - Alignment Research Center

Estimating Tail Risk in Neural Networks

Machine learning systems are typically trained to maximize average-case performance. However, this method of training can fail to meaningfully control the probability of tail events that might cause significant harm. For instance, while … »

Backdoors as an analogy for deceptive alignment

ARC has released a paper on Backdoor defense, learnability and obfuscation in which we study a formal notion of backdoors in ML models. Part of our motivation for this is an analogy between … »

ARC is hiring theoretical researchers

The Alignment Research Center’s Theory team is starting a new hiring round for researchers with a theoretical background. Please apply here. Update January 2024: we have paused hiring and expect to reopen … »

Formalizing the presumption of independence

ARC has released a paper on Formalizing the presumption of independence, an open problem currently central to our approach to Eliciting Latent Knowledge (ELK). … »

ELK prize results

From January - February the Alignment Research Center offered prizes for proposed algorithms for eliciting latent knowledge. In total we received 197 proposals and are awarding 32 prizes of $5k-20k. We are also giving 24 proposals honorable mentions of $1k, for a total of $274,000. … »

ELK First Round Contest Winners

Thank you to all those who have submitted proposals to the ELK proposal competition. We evaluated 30 distinct proposals from 25 people. We awarded a total of $70,000 for proposals from 8 people. … »

ARC is hiring!

This is an archived version of the early 2022 hiring round page. … »