Eric Zelikman

I’m fascinated by how (and whether) algorithms can learn meaningful representations and reason. I'm exploring these questions at xAI. Previously, I was a Ph.D. candidate at Stanford, advised by Nick Haber and Noah Goodman. Perhaps the most daunting exciting gap between human and machine learning is the ease with which we learn concepts about our world from experience: not only do we need much less experience to construct and apply these concepts, but our understanding is flexible in novel situations.

I believe that machine learning can draw lessons from human learning (and vice versa) and that these advances can benefit everyone.

If you're looking for advice or feedback, feel free to schedule a short (non-commercial) research chat. Note I can't give author-level guidance or discuss my current work.

Selected Works

  1. COLM 2024
    Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking
    Eric Zelikman, Georges Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, Noah D. Goodman
    COLM 2024
  2. COLM Oral Spotlight
    Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation
    Eric Zelikman, Eliana Lorch, Lester Mackey, Adam Kalai
    COLM 2024, NeurIPS Workshop on Optimization for Machine Learning 2023
  3. ICLR 2024
    Hypothesis Search: Inductive Reasoning with Language Models
    Ruocheng Wang*, Eric Zelikman*, Gabriel Poesia, Yewen Pu, Nick Haber, Noah D. Goodman
    ICLR 2024
  4. ICLR 2024
    ContextRef: Evaluating Referenceless Metrics For Image Description Generation
    Elisa Kreiss*, Eric Zelikman*, Christopher Potts, Nick Haber
    ICLR 2024
  5. NeurIPS 2024
    Self-Supervised Alignment with Mutual Information: Learning to Follow Principles without Preference Labels
    Jan-Philipp Fränken, Eric Zelikman, Rafael Rafailov, Kanishk Gandhi, Tobias Gerstenberg, Noah D. Goodman
    NeurIPS 2024
  6. TMLR 2024
    Certified Reasoning with Language Models
    Gabriel Poesia, Kanishk Gandhi*, Eric Zelikman*, Noah D. Goodman,
    TMLR 2024
  7. NeurIPS Spotlight
    Parsel 🐍: Algorithmic Reasoning with Language Models by Composing Decompositions
    Eric Zelikman, Qian Huang, Gabriel Poesia, Noah D. Goodman, Nick Haber
    NeurIPS 2023
  8. EMNLP 2023
    Generating and Evaluating Tests for K-12 Students with Language Model Simulations: A Case Study on Sentence Reading Efficiency
    Eric Zelikman*, Wanjing Anya Ma*, Jasmine E. Tran, Diyi Yang, Jason D. Yeatman, Nick Haber,
    EMNLP 2023
  9. NeurIPS Spotlight
    Lexinvariant Language Models
    Qian Huang, Eric Zelikman, Sarah Li Chen, Yuhuai Wu, Gregory Valiant, Percy Liang,
    NeurIPS 2023
  11. Applied Energy
    SkyGPT: Probabilistic Ultra-short-term Solar Forecasting Using Synthetic Sky Images from Physics-constrained VideoGPT
    Yuhao Nie, Eric Zelikman, Andea Scott, Quentin Paletta, Adam Brandt,
    Advances in Applied Energy
  12. NeurIPS 2022
    STaR: Bootstrapping Reasoning With Reasoning
    Eric Zelikman*, Yuhuai (Tony) Wu*, Jesse Mu, Noah D. Goodman,
    NeurIPS 2022
  13. EMNLP 2022
    Context Matters for Image Descriptions for Accessibility: Challenges for Referenceless Evaluation Metrics
    Elisa Kreiss, Cynthia Bennett, Shayan Hooshmand, Eric Zelikman, Meredith Ringel Morris, Christopher Potts,
    EMNLP 2022
  14. TMLR 2022
    Holistic evaluation of language models (reasoning team)
    Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, ... , Eric Zelikman, ...
    TMLR 2022
  15. ICLR 2021
    Evaluating the Disentanglement of Deep Generative Models through Manifold Topology
    Sharon Zhou, Eric Zelikman, Fred Lu, Andrew Y Ng, Gunnar Carlsson, Stefano Ermon,
    International Conference on Learning Representations 2021
  16. ICML Workshop
    CRUDE: Calibrating Regression Uncertainty Distributions Empirically
    Eric Zelikman, Christopher Healy, Sharon Zhou, Anand Avati,
    ICML Workshop on Uncertainty & Robustness in Deep Learning 2020
  17. NeurIPS Workshop
    Short-Term Solar Irradiance Forecasting Using Calibrated Probabilistic Models
    Eric Zelikman*, Sharon Zhou*, Jeremy Irvin*, Cooper Raterink, Hao Sheng, Jack Kelly, Ram Rajagopal, Andrew Y Ng, David Gagne,
    NeurIPS Workshop on Tackling Climate Change with Machine Learning 2020
  18. CVPR Workshop
    Transferable Adversarial Attacks on Deep Reinforcement Learning
    Xinlei Pan, Yulong Cao, Xindi Wu, Eric Zelikman, Chaowei Xiao, Yanan Sui, Rudrasis Chakraborty, Ronald S. Fearing,
    Short Paper in CVPR Workshop on Adversarial Machine Learning in Computer Vision 2020
  19. Honors Thesis
    Learning is its Own Reward: Exploring Worlds with Curiosity-driven Spiking Neural Networks
    Eric Zelikman (advised by Nick Haber),
    Undergraduate Honors Thesis 2020
  20. Contextual Salience for Fast and Accurate Sentence Vectors
    Eric Zelikman, Richard Socher,
    arXiv 2018

Education

Stanford University
Doctoral Student
Computer Science
September 2021 - ??? (On leave at xAI as of March 2024)
Stanford University
Bachelor of Science
Symbolic Systems with Honors
September 2016 - June 2020

Industry

Member of Technical Staff xAI March 2024 - Present
Student Researcher Microsoft Research June 2023 – September 2023
Student Researcher Blueshift @ X & Google Research June 2022 – September 2022
Deep Learning Engineer Lazard July 2020 – September 2021
GANs Curriculum Developer DeepLearning.AI June – October 2020
Machine Learning Intern Argo AI June – September 2019
Machine Learning Intern Uncountable June – September 2018
& April – June 2019
Software Engineer Philometrics October 2016 – July 2017

Other

Reviewer Highlighted (8%) @ ICLR 2022, NeurIPS 2022, COLING 2022, EMNLP 2022, ICML 2023, Best Reviewer Award (1-1.5%) @ ACL 2023, NeurIPS 2023, EMNLP 2023, ICLR 2024 2022-2024
Editor and 2019 Editor-in-Chief Stanford Undergraduate Research Journal 2016 – 2020
Teacher Stanford Splash 2016 - 2020
Section Leader Stanford Code in Place April - May 2020
TreeHacks, HopHacks, Cal Hacks, etc. Hackathons 2016 - 2020