Eric Zelikman

Iā€™m fascinated by how (and whether) algorithms can learn meaningful representations and reason. I'm exploring these questions at xAI. Previously, I was a Ph.D. candidate at Stanford, advised by Nick Haber and Noah Goodman. Perhaps the most daunting exciting gap between human and machine learning is the ease with which we learn to create and apply concepts about our world from experience: not only do we need much less experience to construct these concepts, but our understanding is flexible in novel situations and robust to slight changes.

I believe that machine learning can draw lessons from human learning (and vice versa) and that these advances can benefit everyone.

If you're looking for advice or collaboration, feel free to schedule a short (non-commercial) research chat.

Selected Works

  1. 2024
    Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking
    Eric Zelikman, Georges Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, Noah D. Goodman
    2024
  2. ICLR 2024
    Hypothesis Search: Inductive Reasoning with Language Models
    Ruocheng Wang*, Eric Zelikman*, Gabriel Poesia, Yewen Pu, Nick Haber, Noah D. Goodman
    ICLR 2024
  3. ICLR 2024
    ContextRef: Evaluating Referenceless Metrics For Image Description Generation
    Elisa Kreiss*, Eric Zelikman*, Christopher Potts, Nick Haber
    ICLR 2024
  4. NeurIPS Spotlight
    Parsel šŸ: Algorithmic Reasoning with Language Models by Composing Decompositions
    Eric Zelikman, Qian Huang, Gabriel Poesia, Noah D. Goodman, Nick Haber,
    NeurIPS 2023
  5. EMNLP 2023
    Generating and Evaluating Tests for K-12 Students with Language Model Simulations: A Case Study on Sentence Reading Efficiency
    Eric Zelikman*, Wanjing Anya Ma*, Jasmine E. Tran, Diyi Yang, Jason D. Yeatman, Nick Haber,
    EMNLP 2023
  6. NeurIPS Spotlight
    Lexinvariant Language Models
    Qian Huang, Eric Zelikman, Sarah Li Chen, Yuhuai Wu, Gregory Valiant, Percy Liang,
    NeurIPS 2023
  7. NeurIPS Workshop
    Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation
    Eric Zelikman, Eliana Lorch, Lester Mackey, Adam Kalai
    NeurIPS Workshop on Optimization for Machine Learning 2023
  8. 2023
  9. 2023
    Certified Reasoning with Language Models
    Gabriel Poesia, Kanishk Gandhi*, Eric Zelikman*, Noah D. Goodman,
    2023
  10. Applied Energy
  11. NeurIPS 2022
    STaR: Bootstrapping Reasoning With Reasoning
    Eric Zelikman*, Yuhuai (Tony) Wu*, Jesse Mu, Noah D. Goodman,
    NeurIPS 2022
  12. EMNLP 2022
    Context Matters for Image Descriptions for Accessibility: Challenges for Referenceless Evaluation Metrics
    Elisa Kreiss, Cynthia Bennett, Shayan Hooshmand, Eric Zelikman, Meredith Ringel Morris, Christopher Potts,
    EMNLP 2022
  13. TMLR
    Holistic evaluation of language models (reasoning team)
    Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, ... , Eric Zelikman, ...
    Transactions on Machine Learning Research 2022
  14. ICLR 2021
    Evaluating the Disentanglement of Deep Generative Models through Manifold Topology
    Sharon Zhou, Eric Zelikman, Fred Lu, Andrew Y Ng, Gunnar Carlsson, Stefano Ermon,
    International Conference on Learning Representations 2021
  15. ICML Workshop
    CRUDE: Calibrating Regression Uncertainty Distributions Empirically
    Eric Zelikman, Christopher Healy, Sharon Zhou, Anand Avati,
    ICML Workshop on Uncertainty & Robustness in Deep Learning 2020
  16. NeurIPS Workshop
    Short-Term Solar Irradiance Forecasting Using Calibrated Probabilistic Models
    Eric Zelikman*, Sharon Zhou*, Jeremy Irvin*, Cooper Raterink, Hao Sheng, Jack Kelly, Ram Rajagopal, Andrew Y Ng, David Gagne,
    NeurIPS Workshop on Tackling Climate Change with Machine Learning 2020
  17. CVPR Workshop
    Transferable Adversarial Attacks on Deep Reinforcement Learning
    Xinlei Pan, Yulong Cao, Xindi Wu, Eric Zelikman, Chaowei Xiao, Yanan Sui, Rudrasis Chakraborty, Ronald S. Fearing,
    Short Paper in CVPR Workshop on Adversarial Machine Learning in Computer Vision 2020
  18. Honors Thesis
    Learning is its Own Reward: Exploring Worlds with Curiosity-driven Spiking Neural Networks
    Eric Zelikman (advised by Nick Haber),
    Undergraduate Honors Thesis 2020
  19. Contextual Salience for Fast and Accurate Sentence Vectors
    Eric Zelikman, Richard Socher,
    arXiv 2018

Education

Stanford University
Doctoral Student
Computer Science
September 2021 - Present
Stanford University
Bachelor of Science
Symbolic Systems with Honors
September 2016 - June 2020

Industry

Member of Technical Staff xAI March 2024 - Present
Student Researcher Microsoft Research June 2023 ā€“ September 2023
Student Researcher Blueshift @ X & Google Research June 2022 ā€“ September 2022
Deep Learning Engineer Lazard July 2020 ā€“ September 2021
GANs Curriculum Developer DeepLearning.AI June ā€“ October 2020
Machine Learning Intern Argo AI June ā€“ September 2019
Machine Learning Intern Uncountable June ā€“ September 2018
& April ā€“ June 2019
Software Engineer Philometrics October 2016 ā€“ July 2017

Other

Reviewer Highlighted (8%) @ ICLR 2022, NeurIPS 2022, COLING 2022, EMNLP 2022, ICML 2023, Best Reviewer Award (1-1.5%) @ ACL 2023, NeurIPS 2023, EMNLP 2023, ICLR 2024 2022-2023
Editor and 2019 Editor-in-Chief Stanford Undergraduate Research Journal 2016 ā€“ 2020
Teacher Stanford Splash 2016 - 2020
Section Leader Stanford Code in Place April - May 2020
TreeHacks, HopHacks, Cal Hacks, etc. Hackathons 2016 - 2020
Cofounder - Machine Learning Lead DeepDesigns.ai (defunct) January - July 2020