Newsletter
Innovator profile
JF

Jim Fan

Co-lead, Generalist Embodied Agent Research (GEAR), NVIDIA

Embodied AIRoboticsAI agentsFoundation models

Education and early research

Linxi "Jim" Fan obtained his PhD at the Stanford Vision Lab, advised by Fei-Fei Li[1][2]. Before that he graduated from Columbia University as valedictorian of the class of 2016 and received the Illig Medal[1].

He previously did research internships at OpenAI, Baidu's AI lab, Mila, Google Cloud AI and NVIDIA[1][2]. His website says he was OpenAI's first intern, in 2016[1]. That summer he worked on World of Bits, an agent that perceives the web browser as pixels and outputs keyboard and mouse controls[1].

Agents in Minecraft and simulation

Fan was the first author of MineDojo, a framework built on Minecraft for training generalist agents, submitted to arXiv in June 2022[3]. MineDojo combines a simulation suite of thousands of open-ended tasks with an internet-scale knowledge base of Minecraft videos, tutorials, wiki pages and forum discussions[3]. The paper won the Outstanding Paper Award at NeurIPS 2022[1][2].

He was a co-author of Voyager, published in May 2023, which the authors describe as the first LLM-powered embodied lifelong learning agent in Minecraft[4]. Voyager uses an automatic curriculum, a growing library of executable code skills, and an iterative prompting method that feeds back environment results and errors, and it works with GPT-4 without fine-tuning the model[4].

In October 2023 Fan and colleagues published Eureka, an algorithm that uses large language models such as GPT-4 to write and evolve reward functions for reinforcement learning[5]. The paper reports that Eureka's reward functions outperformed expert human-written rewards across a suite of 29 environments with 10 robot body types, and it taught a five-finger robot hand to spin a pen[5][1]. He also led VIMA, a model for robot manipulation that takes multimodal prompts[1].

GEAR and Project GR00T at NVIDIA

At NVIDIA, Fan co-leads the Generalist Embodied Agent Research group, GEAR, with Yuke Zhu[6]. The group's stated mission is to build foundation models for embodied agents in virtual and physical worlds[6]. Its research covers multimodal foundation models, general-purpose robots, foundation agents for games and simulations, and simulation and synthetic data[6].

NVIDIA announced Project GR00T, a general-purpose foundation model for humanoid robots, at GTC on 18 March 2024[7]. GR00T stands for Generalist Robot 00 Technology, and NVIDIA said robots powered by it would be designed to understand natural language and learn skills by observing human actions[7]. In March 2025 NVIDIA researchers, including Fan, released GR00T N1, described as an open foundation model for generalist humanoid robots[8]. GR00T N1 is a vision-language-action model with two parts: a vision-language module that interprets the scene and instructions, and a diffusion transformer that generates motor actions in real time[8]. GR00T is listed as a project of the GEAR group[6].

Views on robotics data

In an episode of Sequoia Capital's Training Data podcast, Fan described NVIDIA's data strategy for robotics as combining internet-scale data, simulation data and real-world robot data[9]. He said he believes it will be possible within a few years to build a foundation agent that generalizes across skills, robot bodies and both physical and virtual worlds[9]. The episode summary says he expects intelligent robots to become as common as smartphones[9].

Known for

Sources

  1. Jim Fan: personal website — checked 6 Nov 2026
  2. NVIDIA Research: Linxi "Jim" Fan profile — checked 6 Nov 2026
  3. arXiv: MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge, June 2022 — checked 6 Nov 2026
  4. arXiv: Voyager: An Open-Ended Embodied Agent with Large Language Models, May 2023 — checked 6 Nov 2026
  5. arXiv: Eureka: Human-Level Reward Design via Coding Large Language Models, October 2023 — checked 6 Nov 2026
  6. NVIDIA Research: GEAR group home page — checked 6 Nov 2026
  7. NVIDIA Newsroom: NVIDIA Announces Project GR00T Foundation Model for Humanoid Robots, March 2024 — checked 6 Nov 2026
  8. arXiv: GR00T N1: An Open Foundation Model for Generalist Humanoid Robots, March 2025 — checked 6 Nov 2026
  9. Sequoia Capital, Training Data podcast: Jim Fan on Nvidia's Embodied AI Lab — checked 6 Nov 2026
Editorial profile compiled from public sources. Not an endorsement, and listing does not imply participation in this site. To correct or remove this profile, contact us.