Physical reasoning is a remarkable human ability that enables rapid learning and generalization from limited experience. Current AI models, despite extensive training, still struggle to achieve similar generalization, especially in Out-of-distribution (OOD) settings. This limitation stems from their inability to abstract core physical principles from observations. A key challenge is developing representations that can efficiently learn and generalize physical dynamics from minimal data. Here we present Neural Force Field (NFF), a framework extending Neural Ordinary Differential Equation (NODE) to learn complex object interactions through force field representations, which can be efficiently integrated through an Ordinary Differential Equation ( ODE) solver to predict object trajectories. Unlike existing approaches that rely on discrete latent spaces, NFF captures fundamental physical concepts such as gravity, support, and collision in continuous explicit force fields. Experiments on three challenging physical reasoning tasks demonstrate that NFF, trained with only a few examples, achieves strong generalization to unseen scenarios. This physics-grounded representation enables efficient forward-backward planning and rapid adaptation through interactive refinement. Our work suggests that incorporating physics-inspired representations into learning systems can help bridge the gap between artificial and human physical reasoning capabilities.

We present the learned force fields in both the intuitive physical reasoning task and the n-body task, demonstrating various fundamental physical concepts and principles.

I-PHYRE training data: 10 basic scenarios (support, hinder, direction, hole, fill, seesaw, angle, impulse, pendulum, and spring), each with 10 random action sequences.

I-PHYRE within-scenario: scenarios identical to the training data but with different action sequences.

I-PHYRE cross-scenario: scenarios that differ from the training data, including noisy, compositional, and multi-ball environments.

N-body training data: 200 trajectories involving 2-body and 3-body systems (such as planets or comets).

N-body within-scenario: 2-body and 3-body scenarios with different initial conditions (in-distribution positions and masses).

N-body cross-scenario: 9-body scenarios with larger orbital radii and extended time steps.

PHYRE training data: 12,000 (20 templates * 20 tasks * 30 actions) videos with object masks for our NFF. Other models utilize up to 3.2 million samples.

PHYRE within-scenario: same templates in training but different tasks.

PHYRE cross-scenario: unseen templates.

Forward planning: given randomly sampled action sequences, simulate the outcome trajectories using the trained NFF and select the best that can achieve the goal (make the ball into the abyss by eliminating the white blocks).

Refinement: update the model after each round of trials.

Backward planning: utilize the invertible information flow of ODE integration to compute the initial state directly from the final state.