Robotics Briefing /
Can a chatbot drive a robot arm?
Hide a red lid under one of three upturned mugs, shuffle them for half a minute, then ask a robot to lift the right one. Researchers at MIT and Harvard gave a general-purpose AI model the controls. The hard part was keeping track of the mug while the model took seconds to answer. A second group tried a different approach on eight physical tasks, including plugging in a charger. Its AI wrote and tested the program before the robot had to perform.
Your quick takeaways
- A general-purpose model using measuring tools and a second copy watching the cameras completed 22 of 25 physical tabletop trials. It picked the right mug in five of five shuffles, falling to two of five without the second observer.
- Programs written by a coding agent completed 221 of 240 physical trials across eight tasks. People could guide the arms during development. The finished programs ran without the agent or a human operator.
- GXO and Apptronik signed a letter of intent that could put up to 1,000 Apollo humanoids in warehouses over four years. The number depends on trials that begin in 2027.
Papers
Three mugs, one hidden lid
Pengfei Ye, Edward Adelson and colleagues at MIT and Harvard · November 12 preprint · Physical robot tests
A general-purpose AI model can plan a task in words, but it works blind in two ways. It cannot see while the arm is moving, and it cannot see while it is thinking, which can take many seconds. The team’s system, Reins, gives the model 18 tools for a pair of small arms. Some find an object and measure its position to within about 1.6 millimeters. Others move until the motors feel contact. The model can string up to eight actions together and attach a condition to each, so the sequence stops at the first step that goes wrong. A second copy of the model, the narrator, watches one camera frame per second and reports what changed.
With plain-language instructions and no task-specific code, OpenAI’s GPT-6.1 Sol succeeded in 22 of 25 attempts: all five at the mug game, opening and filling a drawer, and sweeping objects into a dustpan, four of five at folding a towel, and three of five at folding a shirt. Without the narrator it won the mug game twice in five tries. Given only raw images and basic moves, it never lifted the right mug. Two other models, GPT-6 Astra and Claude Opus 5.5, also opened the drawer every time. In the mug game Astra won four of five and Opus two, and the authors count both of Opus’s wins as guesses.
Five attempts per condition is a small sample, on one robot, with the operator judging success. Sol ran on a faster service tier than the other two models, so part of the gap in the mug game reflects speed of response. A shirt took a median of 15.5 minutes and about $15.51 in model fees. The authors say code will be released on publication.
Write the program once, then step away
The ACE-CaP Team at ACE Robotics, with three universities · November 12 preprint · Physical robot tests
Another way to use a general-purpose model avoids the thinking delay. An AI coding agent, here Claude Opus 5.5, writes a control program for a two-armed robot, runs it, studies what went wrong and revises it. While it works it can ask a person to physically guide the arms through the task, and it can try ideas in simulation. The finished program is then frozen and runs on its own, using its cameras and control software without further calls to the coding agent.
Across eight tasks, each run 30 times, the frozen programs succeeded in 221 of 240 trials. Plugging in a charger worked every time, and packing bags of bread into a box was weakest at 25 of 30. Average run times ranged from about one minute to a little over two. During development the agent was allowed at most ten scene resets by a person per task.
Each program came from a single development run, so the paper cannot say whether a second run would produce one as reliable. Resets needed to record demonstrations were not counted against the budget of ten. The authors list testing the programs under new conditions as future work.
A humanoid estimates how hard it is pressing
Liangjun Xing, Ke Fan, Kailin Li and colleagues at Shanghai AI Laboratory and four universities · November 12 preprint · Physical humanoid tests
A humanoid carrying a box has to squeeze hard enough to hold it and no harder. Force sensors at the wrists would measure this directly, and many humanoids lack them. FORGE, a controller for the Unitree G1, estimates the force at each wrist from the arm’s joint positions and motor commands. It starts with a physics formula and adds a correction learned in simulation, then shifts the hand’s target position to keep the push near a requested level.
Checked against a handheld force gauge at loads up to 20 newtons, about the weight of two kilograms, the combined estimate was off by 0.8 to 2.0 newtons depending on direction. The formula alone was off by 2.8 to 5.2. With a camera-driven policy sending commands to FORGE, the robot squatted, lifted and carried a box in 20 of 20 trials when it was empty or held 0.5 kilograms, and in 18 of 20 at one and two kilograms.
The estimate covers slow, steady forces only. At heavier loads of 35 to 50 newtons its error grew to between 8 and 12 newtons. The public repository has training and evaluation scripts and a trained model for one stage, and it marks the scripts for running on a physical robot as not yet released.
Thirty-six claims, one €1,500 arm
Ravindu Somawansa at Shisso Robotics · November 13 preprint on alphaXiv · One physical arm, with many checks run on recordings
Of the 21 papers Ravindu Somawansa selected, 16 used simulation or equipment costing more than $20,000. This author asked which claims survive on a hobbyist SO-101 arm and a small onboard computer, about €1,500 in total, with 34 minutes of recorded practice at putting one bowl inside another. For each claim he wrote down a pass mark before testing, and checked a simple alternative where one was available. Eleven claims had such a comparison.
Of 36 claims from 21 papers, 12 held, 18 failed and six were left undecided or untested. All three general robot models he tried could do the bowl task, two of them in nine of ten attempts. All eight methods meant to spot a failing attempt fell short. The cheap alternatives decided several verdicts: one model predicted the arm’s next state well, yet a check that only compared commanded joint angles beat it at identifying the movement the arm had made.
The author says a failure here does not mean a paper is wrong. Seven of the 18 failures involved his own variant of a method, his software, or a task outside what the paper covered. The task lasts about 11 seconds, only eight of 50 test episodes were failures, and he judged every outcome himself. This is one person’s unreviewed work. The code, data and each number’s source are public.
News
GXO and Apptronik set conditions for 1,000 humanoids
GXO Logistics · November 12 announcement · Letter of intent, with trials planned for 2027
The logistics company GXO signed a letter of intent with Apptronik to expand trials of the Apollo humanoid in working warehouses, covering picking, moving cases and totes, packing and sorting. Trials begin in 2027. If they meet agreed productivity and safety targets, the companies intend to roll out up to 1,000 robots over four years, starting in North America in late 2027. GXO would pay for the robots as a service.
A letter of intent is not an order, and the release gives no results from the pilot that finished earlier this year, which involved picking small consumer goods.
OpenAI folds its robotics team into other groups
Fortune report · November 13 · Reorganization confirmed by the company
OpenAI confirmed to Fortune that it had reassigned its robotics researchers to its main research group and its applied robotics and hardware staff to its applied organization.
Read this first
Watch the shell-game video on the Reins project page. Follow one mug through the shuffle, then look at the timestamps on the narrator’s updates. The arm needs a record of where the mug went during the seconds its controller spent deciding what to do.
Read the paper