Rubik's Cube Solving Robot Hand Sparks Debate in the AI Community

What does it mean to actually “solve” a puzzle? Can it be a purely mechanical process? Or does there need to be cognition involved? This is the question at the core of a debate that has raged on social media regarding a recent breakthrough by OpenAI.

Design News recently reported on a paper released by OpenAI, one of the more high-profile firms in the AI research space. In their paper, OpenAI researchers describe how they were able to use neural networks to enable a robot hand to solve a Rubik's Cube. OpenAI hopes that this research will be a step down the long road toward creating AI and robots that can perform more generalized tasks over purpose-built applications.

The research gained a lot of attention, but has also sparked some controversy on social media as engineers and experts in the artificial intelligence space are debating the true significance of OpenAI's achievement.

The debate seems to have been set off by a series of tweets from Gary Marcus, a cognitive scientist and author of the recent book, Rebooting AI: Building Artificial Intelligence We Can Trust. Marcus, a self-proclaimed contrarian when it comes to artificial intelligence, criticized OpenAI for overstating the claims made of its Rubik's Cube solving robot.

“Since @OpenAI still has not changed misleading blog post about 'solving the Rubik's cube', I attach detailed analysis, comparing what they say and imply with what they actually did. IMHO most would not be obvious to nonexperts. Please zoom in to read & judge for yourself,”[sic] Marcus tweeted.

Marcus' chief criticism is that the hand was using an algorithm that was already taught to solve the cube, rather than actually figuring out how to solve the cube on its own. He also levied criticisms at the low success rate of the robot hand (20% for the most difficult cubes), the fact that the object manipulation was not generalized to other objects (the ultimate aim of this particular research by OpenAI), and that both the robot hand and the Rubik's Cube used in the tests were augmented.

“I will say again that the work itself is impressive, but mischaracterized, and that a better title would have been 'manipulating a Rubik's cube using reinforcement learning' or 'progress in manipulation with dextrous robotic hands" or similar lines,” [sic] Marcus wrote in a follow up tweet.

Others on Twitter agreed with Marcus' assessment, leading to a response from Wojciech Zaremba, OpenAI's co-founder and Head of Robotics:

Zaremba also followed up with a thread addressing each of Marcus' concerns and also addressed criticisms and questions from other Twitter users.

In regards to the Rubik's Cube itself being instrumented (equipped with sensors) for example, Zaremba wrote, “We have results without instrumentation, but the results are weaker. The current pipeline is limited by vision state estimation capabilities, which should be improvable.” [sic] Marcus went on to claim that, “In the fully unscrambled cube condition, with an uninstrumented cube (ie no sensors), success rate was zero.” [sic] To which Zaremba simply responded, “Not true Gary.”

The debate spilled over into other social media, onto thread on the /r/MachineLearning subreddit on Reddit.

“Gary's summary is much more misleading than the blog post,” one Reddit user said in the most-upvoted comment on the thread.

What critics are taking the most issue with is the phrasing of OpenAI's paper. Terms like “solved” and “trained” to some would seem to imply the OpenAI team created a system much smarter and more capable than they actually did. There is also the lack of a vision component to the system (which OpenAI does address in its research paper).

“Isn't vision (or state estimation by vision) fundamental part of manipulation? I guess with Bluetooth instrumentation OpenAI showed manipulation 'would have worked' if vision was working. But they couldn't get vision working,” another Reddit user commented.

RELATED ARTICLES:

OpenAI has not made claims that it was able to create a robot or neural network smart enough to solve a Rubik's Cube on its own without any training. What the company did was teach an AI the rules and methods of solving a cube via simulation and port that capability into a physical device (the robot hand). The true advancement is less about the Rubik's Cube itself and more about the ability to move trained AI for more generalized tasks into physical robots and devices.

As the OpenAI researchers state in the conclusion to their paper:

Purpose-made robots have already been able to solve Rubik's Cubes in astounding times. MIT created a robot in 2018 that can solve a cube in 0.38 seconds. But to be able to move the task over to a more generalized, multipurpose machine (like a hand) is where the true potential lies.

However, one can argue it would be easy for overzealous groups or individuals to assume OpenAI had created some sort of sophisticated robot smart enough to solve a Rubik's Cube.

Social media drama aside, this does serve as a lesson for all of us to remain sober and realistic about the capabilities of AI. There is truly impressive work being done today, but we shouldn't expect science fiction to become reality any time soon.

Chris Wiltz is a Senior Editor at  Design News covering emerging technologies including AI, VR/AR, blockchain, and robotics.