An Artificial Intelligence System Just Took the Math SAT

Artificial Intelligence is gaining on us. An A.I. system called GeoS just took some Math SAT tests, and correctly answered 49% of geometry questions from actual tests and 61% of practice questions—which is about average for an eleventh grader. Though computers have been performing complex math problems for years, this experiment was unique because the system was not only expected to solve equations, but to read and understand the questions themselves. 

The software, developed by the Allen Institute for Artificial Intelligence (AI2) and the University of Washington, employed machine vision. This allowed the system to read problems and look at diagrams.

Ali Farhadi, who is the research manager at AI2, told The Washington Post, “Our biggest challenge was converting the question to a computer-understandable language.” That is, the most difficult part of the test for GeoS was reading comprehension: once the system figured out what a math problem was asking, finding the answer was simple. 

Developing a computer capable of understanding and interpreting human language—especially language as complex as a geometry word problem—is no easy task. According to The New York Times, many of the most advanced A.I. systems still struggle to grasp the meaning of words and symbols (like understanding what an arrow represents in the context of a diagram). So while A.I. is certainly becoming more sophisticated, it’s still far from achieving anything close to human intelligence. GeoS may be able to complete the Math SAT at a high school level, but it’s not exactly thinking like a teenager.