Sabine
patreon
AI makes stunning progress in logical reasoning
🕑 Added 2024-01-30 16:00:10 +0000 UTC[This is a transcript with links to references.]
Google has unveiled a new artificially intelligent system, AlphaGeometry, that can solve problems of mathematical geometry. It’s the first computer program to surpass the average performance of participants at the International Mathematical Olympiad. That might sound like an incremental improvement, just one more thing that AI is really good at, but mathematics isn’t just one more thing, it’s everywhere. This makes Google’s recent development a significant step forward. Let’s have a look.
This new research was done by scientists at Google DeepMind and Google Research and just published in Nature. They tested their new program AlphaGeometry with a set of problems posed at the international mathematical Olympiad during the years 2000 to 2022.
After the program was trained, it was given 30 Olympiad geometry problems. The AI solved 25 of them correctly within the standard Olympiad time limit. This performance far surpassed the capabilities of the previous state-of-the-art system, which could only solve 10 of these geometry problems. To put this into perspective, the average participant at these Olympiads solves about 15 problems correctly. A gold medallist typically solves almost 26. So, the new AI is better than the average, but not quite beating the smartest of the smart humans.
Clearly this AI is doing very well, but how does it work? AlphaGeometry uses what’s called a neuro-symbolic approach. That means it combines a neural language model,like ChatGPT, with symbolic deduction like that used by software like Mathematica.
The neural language models are good at identifying general patterns and relationships in data. This makes it possible to quickly come up with potentially useful ideas. These symbolic deduction programs on the other hand basically allow to infer logical relationships. The combination of both makes for a very, very powerful system. It’s indeed much more similar to how the human brain works than just neural networks because it combines "intuitive" ideas that we extract from input, with more deliberate, rational decision-making. Similar to Kahneman’s system 1 and system 2 thinking.
It’s like basically AlphaGeometry has what ChatGPT is missing, it understands logic.
One of the problems that has so far prevented AI from becoming good at maths in general or geometry in particular is the lack of training data. There are only so many proofs that humans have written down that you can train them on. The Google researchers have addressed this by first generating a vast pool of synthetic proofs that added up to as much as 100 million examples. Thanks to this, AlphaGeometry could train without relying on human demonstrations.
Better still, AlphaGeometry doesn’t just spit out a result, it delivers a human-readable step-by-step proof. Basically it works by trying to find a sequence of steps that logically fit together and that also solves the problem. Though it isn’t irrelevant to mention that most of the proofs it found were considerably longer than those from humans.
But the significance of AlphaGeometry's achievement goes beyond solving geometry problems. It generally highlights AI's growing ability to reason logically, discover new knowledge, and verify solutions. And not only this, it can also explain how it arrived at the conclusion. Such a type of AI system has uses that extend way beyond just geometry. Not only can this achievement be generalized across various mathematical domains, but it will without doubt also come in handy in other areas where rational thinking and logical deduction is of use. Like, eh, everything.
Now throw in a bunch of data in addition to the reasoning skills and you have the perfect scientist, unbiased, clearheaded, able to digest huge amounts of data and draw logical conclusions from it.
That AlphaGeometry is able to explain how it arrived at a conclusion should also help alleviate the fears of people who think that AI will be a black box that no human can follow. You see, the thing with mathematical proofs or arguments in general is that it’s difficult to come up with correct chains of reasoning, but once you have the proof it’s much simpler to check whether its correct. Though simpler doesn’t mean simple…
These developments in AI also raise an interesting philosophical question, which is whether there is anything humans can do which AI will not eventually also be able to do. At the moment, the major protection for jobs from being taken over by AI is that a lot of human labour requires sensor input or physical skills that robots don’t have, or AI can’t cope with. Yet. Because robots are developing incredibly quickly, and if you combine this development with AI, there really won’t be much left for us to do.
Maybe we’ll all just make YouTube videos? Well, actually, Google is also working on a text-to-video system that isn’t yet publicly available but from which they shared some amazing examples last week. So soon enough I can replace myself with an AI which will make total sense because all my viewers will be robots.
Comments
Good questions. I think the gist is that the program will produce whatever is conventionalized, not more. It could probably sort a Copernican and Ptolemaic, but having it choose? Seems like it could be good for engineering ideas. Typically necessity seems to drive invention and then choices, and necessity is more of a value. So its training would mean learning value and value comparisons and applying them. Very metaphysical. BTW Thanks for your helping me with E and M!
How is this program in the finding of new solutions? If one would input the position and the motion of our planets, would it find the Ptolemaic solution or the Copernican solution for the rules of this system? And if this system has to be trained (which is necessary for an AI system), which training would be done to enable it for such task?
Yes, the phone answering bots are pretty dumb and can't answer many questions, so they should definitely be improved. On the other hand, maybe we need to change our paradigm of what things humans should be doing, even if robots can do them better.
There are things we can do almost instantly, but a normal (von Neumann) computer can't do, unless it is first coded to compute the relevant equations describing the problem. For example, figuring out if an object's surface is concave or convex, or parts of it are one or the other, for anyone who knows what "concave" or "convex" means, a simple quick visual inspection is enough. But not with a regular computer. So this new type of robot Sabine has described is really a big step forward towards something that is *potentially* very useful. While I must confess of being weary of what powerful people and groups could do to a society if they had really intelligent robots at their beck and call, I do think that much can be harmlessly improved in the robots that, post-Covid, seem to have replaced most human telephone operators. Shouting "Representative" several times in a row or, sometimes punching "0" repeatedly is, in the end, the only way to get to an actual person, at least here in the USA. While getting seriously annoyed in the process.
