Mediaboss Marketing

By now you've heard about the new kid on the chess-engine block, AlphaZero, and its crushing match win vs Stockfish, the strongest open-source chess engine.

The reactions from the chess community to this match ranged from admiration to utter disbelief.

But how does AlphaZero actually work?

How is it different from other engines and why is it so much better? In this two-part article Ill try to explain a bit of what goes on under AlphaZeros hood.

First, lets reflect on what happened. AlphaZero was developed by DeepMind (a Google-owned company) to specialize in learning how to play two-player, alternate-move games. It was primed with the rules of chess, and nothing else.

It then started learning chess by playing games against itself. Game one would have involved totally random moves. At the end of this game, AlphaZero had learned that the losing side had done stuff that wasnt all that smart, and that the winning side had played better. AlphaZero had taught itself its first chess lesson. The quality of chess in game two was a just a tiny bit better than the first.

Nine hours and 44 million games of split-personality chess later, AlphaZero had (very possibly) taught itself enough to become the greatest chess player, silicon- or carbon-based, of all time.

How on earth did it do it?

Google headquarters in London from inside, with the DeepMind section on the eighth floor. | Photo: Maria Emelianova/Chess.com.

It didnt calculate more variations than Stockfish.

Quite the opposite in fact: Stockfish examined 70 million positions per second while AlphaZero contented itself with about 0.1 percent of that: 80,000 per second. This brings to mind a remark made by Jonathan Rowson after Michael Adams crushed him in a match in 1998: I was amazed at how little he saw.

Stronger players tend to calculate fewer variations than weaker ones. Instead their highly-honed intuition guides them to focus their calculation on the most relevant lines. This is exactly what AlphaZero did. It taught itself chess in quite a human-like way, developing an intuition like no other chess machine has ever done, and it combined this with an amount of cold calculation.

Lets see how it did that.

IM Danny Rensch explains the AlphaZero match in a series of videos on Twitch.

The Analysis Tree

Chess engines use a tree-like structure to calculate variations, and use an evaluation function to assign the position at the end of a variation a value like +1.5 (Whites advantage is worth a pawn and a half) or -9.0 (Blacks advantage is worth a queen). AlphaZeros approach to both calculating variations and evaluating positions is radically different to what other engines do.

All popular chess engines are based on the minimax algorithm, which is a fancy name that simply means you pick the move that gives you the biggest advantage regardless of what the opponent plays. Minimax is invariably enhanced with alpha-beta pruning, which is used to reduce the size of the tree of variations to be examined. Heres an extreme example of how this pruning works: Say an engine is considering a move and sees its opponent has 20 feasible replies. One of those replies leads to a forced checkmate. Then the engine can abandon (or cutoff) the move it was considering, no matter how well it would stand after any of the other 19 replies.

Another issue is that if an engine prunes away moves that only seem bad, e.g. those that lose material, it will fail to consider any kind of sacrifice, which is partly why early engines were so materialistic. In current engines like Stockfish, alpha-beta pruning is combined with a range of other chess-specific enhancements such the killer-move heuristic (a strong move in another similar variation is likely to be strong here), counter-move heuristic (some moves have natural responses regardless of position I bet youve often met axb5 with axb5, right?) and many others.

AlphaZero, in contrast, uses Monte Carlo Tree Search, or MCTS for short. Monte Carlo is famous for its casinos, so when you see this term in a computing context it means theres something random going on. An engine using pure MCTS would evaluate a position by generating a number of move sequences (called playouts) from that position randomly, and averaging the final scores (win/draw/loss) that they yield. This approach may seem altogether too simple, but if you think about it youll realize its actually quite a plausible way of evaluating a position.

The Monte Carlo Casino.

AlphaZero creates a number of playouts on each move (800 during its training). It also augments pure MCTS by preferring moves that it has not tried (much) already, that seem probable and that seem to lead to good positions, where good means that the evaluation function (more on this next article) gives them a high value. Its really creating semi-random playouts, lines that seem appropriate to its ever-improving evaluation function. Isnt this quite like how you calculate? By focussing on plausible lines of play?

Notice that so far theres absolutely nothing chess-specific in what AlphaZero is doing. In my next article, when we look at how AlphaZero learns to evaluate chess positions, well see theres absolutely nothing chess-specific there either!

Like a newborn baby, AlphaZero came into the world with little knowledge, but is massively geared to learn. One weakness of MCTS is that since its based on creating semi-random playouts, it can get it completely wrong in tense positions where there is one precise line of optimal play. If it doesnt randomly select this line, it is likely to blunder. This blindness was probably what caused AlphaZeros Go playing predecessor, AlphaGo, to lose a game to 18-time world Go champion Lee Sedol. It seems not to have been an issue in the match with Stockfish, however.

MCTS has been used previously for two-player gameplay, but was found to perform much worse than the well-established minimax plus alpha-beta approach. In AlphaZero, MCTS combines really well with the employed neural network-based evaluation function.

In my next article, Ill explain more about this neural network and especially the fascinating way it learns, on its own, how to evaluate chess positions. Ill also describe the hardware AlphaZero runs on, and make some predictions about how all this will impact chess as we know it.

What do you think about how AlphaZero plays chess? Let us know in the comments.

Corrections: AlphaZero creates a number of playouts on each move, not 800. That was during training.

More:
How Does AlphaZero Play Chess? - Chess.com

Elizabeth Spelke, a cognitive psychologist at Harvard, has spent her career testing the worlds most sophisticated learning systemthe mind of a baby.

Gurgling infants might seem like no match for artificial intelligence. They are terrible at labeling images, hopeless at mining text, and awful at videogames. Then again, babies can do things beyond the reach of any AI. By just a few months old, they've begun to grasp the foundations of language, such as grammar. They've started to understand how the physical world works, how to adapt to unfamiliar situations.

Yet even experts like Spelke don't understand precisely how babiesor adults, for that matterlearn. That gap points to a puzzle at the heart of modern artificial intelligence: We're not sure what to aim for.

Consider one of the most impressive examples of AI, AlphaZero, a program that plays board games with superhuman skill. After playing thousands of games against itself at hyperspeed, and learning from winning positions, AlphaZero independently discovered several famous chess strategies and even invented new ones. It certainly seems like a machine eclipsing human cognitive abilities. But AlphaZero needs to play millions more games than a person during practice to learn a game. Most tellingly, it cannot take what it has learned from the game and apply it to another area.

To some members of the AI priesthood, that calls for a new approach. What makes human intelligence special is its adaptabilityits power to generalize to never-seen-before situations, says Franois Chollet, a well-known AI engineer and the creator of Keras, a widely used framework for deep learning. In a November research paper, he argued that it's misguided to measure machine intelligence solely according to its skills at specific tasks. Humans don't start out with skills; they start out with a broad ability to acquire new skills, he says. What a strong human chess player is demonstrating isn't the ability to play chess per se, but the potential to acquire any task of a similar difficulty. That's a very different capability.

Chollet posed a set of problems designed to test an AI program's ability to learn in a more generalized way. Each problem requires arranging colored squares on a grid based on just a few prior examples. It's not hard for a person. But modern machine-learning programstrained on huge amounts of datacannot learn from so few examples. As of late April, more than 650 teams had signed up to tackle the challenge; the best AI systems were getting about 12 percent correct.

A self-driving car cannot intuit from common sense what will happen if a truck spills its load.

It isn't yet clear how humans solve these problems, but Spelke's work offers a few clues. For one thing, it suggests that humans are born with an innate ability to quickly learn certain things, like what a smile means or what happens when you drop something. It also suggests we learn a lot from each other. One recent experiment showed that 3-month-olds appear puzzled when someone grabs a ball in an inefficient way, suggesting that they already appreciate that people cause changes in their environment. Even the most sophisticated and powerful AI systems on the market can't grasp such concepts. A self-driving car, for instance, cannot intuit from common sense what will happen if a truck spills its load.

Josh Tenenbaum, a professor in MIT's Center for Brains, Minds & Machines, works closely with Spelke and uses insights from cognitive science as inspiration for his programs. He says much of modern AI misses the bigger picture, likening it to a Victorian-era satire about a two-dimensional world inhabited by simple geometrical people. We're sort of exploring Flatlandonly some dimensions of basic intelligence, he says. Tenenbaum believes that, just as evolution has given the human brain certain capabilities, AI programs will need a basic understanding of physics and psychology in order to acquire and use knowledge as efficiently as a baby. And to apply this knowledge to new situations, he says, they'll need to learn in new waysfor example, by drawing causal inferences rather than simply finding patterns. At some pointyou know, if you're intelligentyou realize maybe there's something else out there, he says.

This article appears in the June issue. Subscribe now.

Let us know what you think about this article. Submit a letter to the editor at mail@wired.com.

Special Series: The Future of Thinking Machines

Original post:
It's Called Artificial Intelligencebut What Is Intelligence? - WIRED

Game-playing artificial intelligence

AlphaZero is a computer program developed by artificial intelligence research company DeepMind to master the games of chess, shogi and go. This algorithm uses an approach similar to AlphaGo Zero.

On December 5, 2017, the DeepMind team released a preprint introducing AlphaZero, which within 24 hours of training achieved a superhuman level of play in these three games by defeating world-champion programs Stockfish, elmo, and the 3-day version of AlphaGo Zero. In each case it made use of custom tensor processing units (TPUs) that the Google programs were optimized to use.[1] AlphaZero was trained solely via "self-play" using 5,000 first-generation TPUs to generate the games and 64 second-generation TPUs to train the neural networks, all in parallel, with no access to opening books or endgame tables. After four hours of training, DeepMind estimated AlphaZero was playing at a higher Elo rating than Stockfish 8; after 9 hours of training, the algorithm defeated Stockfish 8 in a time-controlled 100-game tournament (28 wins, 0 losses, and 72 draws).[1][2][3] The trained algorithm played on a single machine with four TPUs.

DeepMind's paper on AlphaZero was published in the journal Science on 7 December 2018.[4] In 2019 DeepMind published a new paper detailing MuZero, a new algorithm able to generalise on AlphaZero work playing both Atari and board games without knowledge of the rules or representations of the game.[5]

AlphaZero (AZ) is a more generalized variant of the AlphaGo Zero (AGZ) algorithm, and is able to play shogi and chess as well as Go. Differences between AZ and AGZ include:[1]

Comparing Monte Carlo tree search searches, AlphaZero searches just 80,000 positions per second in chess and 40,000 in shogi, compared to 70 million for Stockfish and 35 million for elmo. AlphaZero compensates for the lower number of evaluations by using its deep neural network to focus much more selectively on the most promising variation.[1]

AlphaZero was trained solely via self-play, using 5,000 first-generation TPUs to generate the games and 64 second-generation TPUs to train the neural networks. In parallel, the in-training AlphaZero was periodically matched against its benchmark (Stockfish, elmo, or AlphaGo Zero) in brief one-second-per-move games to determine how well the training was progressing. DeepMind judged that AlphaZero's performance exceeded the benchmark after around four hours of training for Stockfish, two hours for elmo, and eight hours for AlphaGo Zero.[1]

In AlphaZero's chess tournament against Stockfish 8 (2016 TCEC world champion), each program was given one minute per move. Stockfish was allocated 64 threads and a hash size of 1 GB,[1] a setting that Stockfish's Tord Romstad later criticized as suboptimal.[6][note 1] AlphaZero was trained on chess for a total of nine hours before the tournament. During the tournament, AlphaZero ran on a single machine with four application-specific TPUs. In 100 games from the normal starting position, AlphaZero won 25 games as White, won 3 as Black, and drew the remaining 72.[8] In a series of twelve, 100-game matches (of unspecified time or resource constraints) against Stockfish starting from the 12 most popular human openings, AlphaZero won 290, drew 886 and lost 24.[1]

AlphaZero was trained on shogi for a total of two hours before the tournament. In 100 shogi games against elmo (World Computer Shogi Championship 27 summer 2017 tournament version with YaneuraOu 4.73 search), AlphaZero won 90 times, lost 8 times and drew twice.[8] As in the chess games, each program got one minute per move, and elmo was given 64 threads and a hash size of 1GB.[1]

After 34 hours of self-learning of Go and against AlphaGo Zero, AlphaZero won 60 games and lost 40.[1][8]

DeepMind stated in its preprint, "The game of chess represented the pinnacle of AI research over several decades. State-of-the-art programs are based on powerful engines that search many millions of positions, leveraging handcrafted domain expertise and sophisticated domain adaptations. AlphaZero is a generic reinforcement learning algorithm originally devised for the game of go that achieved superior results within a few hours, searching a thousand times fewer positions, given no domain knowledge except the rules."[1] DeepMind's Demis Hassabis, a chess player himself, called AlphaZero's play style "alien": It sometimes wins by offering counterintuitive sacrifices, like offering up a queen and bishop to exploit a positional advantage. "It's like chess from another dimension."[9]

Given the difficulty in chess of forcing a win against a strong opponent, the +28 0 =72 result is a significant margin of victory. However, some grandmasters, such as Hikaru Nakamura and Komodo developer Larry Kaufman, downplayed AlphaZero's victory, arguing that the match would have been closer if the programs had access to an opening database (since Stockfish was optimized for that scenario).[10] Romstad additionally pointed out that Stockfish is not optimized for rigidly fixed-time moves and the version used is a year old.[6][11]

Similarly, some shogi observers argued that the elmo hash size was too low, that the resignation settings and the "EnteringKingRule" settings (cf. shogi Entering King) may have been inappropriate, and that elmo is already obsolete compared with newer programs.[12][13]

Papers headlined that the chess training took only four hours: "It was managed in little more than the time between breakfast and lunch."[2][14] Wired hyped AlphaZero as "the first multi-skilled AI board-game champ".[15] AI expert Joanna Bryson noted that Google's "knack for good publicity" was putting it in a strong position against challengers. "It's not only about hiring the best programmers. It's also very political, as it helps make Google as strong as possible when negotiating with governments and regulators looking at the AI sector."[8]

Human chess grandmasters generally expressed excitement about AlphaZero. Danish grandmaster Peter Heine Nielsen likened AlphaZero's play to that of a superior alien species.[8] Norwegian grandmaster Jon Ludvig Hammer characterized AlphaZero's play as "insane attacking chess" with profound positional understanding.[2] Former champion Garry Kasparov said "It's a remarkable achievement, even if we should have expected it after AlphaGo."[10][16]

Grandmaster Hikaru Nakamura was less impressed, and stated "I don't necessarily put a lot of credibility in the results simply because my understanding is that AlphaZero is basically using the Google supercomputer and Stockfish doesn't run on that hardware; Stockfish was basically running on what would be my laptop. If you wanna have a match that's comparable you have to have Stockfish running on a supercomputer as well."[7]

Top US correspondence chess player Wolff Morrow was also unimpressed, claiming that AlphaZero would probably not make the semifinals of a fair competition such as TCEC where all engines play on equal hardware. Morrow further stated that although he might not be able to beat AlphaZero if AlphaZero played drawish openings such as the Petroff Defence, AlphaZero would not be able to beat him in a correspondence chess game either.[17]

Motohiro Isozaki, the author of YaneuraOu, noted that although AlphaZero did comprehensively beat elmo, the rating of AlphaZero in shogi stopped growing at a point which is at most 100~200 higher than elmo. This gap is not that high, and elmo and other shogi software should be able to catch up in 12 years.[18]

DeepMind addressed many of the criticisms in their final version of the paper, published in December 2018 in Science.[4] They further clarified that AlphaZero was not running on a supercomputer; it was trained using 5,000 tensor processing units (TPUs), but only ran on four TPUs and a 44-core CPU in its matches.[19]

In the final results, Stockfish version 8 ran under the same conditions as in the TCEC superfinal: 44 CPU cores, Syzygy endgame tablebases, and a 32GB hash size. Instead of a fixed time control of one move per minute, both engines were given 3 hours plus 15 seconds per move to finish the game. In a 1000-game match, AlphaZero won with a score of 155 wins to 6 losses, with the rest drawn. DeepMind also played a series of games using the TCEC opening positions; AlphaZero also won convincingly.

Similar to Stockfish, Elmo ran under the same conditions as in the 2017 CSA championship. The version of Elmo used was WCSC27 in combination with YaneuraOu 2017 Early KPPT 4.79 64AVX2 TOURNAMENT. Elmo operated on the same hardware as Stockfish: 44 CPU cores and a 32GB hash size. AlphaZero won 98.2% of games when playing black (which plays first in shogi) and 91.2% overall.

Human grandmasters were generally impressed with AlphaZero's games against Stockfish.[20] Former world champion Garry Kasparov said it was a pleasure to watch AlphaZero play, especially since its style was open and dynamic like his own.[21][22]

In the chess community, Komodo developer Mark Lefler called it a "pretty amazing achievement", but also pointed out that the data was old, since Stockfish had gained a lot of strength since January 2018 (when Stockfish 8 was released). Fellow developer Larry Kaufman said AlphaZero would probably lose a match against the latest version of Stockfish, Stockfish 10, under Top Chess Engine Championship (TCEC) conditions. Kaufman argued that the only advantage of neural networkbased engines was that they used a GPU, so if there was no regard for power consumption (e.g. in an equal-hardware contest where both engines had access to the same CPU and GPU) then anything the GPU achieved was "free". Based on this, he stated that the strongest engine was likely to be a hybrid with neural networks and standard alphabeta search.[23]

AlphaZero inspired the computer chess community to develop Leela Chess Zero, using the same techniques as AlphaZero. Leela contested several championships against Stockfish, where it showed similar strength.[24]

In 2019 DeepMind published MuZero, a unified system that played excellent chess, shogi, and go, as well as games in the Atari Learning Environment, without being pre-programmed with their rules.[25][26]

The match results by themselves are not particularly meaningful because of the rather strange choice of time controls and Stockfish parameter settings: The games were played at a fixed time of 1 minute/move, which means that Stockfish has no use of its time management heuristics (lot of effort has been put into making Stockfish identify critical points in the game and decide when to spend some extra time on a move; at a fixed time per move, the strength will suffer significantly). The version of Stockfish used is one year old, was playing with far more search threads than has ever received any significant amount of testing, and had way too small hash tables for the number of threads. I believe the percentage of draws would have been much higher in a match with more normal conditions.[7]

The rest is here:
AlphaZero - Wikipedia

Photo by Luiz Hanfilaque on Unsplash

Modern civilization and the trappings of technology has lead to the decline of our own intuition. Many of us have become unaware of its value or even its very existence. Intuition as a basis of complex computation is easily dismissed as an approach outside of the conventional. This lack of conventionality leads many researchers to ignore its potential.

The intuitive mind is a sacred gift and the rational mind is a faithful servant. We have created a society that honors the servant and has forgotten the gift. Albert Einstein

The research that I do in Artificial Intelligence (AI) revolves around the idea that advanced cognitive machines will use intuition as the substrate of its intelligence (see: artificial intuition). Our own human minds provide ample evidence for general intelligence. Humans are fundamentally intuition machines and our rational (and conscious) self are just a simulation layered on top of intuition-based machinery (see: cognitive stack). This is in stark contrast to Descartes famous saying I think. therefore I am (Cogito ergo sum), which implies that our rational thinking is what separates us from all of biology. We thus have a cognitive bias to demand technologies and methodologies that are driven by logical machinery. This is indeed the reason for multi-decade failure of Good Old Fashioned AI (GOFAI) which attempted to solve the problem of intelligence from formal logic as its starting point.

One of the counter-intuitive predictions of intuition based machines is how can logical thought arise from intuition machines? Since 2012, we have seen the incredible advances of Deep Learning technology. Deep Learning networks are intuition machines. These systems learn to perform inference (or make predictions) by using induction. Deep Learning systems have been able to perform tasks that are usually reserved for biological brains. Tasks that have known to be difficult for conventional computing, such as facial and speech recognition, can be performed at super human levels by these machines.

Deep Learning networks however are incapable of performing logical tasks such as long division. One should not expect to be able to teach an animal (i.e. your dog) to perform multiplication much less addition or subtraction. However, human brains are able to perform all sorts of logical problems. We have to ask though, can a caveman be able to do multiplication? Are we innately capable of advanced logical cognition or is this capability something we learned as a consequence of our advanced civilization?

The big chasm that needs to be crossed to achieve more general artificial intelligence is what is known as the semantic gap. How do we fuse the capabilities of Deep Learning (sub-symbolic) system with logical (symbolic) systems?

Human minds are capable of performing great feats of logical reasoning. How are our minds able to do this if our machinery is all intuition based? I am going to make the assumption here that we dont have any innate logical machinery. It is unlikely that Homo sapiens have evolved this cognitive machinery in the short time weve existed in this planet. Therefore, to bridge the semantic gap, we need to bridge it using intuition only mechanisms. What this means is that we dont need to perform a fusion of logical components with intuition components. All we ever need is intuition components.

Therefore we need to show ample evidence that complex logical thinking can be performed by an intuition machine.

This is where AlphaZero makes its revolutionary revelation. AlphaZero is the latest evolution of DeepMindss Go play program. I have written previously about AlphaGo Zero (different from AlphaZero) and how it was able to learn to master the game of Go from scratch (without human knowledge). 99% of Westerners have never played the game of Go and simply dont understand it at all. So the relevance of DeepMinds AlphaGo Zero achievement has been muted. We dont understand the enormity of the achievement. Go however has been known to be a game of intuition. So its somewhat (ignorantly) unsurprising that an intuition machine (one based on Deep Learning) is able to master the game.

However, what DeepMinds new incarnation (AlphaZero) is able to do is play the game of chess. This of course may not be surprising to many since the game of chess has been solved by computer ever since IBMs DeepBlue bested Kasparov in 1996. It may not be remarkable for the uninitiated that it took AlphaZero a few hours to master the game of chess from scratch. It may not be remarkable that AlphaZero was able to destroy the best chess playing program (Stockfish) in 100 games.

What is truly remarkable is how AlphaZero played in dismantling its more logical opponent. To give you an idea, I will quote some impressions from the chess playing community.

It approaches the Type B, human-like approach to machine chess dreamt of by Claude Shannon and Alan Turing instead of brute force. Gary Kasparov.

I always wondered how it would be if a superior species landed on earth and showed us how they play chess. I feel now I know. Peter Heine Nielsen

It doesnt play like a human, and it doesnt play like a program. It plays in a third, almost alien, way. Demis Hassabis (who also plays chess)

For those who understand chess play, its probably best to watch the actual game play of AlphaZero versus Stockfish. What you will see is how an intuition based system dismantles an opponent that is based on logic (that is, one that cant refuse a gambit). Below are games with expert commentary:

AlphaZero plays a very different game of chess. It is willing to sacrifice pieces in order to gain a positional advantage over its opponent. It is playing a kind of chess judo where it uses an opponents eagerness in achieving an immediate gain against itself. It sets up its opponent into what is known in chess as zugzwang, where every move that one makes leads to a worse outcome. It seems to have a more holistic sense of the game of chess where all its pieces move in a highly coordinated manner. AlphaZero plays a game that maximizes its creativeness against a logical opponent that is unable to see beyond short term gains. It plays a game of chess that is not only unimaginable, but would in the past been placed in a pedestal for all to marvel.

The paper about AlphaZero was presented in the recently concluded NIPS 2017 conference. It is an extremely short paper, the main body is only 7 pages long. It provides an interesting detail about how extensively it evaluates the board position to decide on its move.

AlphaZero searches just 80 thousand positions per second in chess, compared to 70 million for Stockfish.

The intuition machine is using 1,000 times less evaluations than the logical opponent.

What you are witnessing here with AlphaZero is validation of my original thesis about intuition machines and their ability to perform logical reasoning. This is the semantic gap being bridged. This is an extremely difficult AGI milestone being surmounted at a record pace. I doubt anyone in the AI community expected this kind of progress to be achieved so quickly. Yet is has happened and the landscape has been changed forever.

Go here to see the original:
AlphaZero: How Intuition Demolished Logic - Intuition ...

Artificial intelligence that plays Go

AlphaGo Zero is a version of DeepMind's Go software AlphaGo. AlphaGo's team published an article in the journal Nature on 19 October 2017, introducing AlphaGo Zero, a version created without using data from human games, and stronger than any previous version.[1] By playing games against itself, AlphaGo Zero surpassed the strength of AlphaGo Lee in three days by winning 100 games to 0, reached the level of AlphaGo Master in 21 days, and exceeded all the old versions in 40 days.[2]

Training artificial intelligence (AI) without datasets derived from human experts has significant implications for the development of AI with superhuman skills because expert data is "often expensive, unreliable or simply unavailable."[3] Demis Hassabis, the co-founder and CEO of DeepMind, said that AlphaGo Zero was so powerful because it was "no longer constrained by the limits of human knowledge".[4] David Silver, one of the first authors of DeepMind's papers published in Nature on AlphaGo, said that it is possible to have generalised AI algorithms by removing the need to learn from humans.[5]

Google later developed AlphaZero, a generalized version of AlphaGo Zero that could play chess and Shgi in addition to Go. In December 2017, AlphaZero beat the 3-day version of AlphaGo Zero by winning 60 games to 40, and with 8 hours of training it outperformed AlphaGo Lee on an Elo scale. AlphaZero also defeated a top chess program (Stockfish) and a top Shgi program (Elmo).[6][7]

AlphaGo Zero's neural network was trained using TensorFlow, with 64 GPU workers and 19 CPU parameter servers.Only four TPUs were used for inference. The neural network initially knew nothing about Go beyond the rules. Unlike earlier versions of AlphaGo, Zero only perceived the board's stones, rather than having some rare human-programmed edge cases to help recognize unusual Go board positions. The AI engaged in reinforcement learning, playing against itself until it could anticipate its own moves and how those moves would affect the game's outcome.[8] In the first three days AlphaGo Zero played 4.9 million games against itself in quick succession.[9] It appeared to develop the skills required to beat top humans within just a few days, whereas the earlier AlphaGo took months of training to achieve the same level.[10]

For comparison, the researchers also trained a version of AlphaGo Zero using human games, AlphaGo Master, and found that it learned more quickly, but actually performed more poorly in the long run.[11] DeepMind submitted its initial findings in a paper to Nature in April 2017, which was then published in October 2017.[1]

The hardware cost for a single AlphaGo Zero system in 2017, including the four TPUs, has been quoted as around $25 million.[12]

According to Hassabis, AlphaGo's algorithms are likely to be of the most benefit to domains that require an intelligent search through an enormous space of possibilities, such as protein folding or accurately simulating chemical reactions.[13] AlphaGo's techniques are probably less useful in domains that are difficult to simulate, such as learning how to drive a car.[14] DeepMind stated in October 2017 that it had already started active work on attempting to use AlphaGo Zero technology for protein folding, and stated it would soon publish new findings.[15][16]

AlphaGo Zero was widely regarded as a significant advance, even when compared with its groundbreaking predecessor, AlphaGo. Oren Etzioni of the Allen Institute for Artificial Intelligence called AlphaGo Zero "a very impressive technical result" in "both their ability to do itand their ability to train the system in 40 days, on four TPUs".[8] The Guardian called it a "major breakthrough for artificial intelligence", citing Eleni Vasilaki of Sheffield University and Tom Mitchell of Carnegie Mellon University, who called it an impressive feat and an outstanding engineering accomplishment" respectively.[14] Mark Pesce of the University of Sydney called AlphaGo Zero "a big technological advance" taking us into "undiscovered territory".[17]

Gary Marcus, a psychologist at New York University, has cautioned that for all we know, AlphaGo may contain "implicit knowledge that the programmers have about how to construct machines to play problems like Go" and will need to be tested in other domains before being sure that its base architecture is effective at much more than playing Go. In contrast, DeepMind is "confident that this approach is generalisable to a large number of domains".[9]

In response to the reports, South Korean Go professional Lee Sedol said, "The previous version of AlphaGo wasnt perfect, and I believe thats why AlphaGo Zero was made."On the potential for AlphaGo's development, Lee said he will have to wait and see but also said it will affect young Go players.Mok Jin-seok, who directs the South Korean national Go team, said the Go world has already been imitating the playing styles of previous versions of AlphaGo and creating new ideas from them, and he is hopeful that new ideas will come out from AlphaGo Zero. Mok also added that general trends in the Go world are now being influenced by AlphaGos playing style. "At first, it was hard to understand and I almost felt like I was playing against an alien. However, having had a great amount of experience, Ive become used to it," Mok said. "We are now past the point where we debate the gap between the capability of AlphaGo and humans. Its now between computers."Mok has reportedly already begun analyzing the playing style of AlphaGo Zero along with players from the national team."Though having watched only a few matches, we received the impression that AlphaGo Zero plays more like a human than its predecessors," Mok said.[18]Chinese Go professional, Ke Jie commented on the remarkable accomplishments of the new program: "A pure self-learning AlphaGo is the strongest. Humans seem redundant in front of its self-improvement."[19]

Future of Go Summit

89:11 against AlphaGo Master

On 5 December 2017, DeepMind team released a preprint on arXiv, introducing AlphaZero, a program using generalized AlphaGo Zero's approach, which achieved within 24 hours a superhuman level of play in chess, shogi, and Go, defeating world-champion programs, Stockfish, Elmo, and 3-day version of AlphaGo Zero in each case.[6]

AlphaZero (AZ) is a more generalized variant of the AlphaGo Zero (AGZ) algorithm, and is able to play shogi and chess as well as Go. Differences between AZ and AGZ include:[6]

An open source program, Leela Zero, based on the ideas from the AlphaGo papers is available. It uses a GPU instead of the TPUs recent versions of AlphaGo rely on.

Read this article:
AlphaGo Zero - Wikipedia