Robert Blincoe, Computing, Tuesday 8 December 2009 at 10:43:00

Robert Blincoe talks to Dr Martin Smith, an artificial intelligence expert and master of online gaming bots

Dr Martin Smith works at the bleeding edge of artificial intelligence (AI) development. It’s the bleeding edge because if he makes a mistake, his company and clients will haemorrhage money.

Smith writes bots to play poker, blackjack, gin rummy, dominoes, backgammon and poker dice for as much as £1,000 a game. He is at the commercial coalface of AI. His bots play 24/7, all comers, all games, simultaneous sessions, multiple opponents, the works.

The UK’s leading bookies are backing Smith’s skills with cold, hard cash, and he is uniquely qualified to handle the responsibility. Smith is a former poker pro, an ex-chess hustler who has ranked in the UK’s top 75 players. He has a PhD in AI, and has spent time as a programmer working on the cutting edge of video gaming. He knows games, he knows gambling, and he knows coding at the top level.

Smith heads up technology for GameAccount Global, and was head-hunted while making a comfortable living, but boring himself stupid, playing online poker. The job appealed. “I have a unique skill set, which isn’t terribly widely usable,” he acknowledges.

GameAccount provides online games of skill and chance for sports book household names, such as Gala Coral, Rank, Stan James, BoyleSports, Sportingbet Europe, William Hill, Betfred and PaddyPower. GameAccount’s punters enter through the virtual front door of their chosen gaming brand, then congregate in a virtual games room lobby and play each other, or bots, for money.

Some 100 web sites feed this big pool of shared customers, which means a lot of cash is in play. All the companies can promise their customers enough opponents, human or bot, to play at their price or skill level, and keep the money flowing. About 1,000 players at a time can crowd it out.

Between January and March 2009, £150m was staked on the GameAccount network, up from £200m for all 2008.

Smith’s day job comes down to a heads-up between him and everyone who takes on his bots on the web sites for which he works. Each morning, straight after he gets out of bed – holidays too – he checks no one has carved them up overnight.

Smith fields a suite of bots. Probot is the strongest, but there are also Rookiebot and Amateurbot which play less-skilled players for lower stakes. The site records players’ performances, so they cannot keep winning against the easy bots. These bots are programmed to play strongly, but to include a statistically modelled number of stupid errors, which mimics human play. Bots are good to learn from as they play well, both strategically and tactically.

The University of Alberta in Canada is the leading academic institution researching AI and games, and has a whole department dedicated to it. Smith’s work, though worthy of several commercially sensitive research papers, is done by himself. He has just been joined by Paul Thomson, a senior AI programmer who worked on the PlayStation for Sony, and also the classic Elixir Studios’ game Republic alongside Smith.

To develop bots at this level, you need AI knowledge, programming experience and an understanding of the strategy of the game you’re planning to win.

Smith says, “I like to think that a large part of the skill that I bring to the job is having a good understanding of the games and selecting the right tools to do that game. You can use neural networks, heuristic search, genetic algorithms, rule-based expert systems – but that’s a bit 80s.”

So can players win against Probot? Yes. All the games Probot plays are a mix of skill and chance. So, if luck is on your side, you can win. Furthermore, Smith spreads himself over a lot of different games, and he is in a commercial environment where new games, new variations and TV game show spin-offs have to be programmed and online under tight schedules. He hasn’t spent the past five years polishing his games, and admits that Probot’s gin rummy needs improvement.

Probot does not play gin rummy at the £1,000 level. Smith says it can beat 99.9 per cent of players, but one player in a thousand, winning £1,000 games, would destroy the money Smith’s bots make playing £1 games. “Beating 99.9 per cent of players in a video game is a triumph,” he says. “But in [the gambling] world it’s an absolute disaster.”

Smith is in an arms race with the smart players. Smith puts up a bot, it wins for a couple of weeks, then the gamers figure out how to beat it and they win for a couple of weeks. Smith runs through the algorithms, probabilistic calculations and search techniques and comes back with a new version, reweighting the various AI techniques he has applied. Bot 2.0 wins for a couple of weeks, and the opposition responds accordingly.

“We have this ratcheting up. It’s very intellectually rewarding,” he says.

Smith also has to consider processing power in his battles with punters. Probot can play 100 different games at the same time, and it has to make its move in about a quarter of a second. It’s not the unlimited processing power of a supercomputer matched with a lengthy decision-making time, where significant AI gaming victories have previously originated.

If all these parameters were changed, Smith would go higher than £1,000 wagers for Probot games. Smith is aware that players make money against Probot. Some even make a living, but his job is to make sure they don’t win large amounts consistently.

Smith was initially brought into GameAccount to detect if rogue bots were being used to take on human backgammon customers. They were. Open-source backgammon software is better than the best humans, so it was obvious people would try a little online hustling, sitting at their PC with champion software running alongside to make their moves. One gaming site, moving from its own backgammon software to GameAccount’s, discovered that its top 14 backgammon players were bots.

“If you want to cheat against GameAccount’s bots,” Smith says, “Bring it on. I rate my bots as pretty decent. We have an automated system which plays over the games and looks for similarities in the play to the way computers play. It’s almost like fingerprinting. Over enough games the only way they could play the moves they have done is with certain bits of software.”

It is not illegal to use a bot, but it is against the sites’ terms and conditions. Smith confiscates winnings before banning the player.

A brief history of computer gaming – from 19th century mechanised chess to super-smart poker bots

1821 Charles Babbage, originator of the concept of a programmable computer, sketches out plans for a chess-playing machine.

1948 British mathematician Alan Turing and American electronic engineer Claude Shannon independently develop the basic algorithms still used in chess programs.

1980 The Othello program The Moor, written by Mike Reeve and David Levy, wins one game in a six-game match against world champion Hiroshi Inoue.

1992 TD-Gammon, developed by IBM’s Gerald Tesauro, plays at a level nearly equal to that of the best human players. The neural learning technique Tesauro developed meant a bot can learn the game by playing hundreds of thousands of games against itself, and master optimal strategy.

1994 Former world champion draughts (checkers) player Marion Tinsley resigns for health reasons in even match against Chinook, a program written by a team from the University of Alberta. Chinook beats grandmaster Don Lafferty the following year in a 32-game match. The final score is 1-0 with 31 draws.

1997 IBM’s Deep Blue beats world champion grandmaster Garry Kasparov at chess. The machine won a six-game match by two wins to one with three draws. Meanwhile, Logistello, written by Michael Buro, defeats the world Othello champion Takeshi Murakami, 6-0.

1998 Scrabble program Maven, developed by Brian Shepherd, beats grandmaster Adam Logan by nine games to five. Average score 417.3 to 388.6.

2008 Poker bots developed by the University of Alberta’s Computer Poker Research Group beat a team of top poker professionals, across a series of matches. The variation of the game was one-against-one limit hold’em, where the betting levels are fixed. Dr Darse Billings, former lead architect of the University of Alberta’s poker software bots, called the result “a milestone for artificial intelligence, and a watershed moment for poker.”

AI bots: is being great at chess a sign of real intelligence?

Are AI bots intelligent? They take on humans, at games which require humans to act with intelligence. But the question takes in psychology, philosophy and computing. Dr Martin Smith’s favourite quote about AI, said in various forms by big names, is that “AI is whatever PCs can’t do yet.”

“Fifty years ago, chess was the acid test for AI. We’ll know we have intelligent machines when we have a program that can beat a grandmaster at chess, they said. Now we have and everyone says that’s not AI, it’s something else,” he says.

Smith’s Probot doesn’t think like a human. Nor did IBM’s Deep Blue when it beat Gary Kasparov in 1997. “In the 70s we thought that to get computers to play chess like a grandmaster they had to think like grandmasters – that turned out to be bollocks. Computers can play fantastic chess and they don’t do anything like what human grandmasters do,” says Smith.

Probot and Deep Blue exploit what computers are good at: carrying out large numbers of simple computations very fast. Smith steers his bots towards trying out a few hundred thousand combinations of moves and picking what works best.

Full published article at: http://feeds.computing.co.uk/c/554/f/10978/s/7a99f89/l/0L0Scomputing0O0Ccomputing0Cfeatures0C22545930Cgaming0Ebets0Efuture0Eai/story01.htm

This entry was posted on Tuesday, December 8th, 2009 at 10:43 am and is filed under Software & Applications. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.