PUBLIC MARKS with tags "blog post" & algorithms

Black and White 2 AI I played Black and White 2 for many hours yesterday. The computer player and I were in a stalemate. The computer kept sending armies against me and I kept defeating them. I had built my town with walls around it, and then put archers on top of the walls. I was building up my strength while defending myself, in preparation for a big attack. I felt pretty safe. After around 40 attacks, I realized that they weren't all the same. The computer wasn't using the same attackers each time. It tried the creature, archers, swordsmen, and catapults. It tried combinations of them. Sometimes it would come through my main entrance, and sometimes it would come around the back entrance to the city. The computer player also destroyed major sections of the city using the “earthquake” power, but I recovered from these too. After a while the enemy creature figured out that he should kick my wall in. His archers and swordsmen stayed back, out of range, while the creature came up and destroyed my wall, including the archers on it. After it breached the wall, the army swarmed into my town and killed half my people. I rebuilt my wall and started to recover, but the computer's newly discovered strategy worked well. It tried several variants but kept going back to the same approach: kick down the wall, then swarm the town. This forced me to try some new strategies. Although being on the wall has advantages, it leaves the archers vulnerable when the enemy creature attacks the wall. So I moved them behind the wall. I've also learned to open my gate, wait for the enemy army to get close, then close the gate and set their army on fire. I have no good strategy for the creature knocking down my wall though, and I'm constantly losing townspeople and then rebuilding. After a long stalemate, the computer AI learned how to attack more effectively, and now I'm having trouble keeping my city safe. I'm very impressed by the AI. I'm not sure how it's programmed, but it tried out many different things and learned which ones work the best. From the game AI techniques I've learned (genetic algorithms, neural networks, fuzzy logic, state machines, etc.), the AI in Black and White 2 seems to match most closely with what I know about reinforcement learning. It's a technique that uses online learning (observing results as the game is played) instead of training (from examples constructed ahead of time), allows both exploration (trying new things in order to learn) and exploitation (taking advantage of what you've learned), and associates rewards (like whether the attack was successful) with actions (like kicking down the wall and keeping the army away from my archers). I recommend Sutton and Barto's book if you want to learn more. It's entirely possible though that the game uses something much simpler that just happens to look impressive, but my guess is that it's using reinforcement learning. — Amit — Monday, December 12, 2005 Comments: Post a Comment Links to this post: