Using Machine Learning to Beat a Video Game

Our brains are always reacting to stimuli and continually adapting those reactions for improved outcomes. In the first years of our lives, this process taught us how to get food, move around efficiently, and learn what was safe to play with. It took a lot of sensory input and repetition before we could eventually use language and carry out more complex tasks. Over time growing up, many of us encountered new, artificial realms where we could learn and improve under different parameters: video games. Now we needed to learn how to run, jump, and escape obstacles through a coded system of pressing buttons and correlating them to what we saw on a screen. By then we'd already had years of practice in cause and effect in real life, so we caught on pretty quickly. And the more we practiced, the better we got.

Machine learning can be set up to essentially do the same thing, learning through trial and error how to adapt to reach a goal. In a video game setting, an artificial intelligence is tasked to make its way through an artificial world---but it's our own human, physical experience that has created the foundations to model this after. By making a system that takes actions and alters them to reach reward outcomes, we can "teach" a machine to beat a video game---or rather, get it to teach itself. Earlier this year we had the privilege of working with Benjamin Pogacar, a high school intern here at Infused Innovations, to take on this challenge.

The Project's Inspiration

A few years ago, YouTuber SethBling posted a video of his neural network machine beating a level of Super Mario World. He named his machine MarI/O. It started out with no knowledge of how the game works---it had to learn the rules itself through a process of neuro-evolution. Specifically, MarI/O was designed using an algorithm called NEAT, or Neuro Evolution of Augmenting Topologies. This allowed it to begin taking simple actions and grow in the connections it made, with its best moves passed on to further generations by "breeding" the top scoring sets. In other words, it used a process just like normal biological evolution.

In the first generation, MarI/O barely knows what to do and spends a good deal of time just standing there. But eventually it makes the connection that its fitness goes up when it moves to the right. The green line in the screenshot above represents an initial "neural" connection, i.e., move to the right. After many generations, this neural network becomes much more complex, like in the picture below.

Ben's Internship: Mario Bros

Rocky Hill High School senior Ben Pogacar was interested in Artificial Intelligence and Machine Learning and wanted to learn more about it in order to decide whether to pursue it further. Having met our CEO Jeff Wilhelm at a Hack-a-thon event, Ben got the idea to recreate SethBling's machine to beat a video game level. An internship was started. Ben worked primarily with Sean Trifero, our resident AI and ML specialist. After some preliminary experiments with Azure Custom Vision, they started the video game challenge that became Ben's senior project. With Sean's guidance, Ben used the same source code SethBling did to make another MarI/O, this time playing Super Mario Bros. He cleaned up the code and made some other adjustments, resulting in a program that he live-streamed for viewers to witness. After over a hundred generations, Ben's MarI/O successfully beat the first level, with an intricate web of neural connections having evolved.

Reflections on Using Machine Learning to Beat a Video Game

Setting up a machine to learn for itself wasn't simple. There were many bumps along the way. As Ben said of his internship, "The thing I learned the most was troubleshooting, like changing things around, moving files. It was never easy, there was always an issue with the steps. But that’s just how computers are." Even once things were running more smoothly, it took many, many repetitions before the machine could learn how to successfully complete the course. Just like with change in biological species, generations of adaptations are necessary for artificial neuro-evolution. As for Ben, the experience has only inspired him to keep learning. He plans to do several more internships as he explores the educational path he'll take.

More Uses for ArtificiaI Intelligence and Machine Learning

If Ben does decide to pursue responsible AI and ML as a career path, he'll find that there are many varied applications for it. All kinds of industries are beginning to make use of its assistance, everywhere from healthcare and music therapy to improving democracy. That's why at Infused Innovations, we're committed to growing in our understanding and application of AI and ML. We also acknowledge and prioritize the importance of doing so responsibly. We live in an age where technology is changing rapidly. The evolution that comes with these changes---both biological and artificial---will surely prove to be even more fascinating than what can happen in a video game.

-

More on Machine Learning:

• What is Computer Vision?
• Discover Potential for Your Business with a Computer Vision Accelerator