'GTA V' Becomes an Unlikely Mentor for Artificial Intelligence; Will Teach Self-Driving Cars to Prevent Obstacles

Grand Theft Auto 5, of all games, will lead the frontier for A.I. and self driving cars.

'GTA V' Becomes an Unlikely Mentor for Artificial Intelligence; Will Teach Self-Driving Cars to Prevent Obstacles

Who said video games are a waste of time? Actually, nobody says that now unless it’s a disgruntled parent with an academically weak son punching joystick buttons 25 hours a day; borrowing one hour from the next day. Sorry to begin with such a weak unoriginal opening statement. But to say, Grand Theft Auto 5 of all games will lead the frontier for A.I is just another dazzling look at Terminator's Skynet. Known to the incessantly complaining & touchy world of 2016 as a violent, immature and sexually suggestive video game, the hope of self-driving cars lies in the hands of Rockstar’s flamboyant ride GTA V. For the record, I have no problems with GTA V. It is what it is. It’s not like anybody expects a deep philosophical 'Terrence Malick' style narrative game about the video game industry sucking the life outta 12 year olds and pushing materialism as the new age God. Rockstar is bloody awesome and not just for GTA series, I’m speaking about their role as a publisher of L.A Noire. Personally speaking, the making of L.A Noire is the Apocalypse Now equivalent of video games and I hope to write in detail about it someday.

Grand Theft Auto V

How does a video game that cheers traffic rage, callous behavior and facetious police chases become a perfect platform for self-driving cars? Two words: machine learning. Video games are basically 3D architectural design and puzzles that follow a pattern. Today’s video game graphics are as perfect as a real-life environment. We live in a world where it’s getting more and more perspiring to choose between a captured DSLR photo and a graphical rendered image.

Grand Theft Auto V neural network

“The scenery in many games is so fantastically realistic that it can be used to generate data that’s as good as that generated by using real-world imagery.”

Getting to the question as to why choose a video game when there are other ways to train an A.I like driving around a neighborhood, capturing details and cataloguing them - this kind of experiment would mean collecting insane amount of data that has to be crunched and sent through the computer to process. It’s manually too much work, monetarily burdening and time consuming. But video game code always “knows” what’s on-screen, and so essentially the images are pre-labeled. So, video games work better than real images in helping researchers label quickly. Also, artificial environments offer better physical data such as lightning and expose climate conditions that help the A.I determine how to drive on a slick, wet road or through an impassable haze.

According to a study, labeling a single image culled from Grand Theft Auto V took seven seconds on average. Manually labelling real-world images, on the other hand, took anywhere from an hour to 90 minutes.

There’re research groups building their own simulations using game engines to generate training data for algorithms, but Mark Schmidt, an Assistant Professor of Computer Science and Alireza Shafaei, a PhD student, both from University of British Columbia have tapped into the programming and graphical information of GTA V to train artificial intelligence mainly to recognize objects—cars, pedestrians, bicycles, buildings, roads and other things a car might encounter in the real world.

GTA V is a massive game world of polygons all modeled with immense research and real image references of North American countries. This training data is fed to machine intelligence which can help traverse a car. The technique allows software developers to come up with inventive and intriguing applications. From twitter bots that study tweets and go rogue to a neural network that turns photos into all forms of art style including Van Gogh and Picasso. The much talked about Google DeepMind’s AlphaGo AI vs professional human player Lee Sedol is an astute example of analyzing the importance of achieving better artificial intelligence through machine learning. The human lost 1-4 in a five match tournament.

I also heard of an A.I that learnt to watch Blade Runner and reconstructed what it saw. Philip K. Dick would’ve been intrigued. You can also watch the science fiction short film scripted by an A.I. Sunspring is what happens when you feed hundreds of sci-fi scripts to a neural network and get a half-baked mishmash draft. Last month, IBM Watson became the first A.I to cut a trailer for Ridley Scott's son first sci-fi horror movie Morgan. The list is endless I fear.

The MIT’s Technology Review reports “The researchers created a software layer that sits between the game and a computer’s hardware, automatically classifying different objects in the road scenes shown in the game. This provides the labels that can then be fed to a machine-learning algorithm, allowing it to recognize cars, pedestrians, and other objects shown, either in the game or on a real street. According to a paper posted by the team recently, it would be nearly impossible to have people label all of the scenes with similar detail manually. The researchers also say that real training images can be improved with the addition of some synthetic imagery.”

Now, what you see up is a neural network driving a car in a video game that doesn’t focus on traffic lights, speed limits and certainly gives no shit about pedestrians and other folks of the road. It’s as if the A.I is a flâneur leisurely observing the human grasp of technology !

fact or fiction
Read next: Are Loot Boxes Gambling?