deepsense.ai and Google Brain Design Artificial Imagination for Reinforcement Learning

WARSAW, March 21, 2019 /PRNewswire/ -- deepsense.ai, Google Brain, the University of Warsaw and the University of Illinois at Urbana-Champaign have concluded a collaborative research project, building neural networks that mimic a simulated environment and effectively enabling artificial intelligence to perform a simulation.

Designed models can be applied in text or video prediction and to solve the problems where mathematical description may be overly complicated.

Reinforcement learning (RL) enables a neural network to develop and hone its skill by interacting with its environment, which is usually a simulated one. The technique is predicted to shine in robotics and the building of autonomous vehicles.

The research team found a way to build a neural network that can mimic the signals that the RL agent usually gets from interacting with the environment. The neural network produces signals that usually would be taken from sensors, like images. "This is one of the important ideas of reinforcement learning. A recent survey done by DeepMind's J.B. Hammrick provides a thorough account of analogies between model-based reinforcement learning and mental simulation as considered by cognitive science," says Henryk Michalewski, the R&D Coordinator at deepsense.ai and a visiting professor in the Computer Science Department of the University of Oxford.

The challenges of education

In the learning process, the RL agent is rewarded for performing a task correctly and punished for making mistakes. Autonomous cars provide perhaps the best example--the agent is rewarded for safe driving and punished for collisions and speeding. The model seeks to maximize the rewards and minimize the punishments. Initially, all actions are random and the neural networks have yet to explore the possible ways to perform the tasks. It is thus possible that more than the first dozen rides will end with a collision with the nearest wall as the model incurs punishment and learns how to brake or avoid it.

This leads to the idea of conducting the trial and error process inside a simulator. "However, building a realistic simulator is a tedious problem for humans", says Błażej Osiński, a Senior Data Scientist at deepsense.ai. "For example, Unreal Engine 4 has several million lines of code. The idea of our project is to let the neural network learn how to simulate the environment. A similar approach was suggested in a recent work from Yann LeCun's lab, where neural networks were employed to simulate dense traffic." 

The research team has built a neural network that emulates the Atari gaming environment, a popular training ground for reinforcement learning models. The network was able to create a version of the games Pong, Freeway and others that were nearly indistinguishable from Atari's.

Ongoing research

The project was run by data scientists and researchers from deepsense.ai including Henryk Michalewski, Piotr Miłoś, Błażej Osiński and fellow researchers from Google Brain, the University of Warsaw and the University of Illinois at Urbana-Champaign. More detailed information about the research, outcomes and possible uses can be found in the Arxiv paper and a detailed blogpost about artificial imagination on deepsense.ai's blog.

The research opens up interesting ways to apply neural networks in business settings. The first is to build an RL agent that can explore a highly complex environment and then emulate it for the needs of models that are to perform tasks within it. The best example may be teaching the model to emulate the real world in all of its complexity and unpredictability.

Another business application would be a video prediction tool that could deliver videos based on only a few frames, effectively reducing the effort animators and designers put into producing full-length videos.

'We consider our research work an essential part of deepsense.ai and a key aspect of the company's development. This time our researchers have really pushed the boundaries of knowledge with no marketing overstatements. The project brings fresh, new idea into AI research, a thing that is worth contributing into." concludes Tomasz Kułakowski, CEO at deepsense.ai.

SOURCE deepsense.ai