The mind's eye of a neural network system – Purdue University News

Nov 22, 2023 ·

Neural networks are rapidly becoming an important tool in artificial intelligence, enabling the development of systems that can perceive and understand the world around them. This article explores a new neural network system developed by researchers at Purdue University which is capable of using vision to form a mental image of its environment.

The system works by taking generic photographs as input and then applying deep learning techniques to interpret the images. By combining both visual and tactile information, it builds up an internal model of the environment which allows it to make predictions about unseen objects and respond appropriately.

The system has been tested on three different environments: a maze, a room with furniture, and a kitchen. In each case, it was able to accurately predict the presence of objects and their locations, even when they were blocked from view. To test how well the system could estimate sizes and distances, researchers also placed objects at various points in the environments and asked it to estimate their size. In this task, the system performed very well, suggesting that it had formed a mental image of the environment.

In addition to its accuracy in making predictions, the system also demonstrated impressive levels of generalization. When presented with an object that differed slightly from those it had seen before, it was still able to accurately predict its location.

Overall, the research shows that neural networks are now capable of forming mental images of their environment and understanding the relationships between objects in the world around them. This could have many potential applications, such as helping robots to navigate in uncertain environments, or understanding spoken instructions. As the technology continues to develop, we may soon see more autonomous robots capable of perceiving and responding to their surroundings.