What is Computer Vision? The Technology Behind this Intelligent Tool

Computer vision is a machine learning technology that allows a computer to process visual images in such a way that it can "understand" and give meaning to them. Then it can perform tasks related to the sorting and identification of related images. Applications for this AI capability are continuing to grow, and forward-thinking companies have already been using it in a number of ways. How exactly does computer vision work? Here's a look at the technology behind it and how it can be used.

What is Computer Vision?

Computer vision is an artificial intelligence (AI) subset that focuses on analyzing images, video, and other visual data. Its foundation is the machine learning processing of large data sets, which eventually allow the computer model to make observations and connections between visual characteristics. These connections lead to a sort of understanding, so that the model gains the ability to quickly detect, identify, classify, and verify objects. This can be helpful in a variety of applications---but care should be taken to ensure that it is used ethically and privacy is protected.

Technologies Behind Computer Vision

Two important technologies allow computer vision to do what it does. These are deep learning and convolutional neural networks (CNNs).

Deep learning is a system in which a neural network, modeled after human ones, uses algorithms to repeatedly process large amounts of data. This information passes through multiple layers in such a network to create complex connections between images. As data is analyzed, the computer begins to learn about it, fine-tuning its understanding along the way. The machine learning algorithms allow this process to happen on its own, without human intervention beyond the initial setup.

A convolutional neural network aids in this process by breaking images down into pixels and labeling them. With these labels it does mathematical operations called convolutions in order to make predictions about the images. It runs many iterations until the predictions are reliably true. While a CNN comes to understand single images, a recurrent neural network (RNN) looks at videos to determine the relationships between a series of quickly moving frames. All of these parts work together to make computer vision that can work with a variety of inputs.

Comparing Computer Vision to Human Vision

Since neural networks are modeled after those in human brains, it's interesting to compare this process to what happens during human vision. When a human sees, observes and identifies, this is the product of both that particular individual's neurons forming connections and centuries of broader human experience that have passed down instinctive learning abilities. Though both of these time scales are much slower than a computer model's learning process, they are similar to computer vision in that it takes a lot of repetition for the right connections to be made and strengthened, and for the wrong ones to be stripped away.

Like the human brain does in a visual process, a convoluted neural network finds edges and simple shapes in an image, and then it runs through its predictions about these shapes. On a dim evening you may catch a glimpse of what looks like a person in your yard outside---the outline and human-like shape will alert you first. Repeated looks and predictions about likelihoods will help you determine whether it is indeed a person, or a small bear on hind legs, or that funny-shaped shrub you've been meaning to trim. Computer vision will also make predictions until the correct answer is achieved.

Using Computer Vision Technology Ethically

There are many great use cases for computer vision, such as quality control, healthcare applications, and developing self-driving vehicles. But as with many of today's AI tools, there is potential for misuse as well. In the wrong hands the technology may be used for harmful purposes like deepfakes. Facial recognition is thus one area that particularly calls for careful use. Even when intentions are good, the issues of bias, consent violations, and maintaining privacy may also arise. With these considerations in mind, those utilizing computer vision can take steps to ensure ethical use.

It's important to clearly define the purpose for the use case and apply the appropriate level of technology, refraining from the use of unnecessary capabilities. This is similar to the Zero Trust principle of least privilege, allowing access only where it is necessary. Implementing strong data protection and making sure to comply with privacy regulations are also crucial. There are other avenues to explore as well, such as Secure Federated Learning, which is a decentralized approach that keeps raw data protected. In any case, these ethical considerations need to be incorporated into plans for computer vision use. When fully thought out and intelligently applied, companies can take advantage of this powerful tool in both responsible and beneficial ways.

To learn more about how to map out a computer vision plan, take a look at our Computer Vision Accelerator for Image Classification and Object Detection.