What Is Image Processing?

Image processing involves using algorithms to modify digital images, enabling tasks such as enhancement, restoration, compression, and analysis. Some basic processes in image processing include contrast adjustments, blur, noise reduction, transformations, or compression for storage. An image can be anything—from a photo, video, or X-ray—but is ultimately a row of pixels that make a matrix. Digital image processing, which uses computers, focuses on algorithms, as opposed to analog signal chain image processing. Image processing is an important field closely related to image analysis and the development of computer vision. 

Explore the types of image processing, how it works, and the many fields it touches. Also, discover its advantages and disadvantages, and how you can start in image processing. 

Types of image processing

While image processing extends to more intensive computer processes, like image analysis and computer vision, every image requires three main types of low-level image processing so that you can view it on a computer:

• Image enhancement

• Image restoration

• Image compression

Image enhancement

In image enhancement, the goal is not to make destructive changes to the image but to make sure it has certain aesthetics that make it viewable for you. These include basic image adjustments like contrast, coloring, sharpness, noise reduction, and filtering. Image filtering creates denoising and sharpening by using low-pass and high-pass filters, respectively, to improve visual appearance. 

Image restoration

Image restoration is the process of restoring image quality lost during the acquisition of the data. While image restoration uses some of the same filtering techniques in image enhancement, restoration restores images to a certain technical quality rather than creatively manipulating aspects of an image for aesthetic enjoyment. Some degradation that may occur includes motion blur, information loss, out-of-focus shots, and excessive noise. Restoration aims to return this image back to its “original” form, whereupon enhancement can occur. 

Image compression

Image compression is the technical process of reducing image data to an efficient form. It maintains the form of the image while reducing the amount of bytes actually required to represent the image. Two kinds of image compression exist depending on your needs:

• Lossless compression: removes image redundancy without data loss in the image, best for archiving photos, videos, and medical images

• Lossy compression: involves data loss in the reconstructed image in order to save space and load faster

Various image compression algorithms exist to produce both lossless and lossy images. Lossless image formats include RAW and PNG files, and some lossy examples include JPEG and H.264 files.

How does image processing work

Imaging processing works because digital images are functions. Take a color image of red (R), green (G), and blue (B), for example. Each color is a value from 0 to 255, with the function looking like:

f(x,y)=r(x,y) g(x, y) b(x,y)

You can use a website like rgb.to to put in values from 0 to 255 for each amount of red, green, and blue to see what you get. Those three values make up every pixel in a digital color image.

The final function represents a vector. Image filtering manipulates an image by changing the value of the red, green, and blue inside the function. Meanwhile, image warping modifies the image by changing the positions of the pixels, such as stretching or distorting the image.

Image analysis and computer vision

Image analysis and computer vision represent mid-level and high-level image processing, respectively. Image analysis uses low-level image processing techniques—like edge detection—to find patterns, track features, track motion, perform image segmentation, motion detection, etc. Its function is to take an image as an input and feature attributes as an output. These techniques are useful for medical imaging, scientific applications, and industrial applications—like counting objects.

In turn, high-level image processing like computer vision uses these attributes in tandem with machine learning, neural networks, or deep learning in order to interpret and analyze images in real time in a similar manner to the human perception of vision. 

Digital vs. analog image processing

The main difference between digital and analog image processing is that digital image processing uses pixel matrices to represent images on a computer. On the other hand, analog image processing works only through the manipulation of two-dimensional electrical signals, significantly reducing the amount of processing you can do with them. 

Who uses image processing?

A large range of industries use image processing for scientific, industrial, and artistic reasons. Some applications of image processing include:

• Biology

• Astronomy 

• Radiology

• Bioinformatics

• Satellite images

• Commercial photo and video

Each industry uses image processing analysis to help them in their field. Below are a few examples of how they use image processing in their work. 

Astronomers

Scientists, like astronomers, use image processing techniques to capture images of stars and galaxies through telescopes. They use a charge-coupled device (CCD) to capture images in greyscale and color information through filters. This enables them to later process the image and reconstruct the color using that information. For complex structures like galaxies, they use broad-band filters to capture as much information as possible. An example of this is the Hubble space telescope, which uses seven broad-band filters. 

Health care professionals

Health care professionals use image processing with MRIs, CAT scans, and PET scans to help them provide patients with accurate diagnoses. The basic image processing techniques help them amplify certain aspects of an image to identify cancers, assist with surgeries, and perform image segmentation. 

Filmmakers

Filmmakers use image processing when using digital cameras to acquire images. Cameras acquire raw images that they pre-process to correct any distortion or color to get a “normal” looking image. This allows them to perform color-grading, which is the creative manipulation of images to create a desired aesthetic look. 

They can then add visual effects (VFX) to the images. They will eventually export and compress the image in various forms depending on what delivery format is needed. Filmmakers use image restoration techniques to restore old films back to their “original state” as close as they can. 

Pros and cons of image processing

Image processing, specifically digital image processing, possesses a lot of advantages by giving its practitioners many techniques to manipulate, analyze, and study images. It has a few disadvantages, but they are mostly related to the hardware and storage requirements. Explore those advantages and challenges below. 

Advantages

The advantages of image processing come in the range of techniques you can use to manipulate digital images. This includes contrast, color, size, noise, sharpness, distortion, compression, and quality. Other advantages include:

• Ability to enhance hard-to-see details or failures in the acquisition process.

• Various color processing models are available for both scientists and creatives.

• Many techniques and algorithms for compressing images to save storage space while maintaining quality.

• Ability to use segmentation to analyze specific components of an image separately.

• Low-level processing prepares images for a processing pipeline that includes feature recognition, classification, and the application of machine learning and neural networks in computer vision.

Challenges

The challenges of image processing arise when considering the hardware, software, and storage required to work with images in this manner. Consider these challenges in image processing:

• Specialized software is needed to work with and process images.

• With specialized software also comes the need for a dedicated graphics processing unit (GPU) in your computer to efficiently work with images.

• Images take up lots of storage, especially when working with video.

How to get started in image processing

If you want to start in basic image processing by manipulating images using a graphic user interface (GUI), consider trying an app like Photoshop or Affinity Photo. By doing this, you can get a feel for the kinds of adjustments you can make to an image. However, if you’re looking to build skills in image processing from a computer science and software developing perspective, consider using the Python library sci-kit image. It contains an open-source library of algorithms for image processing in Python. It has algorithms for exposure change, color manipulation, edge detection, upscaling, feature detection, and segmentation.

Advanced skills in image processing include the knowledge of linear algebra, probability, statistics, and Python programming, especially for those looking to learn more about computer vision. 

Getting started in image processing with Coursera

Image processing is the first step in the pipeline of image analysis and eventually computer vision. Are you looking to learn more about image processing? Try the Image Processing for Engineering and Science Specialization form MathWorks on Coursera to gain basic skills in the field aimed at engineers and scientists who work with images.