More and more embedded systems include image-processing functions, and thanks to the growing performance of embedded processors, these image-processing functions are increasingly complex. Yet embedded systems are usually very cost-sensitive. Image correction and image processing come at a cost, and it is important to consider overall system costs to make the right decision. There are two types of image processing tasks performed in a vision system: Image pre-processing and image processing. Both require processing power to deliver the expected result.
Image pre-processing, advanced processing, and post-processing
Image pre-processing or image correction and optimization is the sum of the processing tasks performed to convert the raw image data delivered by the sensor into an image whose quality meets the requirements of the specific system application. Image pre-processing may include functions such as defect pixel correction, white balance, gain, noise reduction that will optimize the image for the specific task to be performed by the image-processing software. Once the images have been optimized, the advanced processing allows for more complex enhancements to facilitate the application-specific image analysis. Examples of advanced processing are sharpness or color corrections using look-up tables.
In an embedded system, the pre-processing and advanced processing correction tasks are typically performed in an Image Signal Processor (ISP). The purpose of these corrections is to optimize the image for the actual image processing application of the system also known as post-processing. For example, enhance contrast for easier edge detection.
This post-processing application can be anything depending on the specific system goal, for example identifying a face in the image, locating and reading a license plate in a traffic monitoring application or checking the quality of objects in an industrial inspection application. These tasks are performed by complex, application-specific software algorithms on the main processor of the embedded board.
Camera module vs. sensor module
Image corrections and image processing both require processing power. Until today, embedded system designers have been relying on low-cost sensor modules to capture the images. These devices should rather be called sensor module because their functionalities are limited to interfacing the image sensor with the host. With a sensor module, both the image correction and the application-specific image processing are performed on the host embedded board.
Camera manufacturers coming from the machine vision industry offer more advanced camera modules that bring the performance and intelligence known in PC-based industrial and scientific image processing. These devices have an ISP on-board and perform advanced correction tasks such as defect pixel corrections, color corrections, cropping to a defined region of interest and many more. Allied Vision revolutionized the embedded vision market with the introduction of the ALVIUM® technology that combines high-performance, low-cost processing hardware with a rich image optimization feature set. Such camera modules deliver ready-to-use images to the host application and do not require an image sensing processor on the host side.
Low-cost sensor modules not suitable for low-cost systems
Until recently, most embedded system designers have been accustomed to working with low-cost sensor modules for image capture and perform all the image processing on their host system. So for particularly cost-sensitive applications, their intuition is to search for the lowest possible cost for these sensor modules.
This may not be the most economical way, though. Low-cost sensor modules require higher processing power on the host side, and it comes at a cost. Advanced camera modules may be slightly more expensive than sensor modules but they actually include an ISP and allow system designers to offload the whole image pre-processing from their main board. As a result, they can cut overall system costs significantly by using lower-performance, lower-cost host processors.
Real price of image processing
As counter-intuitive as it may seem at first sight, low-cost sensor modules are not necessarily the best choice for low-cost systems. Depending on your application and processing needs, spending a few dollars more for a more advanced camera module can save you a few hundred dollars on the host side and help you lower your overall system costs.
The rise of advanced camera modules with built-in ISP means a paradigm change for designers of embedded vision systems: More than ever, they should always consider overall system costs when selecting their imaging components. Whether onboard the camera or on the host side: processing power always comes at a cost. Finding the right balance between the camera and the host for image processing is the key to truly optimized system costs.