CMOS Sensor Operation

Until recently the industrial digital vision sensor market was dominated by the CCD array. However technological advances in CMOS production techniques have led to a gradual increase in the popularity of this sensor type. Like CCD arrays, CMOS sensors are also formed on a silicon substrate but the structure is more akin to that of other CMOS technology such as RAM and ROM memory devices.
The diagram below is that of an actual CMOS sensor showing the active pixel area in green and the area occupied by the on chip circuitry in yellow, which replaces that of the shuttered area on a CCD based sensor. The on chip circuitry actually converts the charge into voltage on each pixel whereas the CCD sensor shifts the charge vertically row by row, and then horizontally pixel by pixel to be converted to voltage when it reaches one or more output nodes. This gives CMOS sensors an advantage when it comes to windowing or a region of interest as the pixels can be read out randomly. CCD sensors can only limit its region of interest vertically with the resulting image always containing the data for the full image width.

The on chip active amplifier and the sampling capacitor give CMOS sensors advantages in terms of speed, full well capacities and much improved response characteristics yet introduce dark current level noise and higher black pixel content. CMOS sensors can also produce higher levels of fixed pattern noise than that of CCD, but this type of noise can be easily removed with a software filter.

The development of CMOS sensor technology has been a rapid and varied process. The initial aim of CMOS sensors was to match the imaging performance of CCD technology, with lower power requirements and at less cost. To achieve this performance it was discovered that a much greater level of manufacturing process adaptation and deeper submicron lithography were required than initially expected. This led to the desired CMOS performance but increased development costs more than anticipated.

At first the low power feature of the CMOS imaging sensors was set to be one of their distinct advantages, however the improved development of CCD sensors means that while CMOS has the advantage in this area, the margin is now much smaller.

The integration of on chip control circuitry with the CMOS imager provides the sensor with greater flexibility and integration, the downside has been the introduction of greater noise levels. Both CMOS and CCD imaging sensors still require support chips to process the image, however CMOS imagers can be produced with more functionality on the sensor chip, as shown below.

The spectral response of a CMOS sensor differs from that of the CCD sensors in that the peak response is sited at around 700Nm. Both sensors operate over the same range, typically 200Nm to 1100Nm.

The main advantages of CMOS imaging sensors still remain as faster response, increased integration flexibility and lower on-chip power demands. However the image quality has yet to match that of the CCD and the supporting chips required to increase the CMOS image quality goes some way to squander its previous advantages. Yet neither sensor is categorically superior to the other. They both have their own advantages and disadvantages and with CMOS developers working on the image quality, and CCD developers aiming to reduce power demands and increase flexibility, the existing margins in place to decide which sensor is most suitable for an application look to narrow further.

CCD vs CMOS: A Short-note.

There has been neck to neck competition between CCD and CMOS imaging technologies. CCD and CMOS imagers were invented in the same era within a span of few years. Yet, CCDs became dominant because of the superior results from the then available fabrication technology. CMOS technologies focused on uniformity and smaller feature sizes, which did not quite happen until 1990s when lithography advanced enough to be support small feature sizes. It was after this that CMOS imagers had a comeback and since then both technologies have fought for market dominance. While CCD sensors are known to offer best image qualities, CMOS imagers offer more functions on the chip and attractive features like lesser power usage making them more popular in mobile phone cameras etc. A comparison can be drawn between the two on various aspects and a sound decision be made depending on the requirements of the application.

Factor	CCD	CMOS
Responsivity	Moderate	Higher
Dynamic Range	High	Moderate
Uniformity	High	Low
Speed	Moderate	Higher
Anti blooming	High	High
Signal out of pixel/chip	Electron Packet/Voltage	Voltage/Bits
System/Sensor Complexity	High/Low	Low/High
Noise	Low	High

Markets have seen rapid decline of the CCDs share owing to the growing popularity of CMOS sensors in cell phones and point and shoot cameras and even industry stalwarts like Canon and Sony who used CCD products primarily are now shifting to CMOS imagers. It is expected that more than 95% of the camera market will switch over to CMOS sensors by 2014. But there still remains and would remain a predominant segment which will continue to bank on CCD sensors, the scientific research and the astronomer community, the biggest example being the Hubble Space Telescope. So, while the light from the CCDs might be fading from the earth, we’d still need a CCD to see what’s out there.

CCD Working Patern.

The main performance parameters of a CCD device are its speed, sensitivity, resolution and cost. However, the choice of architecture is dependent on the application of the devices. While astronomy requires that maximum light being captured, price seldom being a constraint, full frame read-out architecture CCD devices are utilized. But in case of the point and shoot photography cameras where shutter speed and cost counts the most, interline architecture is the most popular.

CCD arrays are only sensitive to intensity and not color. Thus, in order to obtain a color image, filters are used. Colored images may be obtained by using either the Bayer Pattern or 3CCD and a dichroic beam splitter prism.

Bayer Pattern is a special color grid which is placed over the imaging array. Human eyes being most sensitive to green color, two out of four grids have a green filter. The remaining have blue and red, comprising the RGB color pattern. A digital signal processor interpolates the two missing colors from the values of the neighboring pixels. Bayer pattern though simple and cheap, throws information away compromising on image resolution. Also, when used in places with abrupt changes in light intensity Bayer pattern creates artifacts. Some cameras may also use different color patterns for color generation too. Resolution can be improved in specific applications by Microscanning Technology.

A dichroic beam splitter prism splits the image into red, green and blue components which can then be incident separately on three-CCD devices. This offers higher quantum efficiency over Bayer Mask as most of the light is captured by the sensors and not absorbed by intermediate layers of the Mask. Such devices have been used in professional video camcorders.

A CCD Device despite of all the good features it has to offer does have a few drawbacks too. Coupling of the charges along a row of many pixels results in the loss of considerable amount of charge resulting in fading. Improper shutter control, or over bright conditions leading to too many photons reaching the CCD element results in the leakage of charge to the neighboring pixel, which causes blooming.

Further, if photons strike sensors while charge is being transferred due to improper clocking, data loss in the form of smears can happen. The sensitivity of a CCD device is often a function of the temperature of operation. As the temperature increases, the leakage dark current also increases, thereby reducing sensitivity. Noise is inherent to every electronics equipment and in case of devices based on CCD, the SNR degradation may take place due to photon noise, dark noise or read noise or the combination of them. However, intensive research work in this field has yielded positive results and   the development of many improved variants. Devices like Electron Multiplying CCD (EMCCDs) which incorporate on-chip multiplication gain have helped achieve single photon detection sensitivities without compromising on resolution or efficiency of CCD structure while reducing costs. These features are otherwise typical of the costly Electron-Bombarded CCDs which utilizes an image intensifier with a CCD device.

Architecture of CCD.

To carry out the read out from the CCD array, three architectures have been utilized:

1.      Full Frame Read Out: The entire CCD array acts as an active area. This type of device uses mechanical shutter mechanism to stop light from reaching the element to avoid smearing when the charges are passed down the Vertical-CCDs in parallel and later serially moved out using the Horizontal-CCDs. This process is quite time consuming.

2.      Frame Transfer: Half of the contiguous array area is used for exposure and the remaining half is opaque. The charges are transferred from the active area to the opaque area in very less time and then are read out from there. This process is faster than the full frame read out as during the time charges are being read out of the opaque area, the active area can be used for capturing new image. It also has an advantage of reduced smearing and light contamination while charge transfer. However, this has a disadvantage of using double the silicon area.

3.      Interline Transfer: In this type of architecture, each pixel has an active area and an opaque area adjacent to it. The charge is quickly transferred from the light sensitive Photodiode to the adjacent Vertical-CCD unit. This uses only one transfer cycle to hide the entire image thus allowing very high shuttering speeds and minimum smear. This too has the disadvantage of increased silicon estate, but modern advancements have tried to increase the quantum efficiency of the array by using microlenses which redirect the light away from the opaque regions. The use of microlenses has increased the fill factor to about 90% of the other architectures without compromising on the speed.

Elements & Working System.

There lies the main challenge, to read out these charges. Many schemes named binning, two-phase, three-phase, four-phase etc. corresponding to the type of clocking used have been used for transferring the charge packets cell to cell in the bucket brigade style while protecting the integrity of each packet. Considering a 3-phase scheme, there would be 3 control wires passing each cell, each wire being connected to one phase of the clock. These wires control the height of the potential wells which pushes and pulls the charge packets along the line. For a three-phase scheme, each pixel contains one storage and two barrier gates. Each storage gate is connected to the same phase and sequentially, every barrier gate is connected to their respective phase clocks. A charge packet moves from one region into the next region when the second gate goes to a higher potential and the first gate goes low. This sequential movement of charges can be demonstrated as in the figure below.

Attempts were made to use CCD devices as memory devices. But soon after, other techniques took over and after that CCD devices have been used primarily in scanning, microscopy and photography. One dimensional array has been utilized in flatbed scanners where one line is scanned at a time and the charges are read out after that. The array is moved mechanically over the entire page to create the 2 dimensional scanned images. Two dimensional arrays have been used to capture the entire scene in one go and then transfer the charges to the output.

 

One dimensional linear array CCD

CCD : Charge Coupled Devices

It was a historic moment for digital cameras when their shutters opened for the fraction of a second to capture the committee, award the coveted Nobel Prize to the ones who invented the electronic eyes. Had it not been the duo Willard Boyle and George Smith, those cameras might still have been shuttering light on a photosensitive film and then drying them on rails in the dark room. By inventing the Charge Coupled Device and predicting its applications way back in 1969, Boyle and Smith opened up the domain of solid state devices for imaging and memory applications.

Though Boyle and Smith invented the method to convert light into an electrical signal, the entire research would not have been possible had it not been Albert Einstein’s groundbreaking explanation of the Photoelectric effect. Einstein explained how materials could absorb incident radiation and the eventual knocking out of the electrons from the surface. Boyle and Smith studied how this incidence of light caused disturbances in electrons and how it could be utilized. They succeeded in doing this by grouping a number of capacitors into an array, something which makes up the pixels of a digital image.

Initially named as Charge Bubble Devices, the device’s operation as a shift register and a linear imaging device was recorded. It was based on a principle similar to Bucket-Brigade Devices, where charge is transferred from one capacitor to another along the semiconductor surface. Bell labs succeeded in building the first solid state video camera in 1970. By 1971, Michael F. Tompsett and other researchers at Bell Labs succeeded in capturing images with linear CCD arrays. After that, many semiconductor majors like Texas Instruments, Fairchild Semiconductors, Sony etc. started investing in CCD technologies. One of the first commercial CCD devices was built by Fairchild in 1974 which was a 100x 100 pixel device having about 500 CCDs array. The first CCD based reconnaissance satellite KH-11 KENNAN was launched in 1976. By 1983, CCDs had started to replace photographic plates in astronomical telescopes too. Companies like Kodak had been manufacturing CCD based professional cameras since 1985, but by 1995, cost effective high resolution CCD cameras started flooding the markets.

A CCD can be thought of as a subset of Charge Transfer Devices. These are based upon Metal Oxide Semiconductor (MOS) capacitors. Two types, viz surface channel and buried channel MOS capacitors have been used in CCD, but primarily buried channel capacitors have been used for manufacturing since these do not have problems caused by surface irregularities at oxide-semiconductor interface. A thin n-type buried channel is formed on a p-type substrate through ion implantation. The Silicon Dioxide insulator layer is grown on top of the n-region, and to complete the capacitor, gates of metal or heavily doped polycrystalline silicon are placed on top of the insulating SiO₂ using CVD process. To isolate the charge packets of one column from other, thermally grown oxide ‘channel stops’ are placed parallel to the channels.

 

Buried Channel MOS

But if a CCD cell were to pass along charges, it would just be any other MOS capacitor. The additional property that it is sensitive to light makes a CCD unique. The response to photons is through the epitaxial layers of doped silicon grown on the substrate. When photons are incident on the semiconductor surface, they dislodge electrons which create charge that is proportional to the light falling on the surface. A single CCD cell performs four functions:

1.      Receive charge from the cell above it in the array.

2.      Hold that charge for sometime without much loss.

3.      Pass that charge to the cell below it in array.

4.      Respond to outside stimulus like light and generate its own charge.

IP Camera: Next generation surveillance

Overview of a network video system

 
Network video, often also called IP-based video surveillance or IP- Surveillance as it is applied in the security industry, uses a wired or wireless IP network as the backbone for transporting digital video, audio and other data. When Power over Ethernet (PoE) technology is applied, the network can also be used to carry power to network video products.
A network video system allows video to be monitored and recorded from anywhere on the network, whether it is, for instance, on a local area network (LAN) or a wide area network (WAN) such as the Internet.
The core components of a network video system consist of the network camera, the video encoder (used to connect to analog cameras), the network, the server and storage, and video management software. As the network camera and the video encoder are computer-based equipment, they have capabilities that cannot be matched by an analog CCTV camera. The network camera, the video encoder and the video management software are considered the cornerstones of an IP-Surveillance solution.
The network, the server and storage components are all standard IT equipment. The ability to use common off-the-shelf equipment is one of the main benefits of network video. Other components of a network video system include accessories, such as camera housings and PoE mid spans and active splitters.
Benefits of network video surveillance
The digital, network video surveillance system provides a host of benefits and advanced functionalists that cannot be provided by an analog video surveillance system. The advantages include remote accessibility, high image quality, event management and intelligent video capabilities, easy integration possibilities and better scalability, flexibility and cost-effectiveness.
Remote accessibility
Network cameras and video encoders can be configured and accessed remotely, enabling multiple, authorized users to view live and recorded video at any time and from virtually any networked location in the world. This is advantageous if users would like a third-party company, such as a security firm, to also gain access to the video. In a traditional analog CCTV system, users would need to be at a specific, on-site monitoring location to view and manage video, and off-site video access would not be possible without such equipment as a video encoder or a network digital video recorder (DVR). A DVR is the digital replacement for the video cassette recorder.
High image quality
In a video surveillance application, high image quality is essential to be able to clearly capture an incident in progress and identify persons or objects involved. With progressive scan and megapixel technologies, a network camera can deliver better image quality and higher resolution than an analog CCTV camera.
Image quality can also be more easily retained in a network video system than in an analog surveillance system. With analog systems today that use a DVR as the recording medium, many analog-to-digital conversions take place: first, analog signals are converted in the camera to digital and then back to analog for transportation; then the analog signals are digitized for recording. Captured images are degraded with every conversion between analog and digital formats and with the cabling distance. The further the analog video signals have to travel, the weaker they become.
In a fully digital IP- Surveillance system, images from a network camera are digitized once and they stay digital with no unnecessary conversions and no image degradation due to distance traveled over a network. In addition, digital images can be more easily stored and retrieved than in cases where analog video tapes are used.
Event management and intelligent video
There is often too much video recorded and lack of time to properly analyze them. Advanced network cameras and video encoders with built-intelligence or analytics take care of this problem by reducing the amount of uninteresting recordings and enabling programmed responses. Such functionalists are not available in an analog system.
Techno Crate network cameras and video encoders have built-in features such as video motion detection, audio detection alarm, active tampering alarm, I/O (input/output) connections, and alarm and event management functionalists. These features enable the network cameras and video encoders to constantly analyze inputs to detect an event and to automatically respond to an event with actions such as video recording and sending alarm notifications.
Event management functionalists can be configured using the network video product’s user interface or a video management software program. Users can define the alarms or events by setting the type of triggers to be used and when. Responses can also be configured (e.g., recording to one or multiple sites, whether local and/or off-site for security purposes; activation of external devices such as alarms, lights and doors; and sending notification messages to users).
Easy, future-proof integration
Network video products based on open standards can be easily integrated with computer and Ethernet-based information systems, audio or security systems and other digital devices, in addition to video management and application software. For instance, video from a network camera can be integrated into a Point of Sales system or a building management system.
Scalability and flexibility
A network video system can grow with a user’s needs. IP-based systems provide a means for many network cameras and video encoders, as well as other types of applications, to share the same wired or wireless network for communicating data; so any number of network video products can be added to the system without significant or costly changes to the network infrastructure. This is not the case with an analog system. In an analog video system, a dedicated coaxial cable must run directly from each camera to a viewing/recording station. Separate audio cables must also be used if audio is required. Network video products can also be placed and networked from virtually any location, and the system can be as open or as closed as desired.
Cost-effectiveness
An IP- Surveillance system typically has a lower total cost of ownership than a traditional analog CCTV system. An IP network infrastructure is often already in place and used for other applications within an organization, so a network video application can piggyback off the existing infrastructure. IP-based networks and wireless options are also much less expensive alternatives than traditional coaxial and fiber cabling for an analog CCTV system. In addition, digital video streams can be routed around the world using a variety of interoperable infrastructure. Management and equipment costs are also lower since back-end applications and storage run on industry standard, open systems-based servers, not on proprietary hardware such as a DVR in the case of an analog CCTV system.
Furthermore, Power over Ethernet technology, which cannot be applied in an analog video system, can be used in a network video system. PoE enables networked devices to receive power from a PoE-enabled switch or midspan through the same Ethernet cable that transports data (video). PoE provides substantial savings in installation costs and can increase the reliability of the system.
What is a network camera?
A network camera, often also called an IP camera, can be described as a camera and computer combined in one unit. The main components of a network camera include a lens, an image sensor, one or several processors, and memory. The processors are used for image processing, compression, video analysis and networking functionalist. The memory is used for storing the network camera’s firmware (computer program) and for local recording of video sequences.
Like a computer, the network camera has its own IP address, is connected directly to a network and can be placed wherever there is a network connection. This differs from a web camera, which can only operate when it is connected to a personal computer (PC) via the USB or IEEE 1394 port, and to use it, software must be installed on the PC. A network camera provides web server, FTP (File Transfer Protocol), and e-mail functionalist, and includes many other IP network and security protocols.
A network camera can be configured to send video over an IP network for live viewing and/or recording either continuously, at scheduled times, on an event or on request from authorized users. Captured images can be streamed as Motion JPEG, MPEG-4 or H.264 video using various networking protocols, or uploaded as individual JPEG images using FTP, e-mail or HTTP (Hypertext Transfer Protocol).
In addition to capturing video, Techno Crate network cameras provide event management and intelligent video functionalists such as video motion detection, audio detection, active tampering alarm and auto-tracking. Most network cameras also offer input/output (I/O) ports that enable connections to external devices such as sensors and relays. Other features may include audio capabilities and built-in support for Power over Ethernet (PoE). Techno Crate network cameras also support advanced security and network management features.

  To be Continue.