Concepts in Digital Imaging Technology

Charge-coupled devices (CCDs), like microprocessor and memory integrated circuits, are fabricated on silicon wafers in a series of elaborate steps using photolithography to define and build various functional elements within the microcircuitry. Each wafer contains tens to hundreds of identical devices, each fully capable of producing a single CCD chip for use in digital cameras. This section of the Molecular Expressions Photomicrography Primer contains links to illustrated discussions of important CCD concepts, which are critical in forming a complete understanding of digital imaging techniques.
Anatomy of a Charge-Coupled Device - Digital image sensor technology is centered around the semiconductor charge-coupled device, which is fabricated in a manner similar to that used in the production of integrated circuits ranging from microprocessors to memory chips. This section discusses common features of CCD anatomy and the basic principles of device operation.
Binning - Pixel binning is a clocking scheme used to combine the charge collected by several adjacent CCD pixels, and is designed to reduce noise and improve the signal-to-noise ratio and frame rate of digital cameras. The binning process is performed by on-chip timing circuitry that assumes control of the serial and parallel shift registers prior to amplification of the CCD analog signal.
CCD Saturation and Blooming - Saturation and blooming are related phenomena that occur in all CCD image sensors under conditions in which either the finite charge capacity of individual photodiodes, or the maximum charge transfer capacity of the CCD, is reached. Once saturation occurs at a charge collection site, accumulation of additional photo-generated charge results in overflow, or blooming, of the excess electrons into adjacent device structures. A number of potentially undesirable effects of blooming may be reflected in the sensor output, ranging from white image streaks and erroneous pixel signal values to complete breakdown at the output amplification stage, producing a dark image.
Dynamic Range - In a charge-coupled device (CCD) or complementary metal oxide semiconductor (CMOS) image sensor, dynamic range is typically specified as the maximum achievable signal divided by the camera noise, where the signal strength is determined by the full-well capacity, and noise is the sum of dark and read noises. As the dynamic range of a device is increased, the ability to quantitatively measure the dimmest intensities in an image (intrascene performance) is improved. The interscene dynamic range represents the spectrum of intensities that can be accommodated when detector gain, integration time, lens aperture, and other variables are adjusted for differing fields of view.
Quantum Efficiency - The quantum efficiency of a charge-coupled device (CCD) is a property of the photovoltaic response defined as the number of electron-hole pairs created and successfully readout by the device for each incoming photon. This property is especially important for low-light imaging applications such as fluorescence microscopy where illumination is sparse and secondary emission photon wavelengths are often in the 375-550 nanometer range and have a relatively high absorption coefficient.
CCD Noise Sources and Signal-to-Noise Ratio - Charge-coupled device (CCD) sensors have numerous advantages over photographic film in scientific imaging applications such as astronomy and optical microscopy. By directly producing images in digital format, suitable for immediate computer processing, CCD-based image capture systems are ideally suited to a wide range of current microscopy and image analysis methods. In particular, the much greater sensitivity of such sensors compared to film is invaluable in low-light techniques, for which every available signal photon may be significant. Noise, arising from a variety of sources, is inherent to all electronic image sensors, and careful control of noise components, both in the design and operation of the CCD system, is necessary to ensure that the signal level relative to noise is adequate to allow capture of accurate image information. For any electronic measuring system, the signal-to-noise ratio (SNR) characterizes the quality of a measurement and determines the ultimate performance of the system.
Charge-Coupled Device (CCD) Linearity - An important characteristic of a scientific imaging system is the linearity in response to incident light, particularly when applied for quantitative photometric analysis. In digital camera systems employing charge-coupled device (CCD) sensors, the fundamental function of the CCD is to convert photons carrying image information into an electronic signal. After digitization, the signal output should ideally be linearly proportional to the amount of light incident on the sensor.
Electronic Shutters - Electronic shutters are employed in charge-coupled devices (CCDs) to control integration time (exposure) of the photodiode array and reduce smear when capturing moving objects in the microscope.
Charge Transfer Clocking Schemes - Charge transfer through CCD shift registers occurs after integration to relocate accumulated charge information to the sense amplifier, which is physically separated from the parallel pixel array. Several clocking schemes, three of which are discussed below, are utilized to transfer charge from the collection gates to the output node.

• Four Phase CCD Clocking - A four phase CCD incorporates four individual polysilicon gate electrodes in each pixel cell, each of which requires a separate input clock signal to properly transport accumulated charge.
• Three Phase CCD Clocking - Three phase CCD clocking improves spatial resolution over that obtained in four phase devices, yet requires only three gates per pixel. This scheme differs from four phase clocking by using only one storage gate and two barrier gates, which allows for faster frame rates and the fabrication of higher density and resolution CCDs.
• Two Phase CCD Clocking - A two phase charge transfer CCD clocking scheme employs four gates for each pixel, with adjacent gates connected together as pairs. Each gate pair is connected to an alternate clock line and one of the gates in each pair is designed with an increased n-type doping level beneath the gate. When voltage is applied to the gate pair, the gate having the increased doping level has a more positive potential, which increases the depth of the charge storage area and results in a "step" in the potential energy profile.

CCD Scanning Formats - Charge-coupled device (CCD) digital imaging sensors are capable of acquiring images in one of three formats: point scanning, line scanning, and area scanning. Each of these formats has specific applications in digital photography and scanning of documents and images.
Full-Frame CCD Architecture - Full-frame charge-coupled devices feature high-density pixel arrays capable of producing digital images with the highest resolution currently available. This CCD architecture has been widely adopted due to the simple design, reliability, and ease of fabrication.
Frame-Transfer CCD Architecture - Frame-Transfer charged coupled image sensors have an architecture similar to that of full-frame CCDs. These devices have a parallel register that is divided into two separate and identical areas, termed the Image and Storage arrays.
Interline Transfer CCD Architecture - Interline charge-coupled device architecture is designed to compensate for many of the shortcomings of frame-transfer CCDs. These devices are composed of a hybrid structure incorporating a separate photodiode and an associated CCD storage region into each pixel element.
Digital Camera Readout and Frame Rates - Recent imaging applications in widefield fluorescence and confocal microscopy have increasingly centered on the demanding requirements of recording rapid transient dynamic processes that may be associated with a very small photon signal, and which often can only be studied in living cells or tissues. Technological advances in producing highly specific fluorescent labels and antibodies, as well as dramatic improvements in camera, laser, and computer hardware have contributed to many breakthrough research accomplishments in a number of fields. As high-performance camera systems, typically employing low-noise cooled charge-coupled device (CCD) detectors, have become more capable of capturing even relatively weak signals at video rates and higher, certain performance factors necessarily take on greater importance. A camera system's readout rate and frame rate are interrelated parameters that are crucial to the ability of the system to record specimen data at high temporal frequency.
Microlens Arrays - Microlens arrays (also referred to as microlenticular arrays or lenslet arrays) are used to increase the optical fill factor in CCDs, such as interline devices, that suffer from reduced aperture due to metal shielding. These tiny lens systems serve to focus and concentrate light onto the photodiode surface instead of allowing it to fall on non-photosensitive areas of the device, where it is lost from the imaging information collected by the CCD.
The MOS Capacitor - At the heart of all charge-coupled devices (CCDs) is a light-sensitive metal oxide semiconductor (MOS) capacitor, which has three components consisting of a metal electrode (or gate), an insulating film of silicon dioxide, and a silicon substrate.
Avalanche Photodiodes - Avalanche photodiodes provide gain by the generation of electron-hole pairs from an energetic electron that creates an "avalanche" of electrons in the substrate.
Photomultiplier Tubes - A photomultiplier tube, useful for light detection of very weak signals, is a photoemissive device in which the absorption of a photon results in the emission of an electron. These detectors work by amplifying the electrons generated by a photocathode exposed to a photon flux.
Proximity-Focused Image Intensifiers - Image intensifiers were developed for military use to enhance our night vision and are often referred to as wafer tubes or proximity-focused intensifiers. They have a flat photocathode separated by a small gap on the input side of a micro-channel plate (MCP) electron multiplier and a phosphorescent output screen on the reverse side of the MCP.
Electron-Bombarded CCDs - The electron-bombarded charge-coupled device (EBCCD) is a hybrid of the image intensifier and the CCD camera, which has just been recently introduced. In this device, photons are detected by a photocathode similar to that in an image intensifier. The released electrons are accelerated across a gap and impact on the back side of a back-thinned CCD.
Electron Multiplying CCDs - An innovative method of amplifying low-light-level signals above the CCD read noise is employed in electron multiplying CCD technology. By incorporating on-chip multiplication gain, the EMCCD achieves, in an all solid-state sensor, the single-photon detection sensitivity typical of intensified or electron-bombarded CCDs at much lower cost and without compromising the quantum efficiency and resolution characteristics of the conventional CCD structure.
Sequential Color CCD Systems - Three-pass sequential color CCD imaging systems employ a rotating color wheel to capture three successive exposures in order to obtain the desired RGB (red, green, and blue) color characteristics of a digital image. The major advantage of this technique is the ability to fully utilize the entire pixel array of a CCD imaging chip, by using one pass for each color.
Digital Imaging Web Resources - As solid-state CCD technology rapidly overtakes traditional film and camera techniques, the number of digital camera manufacturers is constantly expanding to meet the needs of the industry. The links provided in this section point to the important manufacturers, trade magazines, and technical support information.

Contributing Authors
Mortimer Abramowitz - Olympus America, Inc., Two Corporate Center Drive., Melville, New York, 11747.
Kenneth R. Spring - Scientific Consultant, Lusby, Maryland, 20657.
John C. Long, Kirill I. Tchourioukanov, and Michael W. Davidson - National High Magnetic Field Laboratory, 1800 East Paul Dirac Dr., The Florida State University, Tallahassee, Florida, 32310.

CMOS Sensor Operation

Until recently the industrial digital vision sensor market was dominated by the CCD array. However technological advances in CMOS production techniques have led to a gradual increase in the popularity of this sensor type. Like CCD arrays, CMOS sensors are also formed on a silicon substrate but the structure is more akin to that of other CMOS technology such as RAM and ROM memory devices.
The diagram below is that of an actual CMOS sensor showing the active pixel area in green and the area occupied by the on chip circuitry in yellow, which replaces that of the shuttered area on a CCD based sensor. The on chip circuitry actually converts the charge into voltage on each pixel whereas the CCD sensor shifts the charge vertically row by row, and then horizontally pixel by pixel to be converted to voltage when it reaches one or more output nodes. This gives CMOS sensors an advantage when it comes to windowing or a region of interest as the pixels can be read out randomly. CCD sensors can only limit its region of interest vertically with the resulting image always containing the data for the full image width.

The on chip active amplifier and the sampling capacitor give CMOS sensors advantages in terms of speed, full well capacities and much improved response characteristics yet introduce dark current level noise and higher black pixel content. CMOS sensors can also produce higher levels of fixed pattern noise than that of CCD, but this type of noise can be easily removed with a software filter.

The development of CMOS sensor technology has been a rapid and varied process. The initial aim of CMOS sensors was to match the imaging performance of CCD technology, with lower power requirements and at less cost. To achieve this performance it was discovered that a much greater level of manufacturing process adaptation and deeper submicron lithography were required than initially expected. This led to the desired CMOS performance but increased development costs more than anticipated.

At first the low power feature of the CMOS imaging sensors was set to be one of their distinct advantages, however the improved development of CCD sensors means that while CMOS has the advantage in this area, the margin is now much smaller.

The integration of on chip control circuitry with the CMOS imager provides the sensor with greater flexibility and integration, the downside has been the introduction of greater noise levels. Both CMOS and CCD imaging sensors still require support chips to process the image, however CMOS imagers can be produced with more functionality on the sensor chip, as shown below.

The spectral response of a CMOS sensor differs from that of the CCD sensors in that the peak response is sited at around 700Nm. Both sensors operate over the same range, typically 200Nm to 1100Nm.

The main advantages of CMOS imaging sensors still remain as faster response, increased integration flexibility and lower on-chip power demands. However the image quality has yet to match that of the CCD and the supporting chips required to increase the CMOS image quality goes some way to squander its previous advantages. Yet neither sensor is categorically superior to the other. They both have their own advantages and disadvantages and with CMOS developers working on the image quality, and CCD developers aiming to reduce power demands and increase flexibility, the existing margins in place to decide which sensor is most suitable for an application look to narrow further.