Getting the Most Out of Your Image Sensor: What Is Subsampling?

Full-resolution readout can look good but it may not be what we need. In low light, at high frame rates, or under embedded bandwidth limits, reading every pixel produces a noisier, slower, and more expensive image than reading the sensor in a reduced-resolution mode. Most modern sensors offer several of these modes, and choosing the right one matters more than the megapixel number on the datasheet cover.

Here, subsampling means any sensor readout mode that produces fewer output samples than the full pixel array. Vendors do not always use the word consistently, but in practice this family covers binning, pixel skipping, cropping, and region-of-interest (ROI) readout.^{[1], [2], [3]} They look similar on a block diagram and they all cut active sample count, but they behave very differently on noise, color, and aliasing - and each one presents a different timing profile to whatever FPGA, ISP, or SoC sits on the other end of the link.

Binning, Skipping, Cropping

These modes solve different problems and pay different prices.

Analog binning sums charge or voltage before at least part of the readout noise is added. When the relevant read noise is introduced after summation, it helps most in read-noise-limited scenes: a 2x2 analog bin can approach a 4x (N-times) SNR improvement against that downstream read noise, easing back toward 2x (sqrt(N)) once shot noise dominates.^{[4], [5]} The cost is dynamic range at the bright end, because the summed signal can clip the readout chain sooner.

Digital binning combines pixel values after the ADC. Each pixel carries its own read noise, and those independent noises add in quadrature - so even when read noise dominates you only get sqrt(N) (2x for a 2x2 bin), not the N-times improvement that true pre-readout summation can approach.^[5] Resolution and active sample count shrink the same way as analog binning, which is why analog binning is the mode worth paying for when light is scarce.

Pixel skipping discards pixels rather than combining them. The skipped photons are lost, which is bad for SNR, and the surviving samples form a sparser grid with no anti-alias prefiltering. Moire, jaggies, and false-color on fine textures are the expected result. Skipping wins only when frame rate matters more than image quality: fast viewfinder, AF/AE preview, coarse tracking.^[2]

Cropping / ROI reads part of the array at native pixel quality. The field of view shrinks, but resolution inside the window is preserved. This is the right mode when the application already only cares about a narrow region - barcode scanning, eye tracking, alignment fiducials, high-speed inspection of a fixed feature.^[3]

Bayer Sensors Need Care

On a color sensor, the pixel array sits under a Bayer filter, so adjacent pixels do not represent the same color. A naive 2x2 average mixes R, G, and B and damages color. The workarounds matter:^{[1], [6]}

• Color-aware binning combines same-color pixels across a larger neighborhood, typically a 4x4 footprint that contains four 2x2 same-color groups.
• Quad-Bayer / Tetracell / Nonacell sensors physically group same-color pixels into 2x2 or 3x3 clusters. Binning is a native operating mode; full resolution is reconstructed by remosaicing in the sensor or ISP. These are common in modern smartphone sensors and also appear in specialized high-resolution cameras, so the assumption "1 pixel = 1 color sample" is no longer safe.
• Some sensors change the Bayer phase at the output origin between modes. If the ISP keeps assuming the old phase, the result is shifted color planes that look exactly like a debug artifact.

Two sensors both advertising "2x2 binning" can produce very different color, noise, and resolution behavior. Trust the mode tables in the datasheet, not the marketing block.

What Subsampling Does to the FPGA Side

This is where most subsampling problems actually show up. The image-quality tradeoff is well understood; the timing-profile change rarely is.

A subsampling mode is not a smaller version of the same frame. It changes the shape of everything downstream:

• active width and height^[3]
• line length, frame length, horizontal and vertical blanking
• pixel clock, MIPI CSI-2 data type, lane count, lane rate^[7]
• Bayer phase at the output origin and pixel order in the first line
• embedded metadata position and content
• frame-start and line-start timing relative to the trigger
• downstream buffer stride and DMA alignment

A sensor can advertise the same 12-bit RAW pixel format in two modes and still produce different output geometry or readout behavior between them.^{[1], [3]} If the FPGA ISP, line buffer, or DMA engine was sized for the original mode, the symptoms are easy to misread as algorithm bugs: shifted color, hue-swap on alternate lines, occasional torn frames, intermittent timing failures that only appear at certain frame rates.

Treat each subsampling mode as its own video timing profile. Capture the parameters above from the sensor in that mode, not from the full-resolution datasheet page, and reprogram the pipeline to match.

If the pipeline supports live mode switches, make the switch explicit: stop the stream, update sensor registers, flush in-flight packets and FIFOs, reprogram timing and DMA parameters, then restart. The sequence is unglamorous and prevents most intermittent bring-up issues.

Binning Is Not Software Scaling

Resizing in software runs after the sensor has already read out the full frame. The sensor still pays the readout time, the link still carries the full data rate, and the noise is whatever the full-resolution mode produced. Sensor-level subsampling cuts all of that upstream of the processor - which is why it is worth the configuration effort.

Practical Checklist

Before locking in a subsampling mode, confirm:

• whether the binning is analog, digital, or actually pixel skipping
• whether same-color (Bayer-aware) combination is preserved, and whether the sensor is quad-Bayer
• output resolution, field of view, and maximum frame rate in that mode
• MIPI CSI-2 lane rate, data type, and blanking against the FPGA link budget
• Bayer phase, line stride, and DMA alignment at the new output dimensions
• exposure, gain, black-level, defect correction, and HDR behavior in the mode - these often differ
• whether the downstream algorithm gains more from lower noise than it loses from lower resolution

The Bottom Line

Subsampling trades spatial detail for sensitivity, frame rate, and bandwidth headroom. Done well - analog binning where light is scarce, ROI where the scene is small, skipping only where image quality is genuinely disposable - it turns a sensor that does not quite fit the system into one that does. Done blindly, it produces noisy, aliased, color-shifted images that look like a different bug entirely. The mode is part of the design, not a runtime afterthought.

References

1. Basler Product Documentation, "Binning".
2. Basler Product Documentation, "Decimation".
3. Thomas J. Fellers and Michael W. Davidson, "Digital Camera Readout and Frame Rates", Hamamatsu Learning Center.
4. EMVA, "EMVA 1288 Standard for Measurement and Presentation of Specifications for Machine Vision Sensors and Cameras".
5. Anqi Yang, Eunhee Kang, Wei Chen, Hyong-Euk Lee, and Aswin C. Sankaranarayanan, "Towards Spatially-Varying Gain and Binning", arXiv, 2025.
6. Haijin Zeng et al., "Inheriting Bayer's Legacy - Joint Remosaicing and Denoising for Quad Bayer Image Sensor", arXiv, 2023.
7. MIPI Alliance, "MIPI CSI-2", interface overview.