FPGA ISP IP Core

What the ISP does

Image sensors don’t produce pictures — they produce raw Bayer data: one color sample per pixel, offset by a black level, speckled with defective pixels, and shaded darker toward the lens edges. An image signal processor (ISP) is the pipeline that turns that raw stream into video the rest of the system can actually use. Running it in the FPGA keeps the whole chain deterministic and low-latency, on the same device that already handles capture — with no dependency on a specific SoC’s built-in ISP or its driver stack.

The processing pipeline

• MIPI CSI-2 input. Raw video enters from a CSI-2 receiver, a GMSL2/FPD-Link deserializer, or our MIPI Aggregator IP core — single sensor or aggregated multi-camera streams.
• Sensor corrections. Black level subtraction, defective pixel correction, and lens shading correction clean up the raw data before any color processing — fixing the artifacts every real sensor and lens introduces.
• Debayer (demosaic). Full RGB is reconstructed at every pixel from the Bayer mosaic, with edge-aware interpolation to avoid the zipper and false-color artifacts of naive approaches.
• White balance. Per-channel gains neutralize the scene illuminant, with statistics collection to support automatic white balance control loops in software.
• Color correction and gamma. A color correction matrix maps the sensor’s color response to the target color space, and gamma encoding matches the output to displays and downstream consumers.
• Statistics for tuning. Per-stage histograms and counters exposed over registers, so image quality tuning and bring-up are driven by data.
• MIPI CSI-2 output. Processed video leaves as standard CSI-2 (or stays on-chip as AXI4-Stream), feeding an SoC capture port, an encoder, or an on-FPGA vision pipeline.

Integration and delivery

• Standard AXI interfaces. Video moves over AXI4-Stream, and every stage is configured and monitored through an AXI4-Lite register interface — so the core drops into existing Vivado/Platform Designer block designs and is controlled from software like any other memory-mapped peripheral.
• Modular pipeline. Stages are sized and selected against your sensor, resolution, and frame rate — you don’t pay in logic for processing you don’t need.
• Vendor-portable RTL. Targeted at AMD/Xilinx devices and portable to Intel/Altera, Lattice, Microchip, and Efinix families.
• Delivered as a working system. RTL, verification environment, integration documentation, and engineering support — including sensor bring-up and image quality tuning. Our image and video pipeline services cover the rest of the chain, from capture to edge AI inference on the processed streams.

Applications

• Embedded and machine vision. Clean, repeatable color video for inspection, measurement, and classification pipelines.
• Automotive and robotics cameras. Deterministic raw-to-color processing for perception stacks, viewing systems, and driver monitoring.
• Custom camera products. Sensor-to-CSI-2 camera modules where the FPGA is the camera — ISP included, no external processor required.
• Multi-camera systems. Combined with the MIPI Aggregator, one ISP pipeline serving several aggregated sensors.

Frequently asked questions

What does an ISP do, and why put it in an FPGA?

An image signal processor turns the raw Bayer data a sensor actually produces into usable color video: correcting sensor artifacts, reconstructing full color at every pixel, and balancing color for the scene. Running it in an FPGA gives deterministic, low-latency processing at the full sensor rate, on the same device that already handles capture — without depending on a specific SoC’s hardwired ISP or its driver stack.

Which processing stages are included?

The pipeline covers the core raw-to-color chain: black level subtraction, defective pixel correction, lens shading correction, debayer (demosaic), white balance, color correction matrix, and gamma, with per-stage statistics for tuning and bring-up. Stages are configurable at run time over AXI4-Lite, and the pipeline is sized to your sensor and application during evaluation.

Does the ISP work with the MIPI Aggregator and GMSL2 cameras?

Yes. The ISP consumes and produces standard CSI-2-style video over AXI4-Stream, so it drops in behind a MIPI CSI-2 receiver, a GMSL2 or FPD-Link deserializer, or our MIPI Aggregator IP core — processing aggregated multi-camera streams as well as a single sensor.

How is the core delivered and licensed?

The core is delivered as RTL with a verification environment, integration documentation, and engineering support, including help with sensor bring-up and image quality tuning. Licensing terms depend on the project — request the product brief for details and current deliverables.