Real-Time Refocusing using an FPGA-based Standard Plenoptic Camera

Plenoptic cameras are receiving increasing attention in scientific and commercial applications because they capture the entire structure of light in a scene, enabling optical transforms (such as focusing) to be applied computationally after the fact, rather than once and for all at the time a picture is taken. In many settings, real-time interactive performance is also desired, which in turn requires significant computational power due to the large amount of data required to represent a plenoptic image. Although GPUs have been shown to provide acceptable performance for real-time plenoptic rendering, their cost and power requirements make them prohibitive for embedded uses (such as in-camera). On the other hand, the computation to accomplish plenoptic rendering is well-structured, suggesting the use of specialized hardware. Accordingly, this paper presents an array of switch-driven Finite Impulse Response (FIR) filters, implemented with FPGA to accomplish high-throughput spatial-domain rendering. The proposed architecture provides a power-efficient rendering hardware design suitable for full-video applications as required in broadcasting or cinematography. A benchmark assessment of the proposed hardware implementation shows that real-time performance can readily be achieved, with one order of magnitude performance improvement over a GPU implementation and three orders of magnitude performance improvement over a general-purpose CPU implementation.

Citation

Hahne, C., Lumsdaine, A., Aggoun, A. and Velisavljevic, V. (2018) Real-time refocusing using an FPGA-based standard plenoptic camera, IEEE Transactions on Industrial Electronics, 65 (12), pp. 9757-9766, doi: 10.1109/TIE.2018.2818644

Type

Journal article

Language

en

Description

This is an accepted manuscript of an article published by IEEE in IEEE Transactions on Industrial Electronics on 22/03/2018, available online: https://doi.org/10.1109/TIE.2018.2818644 The accepted version of the publication may differ from the final published version.

ISSN

0278-0046

EISSN

1557-9948

Collections

Faculty of Science and Engineering