Colorful image reconstruction from neuromorphic event cameras with biologically inspired deep color fusion neural networks

Hadar Cohen-Duwek, Elishai Ezra Tsur

نتاج البحث: نشر في مجلةمقالةمراجعة النظراء

ملخص

Neuromorphic event-based cameras communicate transients in luminance instead of frames, providing visual information with a fine temporal resolution, high dynamic range and high signal-to-noise ratio. Enriching event data with color information allows for the reconstruction of colorful frame-like intensity maps, supporting improved performance and visually appealing results in various computer vision tasks. In this work, we simulated a biologically inspired color fusion system featuring a three-stage convolutional neural network for reconstructing color intensity maps from event data and sparse color cues. While current approaches for color fusion use full RGB frames in high resolution, our design uses event data and low-spatial and tonal-resolution quantized color cues, providing a high-performing small model for efficient colorful image reconstruction. The proposed model outperforms existing coloring schemes in terms of SSIM, LPIPS, PSNR, and CIEDE2000 metrics. We demonstrate that auxiliary limited color information can be used in conjunction with event data to successfully reconstruct both color and intensity frames, paving the way for more efficient hardware designs.

اللغة الأصليةالإنجليزيّة
رقم المقال036001
دوريةBioinspiration and Biomimetics
مستوى الصوت19
رقم الإصدار3
المعرِّفات الرقمية للأشياء
حالة النشرنُشِر - 4 مارس 2024

ملاحظة ببليوغرافية

Publisher Copyright:
© 2024 IOP Publishing Ltd.

© 2024 IOP Publishing Ltd.

بصمة

أدرس بدقة موضوعات البحث “Colorful image reconstruction from neuromorphic event cameras with biologically inspired deep color fusion neural networks'. فهما يشكلان معًا بصمة فريدة.

قم بذكر هذا