3D Object Tracking with Neuromorphic Event Cameras via Image Reconstruction

Hadar Cohen Duwek; Avinoam Bitton; Elishai Ezra Tsur

doi:10.1109/BioCAS49922.2021.9644652

3D Object Tracking with Neuromorphic Event Cameras via Image Reconstruction

Hadar Cohen Duwek, Avinoam Bitton, Elishai Ezra Tsur

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

One of the first and most remarkable successes in neuromorphic (brain-inspired) engineering was the development of bio-inspired event cameras, which communicate transients in luminance as events. Here we evaluate the combination of the Channel and Spatial Reliability Tracking (CSRT) algorithm and the LapDepth neural network for the implementation of 3D object tracking with event cameras. We show that following image reconstruction, implemented using the FireNet convolution neural network, visual features are augmented, dramatically increasing tracking performance. We utilized the 3D tracker to neuromorphically represent error-correcting signals. These error-correcting signals can further be used for motion correction in adaptive neurorobotics.

Original language	English
Title of host publication	BioCAS 2021 - IEEE Biomedical Circuits and Systems Conference, Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781728172040
DOIs	https://doi.org/10.1109/BioCAS49922.2021.9644652
State	Published - 2021
Event	2021 IEEE Biomedical Circuits and Systems Conference, BioCAS 2021 - Virtual, Online, Germany Duration: 6 Oct 2021 → 9 Oct 2021

Publication series

Name	BioCAS 2021 - IEEE Biomedical Circuits and Systems Conference, Proceedings

Conference

Conference	2021 IEEE Biomedical Circuits and Systems Conference, BioCAS 2021
Country/Territory	Germany
City	Virtual, Online
Period	6/10/21 → 9/10/21

Bibliographical note

Publisher Copyright:
© 2021 IEEE.

Keywords

CSRT
FireNet
LapDepth
dynamic visual sensors
neural engineering framework
neuromorphic computing

Access to Document

10.1109/BioCAS49922.2021.9644652

Cite this

Duwek, H. C., Bitton, A., & Tsur, E. E. (2021). 3D Object Tracking with Neuromorphic Event Cameras via Image Reconstruction. In BioCAS 2021 - IEEE Biomedical Circuits and Systems Conference, Proceedings (BioCAS 2021 - IEEE Biomedical Circuits and Systems Conference, Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BioCAS49922.2021.9644652

@inproceedings{c8595a2adf5d40eda69374ba1e60aea6,

title = "3D Object Tracking with Neuromorphic Event Cameras via Image Reconstruction",

abstract = "One of the first and most remarkable successes in neuromorphic (brain-inspired) engineering was the development of bio-inspired event cameras, which communicate transients in luminance as events. Here we evaluate the combination of the Channel and Spatial Reliability Tracking (CSRT) algorithm and the LapDepth neural network for the implementation of 3D object tracking with event cameras. We show that following image reconstruction, implemented using the FireNet convolution neural network, visual features are augmented, dramatically increasing tracking performance. We utilized the 3D tracker to neuromorphically represent error-correcting signals. These error-correcting signals can further be used for motion correction in adaptive neurorobotics.",

keywords = "CSRT, FireNet, LapDepth, dynamic visual sensors, neural engineering framework, neuromorphic computing",

author = "Duwek, {Hadar Cohen} and Avinoam Bitton and Tsur, {Elishai Ezra}",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE.; 2021 IEEE Biomedical Circuits and Systems Conference, BioCAS 2021 ; Conference date: 06-10-2021 Through 09-10-2021",

year = "2021",

doi = "10.1109/BioCAS49922.2021.9644652",

language = "אנגלית",

series = "BioCAS 2021 - IEEE Biomedical Circuits and Systems Conference, Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "BioCAS 2021 - IEEE Biomedical Circuits and Systems Conference, Proceedings",

address = "ארצות הברית",

}

Duwek, HC, Bitton, A & Tsur, EE 2021, 3D Object Tracking with Neuromorphic Event Cameras via Image Reconstruction. in BioCAS 2021 - IEEE Biomedical Circuits and Systems Conference, Proceedings. BioCAS 2021 - IEEE Biomedical Circuits and Systems Conference, Proceedings, Institute of Electrical and Electronics Engineers Inc., 2021 IEEE Biomedical Circuits and Systems Conference, BioCAS 2021, Virtual, Online, Germany, 6/10/21. https://doi.org/10.1109/BioCAS49922.2021.9644652

3D Object Tracking with Neuromorphic Event Cameras via Image Reconstruction. / Duwek, Hadar Cohen; Bitton, Avinoam; Tsur, Elishai Ezra.
BioCAS 2021 - IEEE Biomedical Circuits and Systems Conference, Proceedings. Institute of Electrical and Electronics Engineers Inc., 2021. (BioCAS 2021 - IEEE Biomedical Circuits and Systems Conference, Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - 3D Object Tracking with Neuromorphic Event Cameras via Image Reconstruction

AU - Duwek, Hadar Cohen

AU - Bitton, Avinoam

AU - Tsur, Elishai Ezra

PY - 2021

Y1 - 2021

N2 - One of the first and most remarkable successes in neuromorphic (brain-inspired) engineering was the development of bio-inspired event cameras, which communicate transients in luminance as events. Here we evaluate the combination of the Channel and Spatial Reliability Tracking (CSRT) algorithm and the LapDepth neural network for the implementation of 3D object tracking with event cameras. We show that following image reconstruction, implemented using the FireNet convolution neural network, visual features are augmented, dramatically increasing tracking performance. We utilized the 3D tracker to neuromorphically represent error-correcting signals. These error-correcting signals can further be used for motion correction in adaptive neurorobotics.

AB - One of the first and most remarkable successes in neuromorphic (brain-inspired) engineering was the development of bio-inspired event cameras, which communicate transients in luminance as events. Here we evaluate the combination of the Channel and Spatial Reliability Tracking (CSRT) algorithm and the LapDepth neural network for the implementation of 3D object tracking with event cameras. We show that following image reconstruction, implemented using the FireNet convolution neural network, visual features are augmented, dramatically increasing tracking performance. We utilized the 3D tracker to neuromorphically represent error-correcting signals. These error-correcting signals can further be used for motion correction in adaptive neurorobotics.

KW - CSRT

KW - FireNet

KW - LapDepth

KW - dynamic visual sensors

KW - neural engineering framework

KW - neuromorphic computing

UR - http://www.scopus.com/inward/record.url?scp=85124243956&partnerID=8YFLogxK

U2 - 10.1109/BioCAS49922.2021.9644652

DO - 10.1109/BioCAS49922.2021.9644652

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85124243956

T3 - BioCAS 2021 - IEEE Biomedical Circuits and Systems Conference, Proceedings

BT - BioCAS 2021 - IEEE Biomedical Circuits and Systems Conference, Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2021 IEEE Biomedical Circuits and Systems Conference, BioCAS 2021

Y2 - 6 October 2021 through 9 October 2021

ER -

3D Object Tracking with Neuromorphic Event Cameras via Image Reconstruction

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this