PCA - Event-Based Features

Main contributors Himanshu Akolkar


The new event-based visual sensors provide us with a different paradigm for doing visual computation. But even after a decade of their mainstream use in the community very less work has been done towards using the explicit temporal information obtained from events. Here, we have attempted to implement a PCA based feature detection to generate space-time features that use temporal axis as an additional dimension to get more information.

Feature generation

The main idea here is to divide the events from a long recording of data into small event clusters. We then perform PCA on these clusters to obtain new set of axis defined by the principle vectors. Once these PCVs are obtained we find the once that are most different from one-another using an orthogonality test. Finally we obtain a set of principle vectors that can define any new event cluster as shown below.

Please login to view results

Feature testing

To test the features, we took a recording of change events from a ATIS recording for a caricature of a face as stimuli. Then, for each event of this input event-stream, an event-cluster was obtained by taking n-events before and after this event. We then define this cluster as a linear combination of the feature PCVs. To make things easier and to save time, the PCV with the highest weight is considered here as winner and said to define this event. The result is shown below. As can be noted the PCVs indicated by different colours not only are selective to different orientations of the edges but are also selective to the direction in which the edges are moving thus providing a single truly spatio-temporal feature.

Please login to view results