Dense, scale-less descriptors

Establishing correspondences between two images requires matching similar image regions. To do this effectively, local representations must be designed to allow for meaningful comparisons. As we discuss in previous chapters, one such representation is the SIFT descriptor used by SIFT flow. The scale selection required to make SIFT scale invariant, however, is only known to be possible at sparse interest points, where local image information varies sufficiently. SIFT flow and similar methods consequently settle for descriptors extracted using manually determined scales, kept constant for all image pixels. In this chapter we discuss alternative representations designed to capture multiscale appearance, even in image regions where existing methods for scale selection are not effective. We show that (1) SIFTs extracted from different scales, even in low contrast areas, vary in their values and so single scale selection often results in poor matches when images show content at different scales. (2) We propose representing pixel appearances with sets of SIFTs extracted at multiple scales. Finally, (3) low-dimensional, linear subspaces are shown to accurately represent such SIFT sets. By mapping these subspaces to points we obtain a novel representation, the Scale-Less SIFT (SLS), which can be used in a dense manner, throughout the image, to represent multiscale image appearances. We demonstrate how the use of the SLS descriptor can act as an alternative to existing, single scale representations, allowing for accurate dense correspondences between images with scale-varying content.