Neural Space-filling Curves

Abstract

We present Neural Space-filling Curves (SFCs), a data-driven approach to infer a context-based scan order for a single image or a set of images. Linear ordering of pixels forms the basis for many applications such as video scrambling, compression, and auto-regressive models that are used in generative modeling for images. Existing algorithms resort to a fixed scanning algorithm such as Raster scan or Hilbert scan. Instead, our work learns a spatially coherent linear ordering of pixels from the dataset of images using a graph-based neural network. The resulting Neural SFC is optimized for an objective suitable for the downstream task when the image is traversed along with the scan line order. We show the advantage of using Neural SFCs in downstream applications such as image compression.

Publication
European Conference on Computer Vision 2022

Comparison between Hilbert curves and Neural SFCs

Comparison between 1D pixel sequences flattened using Neural SFCs (up) and Hilbert curves (down) on FFHQ dataset

Comparison between 1D pixel sequences flattened using Neural SFCs (up) and Hilbert curves (down) on Fashion-MNIST dataset

Hanyu Wang
Hanyu Wang
Ph.D. Student

My research interests include computer vision and deep learning.