Patternshop: Editing Point Patterns by Image Manipulation

ACM Transactions on Graphics (Proceedings of SIGGRAPH), 2023
1Max-Planck-Institut für Informatik, 2University College London 3CNRS, LIX, Ecole Polytechnique, INRIA

Our framework allows manipulating density and correlations independently. Top row shows the point set synthesized using the density (left) and the correlation map (right) shown in the bottom row. The first column shows a point set when both density and correlation are spatially varying. Second column has constant correlation but spatially varying density. Third column point set reproduces the Siggraph logo just from spatially varying correlations.

Abstract

Point patterns are characterized by their density and correlation. While spatial variation of density is well-understood, analysis and synthesis of spatially-varying correlation is an open challenge. No tools are available to intuitively edit such point patterns, primarily due to the lack of a compact representation for spatially varying correlation. We propose a low-dimensional perceptual embedding for point correlations. This embedding can map point patterns to common three-channel raster images, enabling manipulation with off-the-shelf image editing software. To synthesize back point patterns, we propose a novel edge-aware objective that carefully handles sharp variations in density and correlation. The resulting framework allows intuitive and backward-compatible manipulation of point patterns, such as recoloring, relighting to even texture synthesis that have not been available to 2D point pattern design before. Effectiveness of our approach is tested in several user experiments.





Our method

Firstly, we create a perceptual embedding for point correlations using the following pipeline:

Interpolate start reference image.

The 2D perceptual embedding space can be visualized as spatially-varying point patterns, where each coordinate corresponds to a point correlation. Combining this space with density, we can characterize a point pattern as spatially-varying correlation + density using "3D" coordinates. These "3D" coordinates correspond to a 3-channel raster image (represented by the LAB color format), which can be edited using off-the-shelf image manipulation software such as Adobe Photoshop.

Interpolate start reference image.

The following shows the overview of our editing framework, utilizing the perceptual point correlation embedding. After editing from scratch (ab-initio) or from an existing point pattern (with the help of neural networks), we can get the 3-channel raster image and run our optimizer to synthesize the edited point pattern. More details about the optimizer can be found in the paper.

Interpolate start reference image.




Demos

The videos show the full editing process of Ab-inito point pattern design to generate point patterns with spatially-varying density and correlation. The videos are sped up by up to 16x.


Presentation Video

BibTeX

@article{huang2023patternshop,
      author = {Huang, Xingchang and Ritschel, Tobias and Seidel, Hans-Peter and Memari, Pooran and Singh, Gurprit},
      title = {Patternshop: Editing Point Patterns by Image Manipulation},
      year = {2023},
      issue_date = {August 2023},
      publisher = {Association for Computing Machinery},
      address = {New York, NY, USA},
      volume = {42},
      number = {4},
      issn = {0730-0301},
      url = {https://doi.org/10.1145/3592418},
      doi = {10.1145/3592418},
      journal = {ACM Trans. Graph.},
      month = {jul},
      articleno = {53},
      numpages = {14},
}