We propose a new framework for the sampling, compression, and analysis of distributions of point sets and other geometric objects embedded in Euclidean spaces. Our approach involves constructing a tensor called the RaySense sketch, which captures nearest neighbors from the underlying geometry of points along a set of rays. We explore various operations that can be performed on the RaySense sketch, leading to different properties and potential applications. Statistical information about the data set can be extracted from the sketch, independent of the ray set. Line integrals on point sets can be efficiently computed using the sketch. We also present several examples illustrating applications of the proposed strategy in practical scenarios.
Visualizations of Method
Sampling Visualization with rays and Features
(Row 1) Visualization of a two rays (black) through points sampled from various objects (gray). Closest point pairs are shown in green and red. (Row 2) Plot of distance from points along the ray to the corresponding closest points on the object. (Rows 3-5) The $x$, $y$, and $z$ coordinates of the closest points to the ray.
Visualization of Voronoi Cell with Rays
Two rays (black)
sense nearest neighbors of the point set (blue). Singular
points, such as the tip of the tail, have larger Voronoi cells
(dashed lines) and are more likely to be sampled. Closest
point pairs are shown in green and red.
Visualization of Sampling Frequencies
Rays are more likely to sample salient features in the point cloud. Larger points are repeated more often.
We can control the number of points by increasing the number of rays. Each ray contains 30 sample points.
Downstream Applications
Salient Points Detection
RaySense Sampling with Top 10 Frequencies
Uniformly Random Sampling
Comparison of RaySense sampled points and points from uniformly random sampling of the MNIST dataset. RaySense sampled points are those with the highest RaySense sampling frequencies for each class. These points correspond to digits written using unusual strokes, hence giving a notion of salient points of the dataset.
Radon Transform and Reconstruction
Underlying density and point sets
RaySensed points from uniform lines
Integral transformed / spectrum
Reconstruction results
Approximate Radon transform computed with RaySense from point cloud data (a)–(c) and filtered back projection reconstruction (d). Our theoretical analysis also suggests the line integral error from RaySense approximation of an $N$-size point cloud in $\mathbb R^d$ converge in a rate of $\mathcal O\left(N^{-\frac{1}d}\right)$.
Classification using Deep Neural Networks
Neural network architecture for point cloud classification based on ray signatures.
Comparison of our neural network (RayNN) against other prominent methods on ModelNet40 dataset with missing data.
Our model is robust, even when the data consists of only a few number of points.
Comparison of our deep neural network model against PointNet on ModelNet point cloud classification dataset.
Model
ModelNet10 (2048)
ModelNet40 (2048)
ModelNet40 (1024)
ModelNet40 (4096)
PointNet (paper)
-
-
89.2%
-
PointNet.pytorch
92.07% (on 256 pts)
85.3% (on 1024 pts)
87.52%
85.4%
Ours
95.04%
90.03%
90.6%
90.44%
Sensitivity of the neural network models to the input size. Here, $m$ is the number of rays, $N^\ast$ is the number of points used, and $N$ is the total number of points in the original point set.
Our model on ModelNet10 (N=2048)
$m/32$
100%
50%
25%
$N^\ast/N$
15.38%
8.59%
4.59%
Performance/Accuracy
94.60%
95.04%
94.60%
Our model on ModelNet40 (N=1024)
$m/32$
100%
50%
25%
$N^\ast/N$
25.10%
14.75%
8.17%
Performance/Accuracy
90.56%
90.60%
89.82%
PointNet on ModelNet40 (N=1024)
$N^\ast/N$
100%
50%
12.5%
Performance/Accuracy
89.2%
86.8%
69%
Publications
Liu, L., Ly, L., Macdonald, C. & Tsai, R. (2023) Nearest Neighbor Sampling of Point Sets using Rays. Communication on Applied Mathematics and Computation (CAMC), Focused Issue in Honor of Prof. Stanley Osher on the Occasion of His 80th Birthday. Accepted.
