Embedding and projection

Overview

Embedding and projection methods transform selected dataset features into a lower-dimensional coordinate space.

The resulting representation can be inspected directly, compared across runs, used for two- or three-dimensional visualization, or supplied as the analytical space for clustering.

Supported embedding and projection methods

Different methods preserve or emphasize different aspects of dataset structure.

• Linear projection — PCA: Constructs orthogonal components that capture decreasing amounts of dataset variance. [Linear, Projection]
• Nonlinear embedding — UMAP: Constructs a lower-dimensional representation based on neighborhood relationships. [Nonlinear, Neighborhood based]
• Nonlinear embedding — t-SNE: Creates a representation emphasizing local similarity between observations. [Nonlinear, Local structure]
• Nonlinear embedding — MDS: Places observations in a lower-dimensional space while approximating pairwise distances. [Distance based, Projection]
• Neural representation — Autoencoder: Learns a compressed representation through a neural encoder-decoder architecture. [Neural, Learned representation]
• Neural representation — Variational Autoencoder: Learns a probabilistic latent representation using a variational encoder-decoder architecture. [Neural, Probabilistic]
• Neural representation — DEC: Uses a learned latent representation designed around deep embedded structure. [Neural, Embedded structure]

How embeddings are used

• Visualization — Inspect the relative arrangement of observations in two or three dimensions.
• Embedded-space clustering — Use generated coordinates as the analytical input to a clustering method.
• Cross-run comparison — Compare how alternative embedding methods and configurations represent the same dataset.
• Persisted artifacts — Preserve generated coordinates together with their run configuration and downstream outputs.

Interpretation and limitations

• Visual separation in an embedding is not automatically evidence of a meaningful real-world group.
• Different methods preserve and emphasize different kinds of relationships.
• Results can depend on feature selection, preprocessing, parameters, and random initialization.
• Distances and apparent group sizes do not necessarily have equivalent meaning across embedding methods.

Relationship to clustering

Embeddings can be displayed as two- or three-dimensional projections or used as input to clustering. These uses should not be treated as equivalent without considering the method and configuration.