datafold: data-driven models for point clouds and time series on manifolds

Ever increasing data availability has changed the way how data is analyzed and interpreted in many scientific fields.While the underlying complex systems remain the same, data measurements increase in both quantity and dimension.The main drivers are larger computer simulation capabilities and increasingly versatile sensors.In contrast to an equation-driven workflow, a scientist can use data-driven models to analyze a wider range of systems, including those with unknown or intractable equations.The models can be applied to a variety of data-driven scenarios, such as enriching the analysis of unknown systems or merely serve as an equation-free surrogate by providing fast, albeit approximate, responses to unseen data.However, expanding datasets create challenges throughout the analysis workflow from extracting and processing to interpreting the data.This includes the fact that new data does not always provide completely new and uncorrelated information to existing data.One way to extract the essential information is to understand and parametrize the intrinsic data geometry.An intrinsic geometry is what most data-driven models assume implicitly or explicitly in the available data, and successful machine learning algorithms adapt to this underlying structure for tasks like regression or classification (e.g., Bishop, 2006).This geometry is often of much lower dimension than the ambient data space, and finding a suitable set of coordinates can reduce the complexity of the dataset.We refer to this geometric structure encoded in the data as a "manifold".In mathematical terms, a manifold is a topological space that is locally homeomorphic to Euclidean space.Typically, manifold learning attempts to construct a global parametrization (embedding) of this manifold, in a space of much lower dimension than the original ambient space.The well-known manifold hypothesis states that such manifolds underlie many observations and processes, including time-dependent systems.