pachyderm

Pachyderm tl; dr; A combinator data component that installs Pachyderm, a data lineage and pipelining solution. - Introduction - Test Drive - Usage Introduction Pachyderm is an open-source-driven solution that provides data lineage and pipelines. Data lineage is important for \_provenance\_; knowing the origin of downstream assets. In ML, the assets are often models and the provenance describes how the model became to be. Precise knowledge of what a model was trained upon is important for disaster recovery, auditing, and robustness. Pipelines encode a process. This can be anything from automating pre-processing, to training and deploying models. Pachyderm's solution is unique beacuse it is backed by data lineage; i.e. data driven pipelines, not process driven ones. Test Drive The fastest wa