Estimated reading time: 4 minutes

We all know how crucial training data for data scientists is to build quality machine learning models. But when productionizing Machine Learning, Metadata is equally important.

Consider for example:

• Capture of Lineage Information (e.g., Which dataset influences which Model?)
• Capture of Audit Information (e.g, A given model was trained two months ago with the following training/validation performance)
• Reproducible Model Training
• Model Serving Policy (e.g., Which model should be deployed in production based on training statistics)

Over the past two years, many of our customers have productionized their machine learning pipelines. Most pipeline components create some kind of metadata which is important to learn from.

This metadata is often unstructured (e.g. Tensorflow’s training metadata is different from PyTorch), which fits nicely into the flexibility of JSON, but what creates the highest value for DataOps & Data Scientists is when connections between this metadata is brought into context using graph technology…. so, we had this idea… and made the result open-source.

We are excited to share ArangoML Pipeline with..