This document discusses each of the fields present in a pipeline specification.

Before You Start

  • HPE Machine Learning Data Management’s pipeline specifications can be written in JSON or YAML.
  • HPE Machine Learning Data Management uses its json parser if the first character is {.
  • A pipeline specification file can contain multiple pipeline declarations at once.

Minimal Spec

Generally speaking, the only attributes that are strictly required for all scenarios are and transform. Beyond those, other attributes are conditionally required based on your pipeline’s use case. The following are a few examples of common use cases along with their minimally required attributes.