Release Notes Highlights for MLDM

2.6.1

January 1, 0001

  • Enhancement: Improved handling of symlinks with nullptr check.
  • Enhancement You can now set the name of your connection when using pachctl connect.
  • Fix: Resolved an issue where pachctl list commit was showing inconsistent sizes when used with/without branch.
  • Fix: Resolved an issue that caused the file browser in Pachyderm’s PPS extension to unexpectedly jump back to the top level while inspecting notebook outputs within the /pfs/out directory.

2.6.2

January 1, 0001

  • Enhancement: Projects without repos are now listed within the Jupyterlab Mount Extension UI.
  • Enhancement: A Project field has been added to the Jupyterlab Pipeline Extension. Previously, you would define the project using the Pipeline Name field (for example, project/pipeline). The default value for the Project field is the ‘default’ project.

2.6.3

January 1, 0001

  • Fix: Resolved an issue where some users upgrading from 2.5.0 to 2.6.x would experience the following error: duplicate key value violates unique constraint 'commit_totals_pkey'.

2.6.4

January 1, 0001

  • Enhancement: Provenance migration for 2.6 now skips migrating any missing provenance.
  • Enhancement: The Jupyter Pipeline Extension’s entrypoint has been set to run unbuffered so that output is immediately written to stdout.
  • Enhancement: A returned_for field has been added to roles; This field indicates the resources the role is returned for, even if the role wasn’t initially bound to those specific resources.

2.6.5

January 1, 0001

  • Enhancement: Additional PachD Probe settings (readinessProbe, livenessProbe, and startupProbe) have been added to the Helm Chart values.yaml file for added flexibility. If not specified, the default values are used.
  • Enhancement: General improvements to the upgrade process from 2.4.x > 2.6.5

2.6.6

January 1, 0001

  • Enhancement: The DAG view in Console has been refreshed to have a new look and feel.
  • Refactor: Cron Triggers have been refactored to perform as expected, where you can set up a scheduled reoccurring event on a repo branch that evaluates and fires the trigger. When a Cron Trigger fires, but no new data has been added, there are no new downstream commits or jobs.
  • Fix: Fixed an issue in Console for users using Safari where the scrollbar wasn’t working as expected.

2.6.7

January 1, 0001

  • Enahncement: You can now create service pipelines via the Jupyter Extension.
  • Enhancement: The logs viewer in Console has been improved to be more performant.

2.6.8

January 1, 0001

  • Enhancement: GPU support has been added to the Jupyter Pipeline Extension
  • Enhancement: When you create a service pipeline via the Jupyter Extension, the IP address of the created load balancer is now shown.
  • Fix: Fixed an issue with branch triggers where filtering by source branch was not working as expected.

2.6.9

January 1, 0001

  • Feature: The preflight check features for 2.7.0 have been backported to 2.6.9. HPE Machine Learning Data Management’s helm chart now has a section for preflight checks, allowing you to easily validate whether the upgrade/migrations will be successful via a dry run. This section can be found at pachd.preflightchecks. Simply set enabled: true and set the image.tag to the new version you want to upgrade to. If created the pod named pachyderm-preflight-check shows a status of Completed, you are ready to perform the upgrade. See the Upgrade steps for more information.

2.7.0

January 1, 0001

  • Feature: The new Pachyderm SDK is now available. Check out the reference documentation, install guide, and example starter project.
  • Feature: Console now has a runtime visualization for jobs in your pipeline.
  • Feature: The documentation site now has a chatbot to help you find what you’re looking for. This feature is in beta, so please let us know if you have any feedback through our Slack community.
  • Feature: HPE Machine Learning Data Management’s helm chart now has a section for preflight checks, allowing you to easily validate whether the upgrade/migrations will be successful. This section can be found at pachd.preflightchecks. Simply set enabled: true and set the image.tag to the new version you want to upgrade to. If created the pod named pachyderm-preflight-check shows a status of Completed, you are ready to perform the upgrade. See the Upgrade steps for more information.
  • Enhancement: Console’s scalability has been improved to handle more concurrent users (50+) and power users who have many pipelines.
  • Enhancement: Console’s DAG visualization has been upgraded to include more information about the state of your pipelines.
  • Enhancement: The Jupyterlab Pipeline Specification Extension now supports GPUs.
  • Refactor: The functionality of the Branch Cron Trigger has been refactored to work more intuitively. Previously, cron triggers functioned more like rate limiters; now, they enable you to set up a scheduled reoccurring event on a repo branch that evaluates and fires the trigger. When a Cron Trigger fires, but no new data has been added, there are no new downstream commits or jobs. See our Cron glossary entry for more information on crons in HPE Machine Learning Data Management
  • Deprecation: The original Python SDK (python-pachyderm) will be deprecated in 9 months (May 2024). We recommend that you start trying out the new Pachyderm SDK (pachyderm-sdk) and begin planning your transition.