Watson Studio Pipeline Demo
In today’s world, we are observing a trend to integrate AI/ML models into applications. However, It is critical to synchronize between the application and model pipelines to ensure consistent results. Watson Studio Pipelines helps to automate the AI lifecycle. It builds on these practices:
- DevOps for bringing a machine learning model from creation through training to deployment, and finally to production.
- MLOps for managing the lifecycle of a traditional machine learning model, including evaluation and retraining.
Watson Studio Pipeline extends MLOps to include the routine deployment of machine learning models and the continuous retraining, automated updating, and synchronized development and deployment of more complex machine learning models, such as optimization.
Watson Studio Pipeline allows you to train, deploy, and score machine learning models and then evaluate them for bias and drift. In this article, I will show you how to create a Watson Studio Pipeline demo.
Note: This tool is provided as a beta release and is not supported for use in production environments.
Pre-requisites
- IBM Cloud Pak for Data subscription — Sign up and try for free
- A project created in IBM Cloud Pak for Data subscription
- A deployment space created and a Machine learning service associated with it
- A sample data file uploaded to “Assets” in the “Deployment space” — Download from this location
- Watson Studio Pipeline (Beta Release) access
Step-by-step instructions
- Open a project in the IBM Cloud Pak for Data
- Click “Add to project +” and choose Pipeline.
- Enter a name and optional description and tags.
- Click Create and drag “Create AutoAI experiment” from the palette onto the pipeline canvas.
- Double click on Create AutoAI experiment, It will open up the configuration setting box.
- Give AutoAI experiment Name “Pipeline-Demo.”
- Click on “Select resource” underneath Scope and select the deployment space and click Choose.
- Select “Binary classification” under “Prediction type.”
- Give Prediction column (label) as “Churn.”
- Type 4 under “Algorithms to use”
- Leave all other optional parameters blank and click Save.
- Click on Run and drag “Run AutoAI experiment” from the palette onto the pipeline canvas.
- Connect “Create AutoAI experiment” and “Run AutoAI experiment” by dragging the downward arrow from “Create AutoAI experiment” and connecting to the upward arrow of “Run AutoAI experiment.”
- Double click “Run AutoAI experiment” to configure the setting
- Click the folder icon near the AutoAI experiment and click “Select from another node.”
- Click Select node and click “Create AutoAI experiment.”
- Click “Select resource” underneath Training Data Assets, Click Spaces, click your deployment space name, Click “Data asset,” Click “Customer_churn.csv,” and click choose
- Leave other options blank and click Save.
- Click on Create and drag “Create web service” from the palette onto the pipeline canvas.
- Connect “Run AutoAI experiment” to “Create web service” node
- Double click on Create web service to configure it
- Click folder icon nearby ML asset and click “Select from another node.”
- Select “Run AutoAI experiment” and select “Best Model.”
- Leave other fields blank and click Save.
- Now Click Run and click Trial run.
- The trial run will take around 10–15 mins to complete and deploy web service in the deployment space.
- Once complete, click the hamburger icon on the left-hand side top corner near the IBM Cloud Pak for Data and click on your deployment space name.
- Underneath “Deployments,” you will find your web-service deployment. Click on it to test.
- Click here to copy the test data set in JSON format. Then, click “Test” and paste the data from the downloaded file under the “Enter input data” field.
- Click Predict, and you will see the result in the “Result” box.
If everything is working fine, then it’s time to create a Job that would run on a set schedule interval to deploy the updated ML model into deployment space for the application to use. Follow the below instructions to create a job.
- Go to your project and click on the Pipeline you created earlier
- Click “Run” and click “Create a Job.”
- Give a relevant name to this job and click “Next.”
- Turn on Schedule and click “Start on,” and fill in Date and Time
- Click Repeat click appropriate Frequency from “Minutes, hours, day, week, month.”
- If needed, click exclude days and select days you want to exclude
- Click End on and fill Date and Time and Click Next
- Finally, click “Create” to create a job. It will take you to Job monitoring page to monitor jobs
After job creation, you need to upload the data with the same file name into “Assets” in the deployment space. Waston Studio Pipeline will take care of the rest.
I hope this article will help you set up your MLOps pipeline using Watson Studio Pipeline. For additional ML experiments, visit the following links.
- Watson Studio Pipeline Documentation
- View a full set of v2 Machine Learning sample notebooks.
- Build an AutoAI model following the step-by-step tutorial.
- Build and deploy an AutoAI model using built-in sample data.
