Deploy Kubeflow using MiniKF on Google Cloud

Deploy Kubeflow with MiniKF (mini Kubeflow) via Google Cloud Marketplace

MiniKF latestversion

This guide describes how to launch a MiniKF virtual machine (VM) on Google Cloud. MiniKF is a single VM solution on the Google Cloud Marketplace, and installs:

Kubernetes (using Minikube)
Kubeflow
Kale, a tool to convert general purpose Jupyter Notebooks to Kubeflow Pipelines workflows (GitHub)
Arrikto’s Rok, a data management software for data versioning and reproducibility

MiniKF supports NVIDIA GPUs.

Before you start

Before installing MiniKF, make sure that your Google Cloud project meets the minimum requirements described below:

Select or create a project on the Google Cloud Console.
Make sure that you have the editor role role for the project. You can add a member to a project and grant them a Cloud IAM role using the Cloud Console.
Make sure that billing is enabled for your project. For more info, see the Modify a Project’s Billing Settings guide.
If you are using the Google Cloud Free Tier or the 12-month trial period with $300 credit, note that you can’t run the default Google Cloud installation of MiniKF, because the free tier does not offer enough resources. You need to upgrade to a paid account.
Read the Google Cloud guide to resource quotas to understand the quotas on resource usage that Compute Engine enforces, and to learn how to check your quota and how to request an increase in quota.

Installing MiniKF on Google Cloud

To install MiniKF on Google Cloud, follow the steps below:

Go to the MiniKF page on Google Cloud Marketplace.
Click on the LAUNCH button
In the Configure & Deploy window, choose:

Deployment name: a name for your MiniKF instance
Zone: the Google Cloud zone that you will deploy MiniKF
Machine type: a machine type with at least 2 vCPUs and 16 GB of RAM. If in doubt, use the default machine type, n1–standard-8, which has 8 vCPUs and 30 GB of RAM.
Boot Disk: a boot disk. This is the disk that holds the images that MiniKF needs. Use the default value of 200 GB SSD Persistent Disk.
Data Disk: a data disk. This is the disk that holds your snapshots. Use an SSD Persistent Disk of at least 500 GB capacity.
GPUs: add one or more GPUs if you are running computationally intensive ML workloads.

Then click Deploy:

Now wait for the VM to boot.
When the VM is up, ensure that MiniKF is up and running. Before visiting the MiniKF dashboard, ensure that Minikube, Kubeflow, and Rok (Snapshot Store) are up and running. To do so, click on the SSH button to connect to the MiniKF VM and follow the on-screen instructions.
MiniKF dashboard, username, and password. Now that Minikube, Kubeflow, and Rok are up-and-running, close the SSH browser window, as you will not need it anymore. On your current view, please find the MiniKF dashboard URL, MiniKF username, and MiniKF password. You will need them to log in to Kubeflow and Rok. You should see a table like the one below:
Log in to MiniKF. Visit the MiniKF dashboard URL and log in to MiniKF using the MiniKF username and password.

Once you log in, you will see the Central Dashboard