R/python-based web tools at CSC for reproducible research and teaching

All material (C) 2022-2023 by CSC -IT Center for Science Ltd. This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 Unported License, http://creativecommons.org/licenses/by-sa/4.0/

Outline

  • Jupyter and RStudio
  • Motivation for using CSC environment
  • Getting started with R/Python notebooks at CSC
  • Selecting right environment for your needs
  • Custom notebooks (advanced)

Notebooks environments - general

Jupyter Notebooks/Lab

RStudio

Motivation: Why use Jupyter/Rstudio?

  • Easy-to-use, “familiar”
  • Visually pleasing interfaces
  • Interactivity
  • Hides complexity

→ Ideal for courses, quick prototyping and testing

Motivation: Why use CSC services for Jupyter/RStudio?

  • Free-of-charge (for Finnish higher education and research institutes)
  • Resource availability and their scalability
  • Easy-to-use services
  • Customization possibilities
  • Reproducibility of enviroment
  • Same starting point for all students
  • CSC specialist support

CSC’s options for teaching R/Python-based courses

→ All provide access via web browser and are customizable

Puhti Web Interface

  • Access to supercomputing power via graphical user interfaces
  • Not limited to Jupyter and RStudio, but many more applications available
  • Requires teacher and students to have CSC account and be part of CSC project with Puhti access
  • Access to GPUs
  • Customization via containers/modules

CSC Notebooks

  • Direct access via HAKA/VIRTU/CSC account for students/teacher
  • Grouped usage support
  • Highly automated
  • Defaults do not require deep technical knowledge
  • Customization via containers

Rahti Container Cloud

  • Platform as a service (PaaS)
  • Requires CSC account and CSC project with Rahti access for teacher
  • Templates available for RStudio/ Jupyter
  • Requires know-how for setup (containers, admin)
  • Highly customizable

Environment selection

Puhti Web Interface Is For You If

  • You can use HAKA/VIRTU/CSC accounts
  • You have a CSC project with access to Puhti service
  • You need a lot of computing resources (CPUs, GPUs…)
  • You want to make use of data/software modules already available on Puhti

CSC Notebooks Environemnt Is For You If

  • You can have even students/collaborators that do not have HAKA/VIRTU/CSC account
  • You don’t need much resources
  • You want to make use of any of the ready-to-use templates available

Rahti Container Cloud is for you if

  • You have CSC account (= CSC project with Rahti service)
  • You have special needs of running job for long time
  • You need light-weight computing (not ideal for heavy computing; ideal for running microservices)
  • You have advanced skills in openshift deployments (templates are available for R and JupyerHub but most of the stuff needs to be figured out by yourself)

Customization possibilities

Custom Notebooks for Your Needs: Puhti Web Interface

  • Use Jupyter for python-based courses
  • Install necessary packages as container using Tykky on Puhti
  • Create a course environment/module(s) using CSC templates (see tutorial link below)
  • Access custom notebook on Puhti web interface via “Jupyter for courses” under “Interactive Sessions”
  • Useful CSC documentation:

Custom Notebooks for Your Needs: CSC Notebooks

  • Login to CSC Notebooks and make sure that you have workspace owner rights
  • Create a custom image and add it to a registry (e.g., Rahti registry)
  • Create an application in the workspace (fill a template for launching your application)
  • Invite and promote the users as needed for your course
  • Useful CSC documentation:

Custom Notebooks for Your Needs: Rahti Container Cloud

  • Login to Rahti Web Interface (or use commandline OC tools)
  • Create a custom image and add it to a registry or build from source
  • Use existing templates on Rahti web interface or OC commnds to launch applications
  • Note: docker images work but the application has to be run under “certain user”
  • Useful CSC documentation: