// JupyterHub

NCCS resources on both Discover and ADAPT are now accessible through JupyterHub. Users are now able to run Jupyter notebooks and JupyterLab sessions leveraging the back-end resources they already access. This instructional will give an overview of how to access our JupyterHub instances, and how you can get started running your work through the new interface.

Instructional Videos

For more information on Anaconda environments, check out the “Anaconda on ADAPT” video in the full collection of NCCS Instructional videos .

Connection

We currently have the following JupyterHubs deployed:

• Discover
• Prism GPU Cluster
• ICESat-2
• RomanST

Please note: Access to some of the JupyterHub sites may only be accessed from the NASA network or via VPN.

When attempting to connect to any of the hubs, you may be prompted to submit a NAMS request. If so, you will have to create a NAMS request for either 'NCCS Discover JupyterHub' for accessing the Discover hub, or 'NCCS Web Services' for any of the ADAPT hubs. Note that you will only be able to submit NAMS requests from the NASA network/VPN. Once you have submitted your request, please send a ticket to NCCS support stating which specific JupyterHub you are requesting and we will ensure that you have access.

Logging In

Once you have connected to one of the above hubs, you will need to log in with your NCCS LDAP username and password. If you have forgotten or need to reset your password, click here.

Session Allocation

Session allocation on JupyterHub is handled through the Slurm scheduler. While you do not need to know how to use Slurm to use JupyterHub, it is useful to know that your Jupyter user session will be effectively a Slurm job running on the backend compute associated with the hub. Once you have logged into one of the hubs, you will be presented with a drop-down menu to select a job profile. These options will provide different tiers of resources to be allocated to your session. Keep in mind that, if there aren’t resources available, you will not be able to get a session. If your use case seems to require a different job profile, please submit a ticket to NCCS Support by messaging [email protected].

For users on the Discover JupyterHub, the session is charged to your default project allocation. We currently do not support non-default allocation selection when spawning Jupyter sessions. We are working on integrating this functionality in the future. At this time when your session starts it will be billed to your default allocation. Please note that any jobs that you submit through the Jupyter session will not be constrained to this default. If you would update your default allocation, please contact support.

Once you have an allocation, whatever you do inside of the session will be constrained to the resources allocated by Slurm. If you are running multiple notebooks, consoles, and/or terminals, they will be competing for your own resources. To see what is running in your session:

• In JupyterLab, click the icon on the left hand side that looks like a circle with a square inside.
• Or in the Classic Notebook, click the running tab on the top of your screen.

It is also important to understand how your session will (and will not) terminate. Regardless of whether you close your browser or not, or even log out, your session will persist, and your work will still run. The session will only terminate if you run out of time in your Slurm allocation or if you click “Stop My Server” on the Hub Control Panel. To access the hub control panel, in JupyterLab go to File > Hub Control Panel, or in the Classic Notebook go to the “Control Panel” button in the upper right hand corner of your window.

Using Jupyter Lab

Once you have logged in and been allocated resources for your session, you will be dropped by default into the JupyterLab IDE. For more information on JupyterLab usage check out documentation here.

We also encourage you to check out the tutorials that are available at ‘JupyterLinks/tutorialNotebooks’ on all of the systems.

Switching To Classic Notebooks

For users who would prefer using the Classic Notebook interface, simply go to the commands tab on the left hand side and go to Help > Launch Classic Notebook.

Accessing Files

When you look at the file tree in either interface, you will notice that the top level directory is your home directory. Due to limitations within Jupyter and the interaction with file permissions, you will have to use symlinks to access data in other locations. When you start your session, the ~/JupyterLinks directory will be automatically generated with a link to your $NOBACKUP, and a link to CSS. On ADAPT a link to tutorial notebooks will also be generated, while on Discover tutorial notebooks can be found under ~/JupyterLinks/projects/jh_tutorials (Note that this projects link is to the broader project nobackup space).

Environments and Kernels

Kernels in JupyterNotebooks are effectively the same as Anaconda environments. You can use the terminal in Jupyter or an ssh connection to Discover or Adapt to create, destroy, view, and modify environments. For personal environments that you would like to use as a kernel, you must first install ‘ipykernel’ in each environment:
conda activate <your_conda_env>
conda install ipykernel

New environments can be created directly in JupyterHub on Adapt and Prism. However, on Discover, adding packages or environments cannot be done in JupyterHub. Instead, you must first SSH to Discover. Once there, run ‘module load anaconda’, and then create your new Conda environment and ensure you run the commands above to make the conda environment visible to JupyterHub.

Further Questions and Support

If you have any additional questions regarding JupyterHub access or usage, please submit a ticket to NCCS Support at [email protected].