techstaff:aicluster
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
techstaff:aicluster [2020/11/11 16:27] – kauffman | techstaff:aicluster [2020/12/10 12:56] – [Batch] kauffman | ||
---|---|---|---|
Line 65: | Line 65: | ||
==== File Transfer ==== | ==== File Transfer ==== | ||
You will use the FE nodes to transfer your files onto the cluster storage infrastructure. The network connections on those nodes are 2x 10G each. | You will use the FE nodes to transfer your files onto the cluster storage infrastructure. The network connections on those nodes are 2x 10G each. | ||
+ | |||
+ | === Quota === | ||
+ | * By default users are given a quota of 20G. | ||
+ | |||
====== Demo ====== | ====== Demo ====== | ||
Line 146: | Line 150: | ||
</ | </ | ||
+ | |||
+ | ===== Jupyter Notebook Tips ===== | ||
+ | ==== Batch ==== | ||
+ | The process for a batch job is very similar. | ||
+ | |||
+ | jupyter-notebook.sbatch | ||
+ | < | ||
+ | #!/bin/bash | ||
+ | unset XDG_RUNTIME_DIR | ||
+ | NODEIP=$(hostname -i) | ||
+ | NODEPORT=$(( $RANDOM + 1024)) | ||
+ | echo "ssh command: ssh -N -L 8888: | ||
+ | . ~/ | ||
+ | jupyter-notebook --ip=$NODEIP --port=$NODEPORT --no-browser | ||
+ | </ | ||
+ | |||
+ | Check the output of your job to find the ssh command to use when accessing your notebook. | ||
+ | |||
+ | Make a new ssh connection to tunnel your traffic. The format will be something like: | ||
+ | |||
+ | '' | ||
+ | |||
+ | This command will appear to hang since we are using the -N option which tells ssh not to run any commands including a shell on the remote machine. | ||
+ | |||
+ | Open your local browser and visit: '' | ||
+ | ==== Interactive ==== | ||
+ | - '' | ||
+ | - '' | ||
+ | - '' | ||
+ | - '' | ||
+ | - '' | ||
+ | - '' | ||
+ | - Make a new ssh connection with a tunnel to access your notebook | ||
+ | - '' | ||
+ | - This will make an ssh tunnel on your local machine that fowards traffic sent to '' | ||
+ | - Open your local browser and visit: '' | ||
/var/lib/dokuwiki/data/pages/techstaff/aicluster.txt · Last modified: 2021/01/06 16:11 by kauffman