slurm:ai
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
slurm:ai [2021/06/25 14:41] – [Infrastructure] kauffman | slurm:ai [2022/04/04 10:58] (current) – fix typos and add code snippet for interactive jupyter notebook chaochunh | ||
---|---|---|---|
Line 1: | Line 1: | ||
====== AI Cluster - Slurm ====== | ====== AI Cluster - Slurm ====== | ||
- | Cluster is up and running now. Anyone with a CS account who wishes | + | Please send in a ticket requesting |
Line 17: | Line 17: | ||
* [[ http:// | * [[ http:// | ||
* [[ https:// | * [[ https:// | ||
- | * Use `guest` as the username and password to login. | + | * Use '' |
===== Computer/ | ===== Computer/ | ||
Line 34: | Line 34: | ||
* 512G RAM | * 512G RAM | ||
* 4x Nvidia A40 | * 4x Nvidia A40 | ||
- | * Note that not all nodes are online yet. | ||
* all: | * all: | ||
Line 47: | Line 46: | ||
* uplink to cluster network: 2x 25G | * uplink to cluster network: 2x 25G | ||
* / | * / | ||
- | * We intend to set user quotas, however, there are no quotas right now. | + | * 20G quota per user. |
* / | * / | ||
* Lives on the home directory server. | * Lives on the home directory server. | ||
Line 66: | Line 65: | ||
====== Login ====== | ====== Login ====== | ||
- | There are a set of front end nodes that give you access to the Slurm cluster. You will connect through these nodes and need to be on these nodes to submit jobs to the cluster. | ||
- | ssh cnetid@fe.ai.cs.uchicago.edu | + | Anyone with a CS account who has previously sent in a ticket to request access to be added is allowed to login. |
- | * Requires | + | There are a set of front end nodes that give you access to the Slurm cluster. You will connect through these nodes and need to be on these nodes to submit jobs to the cluster. |
+ | ssh cnetid@fe.ai.cs.uchicago.edu | ||
==== File Transfer ==== | ==== File Transfer ==== | ||
You will use the FE nodes to transfer your files onto the cluster storage infrastructure. The network connections on those nodes are 2x 10G each. | You will use the FE nodes to transfer your files onto the cluster storage infrastructure. The network connections on those nodes are 2x 10G each. | ||
Line 169: | Line 168: | ||
Make a new ssh connection to tunnel your traffic. The format will be something like: | Make a new ssh connection to tunnel your traffic. The format will be something like: | ||
- | '' | + | '' |
This command will appear to hang since we are using the -N option which tells ssh not to run any commands including a shell on the remote machine. | This command will appear to hang since we are using the -N option which tells ssh not to run any commands including a shell on the remote machine. | ||
Line 176: | Line 175: | ||
==== Interactive ==== | ==== Interactive ==== | ||
- '' | - '' | ||
- | - '' | + | - '' |
- '' | - '' | ||
- '' | - '' | ||
- '' | - '' | ||
- | - '' | + | - '' |
- Make a new ssh connection with a tunnel to access your notebook | - Make a new ssh connection with a tunnel to access your notebook | ||
- '' | - '' | ||
- | - This will make an ssh tunnel on your local machine that fowards | + | - This will make an ssh tunnel on your local machine that forwards |
- Open your local browser and visit: '' | - Open your local browser and visit: '' | ||
+ | Copy the following code snippt to the interactive node directly: | ||
+ | < | ||
+ | unset XDG_RUNTIME_DIR | ||
+ | NODEIP=$(hostname -i) | ||
+ | NODEPORT=$(( $RANDOM + 1024)) | ||
+ | echo "ssh command: ssh -N -L 8888: | ||
+ | jupyter-notebook --ip=$NODEIP --port=$NODEPORT --no-browser | ||
+ | </ | ||
====== Contribution Policy ===== | ====== Contribution Policy ===== | ||
This section can be ignored by most people. [[techstaff: | This section can be ignored by most people. [[techstaff: | ||
/var/lib/dokuwiki/data/pages/slurm/ai.txt · Last modified: 2022/04/04 10:58 by chaochunh