Differences

This shows you the differences between two versions of the page.

--- techstaff:slurm [2016/05/09 15:06] – kauffman
+++ techstaff:slurm [2017/08/24 14:53] – [Notice] kauffman
@@ Line 1: / Line 1: @@
-====== DRAFT | Peanut Job Submission Cluster ======
+===== Notice =====
+All users should read this message.
-We are currently **alpha** testing and gauging user interest in a cluster of machines that allows for the submission of long running compute jobs. Think of these machines as a dumping ground for discrete computing tasks that might have been rude or disruptive to execute on the main (shared) shell servers (i.e., linux1, linux2, linux3).
+The SLURM cluster will become unavailable starting 2017-08-22 for an upgrade. Normal service should resume on 2017-08-25. Please check back here for status updates.
+**2017-08-22 1800**: Main cluster upgraded. You can try to use it now but I can't guarantee that I won't kill your job tomorrow or Firday.
+**2017-08-23 1345**: GPU servers upgraded and added back to the cluster. They may be missing some software that was not automatable at previous time of installation. Send me an email if you find anything missing.
+We still run systems with Ubuntu 14.04 installed. As of right now these systems cannot submit jobs to the cluster. This is on purpose. The slurm version jump between 14.04 and 16.04 was so huge that this was unavoidable. This means you should prefer to use linux.cs.uchicago.edu or any CS machine that run Ubuntu 16.04.
+**2017-08-24**: Everything seems to be working as expected. Please start using the cluster again. Email techstaff@cs.uchicago.edu if you run into any problems.
+====== Peanut Job Submission Cluster ======
+We are currently **alpha** testing and gauging user interest in a cluster of machines that allows for the submission of long running compute jobs. Think of these machines as a dumping ground for discrete computing tasks that might be rude or disruptive to execute on the main (shared) shell servers (i.e., linux1, linux2, linux3).
 For job submission we will be using a piece of software called [[http://slurm.schedmd.com|SLURM]]. Simply put, SLURM is a queue management system and stands for **S**imple **L**inux **U**tility for **R**esource **M**anagement; it was developed at the Lawrence Livermore National Lab. It currently supports some of the largest compute clusters in the world. The best description of SLURM can be found on its homepage:
@@ Line 228: / Line 240: @@
 ====== Using the GPU ======
+===== Paths =====
+You will need to add the following to your $PATH and $LD_LIBRARY_PATH.
+  export PATH=$PATH:/usr/local/cuda/bin
+  export LD_LIBRARY_PATH=$LD_LIBRARY_PATH=/usr/local/cuda/lib
 ===== Example =====
 This sbatch script will get device information from the installed Tesla gpu.
@@ Line 240: / Line 259: @@
 #SBATCH --partition=gpu
 #SBATCH --job-name=get_tesla_info
+export PATH=$PATH:/usr/local/cuda/bin
+export LD_LIBRARY_PATH=$LD_LIBRARY_PATH=/usr/local/cuda/lib
 cat << EOF > /tmp/getinfo.cu
@@ Line 268: / Line 290: @@
 rm /tmp/getinfo.cu
 </code>
+==== Output ====
+STDOUT will look something like this:
+<code>
+cnetid@linux1:~$ cat $HOME/slurm/slurm_out/12567.gpu1.stdout
+Device Number: 0
+  Device name: Tesla M2090
+  Memory Clock Rate (KHz): 1848000
+  Memory Bus Width (bits): 384
+  Peak Memory Bandwidth (GB/s): 177.408000
+</code>
+STDERR should be blank.
 ====== More ======
 If you feel this documentation is lacking in some way please let techstaff know. Email [[techstaff@cs.uchicago.edu]], call (773-702-1031), or stop by our office (Ryerson 154).