= HPC Overview by Harry Mangalam v1.4, Sept 27, 2017 == What HPC is The HPC facility is a ~280 node, ~9440 core, ~2PB 'Condo Cluster' available to all researchers at UCI (no School or Dept affiliation required). 'Condo' means that most of the compute nodes are individually owned by contributing Schools, Depts, or individual PIs, and are available to them preferentially, but when not so used, are available to the rest of campus users, using the Free Queue[1] system. This allows all users to effectively get more computational power than they paid for, since usage among owners is quite spiky and the 'Free Queue' allows us to harvest otherwise unused cycles. While the HPC facility is not HIPAA-compliant [2], we have been told that appropriately anonymized human data are being reduced and analyzed through various pipelines on HPC. In summary, HPC offers very large computing resources to all UCI researchers to allow them to store and analyze data of a size that would not otherwise be possible locally, and personal assistance in resolving specific computational problems. Should UCI researchers require even larger compute power, Harry Mangalam is an XSEDE Campus Champion [3], who can help to move your problems to the larger XSEDE resources. == Hardware The HPC cluster is now the largest on campus with about 9440 64bit CPU cores, ~55TB aggregate RAM, 30 Nvidia GPUs [4], with Quad Data Rate (40Gbs) Infiniband interconnects. There is ~2PB of storage with >1PB in 2 BeeGFS [5] distributed filesystems, and the rest available from ~34 public and private NFS automounts. We have ~2100 users, of whom ~100 are logged in at any one time on 3 login nodes and >250 use the cluster in an average month. The cluster generally has 65%-90% queue usage with several thousand jobs running and queued at any one time (>14000 as I write this; 20,000 is not unusual). == Software The HPC staff can usually add requested Open Source Software within a day; our 'module' system has >1000 external applications (various versions), libraries, and compilers/interpreters. These include image processing (freesurfer/fsl), statistics (R, SAS, STATA) numerical processing (MATLAB, Mathematica, Pylab/Numpy/SciPy, Scilab, GSL, Boost, etc), molecular dynamics (Gromacs, NAMD, AMBER, VMD), as well as a number of Engineering applications (OpenSees, Openfoam). However, since our largest user group is biologists, we have an exceptionally large group of bioinformatics tools including the bam/sam/bedtools, tophat, bowtie, bwa, wgs, trimmomatic, repeatmasker, fastx toolkit, emboss, tacg, Bioconductor, BLAST, Trinity, GATK, Annovar, velvet, vcftools, cufflinks, smrtanalysis, the SOAP tools, picard, the Bio(Perl|Python|Java) toolkits, etc. Our compilers (GCC, PGI, Intel) support parallel approaches including OpenMP, OpenMPI, Gnu Parallel, R's SNOW, and CUDA. We support debugging and profiling with TotalView, PerfSuite, HPCToolkit, Oprofile, and PAPI. We also support a local installation of Galaxy, integrated into our GridEngine scheduler. Our module library consists of this list [6]. Note the date on top for the latest update. == Classes We run classes and tutorials in using Linux and the HPC cluster as well as Bioinformatics (in conjunction with the Genomic High Throughput Facility [7]) and also offer classes in 'BigData' processing (with Padraic Smith of Information & Computer Sciences). == Staff HPC is supported by ~5 FTEs, including 1 PhD (in Molecular Biology & Bioinformatics), as well as 1 student assistant and these support staff regularly assist with debugging workflows and performance problems in all domains (~700 domain-specific emails per month). We very much welcome new computational and research challenges and our interaction with scientists. Please feel free to mail all of us at . 'Harry Mangalam', PhD, for the Research Computing Support Staff + 'Joseph Farran', Senior Sysadmin, 30+ years IT experience + 'Imam Toufique', Sysadmin, Virtualization & Network Gymnast + 'Francisco Lopez, Sysadmin, Security, Cloud + 'James Walker', Sysadmin + 'Farid Chahla', Student Assistant + 'Emeritus Staff': + 'Garr Updegraff', PhD, + 'Edward Xia' (student) + == References [1] http://hpc.oit.uci.edu/free-queue [2] http://goo.gl/Qqcez [3] https://www.xsede.org/campus-champions [4] http://www.nvidia.com/object/tesla-servers.html [5] http://www.beegfs.com/cms/ [6] http://hpc.oit.uci.edu/all.modules [7] http://ghtf.biochem.uci.edu/