Building HPCC 1.0.0 with the PathScale compilers

To build HPCC, you will need to download the source, configure it to use the PathScale compilers, and then build it. HPCC uses a BLAS library for a few of its benchmarks; we recommend the free GOTO BLAS library appropriate for your architecture.

HPCC can be downloaded from the website:

http://icl.cs.utk.edu/hpcc/software/index.html

These notes are specific to the 1.0.0 version of HPCC released on June 11th 2005.
You will also need a BLAS library. We recommend the GOTO BLAS library, which can be found at:

http://www.tacc.utexas.edu/resources/software/#blas

The following steps are done in the top-level GotoBLAS directory.

After downloading and unpacking the source code, you will need to modify the file Makefile.rule so that F_COMPILER = PATHSCALE is uncommented, as well as BINARY64.

Attached is a Makefile.rule from the GOTO 1.0.0 release that can be used as an example. We do not recommend just dropping this file in to any other GOTO releases, as things may have changed, but it can be used as a guideline.

After modifying the file Makefile.rule, run the following commands to create static and dynamic libraries (.a and .so)

make
cd exports
make so
cd ..

This should create a library that looks something like:

libgoto_opteron-r1.0.0.so

where opteron will be replaced by your architecture if you are using another architecture.

After downloading, unpack the source code:

#example for release hpcc-1.0.0
tar -xvzf hpcc-1.0.0.tar.gz

You will now have a top-level directory named hpcc-1.0.0.

Configure the make system to build with PathScale compilers. To do this, insert either the attached Make.Linux_PSC_Opteron_GOTO, or the attached Make.Linux_PSC_em64t_GOTO file, as appropriate for your cluster, into the hpl directory.

If you want to customize the compilation flags used, this is the file to modify. You will need to edit it to set the variable LAdir to point to the correct location for your GOTO BLAS library. If you chose to use a different BLAS library, that is also the location to set it.

At this point, you may want skip to the OPTIONAL STREAM performance boost section, but otherwise you are ready to build. Building is achieved by the command:

make arch=Linux_PSC_Opteron_GOTO

The final step before running is to customize the input file hpccinf.txt file for the number of processes you will be running on, and how much memory they have. P * Q must be equal to your np. Tuning information can be found at:

http://icl.cs.utk.edu/hpcc/faq/index.html

As a start, we note that you can calculate memory usage per process with the calculation:

mem/process = (N^2 * 8 bytes) / num processes

The FAQ referenced above suggests aiming for about 80% of your total memory per process. We would be interested in hearing your experiences with this.

The version of STREAM in HPCC was reworked in such a way as to give compilers difficulty. It has become common for people wanting to show the very best STREAM performance to modify the make system to allow STREAM to be compiled with additional flags that are not applied to the rest of HPCC. This is completely permitted by the rules for a base submission as found at:

http://icl.cs.utk.edu/hpcc/overview/index.html

The flags we recommend for this are are defined in the files Make.Linux_PSC_Opteron_GOTO and Make.Linux_PSC_em64t_GOTO as STREAMCCFLAGS. However, in order for them to actually be picked up and used, the attached Makefile.hpcc.patch needs to be applied to hpl/lib/arch/build/Makefile.hpcc via this command:

# from top-level directory for hpcc (hpcc-1.0.0 for current version)
# (assuming we previously put Makefile.hpcc.patch in this directory)
patch hpl/lib/arch/build/Makefile.hpcc Makefile.hpcc.patch

None known.

We recommend you record what steps you took, along with their output, in case of problems. You can include this in your email to support. Here is an example of changing a make command to record the output in a text file:

make foo 2>&1 | tee make-foo-log.txt

If you have any comments or suggestions about additions to these pages, please contact support@pathscale.com.