Table of Contents
- Introduction and compiling with the Intel compiler
- Serial Hello World
- Parallel Hello World
- Logging Which Nodes are Used
This video recording on compiling and running code on chip provides a live demonstration of the steps that are written out in the following text, including the run tutorial page.
This Compile (and Run) tutorial starts at the point that you are able to log in to the chip cluster in the UMBC High Performance Computing Facility. For information, how to log in and use your account up to this point, please visit first the page How to Use Your chip Account.
Introduction and Compiling with the Intel Compiler
In this tutorial, we will illustrate how to compile C source code and run the resulting executable on the CPU cluster chip-cpu in chip in the UMBC High Performance Computing Facility. Working on a distributed cluster like chip is fundamentally different from working on a standard server (like gl.umbc.edu) or a personal computer, so please make sure to read and understand this material. We will first start with a classical serial example, and work our way to compiling parallel code using the most important parallel processing library MPI (Message Passing Interface). We will assume that you know some basic programming concepts, so the code will not be explained in explicit detail. More details can be found in manual pages on the system that are available for Linux commands (e.g., try “man mkdir”, “man cd”, “man pwd”, “man ls”) as well as for C functions (e.g., try “man fprintf”).
Since we want to use parallel code with MPI, we need to use the Intel compiler suite, since that has MPI available with it. See the previous documentation how to log in to the usernode of chip. This assumes that you are logged in to your account on chip.
After logging in to chip’s usernode, you need to be on a compute node to compile with the Intel compiler and with MPI, since the module for the Intel compiler suite needs to be loaded (and loading of modules cannot be done on the usernode).
To find out the exact name of the module to load, do “module spider Intel“; this command can be issued on the usernode or a compute node, but the actual “module load ...” can only be done on a compute node. I show the srun for an interactive session on a 2024 node:
srun --cluster=chip-cpu --account=pi_gobbert --partition=2024 --qos=shared --time=07:00:00 --mem=16G --pty $SHELL
Before issuing this command, use <kbd>sinfo</kbd> to see if there are resources available in the desired partition. Here is a sample output:
[gobbert@chip ~]$ sinfo --cluster=chip-cpu CLUSTER: chip-cpu PARTITION AVAIL TIMELIMIT NODES STATE NODELIST 2024 up infinite 12 alloc c24-[29-40] 2024 up infinite 39 idle c24-[01-28,41-51] 2021 up infinite 5 mix c21-[01-05] 2021 up infinite 13 idle c21-[06-18] 2018 up infinite 3 down* c18-[17,19,23] 2018 up infinite 1 drain c18-39 2018 up infinite 5 mix c18-[18,20-21,30,40] 2018 up infinite 33 idle c18-[01-16,22,24-29,31-38,41-42]
which shows that nodes in the 2024 partition are available.
Using the above srun command at the Linux command prompt on the usernode, you should now get a new Linux prompt after possibly some delay and some messages like
[gobbert@chip ~]$ srun --cluster=chip-cpu --account=pi_gobbert --partition=2024 --qos=shared --time=07:00:00 --mem=16G --pty $SHELL The following have been reloaded with a version change: 1) slurm/chip-gpu/23.11.4 => slurm/chip-cpu/23.11.4 [gobbert@c24-01 ~]$
This new prompt indicates that we have an interactive shell on the node with hostname c24-01, not on the usernode chip any more as it said in the original prompt above (where the srun is shown). The tilde “~” means that we are in my home directory in both cases.
The output of “module spider Intel” shows several compiler versions. The latest as of today is 2024a, so the load command is
module load intel/2024a
As you can confirm by which mpiicc, this Intel compiler for C/C++ with Intel MPI (actually a script) is now found on the path. You still need to tell the system the Intel compiler’s name, so also say
export I_MPI_CC=icx
This finishes the setup to use the Intel compiler suite with the icx C compiler. You have to do these steps every time that you want to compile with the Intel compiler.
We also want to demonstrate here that it is a good idea to collect files for a project in a directory. This project is on a serial version of the “Hello, world!” program. Therefore, use the mkdir (= “make directory”) command to create a directory “Hello_Serial” and cd (= “change directory”) into it.
[gobbert@c24-01 ~]$ mkdir Hello_Serial [gobbert@c24-01 ~]$ cd Hello_Serial [gobbert@c24-01 Hello_Serial]$ pwd /home/gobbert/Hello_Serial [gobbert@c24-01 Hello_Serial]$
Notice that the command prompt indicates that I am in directory Hello_Serial now. Use the pwd (= “print working directory”) command any time to confirm where you are in your directory structure and ll (short for “ls -l”) to list the files that are there.
A convenient way to save the example code on this page directly into the current directory of your project uses the wget command as follows. There is a “download” link under each code example in this page farther below. You can copy the link address in your browser (by the right button or similar) and copy it after the wget command in your chip terminal session to download the file to the local directory, as shown here.
[gobbert@taki-usr1 Hello_Serial]$ wget https://userpages.umbc.edu/~gobbert/chip/linux/Hello_Serial/hello_serial.c --2025-06-02 15:08:28-- https://userpages.umbc.edu/~gobbert/chip/linux/Hello_Serial/hello_serial.c Resolving userpages.umbc.edu (userpages.umbc.edu)... 130.85.30.98 Connecting to userpages.umbc.edu (userpages.umbc.edu)|130.85.30.98|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 184 [text/plain] Saving to: ‘hello_serial.c’ hello_serial.c 100%[===================>] 184 --.-KB/s in 0s 2025-06-02 15:08:29 (12.9 MB/s) - ‘hello_serial.c’ saved [184/184]
You can list all files to see that the file is present now:
[gobbert@c24-01 Hello_Serial]$ ll total 24 -rw-rw---- 1 gobbert pi_gobbert 184 Feb 1 2014 hello_serial.c
We have shown the prompt in the examples above to emphasize that a command is being issued. When following the examples, your prompt may look a bit different (e.g., your own username will be there!), but be careful to only issue the command part, not the prompt or the example output.
Serial Hello World
We will now consider a simple “Hello, world!” program that prints the name of the host machine. Here is the code
Download: https://userpages.umbc.edu/~gobbert/chip/linux/Hello_Serial/hello_serial.c
Creating a directory for this project and downloading this code with wget was the example given above in the section Introduction on this page. What the Introduction section also showed was what module to load for the Intel compiler suite and how to set an environment variable for the C compiler icx.
Once you have saved this code to your directory, we have to compile it before we can execute it, since C is a source code programming language. Compile using the Intel icx compiler by
[gobbert@c24-01 Hello_Serial]$ icx hello_serial.c -o hello_serial
The -o option of icx specifies the desired name of the output file; it is customary to use the base filename without extension for an executable in C/C++.
If successful, no errors or warnings will appear and an executable hello_serial will have been created, in addition to the source code in hello_serial.c.
[gobbert@c24-01 Hello_Serial]$ ll total 120 -rwxrwx--- 1 gobbert pi_gobbert 16600 May 27 16:46 hello_serial* -rw-rw---- 1 gobbert pi_gobbert 184 Feb 1 2014 hello_serial.c
Notice that the “x” in the permissions “-rwxrwx—” indicates that hello_serial is an executable; this is also indicated by the asterisk “*” following its name (the “*” is not part of the filename, it is just an indication of an executable from the ls command). When a file is not an executable (or there is no permission to execute it), a dash “-” appears in place of the “x”; the dashes in “-rw-rw—-” for hello_serial.c confirm that this C source code is not executable in its source code form.
To see how to run your serial executable on the cluster, jump to how to run serial programs.
Parallel Hello World
Now we will compile a “Hello, world!” program which can be run in parallel on multiple processors. You may want to create a new directory for this project using “mkdir Hello_Parallel”. Use wget again to save the following code to your directory.
Download: https://userpages.umbc.edu/~gobbert/chip/linux/Hello_Parallel/hello_parallel.c
This version of the “Hello, world!” program collects several pieces of information at each MPI process: the MPI processor name (i.e., the hostname), the process ID, and the number of processes in our job. Notice that we needed a new header file mpi.h to get access to the MPI commands. We also need to call MPI_Init before using any other MPI commands, and MPI_Finalize is needed at the end to clean up. Compile the code with the following command.
[gobbert@c24-41 Hello_Parallel]$ mpiicc hello_parallel.c -o hello_parallel
After a successful compilation with no errors or warnings, an executable “hello_parallel” should have been created, which we confirm by “ll”.
[gobbert@c24-41 Hello_Parallel]$ ll total 120 -rwxrwx--- 1 gobbert pi_gobbert 16776 May 28 11:57 hello_parallel* -rw-rw---- 1 gobbert pi_gobbert 490 Feb 1 2014 hello_parallel.c
To see how to run your parallel executable on the cluster, jump to how to run parallel programs.
Logging Which Nodes are Used
For a parallel program, it is always a good idea to log which compute nodes you have used. We can extend our parallel “Hello, world!” program to accomplish this, namely in addition to printing the information to stdout, we will save it to file. First, the functionality is contained in a self-contained function nodesused() that you can copy also into other programs and then call from the main program, as shown in the code below. Second, we noticed that the processes reported back in a random order to stdout. This is difficult to read for large numbers of processes, so for the output to file, we have the process with ID 0 receive the greeting message from each other process, in order by process ID, and only Process 0 will write the messages to file. Third, the code below actually creates and writes to two files: (i) The file “nodes_used.log” contains only the process ID and hostname, which is the same information as printed stdout already, but ordered. (ii) The file “nodesused_cpuid.log” additionally outputs the CPU ID, that is, the number of the computational core in the two CPUs on the node that the MPI process executed on.
Message sending is accomplished using the MPI_Send() function, and receiving with the MPI_Recv() function. Each process prepares its own message, then execution varies depending on the current process. Process 0 writes its own message first, then receives and writes the others in order by process ID. All other processes simply send their message to process 0. The fprintf function is used to write one line for each MPI process to each of the two output files.
This program is compiled for use with MPI by
[gobbert@taki-usr1 Nodesused]$ mpiicc nodesused.c -o nodesused
To see how to run your parallel executable of the nodesused program on the cluster, use the same slurm script as above and just replace the jobname and executable name by nodesused.
<!– jump to how to run parallel programs on the batch partition. –>