Skip to content

Ansys CFX#

Ansys CFX is a general purpose Computational Fluid Dynamics (CFD) code. It provides a wide variety of physical models for turbulent flows, acoustics, Eulerian and Lagrangian multi-phase flow modeling, radiation, combustion and chemical reactions, heat and mass transfer. It is mostly used for simulating turbomachinery, such as pumps, fans, compressors and gas and hydraulic turbines.

Licensing#

Please note that the clusters do not come with any license. If you want to use Ansys products on the HPC clusters, you have to have access to suitable licenses. These can be purchased directly from RRZE. To efficiently use the HPC resources, Ansys HPC licenses are necessary.

Availability / Target HPC systems#

Different versions of all Ansys products are available via the modules system, which can be listed by module avail ansys. A special version can be loaded, e.g. by module load ansys/2023R2.

Production jobs should be run on the parallel HPC systems in batch mode.

Ansys CFX can also be used in interactive GUI mode on the login nodes. This should only be used to make quick simulation setup changes. It is not permitted to run computationally intensive Ansys CFX simulations or post-processing sessions with large memory consumption on login nodes.

For pre- or post-processing of larger simulation cases via interactive GUI mode, have a look at our visualization node.

Usage#

The (graphical) CFX launcher is started by the command cfx5launch. If you want to use the separate pre- or post-processing capabilities, you can also launch cfx5pre or cfx5post, respectively.

For running simulations in batch mode on the HPC systems, use the

cfx5solve

command. All available parameters for this command can be listed with cfx5solve -help. One example call to use in your batch script is

cfx5solve -batch -par-dist $NODELIST -double -def <solver input file>

The number of processes and the hostnames of the compute nodes to be used are defined in $NODELIST. For how to compile this list, refer to the example script below. Using SMT threads is not recommended.

Best practices#

  • We recommend writing automatic backup files (every 6 to 12 hours) for longer runs to be able to restart the simulation in case of a job or machine failure. This can be specified in Output Control → User Interface → Backup Tab
  • Furthermore, it is recommended to use the "Elapsed Wall Clock Time Control" in the job definition in Ansys CFX Pre (Solver Control → Elapsed Wall Clock Time Control → Maximum Run Time → <24h). Also plan enough buffer time for writing the final output, depending on your simulation, this can take quite a long time!
  • Please note that the default (Intel) MPI startup mechanism is not working for versions below 2023R2. This will lead to the solver hanging without producing any output. Use the option -start-method "Open MPI Distributed Parallel" to prevent this.

Sample job scripts#

All job scripts have to contain the following information:

  • Resource definition for the queuing system (more details here)
  • Load Ansys environment module
  • Generate a file with names of hosts of the current simulation run to tell CFX on which nodes it should run (see example below)
  • Execute cfx5solve with appropriate command line parameters (available options via cfx5solve -help)

Parallel job on Meggie#

The following script runs Ansys CFX on Meggie in parallel on 4 nodes with 20 CPUs each.

#!/bin/bash -l
#SBATCH --job-name=mycfx
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=20
#SBATCH --time=24:00:00
#SBATCH --export=NONE

unset SLURM_EXPORT_ENV
# specify the name of your input-def file
DEFFILE="example.def"

# load environment module
module load ansys/XXXX
# generate node list
NODELIST=$(for node in $( scontrol show hostnames ${SLURM_JOB_NODELIST} | uniq ); do echo -n "${node}*${SLURM_NTASKS_PER_NODE},"; done | sed 's/,$//')
# execute cfx with command line parameters (see cfx5solve -help for all available parameters)  
cfx5solve -batch -double -par-dist "$NODELIST" -def "$DEFFILE" -start-method "Open MPI Distributed Parallel"

Parallel job on Fritz#

The following script runs Ansys CFX on Fritz in parallel on 4 nodes with 72 CPUs each.

#!/bin/bash -l
#SBATCH --job-name=mycfx
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=72
#SBATCH --time=24:00:00
#SBATCH --export=NONE

unset SLURM_EXPORT_ENV
# specify the name of your input-def file
DEFFILE="example.def"

# load environment module
module load ansys/2023R2
# generate node list
NODELIST=$(for node in $( scontrol show hostnames ${SLURM_JOB_NODELIST} | uniq ); do echo -n "${node}*${SLURM_NTASKS_PER_NODE},"; done | sed 's/,$//')
# execute cfx with command line parameters (see cfx5solve -help for all available parameters)  
cfx5solve -batch -double -par-dist "$NODELIST" -def "$DEFFILE"