Scientific Computing

VU: Scientific Computing in Mathematics (2h)

Contents: Extended programming with C++, bash, python, sage, matlab and whatever is needed. Project managment with make, development tools and a version control system. Use of available mathematical libraries. Mathematical problem solving by means of scientific computing tools. Special focus on GPU computing, solvers.

Lecturer: Prof. Gundolf Haase, Heinrichstr. 36, Zi 506, Klappe 5178,

Time and place: Tuesday 12:45-14:15 at Uni Graz: SR 11.34

Modus and deadline (Empfehlungen zu VU):

Deadline for quitting the lecture: 30.03.2025 (afterwards you have to receive a degree!)
The course starts with some recovering your programmings skills, introduction into Linux with make, bash/python and afterwards mathematical projects using GPU computing.

Time table for lectures:

March 4

Preliminary discussion of the lecture (your skills, specialization of students in master studies, laptops?, operating system?, topic of bacc./master thesis?).
Potential projects (2023).

Hardware (login from outside KFU only via VPN):

Mephisto at IMSC, or via jupyter-Server des IMWR.

Remote login to servers:

VPN to KFU is needed: install via VPN Service the software AnyConnect (configure as server: https://univpn.uni-graz.at; login: KFU E-mail)

Linux: use ssh -X 143.50.47.xxx to connect to compute server

Windows: Install WinnSSHTerm with a guided installation of further packages (putty, winscp, X-Server)

Refresh C++ by yourself:

Memory costs of containers (1, 2, 3); deep copy vs. shallow copy (notes); nullptr, const_expression, static_assert;

smart pointer [Examples: variants_pointer, shared, unique];

exception handling (predefined exceptions, tutorial) [Examples: file_io, myvector];

Multi Threading in C++-17 (Bartek's coding blog, sort) [Example: thread_17];
Linker requires option -ltbb

Type-checking at compile time: C++17: type_traits, C++20 concepts,

Ranges in C++20.

Further literature: "The C++ Standard Library"; "C++20: Get the Details"

March 11

bash:

Bash cheatsheet [Rehan, quickme]

Intro into Linux (Wie werde ich UNIX-Guru? (OpenBook)):

command line arguments, redirection, pipelining, tools (example; myvector_ext, BinaryMatrix).

bash scripts (ex)

git: git-Gruppe

git cheatsheets (1,2)

README.md and its basic syntax.

March 18

make:

simple tutorial, cheatsheet (§ 1.4 in pdf),

Preprocessor (Howto)

Example: Mixing of C/C++/Fortran (code)

git: git-Gruppe

git cheatsheets (1,2)

README.md and its basic syntax.

(basics, simple intro, learn git in 15 min, info)

March 25

improving scalar product etc. (all Codes, examples).

firstSteps

skalar (+BLAS, +Thrust), float_skalar, par_skalar (+MPI)

densematrices_libs (+BLAS)

cusolver (+cuSOLVER, +unifies memory)

GPU computing: CUDA, Matlab, python
CUDA: 12.3, docu, Best Practice, Profiler, Debugger,

April 1

pybind11: Interoperability between C++ and Python

github, first steps

Tutorial [Gad, Khajepor]

C++/Python [Rainer Grimm 1, 2]

Call C++ from python (1)

Call a python function from C++ (1, 2)

numpy array in C++ (1)

April 29

Remarks on task sheets 1/2.
Remarks on sheet 3:

Template class VecGPU<FP>

cuDSS (examples, start) direct solver similar class as Chol_GPU (see also transition from cuSparse).
CUDALibrarySamples/cuDSS/simple

AmgX (README)

Solvers for large sparse systems: iterative, multilevel, algebraic multilevel

May 6

Some tools:

PDE solvers:

netgen/ngsolve (tutorials, shape differentiation [paper], AutoDiff)

FEniCS, new: dolfinx; meshing: Gmsh,

OpenFoam: intro, tutorial, download.

Further: DEAL II, FreeFEM++, Dune, ug4,

Graph/mesh partitioning:PT-Scotch, ParMETIS.

Problem coupling: preCICE: intro, docu, quickstart, adapters, fluid-temperature,

Parallel solvers: Paralution (doc), PETSc,

May 13

Solvers for large sparse systems: iterative, multilevel, algebraic multilevel

May 20

no lecture: ASHPC

May 27

June 3

June 10

no lecture at Uni

June 17

Consulting

June 24

Project presentation by students, see example.

Task sheets: Subject to change:

Task 0 Recover your programming skills (C++)
Task 1 some Linux: bash scripts, makefiles
Task 2 GPU and CUDA
Task 3 CUDA libraries
Task 4 projects

Students who are not primarily enrolled at KFU: Guide
Books:

Learn CUDA Programming: A beginner's guide to GPU programming and parallel computing with CUDA 10.x and C/C++; Han/Sharma;Packt; 2019
Scientific Computing - An Introduction using Maple and MATLAB; Gander/Gander/Kwok; Springer; 2014; e-book(UB einloggen, via VPN von daheim)
A Primer on Scientific Programming with Python; Hans Petter Langtangen; Springer, 2014; e-book(UB)
Scientific Computing with MATLAB and Octave; Quarteroni/Saleri/Gervasio; 2014; e-book(UB)
Scriptum of lecture "Programmieren in C++"
B1, B2

C++:

Download of C++compilers and editors for Windows.
Hints for compiler options
Rules for programming; LLVM Coding standards.
C++ Core Guidlines , gcc 7.1,
lexicon: cppreference.com , cplusplus.com (incl. Quiz)
Suggested: CppInsight, Compiler Explorer, WandBox
AI programmers: Phind
"Help!!!!!!!!!": stackoverflow.com
Great C++-page with online code editing/comiling/execution, see e.g., array.
Concurrency with Modern C++ (R. Grimm), bining C and C++ (R. Grimm)
See how your c++ is converted by the compiler: cppInsight.
Memory checker: valgrind, Dr. Memory,
Literature
Boost library (english, german, contents): Graph(doc, english, german)
Bartek's Blog, e.g., execution policy, in-class member initialization.
Format in C++20 by P. Gottschling.

Python:

python: Tutor (interactive),
e-Book by Langtangen.

Unix:

My own Linux: What to do?
How to survive with UNIX (Uni-Graz E-Book, html, PDF ) and more and even more.
Openbook by Willemer
e-book (UB Graz): "Linux - kurz & gut"
make: Tutorial.
bash: first commands and structures, scripts, wild cards, regular expression (try it)
Linux-Subsystem in Windows 11: my hints, heise, microsoft, engl.,

GPU:

available hardware.
CUDA:

Version 12.3, docu, Best Practice, Profiler, Debugger,
libraries: cuBLAS, cuSOLVER and cuSolverSP, cuSPARSE, Thrust (mix with cuBLAS), (AmgX,) (Gunrock)
Exampels: cusolver, cusolverSp,

Matlab:

can by installed on private hardware via Campuslizenz.

R2022b error with Linux (ubuntu): " Inconsistency detected by ld.so: ../elf/dl-tls.c: 517: _dl_allocate_tls_init: Assertion `listp != NULL' failed!":
Bug-Report.
My solution: Don't install Simulink packages if 'ls -al /usr/lib/x86_64-linux-gnu/libc.so.6' returns a version less than 2.34.
R2022b installation poblems with archlinux/manjaro: Follow the fix for installation and for starting problems.

codegen with GPU acceleration: Mandelbrot (improve it!, single prec!?), cuSOLVER and for c++-code (c,project)

Python:

CuPy (NumPy, SciPy)
Numba,
Tutor (general interactive), e-Book by Langtangen.

Solver:

Solver for sparse systems: cuSolverSP, cuSolverRF
special Multilevel/AMG solver:HAZmath, AGMG (Y.Notay), AmgX,
Eikonal (GPU, multiple GPU )

Version control systems:

git: German intro, interactive tutorial

Solvers and other packages (a selection from Arnold Neumaier's page, some literature)

PDE solvers: ngsolve (tutorials, shape differentiation [paper], AutoDiff)
OpenFoam: intro, tutorial, download.
python: numba (JIT compiler), autodiff,
partitioners: PT-Scotch, ParMETIS.
preCICE: intro, docu, quickstart, adapters,
direct solvers: PaStiX, FLAME, PARADISO(O.Schenk, Intel), CHOLMOD [new]
iterative solvers: Paralution (doc), ADI, PETSc, (Aztec)
load balancing, parallel data management: DEAL II, FreeFEM++, Dune, ug4, ( Zoltan)
preconditioners: Hypre, AHMED, PetFMM,
julia (math. script language)
tools: Intel Tools (Advisor, Inspector)
this list will be extended and specified

Automatic differentiation tools for optimization:

Autodiff.org
Books

Andreas Griewank and Andrea Walther: "Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation", Second Edition", SIAM, 2008 (Graz-Lib: ebook and 1st edition: 66:G848)
Uwe Naumann: "The Art of Differentiating Computer Programs", SIAM, 2011 (Graz-Lib: 66:N311 and ebook)
J. Haslinger and R. A. E. Mäkinen: "Introduction to Shape Optimization", SIAM, 2003 (Graz-Lib: 45:H352 and ebook)
Uwe Naumann and Olaf Schenk: "Combinatorial Scientific Computing", Chapman & Hall/CRC Computational Science, 2012 (Graz-Lib: on acquisition)
Alfio Borzì and Volker Schulz: "Computational Optimization of Systems Governed by Partial Differential Equations ", SIAM, 2011 (Graz-Lib: 65:B739 and ebook)

Papers

Bischof et.al,: "Sensitivity Analysis Of Turbulence Models Using Automatic Differentiation", SIAM J. SCI. COMPUT. ,Vol. 26, No. 2, pp. 510-522, 2004
Borggaard et.al: "On Efficient Solutions To The Continuous Sensitivity Equation Using Automatic Differentiation", SIAM J. SCI. COMPUT. ,Vol. 22, No. 1, pp. 39–62, 2000
Pryce et.al: "Fast Automatic Differentiation Jacobians By Compact LU Factorization", SIAM J. SCI. COMPUT. , Vol. 30, No. 4, pp. 1659–1677, 2008
Schanen et.al.: "Interpretative Adjoints for Numerical Simulation Codes using MPI", Procedia Computer Science 1 (2012) 1825–1833, Elsevier
Dayde et.al.: "Element-By-Element Preconditioners For Large Partially Separable Optimization Problems", SIAM J. SCI. COMPUT. , Vol. 18, No. 6, pp. 1767-1787, 1997

May 13, 2025