CS 4170/5170: Introduction to Parallel Programming - Fall 2026

Interactive companions to the lectures. Each one is a single page: concept map, animations, predict-then-reveal questions, step-by-step runs on OSC, and a self-check. Nothing is recorded; progress is kept in your own browser.

Module	Page	What it covers
MPI I	Processes, Messages, and the Machine	Six separate rooms: SPMD, the four-call skeleton, Send/Recv, Probe before Recv, barrier-fenced timing, Slurm and one-process-per-node
MPI II	Splitting the Work, Timing It, Talking to It	The trapezoid split by rank, linear vs tree merge, the cost model and when splitting pays, whose clock counts, who does I/O
	One File, Three Questions	Code walk: the hand merge, designing MPI_Reduce and MPI_Bcast, the n/p bug that prints a plausible answer, the fix, timing rules
MPI III	Moving the Data, Not Just the Answer	Scatter/Gather and the bytes bill, Scatterv counts and displs, the collectives family and rules, t(m) = α + m·β and the 40 KB crossover
	Five Ways to Add Two Vectors	Code walk: the N/p and ANY_SOURCE bugs, counts and displs by hand, the same in three calls, cyclic deals and the pack loop
MPI IV	Describe the Data, Don't Copy It	Struct padding and offsets, size vs extent, Type_vector for a column, resized types, Boost.MPI and what the wrapper costs
	One Record, One Column, One Deal	Code walk: the guessed-offsets bug byte by byte, pack and position, Boost serialize, the column, the resized cyclic scatter
MPI V	Who Am I Talking To?	Comm_split color and key, context and dup, the ragged two-stage sum, groups, Cartesian grids, Cart_shift and PROC_NULL, strips vs blocks
MPI VI	Hang, Wrong, Crash, Silent	Sorting every bug by symptom, eager vs rendezvous, choosing p from the shape, tracing a hang, error handlers, the silent Bcast, one model for R01–R04
MPI Review	MPI Quiz/Exam Review	Messages, costs, and the bugs that pass. Weeks 3–8: SPMD, collectives, the cost model, derived types, communicators, hang/wrong/crash/silent, metrics
Foundations I	Where the Data Lives	The memory ladder measured, the 64-byte line, row vs column traversal, first touch and NUMA, the roofline and where AXPY sits
OpenMP I	One Process, Many Threads	Fork/join, parallel for and the schedule, shared/private/firstprivate, the race, critical vs atomic vs reduction, running it on Cardinal
	The Trapezoid, Three Ways, in Threads	Code walk: TrapRule1/2/3, the int-coordinate and remainder bugs, what a critical section holds, the fixed split, floating-point order, running it on Cardinal
OpenMP II	A Must Precede B	Flow, anti and output dependence, what removing each costs, the prefix-sum scan, odd–even sort and the barrier, why a racy loop passes k of 100 runs, how to detect it