C++11 introduced a standardized memory model, but what exactly does that mean? And how is it going to affect C++ programming?

This article (by Gavin Clarke who quotes Herb Sutter) says that,

The memory model means that C++ code now has a standardized library to call regardless of who made the compiler and on what platform it's running. There's a standard way to control how different threads talk to the processor's memory.

"When you are talking about splitting [code] across different cores that's in the standard, we are talking about the memory model. We are going to optimize it without breaking the following assumptions people are going to make in the code," Sutter said.

Well, I can memorize this and similar paragraphs available online (as I've had my own memory model since birth :P) and can even post as an answer to questions asked by others, but to be honest, I don't exactly understand this.

C++ programmers used to develop multi-threaded applications even before, so how does it matter if it's POSIX threads, or Windows threads, or C++11 threads? What are the benefits? I want to understand the low-level details.

I also get this feeling that the C++11 memory model is somehow related to C++11 multi-threading support, as I often see these two together. If it is, how exactly? Why should they be related?

I don't know how the internals of multi-threading work, and what memory model means in general.

Stephen Ostermiller
  • 23,933
  • 14
  • 88
  • 109
  • 353,942
  • 115
  • 666
  • 851
  • It means that the semantics of multithread programs is now undefined. Previously is was just unspecified. In fact the whole C and C++ semantic model has been destroyed by the introduction of atomics and other primitives that can "time travel". – curiousguy Jul 27 '19 at 01:06
  • In a comment? I asked a Q about it was deleted. Posted an answer it was deleted... Simply put, you can't reason about any program, you can't prove UB doesn't happen, you can't say anything when MT semantics is involved so the so called "memory model" destroyed the C++ std (which was never well written and well defined) – curiousguy Jul 27 '19 at 19:06
  • 7
    @curiousguy: Write a blog then...and propose a fix as well. There is no other way to make your point valid and rationale. – Nawaz Jul 27 '19 at 19:12
  • 6
    I mistook that site as a place to ask Q and exchange ideas. My bad; it's place for conformity where you can't disagree with Herb Sutter even when he flagrantly contradicts himself about throw spec. – curiousguy Jul 27 '19 at 19:46
  • 16
    @curiousguy: C++ is what the Standard says, not what a random guy on the internet says. So yes, there has to be _conformity_ with the Standard. C++ is NOT an open philosophy where you can talk about anything which does not _conform_ to the Standard. – Nawaz Jul 28 '19 at 19:50
  • 9
    _"I proved that no C++ program can have well defined behavior."_. Tall claims, without any proof! – Nawaz Jul 29 '19 at 06:30
  • How do you even reason about a C++ program with MT semantics? When I asked that, the Q was deleted. By you I think – curiousguy Jul 29 '19 at 07:04
  • 2
    No. I've not deleted any question or answer. Anyway, the primitives has certain guarantees, right? If so, then you compose bigger guarantees built on those primitive guarantees. Anyway, do you think it is a problem in C++ (and probably C too) only, or it is a problem in ALL languages? – Nawaz Jul 29 '19 at 09:02
  • There is no guarantee if you have UB; you can't rely on any guarantee because you can't prove there is no UB. Even that simple code isn't well defined: https://stackoverflow.com/q/56673990/963864 – curiousguy Jul 29 '19 at 09:54
  • The answers on [that Q](https://stackoverflow.com/q/56673990/963864) are contradictory. Maybe more ppl could help clear that up. – curiousguy Oct 03 '19 at 23:41

8 Answers8


First, you have to learn to think like a Language Lawyer.

The C++ specification does not make reference to any particular compiler, operating system, or CPU. It makes reference to an abstract machine that is a generalization of actual systems. In the Language Lawyer world, the job of the programmer is to write code for the abstract machine; the job of the compiler is to actualize that code on a concrete machine. By coding rigidly to the spec, you can be certain that your code will compile and run without modification on any system with a compliant C++ compiler, whether today or 50 years from now.

The abstract machine in the C++98/C++03 specification is fundamentally single-threaded. So it is not possible to write multi-threaded C++ code that is "fully portable" with respect to the spec. The spec does not even say anything about the atomicity of memory loads and stores or the order in which loads and stores might happen, never mind things like mutexes.

Of course, you can write multi-threaded code in practice for particular concrete systems – like pthreads or Windows. But there is no standard way to write multi-threaded code for C++98/C++03.

The abstract machine in C++11 is multi-threaded by design. It also has a well-defined memory model; that is, it says what the compiler may and may not do when it comes to accessing memory.

Consider the following example, where a pair of global variables are accessed concurrently by two threads:

           int x, y;

Thread 1            Thread 2
x = 17;             cout << y << " ";
y = 37;             cout << x << endl;

What might Thread 2 output?

Under C++98/C++03, this is not even Undefined Behavior; the question itself is meaningless because the standard does not contemplate anything called a "thread".

Under C++11, the result is Undefined Behavior, because loads and stores need not be atomic in general. Which may not seem like much of an improvement... And by itself, it's not.

But with C++11, you can write this:

           atomic<int> x, y;

Thread 1                 Thread 2
x.store(17);             cout << y.load() << " ";
y.store(37);             cout << x.load() << endl;

Now things get much more interesting. First of all, the behavior here is defined. Thread 2 could now print 0 0 (if it runs before Thread 1), 37 17 (if it runs after Thread 1), or 0 17 (if it runs after Thread 1 assigns to x but before it assigns to y).

What it cannot print is 37 0, because the default mode for atomic loads/stores in C++11 is to enforce sequential consistency. This just means all loads and stores must be "as if" they happened in the order you wrote them within each thread, while operations among threads can be interleaved however the system likes. So the default behavior of atomics provides both atomicity and ordering for loads and stores.

Now, on a modern CPU, ensuring sequential consistency can be expensive. In particular, the compiler is likely to emit full-blown memory barriers between every access here. But if your algorithm can tolerate out-of-order loads and stores; i.e., if it requires atomicity but not ordering; i.e., if it can tolerate 37 0 as output from this program, then you can write this:

           atomic<int> x, y;

Thread 1                            Thread 2
x.store(17,memory_order_relaxed);   cout << y.load(memory_order_relaxed) << " ";
y.store(37,memory_order_relaxed);   cout << x.load(memory_order_relaxed) << endl;

The more modern the CPU, the more likely this is to be faster than the previous example.

Finally, if you just need to keep particular loads and stores in order, you can write:

           atomic<int> x, y;

Thread 1                            Thread 2
x.store(17,memory_order_release);   cout << y.load(memory_order_acquire) << " ";
y.store(37,memory_order_release);   cout << x.load(memory_order_acquire) << endl;

This takes us back to the ordered loads and stores – so 37 0 is no longer a possible output – but it does so with minimal overhead. (In this trivial example, the result is the same as full-blown sequential consistency; in a larger program, it would not be.)

Of course, if the only outputs you want to see are 0 0 or 37 17, you can just wrap a mutex around the original code. But if you have read this far, I bet you already know how that works, and this answer is already longer than I intended :-).

So, bottom line. Mutexes are great, and C++11 standardizes them. But sometimes for performance reasons you want lower-level primitives (e.g., the classic double-checked locking pattern). The new standard provides high-level gadgets like mutexes and condition variables, and it also provides low-level gadgets like atomic types and the various flavors of memory barrier. So now you can write sophisticated, high-performance concurrent routines entirely within the language specified by the standard, and you can be certain your code will compile and run unchanged on both today's systems and tomorrow's.

Although to be frank, unless you are an expert and working on some serious low-level code, you should probably stick to mutexes and condition variables. That's what I intend to do.

For more on this stuff, see this blog post.

  • 70,042
  • 10
  • 116
  • 153
  • 43
    Nice answer, but this is really begging for some actual examples of the new primitives. Also, I think the memory ordering without primitives is the same as pre-C++0x: there are no guarantees. – John Ripley Jun 12 '11 at 00:37
  • 5
    @John: I know, but I am still learning the primitives myself :-). Also I think they guarantee byte accesses are atomic (although not ordered) which is why I went with "char" for my example... But I am not even 100% sure about that... If you want to suggest any good "tutorial" references I will add them to my answer – Nemo Jun 12 '11 at 00:39
  • 6
    @Nemo: Great answer. I've one doubt now: when the output is `37 0` in `Thread 2`, did you not imply that `Thread 1` will execute second statement *before* first statement? If so, would it not produce illogical and undesirable result? I mean, what if the second statement requires the first statement to be executed first to be logically correct? – Nawaz Jun 12 '11 at 14:22
  • 2
    @Nemo: One more thing, I didn't understand the relation betweem C++0x memory model and C++0x multithreading. Could you please emphasise these if you talked about it anywhere in your answer so that I can pay special attention to it? – Nawaz Jun 12 '11 at 14:40
  • 54
    @Nawaz: Yes! Memory accesses can get reordered by the compiler or CPU. Think about (e.g.) caches and speculative loads. The order in which system memory gets hit can be nothing like what you coded. The compiler and CPU will ensure such reorderings do not break _single-threaded_ code. For multi-threaded code, the "memory model" characterizes the possible re-orderings, and what happens if two threads read/write the same location at the same time, and how you excert control over both. For single-threaded code, the memory model is irrelevant. – Nemo Jun 12 '11 at 17:08
  • 29
    @Nawaz, @Nemo - A minor detail: the new memory model is relevant in single-threaded code insofar as it specifies the undefinedness of certain expressions, such as `i = i++`. The old concept of *sequence points* has been discarded; the new standard specifies the same thing using a *sequenced-before* relation which is just a special case of the more general inter-thread *happens-before* concept. – JohannesD Jun 13 '11 at 13:14
  • 2
    Nice answer although note that you would still need to initialize those atomic variables in order to avoid undefined behavior unless somewhere in the new standard integral types are default initialized. Usually only certain compilers in debug builds will set those values to 0 implicitly. – AJG85 Jun 13 '11 at 17:35
  • 21
    @AJG85: Section 3.6.2 of the draft C++0x spec says, "Variables with static storage duration (3.7.1) or thread storage duration (3.7.2) shall be zero-initialized (8.5) before any other initialization takes place." Since x,y are global in this example, they have static storage duration and therefore will zero-initialized, I believe. – Nemo Jun 13 '11 at 20:16
  • 1
    "Under C++11, the result is Undefined Behavior, because loads and stores need not be atomic in general." -- is this "not" really meant to be here? – mlvljr Dec 04 '14 at 00:38
  • 3
    @mlvljr: I am pretty sure I said what I meant... Although I concede this answer would have been better had I used the words "data race" somewhere. – Nemo Dec 04 '14 at 01:39
  • 1
    In most cases there is no sence to implement the double-checked locking initialization in C++11 since local static variable initialization is already thread-safe. – newbie Jun 17 '15 at 21:29
  • @newbie: True, C+11 codified common practice and made the "Meyers singleton" thread-safe. I was just providing an example of the sort of thing you can do with the low-level synchronization primitives. – Nemo Jun 17 '15 at 22:34
  • 1
    @Nemo Could you please explain the 37, 0 part? Whichever way I look at it, x is always assigned before y. So when y is 37, it means x has already been assigned its value. Because if y is 37 and x is 0 then it means single threaded execution is not sequential, which is just wrong. – zindarod Apr 08 '17 at 13:01
  • 4
    @Zindarod: Both the compiler and CPU are free to reorder both loads and stores when you use `memory_order_relaxed`. – Nemo Apr 10 '17 at 03:27
  • 1
    @Nemo: I understand that the CPU can generate the "37, 0" problem because of the inconsistency between main memory and cache levels. So the value read or wrote by one thread might be different to the value read / wrote by the other. However I don't see how a compiler might interfere with the order of the operations in each single thread. Afaik it just "translate" your code into machine instructions in the same order you wrote. So the problem seems to be more related to CPUs, memory levels and threads scheduling than compilers in my opinion. – Bemipefe Oct 31 '17 at 07:46
  • 9
    @Bemipefe: No, the compiler is not obliged to translate your code in the same order you wrote it - it is allowed to re-order operations, provided the overall effect is the same. It might do this, for example, because reordering allows it to produce faster (or smaller) code. – psmears Nov 21 '17 at 16:13
  • @Bemipefe "_nisistency between main memory and cache levels_" No, cache is kept consistent at all time. – curiousguy Jun 07 '18 at 03:10
  • 1
    @psmears "_provided the overall effect is the same_" that statement needs to be qualified. The effect is measurably different! – curiousguy Jun 26 '18 at 17:24
  • 3
    @curiousguy: Whether the caches are kept consistent is CPU-architecture dependent. – psmears Jun 27 '18 at 07:58
  • 1
    @curiousguy: I didn't say the effect is the same - perhaps you missed the word "overall"? (i.e. the intermediate states may be different, but the before/after states will be the same) Obviously that's not the full story, but it's hard to fit the entire language definition into the comment box ;-) – psmears Jun 27 '18 at 08:00
  • @psmears Which commonly used arch where C, C++ or Java can be reasonably supported doesn't have consistent caches? – curiousguy Jun 10 '19 at 05:13
  • Nitpick: std::atomic does not default-initialize to zero. – Karu Jun 12 '19 at 09:32
  • 2
    @Karu: Global variables are always value-initialized. – Nemo Jun 12 '19 at 22:30
  • 1
    Great answer! By the way, I think the minimal overhead to avoid the `"37 0"` would be to `y.store(37,memory_order_release);` in one thread and `y.load(memory_order_acquire);` while still keeping both store and load to the `x` variable `memory_order_relaxed`, right? – Tarc Jul 31 '19 at 19:54
  • @Nemo "_Global variables are always value-initialized_" and? – curiousguy Dec 01 '19 at 16:45
  • 1
    Boom. What an answer. I wanna do pieces like this when I get the knowledge. – Det Feb 22 '20 at 02:00
  • 2
    `"But if you have read this far, I bet you already know how that works"` I don't. I'm just interested in tech that's fascinatedly explained. – Det Feb 22 '20 at 02:03
  • @JohnRipley, This is the code I wrote: `#include #include #include std::atomic x,y; void t1(){ x.store(17,std::memory_order_relaxed); y.store(37,std::memory_order_relaxed); } void t2(){ std::cout << y.load(std::memory_order_relaxed) << " "; std::cout << x.load(std::memory_order_relaxed) << std::endl; } int main(){ std::thread to1(t1); std::thread to2(t2); to1.join(); to2.join(); }` On rpi3 ( a 4 core ARM) always pritnts `0 0` – NameOfTheRose May 30 '20 at 09:29
  • The answer is great, but "37 0" example is simply wrong. Sequential consistency is not enforced on two different atomic variables. This should be corrected before new generation of clueless programmers is repeating mistakes of previous generation – hamilyon Jul 06 '20 at 12:53
  • 3
    @hamilyon: You are mistaken. See e.g. https://stackoverflow.com/a/14851782, or really any tutorial on sequential consistency. By definition, sequential consistency applies to *all* loads and stores, including across multiple memory locations. (It is expensive, not natural, on any modern CPU.) – Nemo Jul 06 '20 at 16:04
  • Ok, maybe in some sense technically this example was somewhat correct and I was wrong. But! It seems to me that pure technical point of view, especially introducing memory barriers to people, is not the best way to serve those who seek answer to this question on this site. Memory barriers are purely architecture-dependent, instruction-set-dependent and thinking in terms of them teaches wrong things. – hamilyon Jul 09 '20 at 15:46
  • Each thread in multi threaded program has different view of the ordering of memory event. https://koheiotsuka701.medium.com/memory-model-basic-d8b5f8fddd5f – rjhcnf Dec 20 '20 at 08:53
  • Do you have a reference for the UB of global write and global read? Is it also UB if thread 2 hasnt been started (and will be started later?) – lalala May 22 '21 at 19:48
  • @Nemo you've mentioned *"Under C++11, the result is Undefined Behavior, because loads and stores need **'not'** be atomic in general."* But shouldn't it be *"... loads and stores need **'to'** be atomic in general."* Thanks in advance! – Milan Jun 07 '21 at 15:51
  • Hi, can I ask to clarify memory_order_relaxed on my example? Imagine 2 threads, both change atomic_int A. 1st do atomic_fetch_add_explicit(&A, +3, ...) 2nd atomic_fetch_add_explicit(&A, +4 ...). Description memory_order_relaxed says it is safe to use with counters, but it does not guarantee synchronization. What does it mean for my example? Will I always have a result = 7? I don't care about the order. All that I want to do is to add some value to the atomic variable, but the value will not be just +1. (My question is mostly about C, in case C++ has a bit different implementation) – Raf Nov 16 '22 at 00:06
  • Or memory_order_relaxed guarantees only that operation `add` will be atomic. For example, in the atomic operation description, I read that CPU can assign 64-bit value in 2 steps: 1st step is the first 32 bits 2nd step last 32 bits or vice versa (maybe it is already not relevant to current times), so memory_order_relaxed with action +4 guarantees that it will not be split into 2 operations. But the final result can be 3 or 4 or 7? – Raf Nov 16 '22 at 00:08

I will just give the analogy with which I understand memory consistency models (or memory models, for short). It is inspired by Leslie Lamport's seminal paper "Time, Clocks, and the Ordering of Events in a Distributed System". The analogy is apt and has fundamental significance, but may be overkill for many people. However, I hope it provides a mental image (a pictorial representation) that facilitates reasoning about memory consistency models.

Let’s view the histories of all memory locations in a space-time diagram in which the horizontal axis represents the address space (i.e., each memory location is represented by a point on that axis) and the vertical axis represents time (we will see that, in general, there is not a universal notion of time). The history of values held by each memory location is, therefore, represented by a vertical column at that memory address. Each value change is due to one of the threads writing a new value to that location. By a memory image, we will mean the aggregate/combination of values of all memory locations observable at a particular time by a particular thread.

Quoting from "A Primer on Memory Consistency and Cache Coherence"

The intuitive (and most restrictive) memory model is sequential consistency (SC) in which a multithreaded execution should look like an interleaving of the sequential executions of each constituent thread, as if the threads were time-multiplexed on a single-core processor.

That global memory order can vary from one run of the program to another and may not be known beforehand. The characteristic feature of SC is the set of horizontal slices in the address-space-time diagram representing planes of simultaneity (i.e., memory images). On a given plane, all of its events (or memory values) are simultaneous. There is a notion of Absolute Time, in which all threads agree on which memory values are simultaneous. In SC, at every time instant, there is only one memory image shared by all threads. That's, at every instant of time, all processors agree on the memory image (i.e., the aggregate content of memory). Not only does this imply that all threads view the same sequence of values for all memory locations, but also that all processors observe the same combinations of values of all variables. This is the same as saying all memory operations (on all memory locations) are observed in the same total order by all threads.

In relaxed memory models, each thread will slice up address-space-time in its own way, the only restriction being that slices of each thread shall not cross each other because all threads must agree on the history of every individual memory location (of course, slices of different threads may, and will, cross each other). There is no universal way to slice it up (no privileged foliation of address-space-time). Slices do not have to be planar (or linear). They can be curved and this is what can make a thread read values written by another thread out of the order they were written in. Histories of different memory locations may slide (or get stretched) arbitrarily relative to each other when viewed by any particular thread. Each thread will have a different sense of which events (or, equivalently, memory values) are simultaneous. The set of events (or memory values) that are simultaneous to one thread are not simultaneous to another. Thus, in a relaxed memory model, all threads still observe the same history (i.e., sequence of values) for each memory location. But they may observe different memory images (i.e., combinations of values of all memory locations). Even if two different memory locations are written by the same thread in sequence, the two newly written values may be observed in different order by other threads.

[Picture from Wikipedia] Picture from Wikipedia

Readers familiar with Einstein’s Special Theory of Relativity will notice what I am alluding to. Translating Minkowski’s words into the memory models realm: address space and time are shadows of address-space-time. In this case, each observer (i.e., thread) will project shadows of events (i.e., memory stores/loads) onto his own world-line (i.e., his time axis) and his own plane of simultaneity (his address-space axis). Threads in the C++11 memory model correspond to observers that are moving relative to each other in special relativity. Sequential consistency corresponds to the Galilean space-time (i.e., all observers agree on one absolute order of events and a global sense of simultaneity).

The resemblance between memory models and special relativity stems from the fact that both define a partially-ordered set of events, often called a causal set. Some events (i.e., memory stores) can affect (but not be affected by) other events. A C++11 thread (or observer in physics) is no more than a chain (i.e., a totally ordered set) of events (e.g., memory loads and stores to possibly different addresses).

In relativity, some order is restored to the seemingly chaotic picture of partially ordered events, since the only temporal ordering that all observers agree on is the ordering among “timelike” events (i.e., those events that are in principle connectible by any particle going slower than the speed of light in a vacuum). Only the timelike related events are invariantly ordered. Time in Physics, Craig Callender.

In C++11 memory model, a similar mechanism (the acquire-release consistency model) is used to establish these local causality relations.

To provide a definition of memory consistency and a motivation for abandoning SC, I will quote from "A Primer on Memory Consistency and Cache Coherence"

For a shared memory machine, the memory consistency model defines the architecturally visible behavior of its memory system. The correctness criterion for a single processor core partitions behavior between “one correct result” and “many incorrect alternatives”. This is because the processor’s architecture mandates that the execution of a thread transforms a given input state into a single well-defined output state, even on an out-of-order core. Shared memory consistency models, however, concern the loads and stores of multiple threads and usually allow many correct executions while disallowing many (more) incorrect ones. The possibility of multiple correct executions is due to the ISA allowing multiple threads to execute concurrently, often with many possible legal interleavings of instructions from different threads.

Relaxed or weak memory consistency models are motivated by the fact that most memory orderings in strong models are unnecessary. If a thread updates ten data items and then a synchronization flag, programmers usually do not care if the data items are updated in order with respect to each other but only that all data items are updated before the flag is updated (usually implemented using FENCE instructions). Relaxed models seek to capture this increased ordering flexibility and preserve only the orders that programmers “require” to get both higher performance and correctness of SC. For example, in certain architectures, FIFO write buffers are used by each core to hold the results of committed (retired) stores before writing the results to the caches. This optimization enhances performance but violates SC. The write buffer hides the latency of servicing a store miss. Because stores are common, being able to avoid stalling on most of them is an important benefit. For a single-core processor, a write buffer can be made architecturally invisible by ensuring that a load to address A returns the value of the most recent store to A even if one or more stores to A are in the write buffer. This is typically done by either bypassing the value of the most recent store to A to the load from A, where “most recent” is determined by program order, or by stalling a load of A if a store to A is in the write buffer. When multiple cores are used, each will have its own bypassing write buffer. Without write buffers, the hardware is SC, but with write buffers, it is not, making write buffers architecturally visible in a multicore processor.

Store-store reordering may happen if a core has a non-FIFO write buffer that lets stores depart in a different order than the order in which they entered. This might occur if the first store misses in the cache while the second hits or if the second store can coalesce with an earlier store (i.e., before the first store). Load-load reordering may also happen on dynamically-scheduled cores that execute instructions out of program order. That can behave the same as reordering stores on another core (Can you come up with an example interleaving between two threads?). Reordering an earlier load with a later store (a load-store reordering) can cause many incorrect behaviors, such as loading a value after releasing the lock that protects it (if the store is the unlock operation). Note that store-load reorderings may also arise due to local bypassing in the commonly implemented FIFO write buffer, even with a core that executes all instructions in program order.

Because cache coherence and memory consistency are sometimes confused, it is instructive to also have this quote:

Unlike consistency, cache coherence is neither visible to software nor required. Coherence seeks to make the caches of a shared-memory system as functionally invisible as the caches in a single-core system. Correct coherence ensures that a programmer cannot determine whether and where a system has caches by analyzing the results of loads and stores. This is because correct coherence ensures that the caches never enable new or different functional behavior (programmers may still be able to infer likely cache structure using timing information). The main purpose of cache coherence protocols is maintaining the single-writer-multiple-readers (SWMR) invariant for every memory location. An important distinction between coherence and consistency is that coherence is specified on a per-memory location basis, whereas consistency is specified with respect to all memory locations.

Continuing with our mental picture, the SWMR invariant corresponds to the physical requirement that there be at most one particle located at any one location but there can be an unlimited number of observers of any location.

  • 1
  • 1
Ahmed Nassar
  • 4,683
  • 2
  • 19
  • 26
  • 61
    +1 for the analogy with special relativity, I've been trying to make the same analogy myself. Too often I see programmers investigating threaded code trying to interpret the behavior as operations in different threads occurring interleaved with one another in a specific order, and I have to tell them, nope, with multi-processor systems the notion of simultaneity between different frames of reference threads is now meaningless. Comparing with special relativity is a good way to make them respect the complexity of the problem. – Pierre Lebeaupin Jun 26 '14 at 19:42
  • 91
    So should you conclude that the Universe is multicore? – Peter K Apr 28 '15 at 11:36
  • 6
    @PeterK: Exactly :) And here is a very nice visualization of this picture of time by physicist Brian Greene: https://www.youtube.com/watch?v=4BjGWLJNPcA&t=22m12s This is "The Illusion of Time [Full Documentary]" at minute 22 and 12 seconds. – Ahmed Nassar Jul 19 '15 at 02:17
  • 4
    Is it just me or is he switching from a 1D memory model (horizontal axis) to a 2D memory model (planes of simultaneity). I find this a bit confusing but maybe that is because I am not a native speaker... Still a very interesting read. – Kami Kaze Jan 12 '17 at 11:31
  • You forgot an essential part: "_by analyzing the results of loads and stores_" ...without using precise timing information. – curiousguy Mar 01 '19 at 16:22
  • I'm finding this very confusing. I understand a 'space-time diagram' to be a two-dimensional diagram with Y axis = time and X axis = space (address space). Then we start talking about 'horizontal slices in the address-space-time diagram', is address-space not the same as (address) space here? You mention how 'address space and time are shadows of address-space-time', so what's the third axis here? Threads? – P.JBoy Nov 10 '19 at 15:09
  • Nice explanation. is it ok if I translate this into a blog @Ahmed Nassar? With reference to you of course ... – Yibo Dec 08 '20 at 12:16
  • 1
    @Yibo Sure. Feel free. – Ahmed Nassar Dec 08 '20 at 20:00
  • 11
    I lived to see relativity being used as a simplified analogy... – Szczepan Hołyszewski Nov 01 '21 at 16:18

This is now a multiple-year old question, but being very popular, it's worth mentioning a fantastic resource for learning about the C++11 memory model. I see no point in summing up his talk in order to make this yet another full answer, but given this is the guy who actually wrote the standard, I think it's well worth watching the talk.

Herb Sutter has a three hour long talk about the C++11 memory model titled "atomic<> Weapons", available on the Channel9 site YouTube - part 1 and part 2. The talk is pretty technical, and covers the following topics:

  1. Optimizations, Races, and the Memory Model
  2. Ordering – What: Acquire and Release
  3. Ordering – How: Mutexes, Atomics, and/or Fences
  4. Other Restrictions on Compilers and Hardware
  5. Code Gen & Performance: x86/x64, IA64, POWER, ARM
  6. Relaxed Atomics

The talk doesn't elaborate on the API, but rather on the reasoning, background, under the hood and behind the scenes (did you know relaxed semantics were added to the standard only because POWER and ARM do not support synchronized load efficiently?).

  • 21,632
  • 6
  • 56
  • 89
  • 4
    @eran do you guys happen to have the slides? links on the channel 9 talk pages do not work. – athos Aug 30 '16 at 02:33
  • 2
    @athos I don't have them, sorry. Try contacting channel 9, I don't think the removal was intentional (my guess is that they got the link from Herb Sutter, posted as is, and he later removed the files; but that's just a speculation...). – Eran Aug 30 '16 at 06:06
  • 1
    Channel 9 went dead, and an available source is YouTube : [part1](https://www.youtube.com/watch?v=A8eCGOqgvH4) and [part2](https://www.youtube.com/watch?v=KeLBd2EJLOU). The slides can be downloaded [here](https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqbkhSNUhtenpKYjcyT3lDeS1ITF9xMGktZXVuUXxBQ3Jtc0trSXdHZ216ekRrNkZySTNKcUpnS3VIc3o4OFpOZWxCS2w1aktoaTBTY3U3R2c5ODFhN3RkeUdsX2pYZnJmQThiNTVvcFlMbjdxa2FSZVhMT0V6dmZIT3N2QlBCVmx6Qnd5TURuZWtpTHRHeVA5UUxXZw&q=https%3A%2F%2F1drv.ms%2Fb%2Fs%21Aq0V7yDPsIZOgcI0y2P8R-VifbnTtw&v=A8eCGOqgvH4). – o_oTurtle Jul 08 '22 at 12:51
  • Thanks @o_oTurtle! fixed the links in the answer. Feel free to make such fixes yourself whenever you come across them. – Eran Jul 10 '22 at 13:48

It means that the standard now defines multi-threading, and it defines what happens in the context of multiple threads. Of course, people used varying implementations, but that's like asking why we should have a std::string when we could all be using a home-rolled string class.

When you're talking about POSIX threads or Windows threads, then this is a bit of an illusion as actually you're talking about x86 threads, as it's a hardware function to run concurrently. The C++0x memory model makes guarantees, whether you're on x86, or ARM, or MIPS, or anything else you can come up with.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
  • 144,682
  • 38
  • 256
  • 465
  • 34
    Posix threads are not restricted to x86. Indeed, the first systems they were implemented on were probably not x86 systems. Posix threads are system-independent, and are valid on all Posix platforms. It's also not really true that it's a hardware property because Posix threads can also be implemented through cooperative multitasking. But of course most threading issues only surface on hardware threading implementations (and some even only on multiprocessor/multicore systems). – celtschk Aug 18 '13 at 19:56
  • This answer is really evasive.. – kingsjester Aug 14 '22 at 14:36

For languages not specifying a memory model, you are writing code for the language and the memory model specified by the processor architecture. The processor may choose to re-order memory accesses for performance. So, if your program has data races (a data race is when it's possible for multiple cores / hyper-threads to access the same memory concurrently) then your program is not cross platform because of its dependence on the processor memory model. You may refer to the Intel or AMD software manuals to find out how the processors may re-order memory accesses.

Very importantly, locks (and concurrency semantics with locking) are typically implemented in a cross platform way... So if you are using standard locks in a multithreaded program with no data races then you don't have to worry about cross platform memory models.

Interestingly, Microsoft compilers for C++ have acquire / release semantics for volatile which is a C++ extension to deal with the lack of a memory model in C++ http://msdn.microsoft.com/en-us/library/12a04hfd(v=vs.80).aspx. However, given that Windows runs on x86 / x64 only, that's not saying much (Intel and AMD memory models make it easy and efficient to implement acquire / release semantics in a language).

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
  • 982
  • 5
  • 5
  • 3
    It is true that, when the answer was written, Windows run on x86/x64 only, but Windows run, at some point in time, on IA64, MIPS, Alpha AXP64, PowerPC and ARM. Today it runs on various versions of ARM, which is quite different memory wise from x86, and nowhere nearly as forgiving. – Lorenzo Dematté Dec 06 '16 at 10:12
  • That link is somewhat broken (says *"Visual Studio 2005 Retired documentation"*). Care to update it? – Peter Mortensen Nov 05 '17 at 23:09
  • 4
    It was not true even when the answer was written. – Ben Dec 02 '17 at 10:14
  • "_to access the same memory concurrently_" to access in a **conflicting** way – curiousguy Jun 13 '18 at 23:22

If you use mutexes to protect all your data, you really shouldn't need to worry. Mutexes have always provided sufficient ordering and visibility guarantees.

Now, if you used atomics, or lock-free algorithms, you need to think about the memory model. The memory model describes precisely when atomics provide ordering and visibility guarantees, and provides portable fences for hand-coded guarantees.

Previously, atomics would be done using compiler intrinsics, or some higher level library. Fences would have been done using CPU-specific instructions (memory barriers).

  • 42,493
  • 9
  • 106
  • 148
  • 21
    The problem before was that there was not such thing as a mutex (in terms of the C++ standard). So the only guarantees you were provided were by the mutex manufacturer, which was fine as long as you did not port the code (as minor changes to guarantees are hard to spot). Now we are get guarantees provided by the standard which should be portable between platforms. – Martin York Jun 12 '11 at 00:09
  • 4
    @Martin: in any case, one thing is the memory model, and another are the atomics and threading primitives that run on top of that memory model. – ninjalj Jun 12 '11 at 00:18
  • 4
    Also, my point was mostly that previously there was mostly no memory model at the language level, it happened to be the memory model of the underlying CPU. Now there is a memory model which is part of the core language; OTOH, mutexes and the like could always be done as a library. – ninjalj Jun 12 '11 at 00:36
  • 3
    It could also be a real problem for the people trying to *write* the mutex library. When the CPU, the memory controller, the kernel, the compiler, and the "C library" are all implemented by different teams, and some of them are in violent disagreement as to how this stuff is supposed to work, well, sometimes the stuff we systems programmers have to do to present a pretty facade to the applications level is not pleasant at all. – zwol Jun 12 '11 at 02:02
  • 12
    Unfortunately it is not enough to guard your data structures with simple mutexes if there is not a consistent memory model in your language. There are various compiler optimizations which make sense in a single threaded context but when multiple threads and cpu cores come into play, reordering of memory accesses and other optimizations may yield undefined behavior. For more information see "Threads cannot be implemented as a library" by Hans Boehm: http://citeseer.ist.psu.edu/viewdoc/download;jsessionid=8923A90891ADBA7C51A8164659100D24?doi= – exDM69 Jun 13 '11 at 12:45
  • @esDM69: it is enough to protect all your data with mutexes, and have the low level fences and atomics isolated in the mutex library. Show me a counterexmaple. – ninjalj Jun 13 '11 at 18:02
  • The paper @exDM69 linked includes all the counterexamples you could want. – zwol Jul 28 '11 at 16:28
  • 1
    @ninjalj there are counterexamples in the paper but in a nutshell: non word-aligned reads and writes and optimizations that combine many small writes to a big one (e.g. 7 one byte writes may be changed into one 64 bit read-modify-write). So if you try to guard individual bits and bytes with a mutex, you're looking for trouble (and a consistent memory model will help). – exDM69 Aug 04 '11 at 11:13
  • @exDM69: If a compiler can be forced/guaranteed to regard actions which acquire and release mutexes as though they might potentially access any object whose address has been exposed to the outside world, that would be sufficient to allow multi-threaded code to use "ordinary" objects for inter-thread communication provided they are guarded by mutexes. Unfortunately, some compilers using Whole Program Optimization may examine the code for the acquisition/release operations and conclude that objects which aren't directly accessed by such code won't be accessed as a consequence... – supercat Dec 03 '17 at 20:50
  • ...of doing a release and an acquire. Unfortunately, the Standard fails to provide any standard means of ensuring that all non-`restrict`-qualified accesses that are executed before a certain point in the code must behave as though they are accessed before any such accesses that follow that point in the code. – supercat Dec 03 '17 at 20:53
  • @supercat For any processor that requires special operation for "acquisition/release operations", you will have either an `asm` statement, or a syscall. Either way, global optimization stops. – curiousguy Jun 26 '18 at 17:46
  • @curiousguy: Some build systems produce information about functions indicating what kinds of side-effects they may have, and then apply optimizations across calls to such functions. Even if it would seem impractical to apply a global optimization across a certain function call, or it would seem like a bad idea, such considerations don't guarantee that a "clever" compiler writer won't find a way to do it anyway. – supercat Jun 26 '18 at 18:10
  • @supercat If you are going to annotate function declarations, you can annotate function calls and describe invariants. So you can describe that a call to `getppid()` is "memory const", it doesn't change any object in memory, and that `getpid()` is functionally "pure" (the result is purely a function of the function arguments), ie it returns a constant value that may be cached. It means that values of objects potentially visible by other threads don't have to reloaded after either calls, and that the value of `getpid()` can even be cached after a call to an arbitrary function. – curiousguy Jun 26 '18 at 19:06
  • @curiousguy: Indeed. Is there any particular reason to believe that a "clever" implementation won't decide that library functions to acquire or release a mutex won't access any objects outside the mutex even if the whole point of using mutexes in the first place is to safely allow for the possibility of outside code accessing objects at arbitrary times prior to mutex acquisition or after mutex release? – supercat Jun 26 '18 at 19:22
  • @supercat If you need a memory fence or LOCK-ed instruction (incl. implicit LOCK) in your function, the compiler will not reorder at least because it doesn't understand assembly. The problem is when you don't need assembly. – curiousguy Mar 01 '19 at 16:27
  • @curiousguy: One of the big reasons C was designed in the first place was to minimize the need to use assembly language, especially since different vendors' assemblers will often require different syntax for the same generated machine code. I find it absurd that one would have to use vendor-specific asm directives to prevent refrain from doing something that every implementation should be capable of refraining from doing. – supercat Mar 01 '19 at 16:57
  • @supercat Now there is `atomic_signal_fence` to force the compiler to treat a NOP as as an ABI barrier (the equivalent of an external call to a NOP function w/o the call) – curiousguy Mar 01 '19 at 21:39
  • 1
    Unfortunately, N1570 7.17.4 [Fences] only defines the behavior of fences as affecting "atomic" operations. Further, nothing would forbid an implementation from having its startup function examine the machine code for an external function, observe whether or not it actually does anything, and then setting a flag which other code could use to ignore any sequencing implications of it if it doesn't. – supercat Mar 01 '19 at 23:00
  • @curiousguy: See above. The only good reason for the C Standard not to have included sequencing-barrier directives is that implementations intended for purposes requiring such semantics could supply them without special directives (e.g. by treating `volatile` accesses or function calls as acquire/release), and there was no reason to believe compiler writers would be incapable of doing so without a mandate. On the other hand, "targeted" barriers could allow better optimization than global ones without imposing any implementation problems. – supercat Mar 01 '19 at 23:07
  • @supercat Since one of the primary use of `volatile` is for async signals, it's a bit silly not to have `atomic_signal_fence` also order regular accesses WRT to volatiles accesses. – curiousguy Mar 02 '19 at 05:26
  • @curiousguy: It's also silly to require that an implementation that claims to offer any of the features in atomic.h must also claim to support all the features, without regard for, or any indication of, whether they can be usefully supported. Many programs require that atomic operations be globally-atomic and non-blocking, but wouldn't care whether a C implementation could guarantee that they are lock-free. Unfortunately, the only quality factor provided for by the Standard is "is lock free". I wonder what the authors of the Standard expect implementations to do with that info? – supercat Mar 02 '19 at 18:38
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/189327/discussion-between-curiousguy-and-supercat). – curiousguy Mar 02 '19 at 19:17

Some of the other answers get at the most fundamental aspects of the C++ memory model. In practice, most uses of std::atomic<> "just work", at least until the programmer over-optimizes (e.g., by trying to relax too many things).

There is one place where mistakes are still common: sequence locks. There is an excellent and easy-to-read discussion of the challenges at https://www.hpl.hp.com/techreports/2012/HPL-2012-68.pdf. Sequence locks are appealing because the reader avoids writing to the lock word. The following code is based on Figure 1 of the above technical report, and it highlights the challenges when implementing sequence locks in C++:

atomic<uint64_t> seq; // seqlock representation
int data1, data2;     // this data will be protected by seq

T reader() {
    int r1, r2;
    unsigned seq0, seq1;
    while (true) {
        seq0 = seq;
        r1 = data1; // INCORRECT! Data Race!
        r2 = data2; // INCORRECT!
        seq1 = seq;

        // if the lock didn't change while I was reading, and
        // the lock wasn't held while I was reading, then my
        // reads should be valid
        if (seq0 == seq1 && !(seq0 & 1))
    use(r1, r2);

void writer(int new_data1, int new_data2) {
    unsigned seq0 = seq;
    while (true) {
        if ((!(seq0 & 1)) && seq.compare_exchange_weak(seq0, seq0 + 1))
            break; // atomically moving the lock from even to odd is an acquire
    data1 = new_data1;
    data2 = new_data2;
    seq = seq0 + 2; // release the lock by increasing its value to even

As unintuitive as it seems at first, data1 and data2 need to be atomic<>. If they are not atomic, then they could be read (in reader()) at the exact same time as they are written (in writer()). According to the C++ memory model, this is a race even if reader() never actually uses the data. In addition, if they are not atomic, then the compiler can cache the first read of each value in a register. Obviously you wouldn't want that... you want to re-read in each iteration of the while loop in reader().

It is also not sufficient to make them atomic<> and access them with memory_order_relaxed. The reason for this is that the reads of seq (in reader()) only have acquire semantics. In simple terms, if X and Y are memory accesses, X precedes Y, X is not an acquire or release, and Y is an acquire, then the compiler can reorder Y before X. If Y was the second read of seq, and X was a read of data, such a reordering would break the lock implementation.

The paper gives a few solutions. The one with the best performance today is probably the one that uses an atomic_thread_fence with memory_order_relaxed before the second read of the seqlock. In the paper, it's Figure 6. I'm not reproducing the code here, because anyone who has read this far really ought to read the paper. It is more precise and complete than this post.

The last issue is that it might be unnatural to make the data variables atomic. If you can't in your code, then you need to be very careful, because casting from non-atomic to atomic is only legal for primitive types. C++20 is supposed to add atomic_ref<>, which will make this problem easier to resolve.

To summarize: even if you think you understand the C++ memory model, you should be very careful before rolling your own sequence locks.

  • 353,942
  • 115
  • 666
  • 851
Mike Spear
  • 832
  • 8
  • 13

C and C++ used to be defined by an execution trace of a well formed program.

Now they are half defined by an execution trace of a program, and half a posteriori by many orderings on synchronisation objects.

Meaning that these language definitions make no sense at all as no logical method to mix these two approaches. In particular, destruction of a mutex or atomic variable is not well defined.

  • 8,038
  • 2
  • 40
  • 58
  • 2
    I share your fierce desire for improvement of the language design, but I think your answer would be more valuable if it were centered on a simple case, for which you showed clearly and explicitly how that behavior violates specific language design principles. After that I would strongly recommend you, if you allow me, to give in that answer a very good argumentation for the relevance of each of those points, because they will be contrasted against the relevance of the inmense productivity benefits perceived by C++ design – Matias Haeussler Nov 21 '19 at 20:33
  • 1
    @MatiasHaeussler I think you misread my answer; I'm not objecting to the definition of a particular C++ feature here (I also have many such pointed criticisms but not here). **I'm arguing here that there is no well defined construct in C++ (nor C).** The whole MT semantics are a complete mess, as you don't have sequential semantics anymore. (I believe Java MT is broken but less.) The "simple example" would be almost any MT program. If you disagree, you are welcome to answer my question about [how to prove correctness of MT C++ programs](https://stackoverflow.com/q/58722848/963864). – curiousguy Nov 21 '19 at 21:49
  • Interesting, I think I understand more what you mean after reading your questiong. If I am right you are refering to *the impossibility of developing proofs for C++ MT programs correctness*. In such a case I would say that for me is something of huge importance for the future of computer programming, in particular for the arrival of artificial inteligence. But I would point too that for the great majority of people asking questions in stack overflow that is not something they are even aware of, and even after understanding what you mean and becoming interested – Matias Haeussler Nov 22 '19 at 00:17
  • they will probably expect for some proof for your remarks. Maybe you could refer to a well stablished source which compares the C capability of proving the correctness of its MT programs vs its C++ counterpart, and ask specific questions related to that or giving answers that addresses facts demostrated inside that source. Right now I am not even sure of the pertinence of that discussion inside stackoverflow. Maybe it would be considered more properly on stack exchange? Indeed I woud be interested in answering this last question, regardless of the answer simplicity: – Matias Haeussler Nov 22 '19 at 00:32
  • 1
    "Should questions about the demostrability of computer programs be posted in stackoverflow or in stackexchange (if in neither, where)?" This one seems to be one for meta stackoverflow, is it not? – Matias Haeussler Nov 22 '19 at 00:35
  • 1
    @MatiasHaeussler 1) C and C++ essentially share the "memory model" of atomic variables, mutexes and multithreading. 2) The relevance on this is about the benefits of having the "memory model". I think the benefit is zero as the model is unsound. – curiousguy Nov 22 '19 at 00:56
  • Maybe when you have time you could give me your opinion about the demonstration method for C++ MT programs given in this papaer https://plv.mpi-sws.org/fsl/ARC/paper.pdf. It is very interesting because they seem to agree with you with the increased hardship for C++ MT programs after the addition of the memory model, however seem to be able to give correctness proofs anyway. In the introduction they give the following reasons for the introduction of weak memory models: – Matias Haeussler Nov 22 '19 at 12:42
  • "Even though sequential consistency is a simple and intuitive concurrency model, it does not match the real world. In practice, no hardware provides us with a sequentially consistent execution environment. In order to improve performance or conserve energy, modern hardware implementations give us what is known as weak memory models; that is, models of concurrency providing weaker guarantees than sequential consistency. " – Matias Haeussler Nov 22 '19 at 12:42
  • @MatiasHaeussler This type of formalisation work is fascinating (I will make some effort to understand it), seems very hard, but *often has hard wired, strong limits on what can be described*. There was an effort to formalize Java threads but it was limited to finite executions. There was a "proof" of correctness of some compilation scheme of some MT language (forgot which) but the proof was restricted by construction; the authors made an informal argument that they couldn't think of a good reason why the same proof wouldn't work in general. But they couldn't generalize the formal proof. Duh. – curiousguy Nov 23 '19 at 01:07
  • It's unclear what you're trying to say with this answer. I'm not sure that "C and C++ used to be defined by an execution trace of a well formed program" is even true in this case. Perhaps that was historically true for a single-threaded program running on a processor with a strong memory model, but I don't think that's been true since the advent of symmetric multi-threading. Is there a specific case where you think the C++11 memory model is a regression? It seems to me it's merely formalising the existing assumptions into a standard model for portability. – Jesse Wyatt Jun 23 '22 at 07:29