patternMinor
Write Serialization for Cache Coherence in the presence of Store Buffers
Viewed 0 times
thecoherenceserializationbufferswritestorecacheforpresence
Problem
One of the requirements for a coherent memory system is write serialization - "two writes to address X by any two processors are observed in the same order by all processors". I am not sure how this condition would be met when the CPU cores have a store buffer into which stores are retired before updating the memory hierarchy.
Suppose Cores A and B both retire a store to address X (which is cached by both A and B), and these stores are now in the store buffers and are yet to update the cache. Now loads from address X on both cores A and B could retire after obtaining the load value from the store buffers ie loads from X on core A see value written by the store to X on core A, similarly, loads from X on core B see value written by the store to X on core B. Say now, the store to X from core A sends a read-upgrade message to core B. What happens next on core B? After coherence transactions have completed for A, the store to X on B updates the local cache. After the stores to X from A and B have completed the final value of X is the store by B.
Above, the order for values seen by A for X are write by A followed by the write to B. But this doesn't seem to be the order seen by B. So the question is how is write serialization enforced when store buffers are being used?
(This is based on my understand of the various material I have read and would appreciate it if anyone can point to sources that discuss these issues in detail)
Suppose Cores A and B both retire a store to address X (which is cached by both A and B), and these stores are now in the store buffers and are yet to update the cache. Now loads from address X on both cores A and B could retire after obtaining the load value from the store buffers ie loads from X on core A see value written by the store to X on core A, similarly, loads from X on core B see value written by the store to X on core B. Say now, the store to X from core A sends a read-upgrade message to core B. What happens next on core B? After coherence transactions have completed for A, the store to X on B updates the local cache. After the stores to X from A and B have completed the final value of X is the store by B.
Above, the order for values seen by A for X are write by A followed by the write to B. But this doesn't seem to be the order seen by B. So the question is how is write serialization enforced when store buffers are being used?
(This is based on my understand of the various material I have read and would appreciate it if anyone can point to sources that discuss these issues in detail)
Solution
From a coherence perspective, I think your example is coherent. All processors believe that the write and read from A happened first, then the write and read from B happened later.
From a consistency standpoint you need to be more careful. (Consistency is the global ordering of memory operations to different addresses.) The way consistency is handled in practice is that the store buffer sits before retirement, not after. The store buffer acts as a little coherent cache for the speculative memory state. A store instruction is not permitted to retire until its processor has acquired write ownership of the appropriate cache line.
The store buffer needs to be aware of all the coherence traffic reaching the cache. If a speculative load gets its value from the store buffer, but the corresponding memory location is invalidated before the load retires, the load (and everything following it in the reorder-buffer) needs to be squashed and reexecuted.
From a consistency standpoint you need to be more careful. (Consistency is the global ordering of memory operations to different addresses.) The way consistency is handled in practice is that the store buffer sits before retirement, not after. The store buffer acts as a little coherent cache for the speculative memory state. A store instruction is not permitted to retire until its processor has acquired write ownership of the appropriate cache line.
The store buffer needs to be aware of all the coherence traffic reaching the cache. If a speculative load gets its value from the store buffer, but the corresponding memory location is invalidated before the load retires, the load (and everything following it in the reorder-buffer) needs to be squashed and reexecuted.
Context
StackExchange Computer Science Q#66093, answer score: 7
Revisions (0)
No revisions yet.