• yacovm's avatar
    [FAB-14691] Add to msgStore and puller atomically · c00203f8
    yacovm authored
    When adding a block to the message store, if it is added and
    not rejected - it is then added to the block puller.
    When the block is removed from the message store, a callback
    is triggered to remove the block from the block puller.
    However, these 2 operations are not atomic.
    Since we can add a block to the message store from both AddToMsgStore
    (which is invoked by Gossip() ) and from HandleMessage,
    we can have the following schedule:
      1) A block with sequence of 100 is gossiped by the upper
        layer of the peer, and AddToMsgStore is called, which
        adds the block to the message store, and the CPU is preempted.
      2) A block message 210 is received via HandleMessage and it causes
         the block 100 to be evicted from the message store, and the
        callback to remove the message from the block puller is called,
        but it is not removed because it is not there yet.
      3) The block 210 is added to the block puller, since it was added
         to the message store.
      4) The CPU is back to perform AddToMsgStore, and adds block 100
         to the block puller.
    Now the block puller has block 100, and the message store doesn't
    have block 100 anymore - which means it will never be evicted from
    the block puller.
    To prevent this we need to make these 2 operations be atomic.
    Change-Id: I3b7d0d013ce8da5d9a0e40f8b0cdbc3edaed22c9
    Signed-off-by: default avataryacovm <yacovm@il.ibm.com>