1. 19 May, 2018 1 commit
    • yacovm's avatar
      [FAB-10222] Honor maximum peers to send if 0 · 9d524719
      yacovm authored
      
      
      When the collection config specifies max peers, it might be 0.
      
      Currently, gossip assumes that if the max peers is 0 then it was
      omitted, and just sets the max peers to all the known peers.
      
      Since collection membership can change in v1.2, we should be
      conservative and enforce 0 max peers as 0.
      
      Change-Id: Idef84b6d5d88a719c737aef05b65a724a3a38468
      Signed-off-by: default avataryacovm <yacovm@il.ibm.com>
      9d524719
  2. 15 Apr, 2018 1 commit
  3. 17 Mar, 2018 2 commits
    • yacovm's avatar
      [FAB-7552] IdentityInfo in gossip identityStore · f4007a41
      yacovm authored
      
      
      The discovery service needs to output identities of known peers.
      
      Gossip stores the identities in an identity mapper,
      which could be extended to return a snapshot of its identities.
      
      This change set makes it so that the identity mapper also saves
      the organizations of peers upon their insertion, and adds
      the needed method.
      
      Change-Id: I94b032781bae5d07a3d9fef586273ac8bfcc2e4d
      Signed-off-by: default avataryacovm <yacovm@il.ibm.com>
      f4007a41
    • Gari Singh's avatar
      [FAB-8061] Update grpc-go to latest version · d09e2f22
      Gari Singh authored
      
      
      Update to grpc-go 1.10.0
      
      We were several releases behind and there were
      changes in behavior and some APIs which required
      tweaking production and test code
      
      Change-Id: I86bc6cd170b5a07777c11db485bb86fdc73e495a
      Signed-off-by: default avatarGari Singh <gari.r.singh@gmail.com>
      d09e2f22
  4. 11 Mar, 2018 1 commit
  5. 05 Mar, 2018 1 commit
    • yacovm's avatar
      [FAB-6741] Deprecate stateInfo.Metadata · 20de11a0
      yacovm authored
      
      
      The stateInfo.Metadata field was used in v1.0 for storing the peer's
      ledger height.
      
      In v1.1 we added a properties proto structure field that would replace
      the metadata for ledger height.
      In v1.2 (next release) we deprecate the metadata and as a result, peers
      in v1.0 won't be able to gossip in channel context with peers
      from v1.2
      
      Change-Id: Ieefa98563500760eb23bf579f93ed786b935acf4
      Signed-off-by: default avataryacovm <yacovm@il.ibm.com>
      20de11a0
  6. 28 Nov, 2017 1 commit
  7. 27 Nov, 2017 1 commit
    • Artem Barger's avatar
      [FAB-2641] Prevent loop while gossiping msgs · eb437dab
      Artem Barger authored
      
      
      Currently while gossip forward the message during distribution it
      selects peers from the current membership view, while not filtering
      the peer which delivered the message. As a consequence message might be
      send to the source peer one again. This commit extends the peers
      selection logic by adding the check which allows to avoid sending same
      message to the source peer back again.
      
      Change-Id: I76ceafba2335c27e1186f511d3cdab8a24d600af
      Signed-off-by: default avatarArtem Barger <bartem@il.ibm.com>
      eb437dab
  8. 12 Sep, 2017 2 commits
  9. 11 Sep, 2017 1 commit
    • Gari Singh's avatar
      FAB-6091 Build with Go 1.9 · 453f4c3f
      Gari Singh authored
      
      
      A prior change made fabric compatible
      with G0 1.9.  This CR now builds fabric
      with Go 1.9 by default by updating the
      baseimage version to 0.4.1 which uses
      Go 1.9.  Also updated any docs to
      indicate Go 1.9 should now be used.
      
      - Fixed linter error with sanitycheck.go
      - added check for testing.tRunner for gossip
      TestEndedGoroutines.  Not sure why I missed
      this earlier
      
      Change-Id: I213cc2f88363c32ccff246b5655afd88b42a7778
      Signed-off-by: default avatarGari Singh <gari.r.singh@gmail.com>
      453f4c3f
  10. 05 Sep, 2017 1 commit
    • yacovm's avatar
      [FAB-5929] Ack based private data dissemination · e5794f3f
      yacovm authored
      
      
      This commit adds to the gossip API an ability to send to a set of peers
      a message according to some criteria, like:
      - Minimum acknowledgement from the peers
      - Timeout to wait
      - Channel
      - Maximum amount of peers to send to
      - A predicate, which can be derived from MSP policies
      
      Change-Id: I69ee27ead3532558fa0213594ece71ec54dc5a07
      Signed-off-by: default avataryacovm <yacovm@il.ibm.com>
      e5794f3f
  11. 30 Aug, 2017 3 commits
    • yacovm's avatar
      [FAB-5752] Gossip identity expiration III · b1c90bb5
      yacovm authored
      
      
      This commit:
      1) Makes gossip's identity package use the new method
         Expiration() that was added to the MessageCryptoService
         in the previous commit.
         Now, when an identity is added to the identity mapper -
         a timer is set and the identity is automatically purged
         after the timer expires.
         The timer is set to the time left until the identity
         is expired.
      2) Adds some tests to the identity package, and makes its
         code-coverage reach 100%.
      3) Moves the logic that periodically purges un-used identities
         from the gossip package to the identity package, to improve
         code cohesion.
      
      Change-Id: I684197ff0f20031e80e9e9341042a5593eaca07a
      Signed-off-by: default avataryacovm <yacovm@il.ibm.com>
      b1c90bb5
    • yacovm's avatar
      [FAB-5752] Gossip identity expiration II · e16c4851
      yacovm authored
      
      
      This commit adds a new method (and its implementation) to
      the MessageCryptoService:
      	Expiration(peerIdentity PeerIdentityType) (time.Time, error)
      
      In the next commit I would adjust gossip to make use of this method
      
      Change-Id: Idbdf1ba3b238a122e9d2db0b84016eb7cc6490a7
      Signed-off-by: default avataryacovm <yacovm@il.ibm.com>
      e16c4851
    • yacovm's avatar
      [FAB-5752] Gossip identity expiration I · 0a03e393
      yacovm authored
      
      
      This commit:
      1) Moves the identity mapper into the gossip package from the service
         package, since it's not needed in the service package.
      2) Extends the constructor of the identity mapper to have a callback
         that is fired whenever an identity is deleted from the identity mapper.
      
      Change-Id: I1c9eaa47c97351518848843a9c4f2e1835f82249
      Signed-off-by: default avataryacovm <yacovm@il.ibm.com>
      0a03e393
  12. 29 Aug, 2017 1 commit
  13. 14 Aug, 2017 1 commit
    • yacovm's avatar
      [FAB-3949] refactor gossip UT waitgroup counter · a434b221
      yacovm authored
      
      
      In the gossip tests there is a waitgroup counter that is
      used for the last test (that checks that goroutines ended)
      to be able to run only when all the other tests have finished.
      
      Currently the waitgroup counter is initialized to the number of
      tests but it should be initialized in a more dynamic way that
      shows which tests are disabled and which tests are not.
      
      Change-Id: I491d7d3676aeb79fbe2a30fadc63b27a37f003c9
      Signed-off-by: default avataryacovm <yacovm@il.ibm.com>
      a434b221
  14. 01 Aug, 2017 1 commit
    • Baohua Yang's avatar
      [FAB-5194] Fix usage problems in code · 76c0dc56
      Baohua Yang authored
      
      
      This patchset helps clean up the code and docs:
      
      * Fix redundant parenthesese.
      * Fix mismatching params in func return.
      * Fix variable name collides with imported package name and builtin
        function or reserved words.
      * Fix word typos.
      
      Change-Id: I30304c233067ead7e74d18e3caf2604b3ed1490a
      Signed-off-by: default avatarBaohua Yang <yangbaohua@gmail.com>
      76c0dc56
  15. 21 Jul, 2017 1 commit
  16. 11 Jun, 2017 1 commit
  17. 10 Jun, 2017 1 commit
    • YACOVM's avatar
      [FAB-3450] Prevent panic on msg signing · d7233d59
      YACOVM authored
      
      
      Currently, the gossip code panics if it's unable to sign a message
      or to marshal it bytes, because the assumption was
      that if the signing failed then something is terrible wrong with the
      peer and it's not recoverable
      If turning a message into bytes fails, it's because of something
      not recoverable too.
      
      However, a user posted a stack trace in which gossip tried constructing
      a message that its envelope has a payload of 2GB.
      
      Now, the cause for a creation of such a message was fixed but in theory
      we may somehow have a situation where a large message is constructed.
      
      In order to prevent these kind of problems, and the crash of the peer,
      we need to make the Signer return an error instead of panicking.
      
      Change-Id: Ife8f4e137c92c167dadc7bbe1bcdf45f93b2b38e
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      d7233d59
  18. 30 May, 2017 1 commit
    • Gari Singh's avatar
      [FAB-4002] Upgrade to latest version of grpc-go · a1c67027
      Gari Singh authored
      
      
      There have been many updates to the grpc-go package
      since it was vendored in v0.6 and then v1.0.0.
      
      There are several defects / issues related to
      inactivity timeouts, connectivity, etc which can
      be more easily fixed using new features such as
      the client keepalive.
      
      This CR updates grpc-go as well as all of its
      dependencies and makes minor updates to the
      code where required as well as recompiles all of
      the protos using the protobuf compiler plugin
      required by this version of grpc-go.
      
      Note that for the updated vendoring, I chose
      to delete the old packages and add the
      updated ones as govendor is not perfect
      when it comes to trying to update packages.
      
      The only "new" code was adding custom server
      transport credentials in the comm package as
      the default provided by the grpc/credentials
      package was changed and breaks the functionality
      required for dynamically updating root certs for
      running grpc servers and new package config
      values for send/receive msg sizes as grpc-go
      now enforces them.  Changes were also made
      to any grpc clients and servers used in the
      code to leverage the new config values.
      
      There is one TODO which is to actually set
      the max send/receive msg sizes based on
      config.  This will need to be done for
      both the peer and the orderer.
      
      Also changed the license header to the new
      format for any new or modified files
      
      Change-Id: I41687a54aa21802203b04a339952881349042b70
      Signed-off-by: default avatarGari Singh <gari.r.singh@gmail.com>
      a1c67027
  19. 24 May, 2017 1 commit
    • YACOVM's avatar
      [FAB-4085] Prevent expiration of self identity · 08a2515c
      YACOVM authored
      
      
      In gossip we have time-based expiration (purge) of identities that haven't
      been used for a long time.
      This may cause an expiration of the peer's own certificate if it doesn't
      gossip with any peers (i.e if it's alone in the world).
      
      The problem is that when an identity expires from the identity store,
      it is also expired from the pull mediator that is used for identities.
      This is important because the only way identities are gossiped
      transitively is via the pull mechanism.
      If a peer's own identity disappears from the pull mediator,
      it will never be sent to peers transitively.
      
      Change-Id: Ic2138ebaa60bdf2454d65d3884f141c3736254a0
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      08a2515c
  20. 16 May, 2017 1 commit
    • YACOVM's avatar
      [FAB-3109] Fix anchor peer connection logic · f966c8c9
      YACOVM authored
      
      
      The current implementation of what happens when the peer connects
      to an anchor peer has a flawed logic.
      
      It assumes that if the probe is successful, the sending is also.
      That's too optimistic.
      Another problem is that after the handshake is performed, the remote peer
      may not distinguish between the closing of the connection of the handshake
      and the closing of the new connection that sends the MembershipRequest,
      since we only support 1 connection between each peer (to prevent DOS).
      
      I re-implemented the logic to do the following:
      1) First, perform a handshake to know whether the remote peer is from
        our org or not.
        Peers in different orgs should not know of our internal endpoints).
        The handshake is performed until it succeeds or too many attempts
        are made.
      2) Now, construct a MembershipRequest and send it until you get
         acked from the remote peer, or when attempts are exhausted.
      
      Added a test that simulates the flow.
      
      Change-Id: I72370d9be816105fdc5af7577ce578faf6e5abdc
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      f966c8c9
  21. 05 May, 2017 2 commits
  22. 03 May, 2017 1 commit
  23. 01 May, 2017 2 commits
    • YACOVM's avatar
      [FAB-3530] Gossip - add block seq# validation · d21cd6db
      YACOVM authored
      
      
      When a peer receives a data message (contains a block) it doesn't verify
      that the sequence of the block matches the sequence inside the marshaled block,
      and just calls VerifyBlock.
      
      The implementation already unmarshals the block, so there is no point
      in un-marshalling it outside.
      I extended the interface VerifyBlock to accept also a sequence number.
      
      Change-Id: I85c6c57f26f5e404fe8e7ece2abffa58eaded625
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      d21cd6db
    • YACOVM's avatar
      [FAB-3497] Removing the hash from gossip dataMsg · 1acb65f2
      YACOVM authored
      
      
      The hash field isn't used anywhere but in the protos comparator function
      to compare 2 ledger blocks disseminated.
      This imposes a security vulnerability:
      A malicious peer can send blocks with an arbitrary hash in the hash field
      of the DataMsg, and then these messages would enter the in-memory stores
      and fill up the memory.
      
      Since we have no use of the block hash of our own message, I am removing it
      entirely.
      
      Change-Id: Ic22ed3d06f102795c8f2a74b27063848affc926a
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      1acb65f2
  24. 30 Apr, 2017 1 commit
    • Gari Singh's avatar
      [FAB-3105] Gossip needs to use comm package CA support · f0acc681
      Gari Singh authored
      
      
      The current gossip implementation is initialized with a
      set of dial options which don't change during the lifecycle
      of the peer.
      
      In order to resolve this, the gossip comm implementation
      has been modified to accept a function which returns the
      appropriate dial options for secure communincations with
      other peers.
      
      When used in the peer, the function passed in
      leverages the core/comm packages CASupport.
      
      Change-Id: I99cc4204e27c67f93f8f1ba91891fdb97081c99e
      Signed-off-by: default avatarGari Singh <gari.r.singh@gmail.com>
      f0acc681
  25. 26 Apr, 2017 1 commit
    • YACOVM's avatar
      [FAB-3114] Gossip identity expiration · 55d96b2e
      YACOVM authored
      
      
      In gossip we have peer identities that are stored in-memory.
      This commit adds 2 timely checks that deletes the identity
      from memory and also closes connections to corresponding peers.
      - That the certificate (identity) didn't expire
      - If the identity hasn't been used for a long time
      
      Change-Id: I901b60e2a09c11202f305106991b7eadd5f7b29c
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      55d96b2e
  26. 24 Apr, 2017 1 commit
  27. 18 Apr, 2017 1 commit
    • YACOVM's avatar
      [FAB-3213] Gossip identity revocation support · 9d121662
      YACOVM authored
      
      
      When a peer gets a new config block and it contains CRLs,
      the gossip layer needs to be notified in order to close existing
      connections to peers that their certificates have been expired.
      
      If this is not done, then these peers are still forwarded data
      like peer membership and channel membership, because the connection
      is already open.
      
      This commit adds an ability to revoke identities by receiving a predicate
      function that: given an identity, it returns whether it is suspected of being
      revoked (i.e, the SN is found within some CRL of some MSP).
      Then- the gossip layer calls ValidateIdentity on the stored identity, and if
      it is found to be invalid- it:
      - deletes the identity from memory
      - closes an active connection to the peer, if such exists.
      
      Currently the implementation of that predicate is the naive/obvious one,
      that suspects all identities.
      In a future commit I'll (hopefully) add code that uses the CRLs themselves.
      
      Change-Id: I56d995a3720a736b1242b13a193f9a7933299345
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      9d121662
  28. 14 Mar, 2017 1 commit
    • YACOVM's avatar
      [FAB-2758] Decouple anchor peers endpoints from orgIDs · f1a88db2
      YACOVM authored
      In the current implementation of the join channel message in the
      gossip layer, each anchor peer has its own organization, and the
      joinChannel invocation enumerates all anchor peers and builds a
      list of organizationIDs from them.
      
      When https://gerrit.hyperledger.org/r/#/c/7105/
      
       will be introduced,
      it will be needed to create a channel only with either no anchor peers
      or anchor peers of the organization of the channel creator.
      This would confuse the gossip layer because it'll derive that each channel
      has only 1 organization instead of the original participants.
      
      I decoupled the anchor peers from the organization IDs by changing
      the JoinChannelMessage interface in api/channel.go to have
      Members() []OrgIdentityType  which would
      return the channel members (organizations)
      and AnchorPeersOf(org OrgIdentityType) []AnchorPeer which would
      return for each organization, its corresponding anchor peers.
      
      Also (of course) updated the tests accordingly.
      
      Change-Id: I90131a0726fc7ca68fd750247f4df558c5cca0fd
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      f1a88db2
  29. 07 Mar, 2017 1 commit
    • YACOVM's avatar
      [FAB-2658] Bug fix: pass correct block data to MCS · 9a5b456b
      YACOVM authored
      
      
      The MessageCryptoService complains:
      verifyBlock -> WARN 0cc Received fabricated block from [...]\
       in DataUpdate: Failed casting SignedBlock to []byte on channel [foo]
      
      Because the object that was passed wasn't a []byte as expected but
      the object that contains the []byte as a field.
      I changed the signature of VerifyBlock to accept []byte instead
      of the previous interface type api.SignedBlock and removed the latter type.
      
      Change-Id: I7320c398b117072c0790f77d9c0f9305b1adf5ea
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      9a5b456b
  30. 05 Mar, 2017 2 commits
    • YACOVM's avatar
      [FAB-2624] Gossip Anchor peer connect bug fix · 48987d23
      YACOVM authored
      
      
      When the PEM was removed from the anchor peers, the Connect()
      method of the discovery layer wasn't updated accordingly.
      As a result, all anchor peers were mapped to the same (empty string) key
      in the discovery layer
      
      This commit:
      1) Changes the logic and semantics of the Connect():
         It now creates a membership request that doesn't
         (or does, up to the caller) have the internal endpoint inside of it,
         according to the param passed.
         This is because we don't have knowledge of whether the
         remote anchor peer is in our organization or not.
      2) Changes the test case of the gossip:ConnectToAnchorPeers
         to simulate a situation that would've caught the bug
         had it been tested in the first place:
         Starting 10 peers, waiting a bit and then starting a *random*
         anchor peer out of the 3 anchor peers given to the 10 peers.
         With the bug- all peers would've mapped all anchor peers to the same
         key, thus unable to connect with probability of 2/3.
      3) Changes the test of TestConnect in the discovery package
         to test that the when the Connect() is invoked with false,
         meaning- not to send the internal endpoint, the first membershipRequest
         that is sent indeed doesn't contain the internal endpoint.
         and also adds a test for each of the options (true, false) of
         the method createMembershipRequest(includeInternalEndpoint)
      
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      Change-Id: Ibd167188dac214951adedfc2ef050d635c6db79f
      48987d23
    • Jason Yellick's avatar
      [FAB-2473] Fix gossip proto style 3/3 · ed7ed802
      Jason Yellick authored
      https://jira.hyperledger.org/browse/FAB-2473
      
      
      
      This is the third and final in a series of changesets to move the gossip
      protos to conform with the official proto style guidelines.
      
      This CR is the most invasive, because it changes pkiID to be pki_id,
      causing the generated code to become PkiId instead of PkiId.  This is
      very similar to the change from ChaincodeID to ChaincodeId, but much
      smaller in scope.
      
      Change-Id: I7ebd2db48ef464ac7f1d024a41fbd11514ca5385
      Signed-off-by: default avatarJason Yellick <jyellick@us.ibm.com>
      ed7ed802
  31. 02 Mar, 2017 2 commits
    • YACOVM's avatar
      Make BCCSP stop complaining in gossip tests · 54f6a82a
      YACOVM authored
      
      
      BCCSP is logging the following message:
      [BCCSP_FACTORY] GetDefault -> WARN 29a Before using BCCSP,\
       please call InitFactories(). Falling back to bootBCCSP.
      
      If it is not initialized manually. Gossip uses BCCSP's hash
      so when running the intensive gossip_test, there are lots of
      BCCSP log entries logged.
      I added a line that initializes the BCCSP to prevent this.
      
      Change-Id: I1dd19794ced995f6cd3699d35aa138511267e9c4
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      54f6a82a
    • YACOVM's avatar
      [FAB-2007] Gossip: External and internal endpoints II · cde26407
      YACOVM authored
      Background:
      ------------
      The previous commit added support in the discovery layer
      for policies that effect the membership response handling logic.
      That logic is based on the selfInformation field which is a signed
      GossipMessage that contains an AliveMessage and is supposed to represent
      the remote peer that sent the membership request.
      
      Even though the message is validated, nothing prevents an attacker
      to replay a signedMessage he recorded.
      
      There is a TODO in the code that says:
      https://github.com/hyperledger/fabric/blob/master/gossip/discovery/discovery_impl.go#L293
      
      
      // TODO: make sure somehow that the membership request is "fresh"
      
      This is to prevent replay attacks, and:
      1) This needs to be addressed for FAB-2007
      2) A replay attack can happen anyway if a malicious peer gets hold
         of such a message early enough befor the attacked peer got the message
      
      What's in this commit?
      ----------------------
      This commit leverages the fact that a membership request is point-to-point,
      and checks that the sender of the membership request is the same peer
      that is on the other side of the connection.
      Also removed the TODO since now it's not needed anymore.
      
      How is this tested?
      --------------------
      I added a test that creates such a replay attack and spoofs
      a membership request, and compares it to a valid membership
      request to demonstrate that the attack prevention works.
      
      Change-Id: I8c994b5627189a1d0fb3f6a7d9edbd9a9c021b2c
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      cde26407
  32. 01 Mar, 2017 1 commit