1. 27 Jul, 2017 1 commit
    • Artem Barger's avatar
      [FAB-5499]: Flatten logic of gossip Peers API · 3e1a17ea
      Artem Barger authored
      
      
      Current implmenetaition makes use of discovery GetMembership API and
      iterates over to construct the result, while this is redundant, since we
      can return GetMembership result right away. This commit replace
      iteration with simply returning result of discovery service.
      
      Change-Id: Idd69acd08560e0fac84bae43cd8acd50e6d8bdd0
      Signed-off-by: default avatarArtem Barger <bartem@il.ibm.com>
      3e1a17ea
  2. 21 Jul, 2017 1 commit
  3. 05 Jul, 2017 1 commit
    • yacovm's avatar
      [FAB-5165] Optimize block verification · 6d56e6eb
      yacovm authored
      
      
      In gossip, when block messages are gossiped among peers the
      signature of the ordering service on them is validated.
      
      This causes a message to be validated in several places:
      
      1) When it is received from the ordering service
      2) When it is received from a peer via forwarding or pull
      3) When it is received from a peer via state tranfer
      
      The problem with (2) is that it is done in an inefficient way:
      - When the block is received from the communication layer it is verified
        and then forwarded to the "channel" module that handles it.
      - The channel module verifies blocks in 2 cases:
        - If the block is part of a "data update" (gossip pull response) message
          the message is opened and all blocks are verified
        - If the block is a block message itself, it is verified again,
          although... it was verified before passed into the channel module.
          This is redundant.
      
      But the biggest inefficiency is w.r.t the handling in the channel module:
      When a block is verified it is then decided if it should be be
      propagated to the state transfer layer (the final stop before it is
      passed to the committer module). It is decided by asking the in-memory
      message store if the block has been already received before, or
      if it is too old.
      
      The problem is that this is done *AFTER* the verification and not *BEFORE*
      and therefore - since in gossip you may get the same block several times
      (from other peers) - we end up verifying the block and then discarding
      it anyway.
      
      Empirical performance tests I have conducted show that for blocks
      of 100KB, the time spent on verifying a block is between 700 micro-seconds
      to 2milliseconds.
      
      When testing a benchmark scenario of 1000 blocks with a single leader
      disseminating to 7 non-leader peers, with propagation factor of 4,
      a block entry rate (to the leader peer) of bursts of 20 blocks every 100ms,
      the gossip network is over committed and starting from block 500 -
      most blocks were dropped because the gossip internal buffers were full
      (we drop blocks in order for the network not to be "deadlocked").
      
      With this change applied, no block is dropped.
      
      Change-Id: I02ef1a203f469d324509a2fdbd1c8b449a9bcf8f
      Signed-off-by: default avataryacovm <yacovm@il.ibm.com>
      6d56e6eb
  4. 04 Jul, 2017 1 commit
    • yacovm's avatar
      [FAB-5157] Optimize peer selection of channel batches · 6c3cb99d
      yacovm authored
      
      
      In gossip whenever a batch of channel-scoped messages
      (either leadership, blocks, stateInfo, etc.) is sent to remote peers,
      a function goes over all existing alive peers and then selects from
      them a subset of peers. This is done in gossipInChan function
      
      In case of leadership messages, the subset is taken from the entire
      membership set without an upper bound (we gossip leadership messages
      to all peers in the channel), and in case of non-leadership messages
      the subset is taken with an upper bound equal to the propogation
      fanout (configurable).
      
      Finding peers that are eligible to receive any channel-related data
      involves cryptographical computations and is non-negligible
      (measured ~ 25ms in a network of 8 peers)
      
      The 2 possibilities (leadership and non-leadership) are calculated even
      though each method invocation of gossipInChan receives only 1 type of message.
      Therefore, it would be beneficial performance wise to just calculate the
      option that is relevant to that method invocation and not calculate
      both options each time.
      
      This commit addresses this by calculating the peers to send according to
      the type of message in the invocation.
      
      Change-Id: If6940182f83ef046c1d1f7186a71946128591e69
      Signed-off-by: default avataryacovm <yacovm@il.ibm.com>
      6c3cb99d
  5. 30 Jun, 2017 1 commit
    • yacovm's avatar
      [FAB-5052] [FAB-5051] Fine tune log messages · 2993534f
      yacovm authored
      
      
      FAB-5052:
      In gossip when a peer obtains an AliveMessage from a peer it hasn't
      received its certificate yet - it complains in the log in WARNING that
      it has received a message from some peer it can't verify its identity.
      
      This is actually a common scenario where a peer is restarted and its
      in-meory certificate store is now empty, but the other peers contact
      the peer and send it alive messages which he can't verify until it syncs
      its certificates with them.
      
      This startles users and I think we should reduce the logging severity
      from WARN to DEBUG.
      
      FAB-5051:
      When a peer is in a partition or is dead for a long enough time,
      gossip prints:
              Haven't heard from", <gibberish>, "for" <elapsed time>
      
      The problem is that the id is a string of a hash and its
      printed in gibberish.
      It's not something humans can read,
      and we need to cast the string back to []byte so it would be human readable.
      
      Change-Id: I94bcad11623e53b5275ef8cecbdd3f9563fa1e77
      Signed-off-by: default avataryacovm <yacovm@il.ibm.com>
      2993534f
  6. 12 Jun, 2017 1 commit
    • YACOVM's avatar
      [FAB-4557] gossip: Do not sign block messages · 961ccabe
      YACOVM authored
      
      
      In gossip we currently sign every message in the following way:
      - Message sent by the Gossip() API call - uses the peer's signing identity
        to sign the message.
      - Message sent via Send() is signed in a NOOP manner (the signature is empty)
      
      Since blocks are already signed by the ordering service
      there is no point in signing them with the peer's signing identity.
      This commit makes such message be NOOP-signed.
      
      Change-Id: I0e5dca73e2ba3e711955829435172ed7f0365b61
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      961ccabe
  7. 10 Jun, 2017 1 commit
    • YACOVM's avatar
      [FAB-3450] Prevent panic on msg signing · d7233d59
      YACOVM authored
      
      
      Currently, the gossip code panics if it's unable to sign a message
      or to marshal it bytes, because the assumption was
      that if the signing failed then something is terrible wrong with the
      peer and it's not recoverable
      If turning a message into bytes fails, it's because of something
      not recoverable too.
      
      However, a user posted a stack trace in which gossip tried constructing
      a message that its envelope has a payload of 2GB.
      
      Now, the cause for a creation of such a message was fixed but in theory
      we may somehow have a situation where a large message is constructed.
      
      In order to prevent these kind of problems, and the crash of the peer,
      we need to make the Signer return an error instead of panicking.
      
      Change-Id: Ife8f4e137c92c167dadc7bbe1bcdf45f93b2b38e
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      d7233d59
  8. 05 Jun, 2017 1 commit
    • YACOVM's avatar
      [FAB-4302] Harden gossip bootstrap peer connection · 3177af10
      YACOVM authored
      
      
      Full details and introduction is in the JIRA item.
      
      Gossip should be protect against mis-configurations that have security
      implications, and therefore I suggest to make the bootstrap peer
      connection to use the same mechanism as the anchor peer connection -
      when a peer connects to an anchor peer, it first performs a handshake
      (which is authenticated), and learns about the organization of the peer
      behind the endpoint and only then decides whether to send its internal
      endpoint or not.
      
      Change-Id: Ieee1abde85f8e22c4ff6adad16dcd6fdab47b57f
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      3177af10
  9. 24 May, 2017 1 commit
    • YACOVM's avatar
      [FAB-4078] StateInfo expiration membership awareness · 23a7d508
      YACOVM authored
      
      
      The stateInfo message store is used in gossip to know the membership
      of the peers of the channel.
      Each entry in the message store is the latest StateInfo message a peer
      has sent either directly or indirectly.
      Peers send StateInfo messages in the following manner:
      - Upon reception of a block, a new StateInfo message is created and is
      propagated to adjacent peers.
      - Once in a while, a peer send StateInfo pull requests to adjacent peers
        and they send back the entire StateInfo message set stored in their
        memory that they have received from other peers (reconciliation).
      
      However - the StateInfo message store is an expirable message store -
      its messages are expired after a limited time.
      
      This causes the following problem:
      If no new blocks are generated in the channel, no new StateInfo messages
      are generated as a result, and eventually -
      he StateInfo messages of peers are cleaned up from memory.
      
      This causes many problems, namely that when a new peers joins a channel
      after no new blocks were created in that channel for a while -
      it will not get StateInfo messages from other peers and thus won't be
      able to pull blocks from them, unless it is configured to use leader
      election, and then it will elect itself as a leader and connect directly
       to the ordering service.
      
      Change-Id: Ifa38a714a448ae3708e9852384b5e4107380e947
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      23a7d508
  10. 16 May, 2017 1 commit
    • YACOVM's avatar
      [FAB-3109] Fix anchor peer connection logic · f966c8c9
      YACOVM authored
      
      
      The current implementation of what happens when the peer connects
      to an anchor peer has a flawed logic.
      
      It assumes that if the probe is successful, the sending is also.
      That's too optimistic.
      Another problem is that after the handshake is performed, the remote peer
      may not distinguish between the closing of the connection of the handshake
      and the closing of the new connection that sends the MembershipRequest,
      since we only support 1 connection between each peer (to prevent DOS).
      
      I re-implemented the logic to do the following:
      1) First, perform a handshake to know whether the remote peer is from
        our org or not.
        Peers in different orgs should not know of our internal endpoints).
        The handshake is performed until it succeeds or too many attempts
        are made.
      2) Now, construct a MembershipRequest and send it until you get
         acked from the remote peer, or when attempts are exhausted.
      
      Added a test that simulates the flow.
      
      Change-Id: I72370d9be816105fdc5af7577ce578faf6e5abdc
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      f966c8c9
  11. 10 May, 2017 1 commit
    • YACOVM's avatar
      [FAB-3755] Gossip:Add remote peer endpoint to ConnInfo · b7e22269
      YACOVM authored
      
      
      In gossip we have the following struct that can be extracted
      from a point-to-point message:
      
      type ConnectionInfo struct {
          ID       common.PKIidType
          Auth     *AuthInfo
          Identity api.PeerIdentityType
      }
      
      In the logs we write the ID of the peer but it's a byte slice,
      and doesn't tell much about the peer.
      
      On the other hand, when a remote peer connects to a peer,
      we extract its endpoint anyway.
      
      I added the endpoint to the struct and added an assignment
      at the handshake time.
      
      Change-Id: If29abda09f2b066332099a45328c26027641215b
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      b7e22269
  12. 03 May, 2017 1 commit
  13. 02 May, 2017 1 commit
  14. 01 May, 2017 1 commit
    • YACOVM's avatar
      [FAB-3530] Gossip - add block seq# validation · d21cd6db
      YACOVM authored
      
      
      When a peer receives a data message (contains a block) it doesn't verify
      that the sequence of the block matches the sequence inside the marshaled block,
      and just calls VerifyBlock.
      
      The implementation already unmarshals the block, so there is no point
      in un-marshalling it outside.
      I extended the interface VerifyBlock to accept also a sequence number.
      
      Change-Id: I85c6c57f26f5e404fe8e7ece2abffa58eaded625
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      d21cd6db
  15. 30 Apr, 2017 2 commits
    • Gari Singh's avatar
      [FAB-3105] Gossip needs to use comm package CA support · f0acc681
      Gari Singh authored
      
      
      The current gossip implementation is initialized with a
      set of dial options which don't change during the lifecycle
      of the peer.
      
      In order to resolve this, the gossip comm implementation
      has been modified to accept a function which returns the
      appropriate dial options for secure communincations with
      other peers.
      
      When used in the peer, the function passed in
      leverages the core/comm packages CASupport.
      
      Change-Id: I99cc4204e27c67f93f8f1ba91891fdb97081c99e
      Signed-off-by: default avatarGari Singh <gari.r.singh@gmail.com>
      f0acc681
    • YACOVM's avatar
      [FAB-3444] Gossip: pulled blocks aren't evicted · 05f811f2
      YACOVM authored
      
      
      In Gossip, the component that handles pulling messages
      of blocks is the pullMediator .
      
      There is a bug that was discovered, that blocks may enter the pull
      mediator's internal map, but are never evicted.
      This is because the eviction logic only worked for blocks that were manually
      added, but blocks that were consumed by update messages were never removed by
      the dedicated callback (that worked for blocks that were manually added)
      
      I fixed the bug by filtering the incoming data request with the help
      of the message store, and thus - blocks that are old don't enter
      the pull mediator.
      
      This effectively makes all blocks of the pull mediator a subset of the blocks
      in the message store (which has a fixed upper bound)
      
      Blocks that are new, cause eviction of old blocks from the pull mediator
      in the callback in channel.go line 185
      
      Change-Id: Idfcaeff68f941cc5ac7e437968ec8fa4d4a8fe31
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      05f811f2
  16. 26 Apr, 2017 2 commits
  17. 22 Apr, 2017 2 commits
    • YACOVM's avatar
      [FAB-2963] Gossip inter-org confidentiality - P4 · 43bcc9a3
      YACOVM authored
      
      
      In gossip, AliveMessages of peers that have no external endpoints
      are only gossiped among peers of the same organization.
      However, no such restriction exists for StateInfo messages and for
      Identities. Therefore- foreign organizations know of peers
      that they can't communicate with, and that the organization had no
      intention of exposing.
      
      This commit addresses this by filtering out StateInfo messages
      of peers without external endpoints when gossiping to peers of foreign
      organizations, and the same for identity messages.
      
      Change-Id: Idf7953b9fa5efd1415ac294ff63e7ea321762a3f
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      43bcc9a3
    • YACOVM's avatar
      [FAB-2061] Gossip inter-org confidentiality - P3 · 9d042696
      YACOVM authored
      
      
      In the previous commit, I extended the pull mechanism to support
      filtering based on the remote peer.
      
      The goal was to ensure that peers only gossip identities of either
      their own organization, or of organizations they are gossiping with.
      (Meaning- if A gossips with B identities, the identities that are sent
      are either identities of A or of B).
      
      In this commit I hook a filter to the constructor of the identity puller
      that extracts the organization of the remote peer, and based on the
      organization of the digest (which is the PKI-ID of the identity),
      the filter ensures the confidentiality explained above.
      
      I also extended the test of the first commit in the series, to inspect
      identities that travel between organizations, to make sure
      that the confidentiality rules are preserved.
      
      I also did the following test, to make sure that state transfer between
      peers still works:
      
      network_setup.sh: Overided the docker-compose file with docker-compose-no-tls
      peer-base/peer-base-no-tls.yaml: I made both options:
            - CORE_PEER_GOSSIP_ORGLEADER=false
            - CORE_PEER_GOSSIP_USELEADERELECTION=false
      And in docker-compose-no-tls.yaml I made ONLY peer0 to be ORGLEADER-true
      
      Then I ran network_setup.sh up and it passed, so this proves that
      peer2 and peer3 got their blocks from peer0 (or peer1).
      
      Change-Id: Ied4979dd5cfc24ef7b0d1bc092653e9f037a64f2
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      9d042696
  18. 20 Apr, 2017 1 commit
  19. 19 Apr, 2017 1 commit
  20. 18 Apr, 2017 3 commits
    • YACOVM's avatar
      [FAB-3213] Gossip identity revocation support · 9d121662
      YACOVM authored
      
      
      When a peer gets a new config block and it contains CRLs,
      the gossip layer needs to be notified in order to close existing
      connections to peers that their certificates have been expired.
      
      If this is not done, then these peers are still forwarded data
      like peer membership and channel membership, because the connection
      is already open.
      
      This commit adds an ability to revoke identities by receiving a predicate
      function that: given an identity, it returns whether it is suspected of being
      revoked (i.e, the SN is found within some CRL of some MSP).
      Then- the gossip layer calls ValidateIdentity on the stored identity, and if
      it is found to be invalid- it:
      - deletes the identity from memory
      - closes an active connection to the peer, if such exists.
      
      Currently the implementation of that predicate is the naive/obvious one,
      that suspects all identities.
      In a future commit I'll (hopefully) add code that uses the CRLs themselves.
      
      Change-Id: I56d995a3720a736b1242b13a193f9a7933299345
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      9d121662
    • YACOVM's avatar
      [FAB-2061] Gossip inter-org confidentiality - P2 · 90b4c721
      YACOVM authored
      
      
      In the previous commit, I adjusted the membership routing so that
      membership would only be disseminated between pairs of organizations.
      
      The only thing left is to deal with identity messages, which are
      disseminated using the pull mechanism.
      
      This commit adds a filtering capability to the pull engine,
      and to its enclosing object- the pull mediator.
      
      The filter acts on a context of a message (that contains info about the
      peer's credentials, hence it's organization) sent from the remote peer
      (helloMsg and requestMsg) and return a function that declares for each
      item (digest or item in the response) whether it should be sent to the remote peer.
      
      In the next commit, I shall write filtering logic to make it so
      that peers would gossip identities with destination orgs only
      if they are their own identities.
      
      Change-Id: I7996bafe5ce2962279c01e59c57868057cfd1d7e
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      90b4c721
    • YACOVM's avatar
      [FAB-2061] Gossip inter-org confidentiality - P1 · 077126ef
      YACOVM authored
      
      
      This change set makes every 2 orgs (that share a channel),
      gossip only alive messages that that originated in one of the two
      organizations.
      
      The motivation is to prevent a third organization from knowing
      that the two orgs share a common channel (unless of course, they all share it).
      
      So, if we have the following situation:
      Channel C0: { orgA, orgB }
      Channel C1: { orgB, orgC }
      [ A ]--C0--[ B ]--C1--[ C ]
      Then orgA has no idea of orgC's existence and vice-versa.
      
      But, also if we have the following situation:
      Channel C0: { orgA, orgB }
      Channel C1: { orgA, orgC }
      Channel C2: { orgB, orgC }
      [ A ]-C0-[ B ]
        |      /
        |     /
        C1   C2
        |   /
        |  /
      [ C ]
      
      Then, alive message from orgA, are never sent
      to orgB via orgC (and the same for orgB, orgC)
      Or, in other words: an Alive message m about a peer in org X
      is sent to org Y, only if X is either in X or in Y.
      
      In the next change set I'll do the same for identity messages
      (messages that contain certificates)
      
      Change-Id: I816e6b23050d320f8b99a64b5c84d619de54aaa8
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      077126ef
  21. 14 Apr, 2017 1 commit
    • YACOVM's avatar
      [FAB-2962] Gossip- Channel name obfuscation · 90573e87
      YACOVM authored
      
      
      This commit changes the stateInfo messages in gossip in the following way:
      Instead of sending the plaintext of the channel in the channel field
      of the GossipMessage - it now sends instead: Hash(channel || PKI-ID).
      
      The added security benefit:
      Given a message M with a channel C,
      a peer not in C needs to do an exhaustive search / dictionary attack
      on channel names, in order to guess C.
      Without knowing C, it cannot ask an ordering service, or some other peer
      to get a block for that channel because sending the hash would be worthless.
      
      On reception of a StateInfo message from a peer with PKI-ID P, a peer
      iterates over all channels (denote CHAN as the current iteration)
      it participates in, and computes Hash(P || CHAN) and this way it
      knows the channel the message relates to.
      - If it is not in the channel, it would simply forward the message
        to peers in the organization (as done today).
      - Else, it forwards the message to peers that their organization is a member
        of the channel (as done today).
      
      Change-Id: I419ed3e440b936de80ec5624698b5d6e52376e98
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      90573e87
  22. 15 Mar, 2017 1 commit
  23. 14 Mar, 2017 2 commits
    • YACOVM's avatar
      [FAB-2758] Decouple anchor peers endpoints from orgIDs · f1a88db2
      YACOVM authored
      In the current implementation of the join channel message in the
      gossip layer, each anchor peer has its own organization, and the
      joinChannel invocation enumerates all anchor peers and builds a
      list of organizationIDs from them.
      
      When https://gerrit.hyperledger.org/r/#/c/7105/
      
       will be introduced,
      it will be needed to create a channel only with either no anchor peers
      or anchor peers of the organization of the channel creator.
      This would confuse the gossip layer because it'll derive that each channel
      has only 1 organization instead of the original participants.
      
      I decoupled the anchor peers from the organization IDs by changing
      the JoinChannelMessage interface in api/channel.go to have
      Members() []OrgIdentityType  which would
      return the channel members (organizations)
      and AnchorPeersOf(org OrgIdentityType) []AnchorPeer which would
      return for each organization, its corresponding anchor peers.
      
      Also (of course) updated the tests accordingly.
      
      Change-Id: I90131a0726fc7ca68fd750247f4df558c5cca0fd
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      f1a88db2
    • YACOVM's avatar
      Adjust gossip logging levels · 13194fac
      YACOVM authored
      
      
      This commit:
      1) Changes an Entering logging event from Info to Debug
      2) Changes a logging event in case of receiving a message (or an API call)
         regarding a channel that isn't known to the peer from Warn to Debug
      
      Change-Id: If10cd1d0ac1153481e905f2c78f9a2f747834ab5
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      13194fac
  24. 08 Mar, 2017 3 commits
    • YACOVM's avatar
      [FAB-2007] Gossip: External and internal endpoints IV · 19e07d78
      YACOVM authored
      
      
      In the previous commit we extended the comm layer to support deep scanning
      of remote peers in order to know whether anchor peers are indeed
      in the organization it is claimed (in the genesis block) they are.
      
      This commit connects between the gossip join-channel logic and this
      capability into the discovery layer, and makes the gossip logic
      pass a predicate into the Connect() method of the discovery layer that
      determines at runtime (as soon as the remote peer is available) - whether
      the anchor peer is indeed in our organization or not.
      
      Also changed the test in a way to check that such a spoofing doesn't work
      anymore:
      I added a 3rd anchor peer which is from orgB but is claimed to be in orgA.
      
      When I ran the test, it passed and the gossip code complained as follows:
      ----------------------------------------------------------
      WARN 007 Anchor peer localhost:11616 isn't in our org, but is claimed to be
      
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      Change-Id: I2d92628cc5428cc4194a0f3909eb13562b7588a4
      19e07d78
    • YACOVM's avatar
      [FAB-2007] Gossip/Comm deep probing · 5eb459ae
      YACOVM authored
      
      
      When a peer receives a join-channel event, it doesn't know whether
      the anchor peers in the genesis block indeed match their said organizations.
      A malicious attacker may create a channel and add an anchor peer of
      its own in each organization, and this way- make peers expose
      their internal endpoints which is not wanted.
      
      This commit introduces another method - Handkshake() that
      performs a handshake to the remote peer, and returns its identity,
      or an error if either a communication or authentication error occures.
      The certificate then can be used to extract the remote peers' organization.
      
      Change-Id: I0a0e71a10cb5831f07c72be28308cadb486c1d3a
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      5eb459ae
    • YACOVM's avatar
      [FAB-2007] External and internal endpoints III · 821c9d8a
      YACOVM authored
      
      
      This is the last commit of the series, in which we enforce
      publishing of external endpoints to peers of different orgs
       and publishing of both external and internal endpoints only
       within the peer's organization.
      
      The commit addesses this only in the gossip layer,
      as the discovery layer has been taken care of in previous commits.
      
      It contains a test in which we:
      - Create 2 orgs with 5 peers each, while the first 2 peers have external
        endpoints, and the rest don't.
      - The first org has a bootstrap peer, the second doesn't.
      - JoinChannel is invoked with the first peer of each org as anchor peers.
      - It is tested that:
        - Peers know peers from other orgs only if they both
          have external endpoints.
        - Peers do not know the internal endpoints of peers of other orgs.
      
      Change-Id: I157c8cea29b35adb84314fcb695413b005f2b236
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      821c9d8a
  25. 07 Mar, 2017 1 commit
    • YACOVM's avatar
      [FAB-2658] Bug fix: pass correct block data to MCS · 9a5b456b
      YACOVM authored
      
      
      The MessageCryptoService complains:
      verifyBlock -> WARN 0cc Received fabricated block from [...]\
       in DataUpdate: Failed casting SignedBlock to []byte on channel [foo]
      
      Because the object that was passed wasn't a []byte as expected but
      the object that contains the []byte as a field.
      I changed the signature of VerifyBlock to accept []byte instead
      of the previous interface type api.SignedBlock and removed the latter type.
      
      Change-Id: I7320c398b117072c0790f77d9c0f9305b1adf5ea
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      9a5b456b
  26. 05 Mar, 2017 4 commits
    • YACOVM's avatar
      [FAB-2624] Gossip Anchor peer connect bug fix · 48987d23
      YACOVM authored
      
      
      When the PEM was removed from the anchor peers, the Connect()
      method of the discovery layer wasn't updated accordingly.
      As a result, all anchor peers were mapped to the same (empty string) key
      in the discovery layer
      
      This commit:
      1) Changes the logic and semantics of the Connect():
         It now creates a membership request that doesn't
         (or does, up to the caller) have the internal endpoint inside of it,
         according to the param passed.
         This is because we don't have knowledge of whether the
         remote anchor peer is in our organization or not.
      2) Changes the test case of the gossip:ConnectToAnchorPeers
         to simulate a situation that would've caught the bug
         had it been tested in the first place:
         Starting 10 peers, waiting a bit and then starting a *random*
         anchor peer out of the 3 anchor peers given to the 10 peers.
         With the bug- all peers would've mapped all anchor peers to the same
         key, thus unable to connect with probability of 2/3.
      3) Changes the test of TestConnect in the discovery package
         to test that the when the Connect() is invoked with false,
         meaning- not to send the internal endpoint, the first membershipRequest
         that is sent indeed doesn't contain the internal endpoint.
         and also adds a test for each of the options (true, false) of
         the method createMembershipRequest(includeInternalEndpoint)
      
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      Change-Id: Ibd167188dac214951adedfc2ef050d635c6db79f
      48987d23
    • YACOVM's avatar
      Gossip: Learn certificate at validation time · bc7f9d8e
      YACOVM authored
      
      
      When an AliveMessage enters the routine that handles all messages,
      It is first validated and only then the certificate may be learned
      from the alive message, but then it is validated again when it is
      learned because the identity store validates the cert when it attempts
      to store it.
      
      This is redundant, we may simply learn the cert at validation time as
      part of storing the cert (gossip/identity/identityMapper:Put also
      validates a cert, and even checks if the calculated PKI-ID matches
      the claimed PKI-ID)
      
      Change-Id: I90900818aef3b51c413384df989f4e63073401f6
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      bc7f9d8e
    • Jason Yellick's avatar
      [FAB-2473] Fix gossip proto style 3/3 · ed7ed802
      Jason Yellick authored
      https://jira.hyperledger.org/browse/FAB-2473
      
      
      
      This is the third and final in a series of changesets to move the gossip
      protos to conform with the official proto style guidelines.
      
      This CR is the most invasive, because it changes pkiID to be pki_id,
      causing the generated code to become PkiId instead of PkiId.  This is
      very similar to the change from ChaincodeID to ChaincodeId, but much
      smaller in scope.
      
      Change-Id: I7ebd2db48ef464ac7f1d024a41fbd11514ca5385
      Signed-off-by: default avatarJason Yellick <jyellick@us.ibm.com>
      ed7ed802
    • Jason Yellick's avatar
      [FAB-2472] Fix gossip proto style 2/3 · 953bb211
      Jason Yellick authored
      https://jira.hyperledger.org/browse/FAB-2472
      
      
      
      This is the second in a series of changesets to fix the gossip protos to
      conform to official proto style guidelines.
      
      This CR modifies the enums to be upper case and updates the
      corresponding go code as well.
      
      Change-Id: Ia75470b1a5fcf3a70128535e9da9e280dca1bd10
      Signed-off-by: default avatarJason Yellick <jyellick@us.ibm.com>
      953bb211
  27. 02 Mar, 2017 2 commits
    • YACOVM's avatar
      [FAB-2007] Gossip: External and internal endpoints II · cde26407
      YACOVM authored
      Background:
      ------------
      The previous commit added support in the discovery layer
      for policies that effect the membership response handling logic.
      That logic is based on the selfInformation field which is a signed
      GossipMessage that contains an AliveMessage and is supposed to represent
      the remote peer that sent the membership request.
      
      Even though the message is validated, nothing prevents an attacker
      to replay a signedMessage he recorded.
      
      There is a TODO in the code that says:
      https://github.com/hyperledger/fabric/blob/master/gossip/discovery/discovery_impl.go#L293
      
      
      // TODO: make sure somehow that the membership request is "fresh"
      
      This is to prevent replay attacks, and:
      1) This needs to be addressed for FAB-2007
      2) A replay attack can happen anyway if a malicious peer gets hold
         of such a message early enough befor the attacked peer got the message
      
      What's in this commit?
      ----------------------
      This commit leverages the fact that a membership request is point-to-point,
      and checks that the sender of the membership request is the same peer
      that is on the other side of the connection.
      Also removed the TODO since now it's not needed anymore.
      
      How is this tested?
      --------------------
      I added a test that creates such a replay attack and spoofs
      a membership request, and compares it to a valid membership
      request to demonstrate that the attack prevention works.
      
      Change-Id: I8c994b5627189a1d0fb3f6a7d9edbd9a9c021b2c
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      cde26407
    • YACOVM's avatar
      [FAB-2007] Gossip: External and internal endpoints I · 4579ed14
      YACOVM authored
      
      
      Intro:
            An organization might want to publish external endpoints for other
            organizations, but use internal endpoints (intranet) for communication
            between peers inside the organization.
      
            At the same time, an organization might not want to leak information
            about its internal addresses to other organizations.
      
            A peer has 2 endpoints when it is configured:
            1) Internal endpoint (exists anyway)
            2) External endpoint (might be configured)
            Only peers that have an external endpoint configured are supposed
            to be visible to peers outside the organization.
      
      What's in this commit?
            This commit addresses this deal in the discovery layer:
            When a membership request message reaches a peer, it grabs all
            alive messages it posseses and sends them to the remote peer
            in a membership response message.
            Both messages are point-to-point (not "gossiped"/broadcasted).
            And need to be created in such a way to:
            1) Not tell about peers that have no external endpoint
            2) Not leak internal endpoints to peers outside the org
      
            This commit adds a policy to the discovery layer that enables:
            1) Filter (Sieve): Only to include peers that hold some
               criteria in the membership response message.
            2) Message mutator (Disjoiner): removes fields of the messages
               sent to remote peers that shouldn't be exposed to the remote
               peer.
      
      How is it tested?
            I wrote a test that simulates 2 organizations, and
            a disclosure policy that fits what is going to be
            done in the next commit in the gossip layer (the layer above).
            The test checks conditions (1) and (2).
      
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      Change-Id: Iade3d32b0d2a58400734b76c30189474c001718b
      4579ed14
  28. 27 Feb, 2017 1 commit
    • YACOVM's avatar
      [FAB-2424] Gossip: Extract connection authInfo in comm · 00a9bd79
      YACOVM authored
      In the previous chapter of FAB-2424(https://gerrit.hyperledger.org/r/#/c/6393/
      
      )
      we took care of channel-based access control by consulting the
      MSP whether a given peer is eligible for a specific channel.
      This works for blocks that are broadcasted, and also for blocks
      that are pulled from peers in the gossip layer,
      but alas - the state transfer layer still stays bare and exposed to
      the mercy of malicious peers!
      
      This commit extends the protos/gossip/extensions.go:ReceivedMessage
      interface and replaces GetPKIID() with GetConnectionInfo() that returns:
      	ID       common.PKIidType
      	Auth     *AuthInfo:
      			SignedData []byte
      			Signature  []byte
      	Identity api.PeerIdentityType
      Using this, in the next commit I'll be able to modify the state
      transfer module by having the predicate it passes to the gossip
      layer when listening for messages from remote peers to also
      call the method provided by the MSP: VerifyByChannel()
      and in this way- to verify that the remote peer
      should indeed receive blocks or not.
      
      Change-Id: I9e2e6f4da430ed062a6fa12bebdfab4add6c4843
      Signed-off-by: default avatarYacov Manevich <yacovm@il.ibm.com>
      00a9bd79