1. 16 Mar, 2019 1 commit
    • Jay Guo's avatar
      FAB-14593 Refine etcdraft parameters · 2a4e15e9
      Jay Guo authored
      
      
      - MaxInflightMsgs is internal to etcd/raft and should be exposed
      to users with a more appropriate name: MaxInflightBlocks
      
      - MaxSizePerMsg is also internal to etcd/raft, and it's defaulted
      to PreferredMaxBytes in BatchSize, so that if a big block is created,
      it is sent in a its own etcd/raft message, instead of being batched
      with other blocks. This parameter takes effect when a batch of entries
      is sent to lagged node. During normal replication, each block is
      sent in its own message.
        It's not necessary to expose this config option to users.
      
      - SnapInterval is renamed to SnapshotIntervalSize
      
      FAB-14593 #done
      
      Change-Id: Icaf2848a41c5f0f0a02f4b0b4a80ba852fddd584
      Signed-off-by: default avatarJay Guo <guojiannan1101@gmail.com>
      2a4e15e9
  2. 14 Mar, 2019 1 commit
    • Jason Yellick's avatar
      FAB-14619 Rename Raft metadata protos · d645c833
      Jason Yellick authored
      
      
      There are presently two etcdraft protos around metadata.  One is the
      metadata stored in the config and it is named 'Metadata', the other is
      the metadata stored in each block, this is named 'RaftMetadata'.  This
      causes confusion when reading the code.  This CR transforms those names
      to be:
      
       Metadata -> ConfigMetadata
       RaftMetadata -> BlockMetadata
      
      Change-Id: Ia0394ebe78f5541996c010c3c67d760f336f75d8
      Signed-off-by: default avatarJason Yellick <jyellick@us.ibm.com>
      d645c833
  3. 11 Mar, 2019 1 commit
    • yacovm's avatar
      [FAB-14578] Sane defaults for block cutting · 609d4ea8
      yacovm authored
      
      
      A max of 10 transactions per block is too low because
      it creates tiny blocks. We should increase it to a
      saner value which is more optimized for throughput
      instead of letting the users always do it themselves.
      
      Correspondingly, 256 blocks circulating the consensus
      is too high and should be tuned down appropriately.
      
      Change-Id: Ib2f2c0f9c1875102e14d73f054f67103036b3d8d
      Signed-off-by: default avataryacovm <yacovm@il.ibm.com>
      609d4ea8
  4. 05 Mar, 2019 1 commit
  5. 03 Mar, 2019 2 commits
    • Jay Guo's avatar
      [FAB-13656] Size-based snapshotting · 566562e7
      Jay Guo authored
      
      
      Instead of taking snapshot every N blocks, this CR
      changes it to taking snapshot every N bytes.
      
      This also sets default SnapshotInterval to 100MB, if
      it's unset. Otherwise data in memory is never compacted
      till OOM.
      
      Meanwhile, DefaultSnapshotCatchUpEntries is shrunk so
      it does not take too much space to preserve extra entries
      every time a snapshot is taken. Slow nodes are catching up
      using blockpuller, which is also efficient.
      
      Change-Id: I79cfeb8652fcbafdeb5793bf4f06267b95a858d6
      Signed-off-by: default avatarJay Guo <guojiannan1101@gmail.com>
      566562e7
    • Jay Guo's avatar
      [FAB-13845] Increase default raft tick interval · ebd9127c
      Jay Guo authored
      
      
      Increase default etcdraft tick interval to 500ms, for several reasons:
      - in a WAN/Cloud environment, this is more realistic
      - WAL sync in CI often exceeds 1s, which causes heartbeats not being
        sent timely. Increasing election timeout can decrease the chance of
        unexpected leader failover.
      
      This CR also increases default timeout of peer cmd, because now
      it takes 5~10s to elect a leader for a newly created channel, and
      `peer channel create` can only retrieve genesis block of that channel
      when leader exists (Deliver API returns error if leaderless).
      
      Note that this value is still configurable by users.
      
      Change-Id: I94fbbc750fa096cce6ef9e2d65eb981c6202b675
      Signed-off-by: default avatarJay Guo <guojiannan1101@gmail.com>
      ebd9127c
  6. 04 Dec, 2018 1 commit
  7. 30 Oct, 2018 1 commit
  8. 29 Oct, 2018 1 commit
  9. 17 Oct, 2018 1 commit
  10. 08 Oct, 2018 1 commit
  11. 24 Aug, 2018 1 commit
    • Matthew Sykes's avatar
      [FAB-9131] wire zap based fabric logger · 97215fa6
      Matthew Sykes authored
      
      
      Replace go-logging with a zap based implementation. The implementation
      is mostly compatible with the legacy format but differs in a two key
      ways:
      
      1. CRITICAL and NOTICE log levels no longer exist. Calls to Critical get
         mapped to error and NOTICE gets mapped to INFO. The methods
         associated with these levels will be removed in a future CR.
      2. The log level constants are now sourced from the zap/zapcore packages.
         These values are incompatible wit the go-logging constants. Please be
         sure to use the appropriate constant when necessary.
      
      The existing go-logging package is still used by the chaincode shim to
      reduce the risk of negatively impacting legacy chaincode.
      
      Change-Id: Iaf5fac807244883a8285892ccd350c5256959782
      Signed-off-by: default avatarMatthew Sykes <sykesmat@us.ibm.com>
      97215fa6
  12. 10 Aug, 2018 1 commit
    • Kostas Christidis's avatar
      FAB-11478 Enable etcd/raft configuration in channel · 808093bb
      Kostas Christidis authored
      
      
      This changeset:
      
      1. Defines consenter metadata for the etcd/raft plugin, creates the
      associated metadata message factory, and registers it with the metadata
      factory map in protos/orderer for protolator to work.
      
      2. Adds supports for etcd/raft configuration to the channel
      configuration encoder.
      
      3. Adds an "etcdraft" section to the configtx sample YAML and the
      backing config structure.
      
      4. Adds a "SampleDevModeEtcdRaft" profile preset to the configtx sample
      YAML. This will be consumed by integration tests.
      
      Change-Id: I7460c89fdb91e68e4217932aeed5ca41a131d47b
      Signed-off-by: default avatarKostas Christidis <kostas@christidis.io>
      808093bb
  13. 18 Jul, 2018 1 commit
    • Jason Yellick's avatar
      FAB-11215 Enable vetting of more print/printf fncs · 2a3ebd57
      Jason Yellick authored
      
      
      Go vet attempts to detect missing format arguments or format strings in
      non-format contexts.  However, the default list of functions does not
      include many of our logging functions (like Warningf, Debugf, etc.).
      This CR enables the additional vetting and fixes the errors that it
      catches.
      
      Change-Id: I4eefdf84d9349f2ae08da9bb0c9b829226219c44
      Signed-off-by: default avatarJason Yellick <jyellick@us.ibm.com>
      2a3ebd57
  14. 07 May, 2018 1 commit
  15. 16 Apr, 2018 1 commit
    • Matthew Sykes's avatar
      [FAB-9494] test package functions to load config · 1d8344d7
      Matthew Sykes authored
      
      
      - Add variadic argument for extra search paths to configtxgen/localconfig
        Load and LoadTopLevel functions. This is primarily intended to be used
        in tests.
      - Create configtxgentest package that loads configurations and profiles
        from the development sampleconfig directory.
      - Add method to configtest to setup and reset FABRIC_CFG_PATH.
      - Update tests to use configtxgentest and/or configtest as appropriate.
      
      Change-Id: I6a66491bfbc0048f5b38babf6a55fb918a439fe6
      Signed-off-by: default avatarMatthew Sykes <sykesmat@us.ibm.com>
      1d8344d7
  16. 13 Apr, 2018 1 commit
  17. 12 Apr, 2018 1 commit
  18. 03 Apr, 2018 1 commit
    • Jason Yellick's avatar
      [FAB-9254] Specify policies in configtx.yaml · f519714c
      Jason Yellick authored
      
      
      This CR adds the ability to specify policies in configtx.yaml.  It is
      one half of two required for users to specify their own policies.  The
      second half is in FAB-9255 (submitted together) which takes these
      additional config file options and encodes them into channel config.
      
      Change-Id: I7ba264a7554d4e3656bf13f2191bd871ba401e21
      Signed-off-by: default avatarJason Yellick <jyellick@us.ibm.com>
      f519714c
  19. 22 Mar, 2018 1 commit
    • Jason Yellick's avatar
      [FAB-9002] Define new application v1.2 capability · 10fdcc9c
      Jason Yellick authored
      
      
      For all the non-backwards compatible work, including the new chaincode
      lifecycle and resources config changes, we must use a new capability to
      determine whether the behavior is enabled or not.
      
      This CR adds one for the application portion of the config.  At the time
      of this writing there are no features which would introduce
      incompatibilities at the orderer or channel level, so those capabilities
      are being left in place.
      
      This CR also removes the assorted 'V11' profiles, as maintaining a set
      of profiles for each version is unmaintainable and confusing.  For this
      and subsequent releases, configtx.yaml will contain only defaults for
      the current version of fabric.  Users may of course override these
      defaults by modifying the config file.
      
      Change-Id: I08729fef76292044440ebc7e4f39145b0640afc4
      Signed-off-by: default avatarJason Yellick <jyellick@us.ibm.com>
      10fdcc9c
  20. 06 Dec, 2017 2 commits
    • Jason Yellick's avatar
      [FAB-7322] configtxgen set default res mod_policy · d0eb6685
      Jason Yellick authored
      
      
      In v1.0, chaincode instantiation was always controlled by a policy which
      required a single signature.  By default, we wish for chaincode
      instantiation to require a quorum of administrators to sign off, but
      doing so may break existing workflows.
      
      This CR adds a field to configtx.yaml which specifies the default
      mod_policy to use throughout the resources tree.  If set to a single
      signature policy like /Channel/Application/Writers, then the old workflow
      of a single signer is still possible.  If set to a multi-sig policy like
      /Channel/Application/Admins, then the user's workflow will need to be
      changed to accomodate the additional signing requirements.
      
      Change-Id: I021801f1540ee5ee25d110682852564650421dbd
      Signed-off-by: default avatarJason Yellick <jyellick@us.ibm.com>
      d0eb6685
    • Jason Yellick's avatar
      [FAB-6234] configtxgen emit resource defaults · 500d3dee
      Jason Yellick authored
      
      
      This changeset at its core, simply enables embedding a default resource
      tree into the output of channel creation transactions which are created
      with v1.1 application capabilities.
      
      Unit tests have been added. Additionally, the new channel creation tx
      behavior can be confirmed by issuing:
      
       configtxgen -profile SampleSingleMSPChannelV1_1 \
        -outputCreateChannelTx foo.tx -inspectChannelCreateTx foo.tx
      
      and observing the resources tree in the JSON output.
      
      Because the utility function for creating the channel creation
      transaction is used widely throughout the tests, and the function
      parameters needed to change significantly, there is a fair amount of
      churn in the tests, but none in production code.
      
      Change-Id: Id719a1ae3f309a0a189f566dc0fef409564b519e
      Signed-off-by: default avatarJason Yellick <jyellick@us.ibm.com>
      500d3dee
  21. 28 Oct, 2017 1 commit
    • Alessandro Sorniotti's avatar
      [FAB-6367] create genesis tx/block with other MSPs · 8139bb47
      Alessandro Sorniotti authored
      
      
      This change set introduces an 'MSPType' field to configtx.yaml, so that the
      type of MSP can be specified. This permits credentials for other MSP types,
      most notably the idemix MSP, to be included in genesis tx/blocks so that they
      can be used as well. If no MSPType is specified, the standard one is
      automatically assumed.
      
      Change-Id: I5a609b57cfe85ae030f7a8fbecf8cfd71d3074ae
      Signed-off-by: default avatarAlessandro Sorniotti <ale.linux@sopit.net>
      8139bb47
  22. 25 Oct, 2017 1 commit
    • Will Lahti's avatar
      [FAB-6747]configtxgen unable to read V1.1 sample profs · 3fea60e0
      Will Lahti authored
      
      
      configtxgen is unable to read in the sample V1.1 profiles due to the
      period in the profile name as defined in sampleconfig/configtx.yaml
      (e.g. SampleSingleMSPChannelV1.1). This CR updates the profile names
      to use underscores instead so Viper is able to locate and load them
      as expected.
      
      Change-Id: Ic7bf03199c20eca466519c406c702221b071602c
      Signed-off-by: default avatarWill Lahti <wtlahti@us.ibm.com>
      3fea60e0
  23. 13 Oct, 2017 2 commits
    • Jason Yellick's avatar
      [FAB-6103] Add printOrg to configtxgen · 6abde4b8
      Jason Yellick authored
      
      
      One of the most common questions when attempting to add an organization
      to a channel is how to get the JSON definition for that organization.
      The only real way to do this at present is to generate a genesis block,
      decode the block, then go look for the org definition.
      
      This CR adds a '-printOrg' flag to configtxgen which will print the
      organization's definition as JSON to make this process easier.
      
      Change-Id: Id12098f36feae8ca4751b82afaf2914166077e50
      Signed-off-by: default avatarJason Yellick <jyellick@us.ibm.com>
      6abde4b8
    • Jason Yellick's avatar
      [FAB-6140] Remove the provisional config encoder · 95b95e90
      Jason Yellick authored
      
      
      With the introduction of FAB-6139 a new more readable and flexible
      config encoder is available.  This means all existing usage of the old
      deprecated provisional encoder should be removed.  This CR does that.
      
      Change-Id: If64e0e7dc920c709cc39a111547c61c3ae4dc3f0
      Signed-off-by: default avatarJason Yellick <jyellick@us.ibm.com>
      95b95e90
  24. 03 Oct, 2017 1 commit
    • Jason Yellick's avatar
      [FAB-6087] Add configtxgen capabilties support · a77389d1
      Jason Yellick authored
      
      
      As a prerequisit to any real usage of the new capabilities support, it
      must be possible for an admin to generate channel creation requests and
      or bootstrap the ordering system network with the new configuration
      keys.
      
      This CR adds support to configtxgen to do this, though there are no
      capabilities actually supported yet.
      
      Change-Id: I9ab962727e446f593ccd262cb31ac89357cdf9d9
      Signed-off-by: default avatarJason Yellick <jyellick@us.ibm.com>
      a77389d1
  25. 10 Sep, 2017 1 commit
  26. 05 Aug, 2017 1 commit
  27. 25 Jul, 2017 1 commit
  28. 12 Jun, 2017 1 commit
    • Kostas Christidis's avatar
      [FAB-4083] Fix filesize-related defaults for orderer · 8bf2492c
      Kostas Christidis authored
      
      
      The `Producer.MaxMessageBytes` default we were setting on the Kafka
      producers was incorrectly set to Kafka's default value for
      `socket.request.max.bytes`. (I actually set it to a value that was just
      a byte shorter than this when I pushed in the refactor changeset a few
      days ago in order to silence the sarama logs, and added a FIXME note for
      this very reason.) This value is incorrect because it doesn't account
      for headers. This changeset assumes a too-generous 1MB for headers and
      decreates `Producer.MaxMessageBytes` accordingly.
      
      This changeset also updates the default value for AbsoluteMaxBytes in
      `configtx.yaml`. The current value is incorrect because it doesn't
      account for the wrapping of messages in the container `KafkaRegular`
      format. At a minimum, it should be decreased compared to
      `Producer.MaxMessageBytes` by an equal padding. This changeset does this
      for all `configtx.yaml` files currently in our codebase.
      
      Finally, it sets the default, suggested value for AbsoluteMaxBytes in
      the sampleconfig `configtx.yaml` file to a much more reasonable 10 MiB,
      well within the constraints we outlined above. The default check in
      `localconfig` has been updated accordingly.
      
      See the JIRA item for a detailed explanation of the issue.
      
      Change-Id: I8821982707282c5069c1a66dd2a121b894b3a3c8
      Signed-off-by: default avatarKostas Christidis <kostas@christidis.io>
      8bf2492c
  29. 04 Jun, 2017 2 commits
    • Jason Yellick's avatar
      [FAB-4157] Remove BCCSP parms from configtx.yaml · a997c304
      Jason Yellick authored
      
      
      Because configtxgen only collects crypto material, it does not actually
      validate it, (ie signing, or checking signatures), the BCCSP information
      attached to the MSP definitions is unnecessary.
      
      Its presence complicates the config file for no gain, and there have
      been issues reported around failing to set these parms properly.
      
      Change-Id: I9728b69a66fbbc21373eba59693481d1e6156615
      Signed-off-by: default avatarJason Yellick <jyellick@us.ibm.com>
      a997c304
    • Jason Yellick's avatar
      [FAB-4328] Fix orderer genesis defaults · eb640dba
      Jason Yellick authored
      
      
      There are a few incorrect things about the default configuration
      produced for the ordering system channel by the configtxgen tool.
      
      The consortiums definitions and channel creation policy cannot be
      modified after creation, and the ordering system channel includes an
      Applications section by default which is inappropriate.
      
      This CR fixes both these issues.
      
      Change-Id: Ie6a934198fafbb37065d703edc0bc8fc9208b9ca
      Signed-off-by: default avatarJason Yellick <jyellick@us.ibm.com>
      eb640dba
  30. 28 Apr, 2017 1 commit
  31. 26 Apr, 2017 4 commits
    • Gregory Haskins's avatar
      [FAB-3428] configtxgen: Report selected configuration · b5430b2d
      Gregory Haskins authored
      
      
      This patch simply adds an INFO level log message to indicate
      which configuration was ultimately selected.  This may be
      helpful to the user since the process for selecting the
      config may be automated by the software.  This could be
      confusing if the user believes one configuration was in
      effect when the system loaded an alternate.
      
      Fixes FAB-3428
      
      Change-Id: I085c4b719c85a9d7d4af86843f89ac819da9b6e4
      Signed-off-by: default avatarGreg Haskins <gregory.haskins@gmail.com>
      b5430b2d
    • Jason Yellick's avatar
      [FAB-1302] (PA) Add channel create authorization · 312628cd
      Jason Yellick authored
      
      
      There is currently no inspection of the channel creation transaction.
      This means that during a channel creation, the creator could
      inappropriately specify config, including items like orderer parameters,
      and the ordering service would silently accept this incorrect
      parameters.  This is especially troublesome from a security perspective,
      as a specially crafted config tx could potentially trick another user
      into believe they were participating in a channel with one member when
      they are actually participating with another.
      
      This CR adds this checking by leveraging the existing underlying
      authorization mechanisms of the configuration transaction.  It ensures
      that the creating members are not only authorized to create the channel,
      but also ensures that all channel config that is set is done so in
      accordance with the underlying governance for that config element.  This
      means that a collection of authorized ordering organizations may choose
      to modifying ordering parameters in a channel creation request while a
      set of application orgs would not be able to.
      
      This CR is somewhat large due to the significant invasive nature of this
      change, and that the entire end to end must be modified, not just a
      single component.  However, given the increase in function with decrease
      in complexity, the diff stat is hopefully acceptable.
      
      Change-Id: I9bb64d3f70d07eb8a8a4ea4893c71cf1112b4bf7
      Signed-off-by: default avatarJason Yellick <jyellick@us.ibm.com>
      Signed-off-by: default avatarKostas Christidis <kostas@christidis.io>
      312628cd
    • Jason Yellick's avatar
      [FAB-2783] (PA) configtxgen specify admin principl · 8288a7fc
      Jason Yellick authored
      
      
      Currently, the configtxgen tool automatically outputs admin policies
      which require the signature of an admin certificate in order to satisfy
      them.
      
      For some deployment scenarios, especially our current e2e scenarios,
      it's not possible to sign with an admin cert, so this CR adds an option
      to specify the admin principal to be of type Role.MEMBER.
      
      Change-Id: If0f05fde2f726b88c54d6bdbc81e210c63bfa959
      Signed-off-by: default avatarJason Yellick <jyellick@us.ibm.com>
      Signed-off-by: default avatarKostas Christidis <kostas@christidis.io>
      8288a7fc
    • Jason Yellick's avatar
      [FAB-2646] (PA) Create consortium configuration · 7998d07b
      Jason Yellick authored
      
      
      In order to support returning the list of channels via the ordering
      system channel configuration, there needs to be additional configuration
      structures representing the groups which may create channels.
      
      We call these groups Consortiums by convention, where each consortium
      represents a group of organizations which may form channels together.
      
      Note, this also include FAB-2647 as combined with FAB-2700 it provides
      testing coverage for the new code.
      
      Change-Id: Iaa7dada39daa153c95613b47def8169b57f27be4
      Signed-off-by: default avatarJason Yellick <jyellick@us.ibm.com>
      Signed-off-by: default avatarKostas Christidis <kostas@christidis.io>
      7998d07b
  32. 24 Apr, 2017 1 commit
    • Gregory Haskins's avatar
      [FAB-3160] Provide config-relative path feature · 8ce10737
      Gregory Haskins authored
      
      
      Introduction
      ======================
      The primary goal of this patch is to create the notion of a
      "config-relative" path reference.  For example, a configuration file
      "/etc/foo/bar.yaml" that contains a key "bat" with a value
      "baz/blaz/blamo" can be used to specify that "baz/blaz/blamo" should
      be considered relative to the configuration file itself.  In this case,
      it would be expected to be found at /etc/foo/baz/blaz/blamo.  FAB-2037
      does a much more thorough job of explaining the rationale on why
      config-relative is considered important/good-form.
      
      This is in stark contrast to what we have today, which is a jumbled
      mess of assumed GOPATH relative, CWD relative, ENVVAR absolute and
      sometimes even ENVVAR relative.  Therefore, an additional positive
      side-effect of this endeavor is that this patch also substantially
      cleans up some technical debt that had been accumulating in the tree
      for some time related to ad-hoc pathing, DRY violations, and just
      general inconsistencies in how configuration files were managed.
      
      Design Details
      ==========================
      This patch refactors the basic configuration system into the notion of
      a tree rooted at a configuration-path.  By default, this path is
      $GOROOT/..../fabric/sampleconfig during dev/test and
      /etc/hyperledger/fabric during runtime.  The root may be overridden
      at any time by specifying the environment variable FABRIC_CFG_PATH.
      (Note that this variable unifies and replaces the former PEER_CFG_PATH
      and ORDERER_CFG_PATH).
      
      The dev/test environment will operate out of the ./fabric/sampleconfig
      configuration root.  The build-system will package that root into
      /etc/hyperledger/fabric in the runtime context with the intention of
      the end-user/admin/deployer replacing parts or all of the sampleconfig
      to suit their application.
      
      Since configuration-relative paths are now possible, the configuration
      files may reference other relative files and they will behave
      appropriately regardless of the context in which they are executed.
      For example, consider the files ./sampleconfig/tls/server.[crt|key].
      A configuration file may contain a key "tls/server.key" and the system
      will properly resolve this relative file even at runtime. This is (IMO)
      far more natural than assuming a path is relative to the CWD of where
      the command is executed, which is how most of the system behaves today
      (or requires awkward and very specific ENVVAR overrides).
      
      This will be conducive to something like a package-installer
      (e.g. RPM/DEB) or a docker environment to augment/replace elements
      of the configuration root and to freely move the configuration around
      as the package/deployer sees fit.
      
      As an example, a deployment on Kubernetes might opt to volume mount
      /etc/hyperledger/fabric to replace the entire config, or it might just use
      a secrets mount on /etc/hyperledger/fabric/peer/tls.  An RPM packager
      might opt to install the configuration files in the default
      /etc/hyperledger/fabric, whereas an unprivledged user might install them
      in ~/hyperledger.  The point is, it shouldn't matter where they are and the
      user shouldn't need a PhD in CORE_* variables to get it to work.
      
      This is part of an overall effort to improve the user-experience as we
      march towards a v1.0 release.
      
      Fixes FAB-3169 as part of FAB-2037
      
      Change-Id: I5f47f554c2f956ec2e1afebd9bd82b0bbb62892a
      Signed-off-by: default avatarGreg Haskins <gregory.haskins@gmail.com>
      8ce10737
  33. 22 Apr, 2017 1 commit
    • Will Lahti's avatar
      [FAB-2351] Update loggers to flogging.MustGetLogger · 7132dd54
      Will Lahti authored
      
      
      This CR updates all loggers throughout the code base to use
      `flogging.MustGetLogger`. This function wraps `logging.MustGetLogger`
      and tracks the logger modules defined in the system. This enables the
      ability to set log levels for modules using regular expressions.
      and make it easy to change the levels for any module and all its
      submodules with a single command (e.g. gossip, ledger, msp).
      
      Change-Id: If5d3229ea2312adb56fc21bfdafbed3d967cf1df
      Signed-off-by: default avatarWill Lahti <wtlahti@us.ibm.com>
      7132dd54