1. 18 Mar, 2019 1 commit
    • David Enyeart's avatar
      [FAB-13196] Make Fabric compatible with CouchDB 2.3 · 3975b5f3
      David Enyeart authored
      
      
      CouchDB 2.3 changed format of fields update_seq and purge_seq.
      This causes a problem when Fabric maps the CouchDB
      JSON resposnes to structs.
      
      Since Fabric doesn't use these fields anyways, they can be removed
      from the Fabric CouchDB structs. This will make Fabric
      compatible with CouchDB 2.3.
      
      Change-Id: I89f789ef79ea485aa74b99ef047af417a8be044e
      Signed-off-by: default avatarDavid Enyeart <enyeart@us.ibm.com>
      3975b5f3
  2. 15 Jan, 2019 1 commit
  3. 09 Dec, 2018 1 commit
  4. 04 Dec, 2018 1 commit
  5. 29 Nov, 2018 3 commits
  6. 22 Sep, 2018 1 commit
  7. 14 Sep, 2018 1 commit
    • Chris Elder's avatar
      [FAB-12014] Log warning for missing index · 38430add
      Chris Elder authored
      
      
      Currently the warning is only seen in peer logs if couchdb debug is enabled.
      
      Peer should log a warning without having to enable couchdb debug.
      
      Log warning to peer when "no matching index found" warning comes from
      couchdb queries.
      
      Change-Id: I46a5fa663c502bbdf74261a4925ce54633384d1f
      Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      38430add
  8. 07 Sep, 2018 1 commit
    • Chris Elder's avatar
      [FAB-9840] CouchDB safe pagination - statecouchdb · ca152f26
      Chris Elder authored
      Currently, CouchDB queries are limited to configurable max number of
      results/docs, based on peer queryLimit config option (default 10000).
      The shim/peer supports pagination but CouchDB query iterators do not.
      
      This change will allow additional controls to prevent a large accidental or
      malicious query from causing performance issues with CouchDB.
      
      Changes to core.yaml:
      
      Changed queryLimit to 1000.  This is part of repurposing this parameter.
      QueryLimit is now to be used as a per query limit to CouchDB instead of the
      limit for the query from the shim.
      
      Added totalLimit parameter.  This parameter will be used to cap the maximum
      number of records returned by the query from the shim.
      
      Changes to couchdb:
      
      Added changes to range query to find the next start key (if exists) for the
      specified range.  The next start key is returned with the query results.
      
      Added changes to rich query to find the next bookmark for the query.  The
      bookmark is returned with the query ...
      ca152f26
  9. 24 Aug, 2018 1 commit
    • Matthew Sykes's avatar
      [FAB-9131] wire zap based fabric logger · 97215fa6
      Matthew Sykes authored
      
      
      Replace go-logging with a zap based implementation. The implementation
      is mostly compatible with the legacy format but differs in a two key
      ways:
      
      1. CRITICAL and NOTICE log levels no longer exist. Calls to Critical get
         mapped to error and NOTICE gets mapped to INFO. The methods
         associated with these levels will be removed in a future CR.
      2. The log level constants are now sourced from the zap/zapcore packages.
         These values are incompatible wit the go-logging constants. Please be
         sure to use the appropriate constant when necessary.
      
      The existing go-logging package is still used by the chaincode shim to
      reduce the risk of negatively impacting legacy chaincode.
      
      Change-Id: Iaf5fac807244883a8285892ccd350c5256959782
      Signed-off-by: default avatarMatthew Sykes <sykesmat@us.ibm.com>
      97215fa6
  10. 07 Aug, 2018 1 commit
  11. 02 Aug, 2018 1 commit
    • Chris Elder's avatar
      [FAB-11308] Allow use of + char in namespace · 851cb6a0
      Chris Elder authored
      
      
      The CouchDB implementation currently does not allow + chars in the
      database name.
      
      Add support in the CouchDB implementation layer for the + char
      be included as part of the database name.
      
      Add unit tests for databases containing a + char.
      
      Change-Id: I59001d1aed19066d65f6e230f41958cd2a8cbd73
      Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      851cb6a0
  12. 10 Jul, 2018 1 commit
  13. 18 Jun, 2018 1 commit
  14. 20 May, 2018 1 commit
    • Chris Elder's avatar
      [FAB-10048] Add authentication to each CouchDB db · fcb9ae5a
      Chris Elder authored
      
      
      Add changes to add database security to each CouchDB database.   This will
      prevent any unauthorized browsing of state data.
      
      Add couchdb methods to expose the CouchDB _security API for PUT and GET to
      respectively create and retrieve the database security permissions.
      
      Add the following structure to couchdb to represent the couchdb security
      structure:
      
      type DatabaseSecurity struct {
      	Admins struct {
      		Names []string `json:"names"`
      		Roles []string `json:"roles"`
      	} `json:"admins"`
      	Members struct {
      		Names []string `json:"names"`
      		Roles []string `json:"roles"`
      	} `json:"members"`
      }
      
      Add changes to couchdb CreateDatabaseIfNotExist() and apply a security
      permission definition based on the configured CouchDB username.  Add the
      username to the Admins and Members section.
      
      Change-Id: I8dadd0e1575babef2143c7fb332b863aa0c6be5b
      Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      fcb9ae5a
  15. 02 May, 2018 1 commit
  16. 21 Feb, 2018 1 commit
    • Chris Elder's avatar
      [FAB-8336] Add create or update msg for CouchDB index · 7c963508
      Chris Elder authored
      
      
      Problem Statement:
      
      If the provided index definitions have more than one (lets say N)
      index with same DesignDoc and indexName but with potentially different indexes,
      the log will contain N messages like `created CouchDB index in the channel...`
      but in reality only 1 index will be created in CouchDB. Fabric goes through
      each file in alphabetic order, creating and (quietly) replacing the index
      created in last round with the same DDoc and indexName, so only the last index
      remains in CouchDB.
      
      Solution:
      
      Part of the confusion can be prevented by updating the message when an index
      is created or updated in CouchDB.  The create index command responds with
      metadata that can be captured and provide a more detailed message and can
      determine if the index has been created or updated.  This can be accomplished
      with the existing http request.
      
      Change-Id: I84674ceed54dadba9470d768a028a6acb3b237f6
      Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      7c963508
  17. 13 Feb, 2018 1 commit
    • Chris Elder's avatar
      [FAB-8019] CouchDB Retry count is misleading · 04e95e1e
      Chris Elder authored
      
      
      The CouchDB retry currently assumes the first pass through is
      actually retry number 1.
      
      It also does not currently allow a setting of zero retries.
      This can be confusing to the end user and will prevent some potential
      use cases that may need to wrap the request handler with zero retry logic.
      
      Change-Id: I74a5628f19fd6d73a24091bad66f955046e28190
      Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      04e95e1e
  18. 09 Feb, 2018 1 commit
    • Chris Elder's avatar
      [FAB-8044] Add config option for defer index warming · c826b64a
      Chris Elder authored
      
      
      CouchDB index warming is executed per block, and as a result it
      may become an unexpected resource hog in some scenarios, such
      as when a high rate of updates going on with very small Batchsize.
      
      A new config option warmIndexesAfterNIntervals has been added to
      prevent index warming from happening on every block.  Default is
      configured as 1.
      
      Change-Id: If7198a4140e27d312ee7caa2f667bf2c80df3657
      Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      c826b64a
  19. 07 Feb, 2018 1 commit
    • Chris Elder's avatar
      [FAB-7954] Errors not handled for JSON marshaling · ba3e8415
      Chris Elder authored
      
      
      Upon code review in statecouchdb.go and couchdb.go, it appears
      there are occurrences of the following error producing functions
      without errors captured and handled:
      
      json.Marshal()
      json.Unmarhsal()
      decoder.Decode()
         e.g. json.NewDecoder(resp.Body).Decode(&dbResponse)
         e.g. decoder.Decode(&genericMap)
      
      Change-Id: Id632d636d7e2de817abb60961f908f7ad5222bec
      Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      ba3e8415
  20. 05 Feb, 2018 1 commit
    • Chris Elder's avatar
      [FAB-6938] Cannot create CouchDB database for channel · c0b6e8cd
      Chris Elder authored
      
      
      In rocket.chat there was a post about an error:
      when i am running join peer request some times it's giving
      Error:file_exists, Status Code:412, Reason:The database could
      not be created, the file already exists
      
      I believe this can happen if peer is retrying a create database
      request against CouchDB.  A prior attempt to create database came
      back as a timeout failure to peer, but in the meantime the database
      got created in CouchDB.  The next retry would get the error above.
      This scenario should be handled in peer code.
      
      Change-Id: I01ecd3a5fde8a9042f6a18181376dec267d0cc15
      Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      c0b6e8cd
  21. 02 Feb, 2018 1 commit
    • Chris Elder's avatar
      [FAB-7969] Remove CouchDB "data" envelope · 5185bada
      Chris Elder authored
      
      
      Chaincode data is saved in CouchDB under a JSON "data" envelope.
      
      For queries, peer automatically re-writes chaincode query to account for
      data envelope.
      
      For indexes, chaincode author must specify the data envelop when referencing
      fields.
      
      Need to make the experience consistent, by removing the "data" envelope.
      This will fix the usability issue, and also simplify the code, eliminating
      other possible defects.
      
      This change is only possible in 1.1, since the upgrade from 1.0 to 1.1 will
      rebuild CouchDB state database using the enhanced 1.1 CouchDB data structure.
      
      Change-Id: I2f7345fe4f4aaa17324c26f1e32bec2b3a5f67db
      Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      5185bada
  22. 20 Jan, 2018 1 commit
  23. 19 Jan, 2018 1 commit
    • David Enyeart's avatar
      [FAB-7816] Fix cc event handling and debug · 66aef3cf
      David Enyeart authored
      
      
      Chaincode event handling was intended to pick up LSCC
      chaincode updates, but was also picking up LSCC
      chaincode collection updates.
      
      This change ignores chaincode collection updates.
      
      Additionally, related chaincode event debug is improved.
      
      Note that two copies of Collecion-specific constants
      were found.  The unused duplicate is removed from lscc.
      
      Change-Id: I8d7c4320d757a7d3327bd2bb22bba0b653febc23
      Signed-off-by: default avatarDavid Enyeart <enyeart@us.ibm.com>
      66aef3cf
  24. 14 Jan, 2018 1 commit
    • Chris Elder's avatar
      [FAB-6855] Add CouchDB index warming · 1cb48fab
      Chris Elder authored
      
      
      CouchDB does not automatically update indexes when records are
      inserted or updated.  The update is performed for the index
      during the query operation.   This can cause delays during
      the query.   Subsequent queries then take advantage of the updated
      index.
      
      This improvement will add an "index warming" step to refresh indexes
      following an insert or update.
      
      The addition is to the EnsureFullCommit step.  This will execute an
      index warming action for each batch.  The index refresh will need to
      be called for each index defined.
      
      Change-Id: I4de7f8049be8d5ff092c84a42d250ab7c361c492
      Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      1cb48fab
  25. 14 Dec, 2017 1 commit
  26. 24 Oct, 2017 1 commit
    • Chris Elder's avatar
      [FAB-6738] GetState in CouchDB fails to return database · f0757eff
      Chris Elder authored
      
      
      For a while composer has created composite keys in a specific
      format and this has worked fine. Since the introduction of a
      change to the GetState protocol, GetState has not been returning
      information.
      
      Additional investigation showed that the error occurs due to a
      bad URL generated when then key contains a colon ":"
      
      Problem is resolved by encoding the path using url EscapedPath()
      method instead of the String() method.
      
      Change-Id: I20ad8177301ea845dd234eb867c95b5cde7a908c
      Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      f0757eff
  27. 29 Sep, 2017 1 commit
    • David Enyeart's avatar
      [FAB-6355] Fix GetPrivateData() error on CouchDB · 41714c2a
      David Enyeart authored
      
      
      The problem was that the retrieve of the public
      hash from couch state db fails due to a URL
      encoding issue of a plus sign in the hashed key.
      
      Keys were already URL encoded using golang URL
      encoding, but the encoding skips plus signs in
      the path component.  CouchDB unencodes the plus
      sign as a space.
      
      The fix is to explicitly URL encode plus character
      when interacting with CouchDB.
      
      Also add unit test to verify the fix and test
      all other URL special characters.
      
      Also clarified the error message received when
      private version does not match public hash.
      
      Change-Id: I9b99d3446542e9eae0196158270205d3ea09db9f
      Signed-off-by: default avatarDavid Enyeart <enyeart@us.ibm.com>
      41714c2a
  28. 21 Aug, 2017 1 commit
  29. 03 Aug, 2017 1 commit
    • Chris Elder's avatar
      [FAB-2960] Transaction Mgr changes batch optimizations · 7d90d014
      Chris Elder authored
      
      
      This is change is part of FAB-2725 CouchDB optimizations
      
      Interactions with CouchDB are currently done individually.
      Need to switch to using bulk operations to get optimal performance
      from CouchDB. Need to performance test and stress test.
      
      This change implements all changes needed by the commit phase to use
      bulk operations in CouchDB.
      
      A subsequent change will implement validation bulk read and caching
      of versions and revisions for CouchDB.
      
      Subsequent changes will also be added to retry errors encountered by
      the bulk update operation and a configuration setting to limit the
      size of the bulk update by number of update documents.
      
      - Add new interface to statedb for bulk operations, the new operations
      will only be implemented for databases that can support bulk operations
      (only CouchDB)
      
      type BulkOptimizable interface {
      	LoadCommittedVersions(keys []*CompositeKey)
      	ClearCachedVersions()
      }
      
      LoadCommittedVersions will bulk load version and revision numbers into
      cache based on the document ID.  Documents missing a version or revision
      will be added to the cache with a nil value.
      
      - Add GetVersion method to VersionedDB to enhance batch
      validation of batches
      
      GetVersion(namespace string, key string) (*version.Height, error)
      
      - Implement the GetVersion method for CouchDB and goleveldb.  The goleveldb
      implementation will be similar to GetState, except it will return the
      version.  CouchDB implementation will attempt a cache lookup, then fall
      back to a database read if not found.
      
      - Enhance the ApplyUpdates method for CouchDB to incorporate the
      ability to retrieve all revision information required for bulk
      updates and deletes for CouchDB.
      
      Change-Id: I65f8525a7a13b1ad7b49fda0e357b9a406fba79d
      Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      7d90d014
  30. 01 Aug, 2017 1 commit
    • Baohua Yang's avatar
      [FAB-5194] Fix usage problems in code · 76c0dc56
      Baohua Yang authored
      
      
      This patchset helps clean up the code and docs:
      
      * Fix redundant parenthesese.
      * Fix mismatching params in func return.
      * Fix variable name collides with imported package name and builtin
        function or reserved words.
      * Fix word typos.
      
      Change-Id: I30304c233067ead7e74d18e3caf2604b3ed1490a
      Signed-off-by: default avatarBaohua Yang <yangbaohua@gmail.com>
      76c0dc56
  31. 13 Jun, 2017 1 commit
    • Chris Elder's avatar
      [FAB-3555] Peer panic on SIGSEV connecting to CouchDB · dc8d45fa
      Chris Elder authored
      
      
      A peer panic was encountered connecting to CouchDB on
      peer startup.   This occurred in the code after multiple
      retries.
      
      The panic was caused by a null pointer when attempting
      to read the status code from an http return object.  The
      http return object was already checked for an error.
      
      The situation causing the null pointer can only be caused
      if the golang http return has a nil for both the response
      and the error.   According to golang spec this is not
      possible.
      
      An additional check is being made to ensure a good error
      message is returned from the CouchDB connection request.
      
      Change-Id: I13740a32231d49faf3a6acf7b37f7bc6e4bba3fa
      Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      dc8d45fa
  32. 30 May, 2017 1 commit
    • Chris Elder's avatar
      [FAB-3686] CouchDB timeout causes error upon retry · b3eef4ca
      Chris Elder authored
      
      
      If a state db document update to CouchDB has a http timeout
      (currently defaulted at 35s), the transaction may eventually
      succeed in CouchDB, but the peer will do a retry using the
      previous CouchDB document revision number. The retry will
      therefore fail due to revision number conflict. The peer will
      think it failed, when in fact it succeeded. Peer panics when
      it can't process the block to completion, since subsequent
      blocks should not be processed. Need to fix retry logic to
      account for this scenario, potentially by introducing a higher
      level retry that could get the new revision number for a final
      attempt.
      
      Implement a retry in the couchdb SaveDoc/DeleteDoc functions to retry
      if the return code is 409 (revision conflict).  If
      the return code is 409, then retrieve the current revision
      for the document ID and retry.   This logic is separate from
      the timeout logic in handleRequest is only used for resolving
      document revision conflicts caused by CouchDB timeouts.
      
      Change-Id: I2ef79b63397a8f76e295dc7c562f5cc7f86da73d
      Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      Signed-off-by: default avatarDavid Enyeart <enyeart@us.ibm.com>
      b3eef4ca
  33. 12 May, 2017 1 commit
    • Chris Elder's avatar
      [FAB-3368] Zero length CouchDB attachment delays · 46f52deb
      Chris Elder authored
      CouchDB attachments with zero length causes CouchDB to return
      successfully but the connection is not available for future
      connections.  This is a bug identified in CouchDB.
      
      Zero length attachments are valid.  When an empty byte array
      is saved, CouchDB stores the value as a zero length attachment.
      
      The next subsequent client request over the same connection times
      out reading headers from server.
      
      This is the CouchDB JIRA item for tracking purposes.
      https://issues.apache.org/jira/browse/COUCHDB-3394
      
      
      
      The fix in this change checks to see if the attachment in
      SaveDoc is zero length.  If the attachment is zero length,
      then request close value is set to true.  This setting will
      not allow the http connection to be reused and will prevent
      connection timeouts.
      
      Change-Id: I1a1a33c44a3b96dfd2bffaf1cec5fe9ece084817
      Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      46f52deb
  34. 24 Apr, 2017 2 commits
  35. 22 Apr, 2017 1 commit
    • Will Lahti's avatar
      [FAB-2351] Update loggers to flogging.MustGetLogger · 7132dd54
      Will Lahti authored
      
      
      This CR updates all loggers throughout the code base to use
      `flogging.MustGetLogger`. This function wraps `logging.MustGetLogger`
      and tracks the logger modules defined in the system. This enables the
      ability to set log levels for modules using regular expressions.
      and make it easy to change the levels for any module and all its
      submodules with a single command (e.g. gossip, ledger, msp).
      
      Change-Id: If5d3229ea2312adb56fc21bfdafbed3d967cf1df
      Signed-off-by: default avatarWill Lahti <wtlahti@us.ibm.com>
      7132dd54
  36. 18 Apr, 2017 1 commit
    • Chris Elder's avatar
      [FAB-3094] CouchDB Errors if system databases missing · 9280d69d
      Chris Elder authored
      
      
      CouchDB log fills with errors if its system databases are not created.
      The system databases should be created upon peer startup,
      if they do not yet exist.
      
      This change makes a simple check on creating the state database to
      check if the system database exist as well.  If they are missing, then
      the system databases are created.
      
      Change-Id: I6dacb8ceefcda8fd0f14a0ef55da921b9160343d
      Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      9280d69d
  37. 17 Apr, 2017 1 commit
    • Chris Elder's avatar
      [FAB-3131] Peer hangs when CouchDB unresponsive · 2e0a61f6
      Chris Elder authored
      
      
      The peer hangs when the CouchDB network connection is unresponsive.
      This can be corrected by adding a timeout to the network client
      connection used for connecting to CouchDB.  If the connection
      times out, then the request to CouchDB will return an error and
      the retry logic will take over.
      
      A new entry is added for CouchDB in core.yaml named connectionTimeout
      
      couchDBConfig:
         couchDBAddress: 127.0.0.1:5984
         username:
         password:
         # Number of retries for CouchDB errors
         maxRetries: 3
         # Number of retries for CouchDB errors during peer startup
         maxRetriesOnStartup: 10
         # CouchDB connection timeout (unit: duration, e.g. 60s)
         connectionTimeout: 60s
      
      This is a duration property configurable by the peer.
      
      Change-Id: I5028029e7f303144465c2bfeebf540c5d8e64d82
      Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      2e0a61f6