Skip to content
GitLab
  1. 03 Dec, 2018 2 commits
  3. 20 Nov, 2018 1 commit
  5. 29 Oct, 2018 1 commit
  7. 25 Oct, 2018 1 commit
  9. 17 Oct, 2018 1 commit
  11. 03 Oct, 2018 1 commit
  13. 24 Sep, 2018 1 commit
  15. 22 Sep, 2018 2 commits
  17. 19 Sep, 2018 1 commit
  19. 07 Sep, 2018 1 commit
    • Chris Elder's avatar
      [FAB-9840] CouchDB safe pagination - statecouchdb · ca152f26
      Chris Elder authored 
      Currently, CouchDB queries are limited to configurable max number of
results/docs, based on peer queryLimit config option (default 10000).
The shim/peer supports pagination but CouchDB query iterators do not.

This change will allow additional controls to prevent a large accidental or
malicious query from causing performance issues with CouchDB.

Changes to core.yaml:

Changed queryLimit to 1000.  This is part of repurposing this parameter.
QueryLimit is now to be used as a per query limit to CouchDB instead of the
limit for the query from the shim.

Added totalLimit parameter.  This parameter will be used to cap the maximum
number of records returned by the query from the shim.

Changes to couchdb:

Added changes to range query to find the next start key (if exists) for the
specified range.  The next start key is returned with the query results.

Added changes to rich query to find the next bookmark for the query.  The
bookmark is returned with the query ...
      ca152f26
  21. 23 Aug, 2018 1 commit
  23. 20 Jun, 2018 1 commit
  25. 16 Jun, 2018 1 commit
    • Chris Elder's avatar
      [FAB-8877] Validate Reserved Fields Name in CouchDB · db1a30e1
      Chris Elder authored 
      

Ledger removed the "data" wrapper when using CouchDB as the
state database.  This change means that documents in the state
database must meet CouchDB's restrictions for top level field
names.

The validation for simulation and commit needs to be updated
for the following invalid field names:

- Any field beginning with an underscore, "_"
- ~version
- ~metadata

Change-Id: I9203691553de8c15888ce93ade99337bad5222a9
Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      db1a30e1
  27. 11 Jun, 2018 1 commit
  29. 21 May, 2018 1 commit
    • Chris Elder's avatar
      [FAB-7779] CouchDB indexes for private data · 9a3b1811
      Chris Elder authored 
      

Any couchdb indexes defined for a chaincode will be deployed in
the chaincode state database.  Indexes for collections of private
data should be applied in the chaincode's collection state database
rather than in the chaincode state database.

Added support for HandleChaincodeDeploy event to common_storage_db.  This
will now process the event from chain code deploy.  The move was necessary
for handling the private data namespace mapping.

Added new function to statecouchdb for processing index creation.

Added new interface to statedb for IndexCapable.  This allows the
common_storage_db to detect that the state database will accept index
creation events.

Updated index validation to allow for indexes under collections directories.

Change-Id: Icfda923c584d953ce5886023f7411bbddb59d595
Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      9a3b1811
  31. 27 Apr, 2018 1 commit
  33. 14 Apr, 2018 1 commit
    • manish's avatar
      [FAB-7692] Refactor statecouchdb impl · 4aba4a81
      manish authored 
      

This CR refactors the code for statecouchdb implementation.
This includes splitting the code into different files and
making the code more modular by spliting it into high level structures and functions
so to make the code more readble and maintainable

The unit tests are not changed in this CR so as to make sure
that things are not lost in the refactoring

Change-Id: I5c937f7d3c541be0b89591175b6946ae32c5833f
Signed-off-by: default avatarmanish <manish.sethi@gmail.com>
      4aba4a81
  35. 13 Apr, 2018 1 commit
  37. 10 Apr, 2018 1 commit
    • manish's avatar
      [FAB-9452] Enhance chaincode event listers · 08516948
      manish authored 
      

   This CR:
 - Allows registeration of multiple listeners for
   chaincode life-cycle events
 - Introduces a call back to the listeners when
   the chaincode deploy transaction commits or
   chaincode install completes

Change-Id: Ic9e26b430d8b0c844219edb454e5d410477c9516
Signed-off-by: default avatarmanish <manish.sethi@gmail.com>
      08516948
  39. 16 Feb, 2018 1 commit
    • manish's avatar
      [FAB-7777] Suppress couchdb index creation errors · 876b2747
      manish authored 
      

This CR changes the statedb impl for the couchdb such that it does
not propagate the errors that are caused by erroneous db artifacts present
in the chaincode package archive. This is because a bad index file could bring
the peer to a halt until the chaincode package in uninstalled

Change-Id: Iecb8c243fcb5c8e01beb31a6ca21a53a1b14a354
Signed-off-by: default avatarmanish <manish.sethi@gmail.com>
      876b2747
  41. 13 Feb, 2018 1 commit
  43. 02 Feb, 2018 1 commit
    • Chris Elder's avatar
      [FAB-7969] Remove CouchDB "data" envelope · 5185bada
      Chris Elder authored 
      

Chaincode data is saved in CouchDB under a JSON "data" envelope.

For queries, peer automatically re-writes chaincode query to account for
data envelope.

For indexes, chaincode author must specify the data envelop when referencing
fields.

Need to make the experience consistent, by removing the "data" envelope.
This will fix the usability issue, and also simplify the code, eliminating
other possible defects.

This change is only possible in 1.1, since the upgrade from 1.0 to 1.1 will
rebuild CouchDB state database using the enhanced 1.1 CouchDB data structure.

Change-Id: I2f7345fe4f4aaa17324c26f1e32bec2b3a5f67db
Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      5185bada
  45. 15 Jan, 2018 1 commit
    • Chris Elder's avatar
      [FAB-6176] Couchdb index management state database · 66d785bd
      Chris Elder authored 
      

Add couchdb index management capabilities to the state database layer.

Add create and delete index functions to statecouchdb.

Add HandleChaincodeDefinition to statecouchdb.

Add unit tests for create and save index functions.

Add unit tests for HandleChaincodeDefinition.

Change-Id: I82f31ced8eb666e607c549f995d0c83fc7364664
Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      66d785bd
  47. 05 Jan, 2018 1 commit
    • senthil's avatar
      [FAB-7131] CouchDB: per channel-chaincode DB · 6cbebf4a
      senthil authored 
      

For every chaincode, use a separate database instead of using
the same channel database. This CR makes the following changes:
(i) creates a database per channel_chaincode and two DBs per
channel_chaincode_collection (one for hashed data and another for
actual private data).
(ii) modifies GetState, RangeQuery, and ExecuteQuery implementation
to use appropriate chaincode DB as per the given namespace.
(iii) modifies commit process (ApplyUpdates) to apply changes to
respective chaincode DBs.

In the subsequent CR, for loadCommittedVersions and ApplyUpdates,
goroutines would be used to process each chaincode batches
parallely.

Change-Id: I05b5f70e6c253cec683382c9e5b2b39b6a6bcce8
Signed-off-by: default avatarsenthil <cendhu@gmail.com>
      6cbebf4a
  49. 26 Oct, 2017 1 commit
  51. 23 Oct, 2017 1 commit
  53. 12 Sep, 2017 1 commit
  55. 06 Sep, 2017 1 commit
  57. 28 Apr, 2017 1 commit
  59. 24 Apr, 2017 1 commit
    • Gregory Haskins's avatar
      [FAB-3160] Provide config-relative path feature · 8ce10737
      Gregory Haskins authored 
      

Introduction
======================
The primary goal of this patch is to create the notion of a
"config-relative" path reference.  For example, a configuration file
"/etc/foo/bar.yaml" that contains a key "bat" with a value
"baz/blaz/blamo" can be used to specify that "baz/blaz/blamo" should
be considered relative to the configuration file itself.  In this case,
it would be expected to be found at /etc/foo/baz/blaz/blamo.  FAB-2037
does a much more thorough job of explaining the rationale on why
config-relative is considered important/good-form.

This is in stark contrast to what we have today, which is a jumbled
mess of assumed GOPATH relative, CWD relative, ENVVAR absolute and
sometimes even ENVVAR relative.  Therefore, an additional positive
side-effect of this endeavor is that this patch also substantially
cleans up some technical debt that had been accumulating in the tree
for some time related to ad-hoc pathing, DRY violations, and just
general inconsistencies in how configuration files were managed.

Design Details
==========================
This patch refactors the basic configuration system into the notion of
a tree rooted at a configuration-path.  By default, this path is
$GOROOT/..../fabric/sampleconfig during dev/test and
/etc/hyperledger/fabric during runtime.  The root may be overridden
at any time by specifying the environment variable FABRIC_CFG_PATH.
(Note that this variable unifies and replaces the former PEER_CFG_PATH
and ORDERER_CFG_PATH).

The dev/test environment will operate out of the ./fabric/sampleconfig
configuration root.  The build-system will package that root into
/etc/hyperledger/fabric in the runtime context with the intention of
the end-user/admin/deployer replacing parts or all of the sampleconfig
to suit their application.

Since configuration-relative paths are now possible, the configuration
files may reference other relative files and they will behave
appropriately regardless of the context in which they are executed.
For example, consider the files ./sampleconfig/tls/server.[crt|key].
A configuration file may contain a key "tls/server.key" and the system
will properly resolve this relative file even at runtime. This is (IMO)
far more natural than assuming a path is relative to the CWD of where
the command is executed, which is how most of the system behaves today
(or requires awkward and very specific ENVVAR overrides).

This will be conducive to something like a package-installer
(e.g. RPM/DEB) or a docker environment to augment/replace elements
of the configuration root and to freely move the configuration around
as the package/deployer sees fit.

As an example, a deployment on Kubernetes might opt to volume mount
/etc/hyperledger/fabric to replace the entire config, or it might just use
a secrets mount on /etc/hyperledger/fabric/peer/tls.  An RPM packager
might opt to install the configuration files in the default
/etc/hyperledger/fabric, whereas an unprivledged user might install them
in ~/hyperledger.  The point is, it shouldn't matter where they are and the
user shouldn't need a PhD in CORE_* variables to get it to work.

This is part of an overall effort to improve the user-experience as we
march towards a v1.0 release.

Fixes FAB-3169 as part of FAB-2037

Change-Id: I5f47f554c2f956ec2e1afebd9bd82b0bbb62892a
Signed-off-by: default avatarGreg Haskins <gregory.haskins@gmail.com>
      8ce10737
  61. 21 Apr, 2017 1 commit
    • denyeart's avatar
      [FAB-3317] Update peer CouchDB timeout · 5628d8e8
      denyeart authored 
      

Occasionally CouchDB doesn't respond for (exactly) 30s.
This is most often seen in create database calls
which happens frequently during unit tests.

The default peer behavior was to retry couch after 20s,
and therefore these occurrences disrupt unit tests and
causes them to fail occasionally.

This change will set the peer CouchDB timeout to 35s for now,
in order to avoid the retries in the peer avoid the unit test problems.

After the root problem in CouchDB is resolved, the default may
change back to 20s.

Change-Id: Id688e2e137b8354512b6543fc63ad3e59a1040eb
Signed-off-by: default avatardenyeart <enyeart@us.ibm.com>
      5628d8e8
  63. 20 Apr, 2017 1 commit
  65. 27 Feb, 2017 2 commits
  67. 11 Feb, 2017 1 commit
    • Chris Elder's avatar
      FAB-2098 Add delete to ledger for CouchDB · c84b8297
      Chris Elder authored 
      

Motivation for this change:
Add DeleteDoc method to the CouchDB layer in order to provide
delete by ID capability.  This function mirrors the delete
capability used by goloveldb.

- Add delete to statecouchdb.go if value is set to null

- Add unit tests for delete in statecouchdb_test.go

Change-Id: I23b8e7e2daf8679dc0fbe8f1a75011530ff8d250
Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      c84b8297
  69. 03 Feb, 2017 1 commit
    • manish's avatar
      Move Blockstorage code under /fabric/common package · 2a16532c
      manish authored 
      https://jira.hyperledger.org/browse/FAB-2022



This changes introduced by this CR
- Moves the block storage code from package
core/ledger/blkstorage to common/ledger/blkstorage

- Splits the ledger_interface.go so as to move common interfaces
and data type to common/ledger package

- Moves some of the util functions to common/ledger package

- Moves core/ledger/ordererledger package to orderer/ledger/fsledger
orderer folks can futher rename/refactor this as seems suitable to them

Change-Id: I759e16f00dc2ec9bb62196121083cf48eae76948
Signed-off-by: default avatarmanish <manish.sethi@gmail.com>
      2a16532c
  71. 27 Jan, 2017 1 commit
    • Chris Elder's avatar
      FAB-1818 Create data wrapper for state data in CouchDB · ead6705d
      Chris Elder authored 
      

Motivation for this change:
Maintain the version inline with the document, without changing the
structure/value of the stored document.  This will prevent unexpected
data values being returned and will prevent possible name collisions
with document values.

- Create a data wrapper for ledger JSON data stored in CouchDB.

- Return the version based on block number and transaction id

The wrapper will be implemented as a new key named "data" which will
contain the JSON data for the state database.

Prior to change example:

"doc": {
 "_id": "2",
 "_rev": "2-8ee0c31b21ad650e5b872c0b98e59ab5",
 "version":"1:2"
 "asset_name": "marble2",
 "color": "red",
 "owner": "tom",
 "size": "25"
}

 Following change:

 "doc": {
  "_id": "2",
  "_rev": "2-8ee0c31b21ad650e5b872c0b98e59ab5",
  "version":"1:2"
  "data": {
   "asset_name": "marble2",
   "color": "red",
   "owner": "tom",
   "size": "25"
  }
 }

Change-Id: I59391ea926531c46c346fc8448e3d041ca5f3fdf
Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      ead6705d
  73. 26 Jan, 2017 1 commit
    • denyeart's avatar
      [FAB-1762] Refactor couchdb history db to leveldb. · 2c982629
      denyeart authored 
      

- Utilize leveldb for history of keys instead of couchdb

- Single leveldb database serves all chains to optimize
  footprint (same structure as leveldb block index database
  and leveldb state database)

- Block commit writes a history key for each key/value
  that is updated in a tran, in the form:
   ns~key~blocknum~trannum

- History database is an index for history of key writes
  over time.  The key values are not included to reduce
  size of database.

- GetHistoryForKey() finds all records starting with ~ns~key
  and returns the transactions that updated the key.

- Subsequent changeset will lookup and return the txid
  and historic value from the block storage.  Client can
  then GetTransactionById to see the historic transactions.

- Since history db size is much reduced, it is now enabled
  by default in core.yaml.

- Upon crash recovery, ledger initialization will ensure
  that both state db and history db in sync with block storage

Reused existing test logic, therefore the changeset
is relatively large to ensure tests still pass.

Change-Id: I79103aa39957f58d246de5b5295fb40a4b9c033b
Signed-off-by: default avatardenyeart <enyeart@us.ibm.com>
      2c982629