1. 03 Feb, 2017 1 commit
    • manish's avatar
      Move Blockstorage code under /fabric/common package · 2a16532c
      manish authored
      https://jira.hyperledger.org/browse/FAB-2022
      
      
      
      This changes introduced by this CR
      - Moves the block storage code from package
      core/ledger/blkstorage to common/ledger/blkstorage
      
      - Splits the ledger_interface.go so as to move common interfaces
      and data type to common/ledger package
      
      - Moves some of the util functions to common/ledger package
      
      - Moves core/ledger/ordererledger package to orderer/ledger/fsledger
      orderer folks can futher rename/refactor this as seems suitable to them
      
      Change-Id: I759e16f00dc2ec9bb62196121083cf48eae76948
      Signed-off-by: default avatarmanish <manish.sethi@gmail.com>
      2a16532c
  2. 27 Jan, 2017 1 commit
    • denyeart's avatar
      [FAB-1885] GetTransactionByID to return Tran Envelope · 9ca80f1f
      denyeart authored
      
      
      QSCC GetTransactionByID should return a Transaction  Envelope,
      not a Transaction. The Envelope contains the signature and
      Payload, which contains the transaction header and Transaction.
      
      Clients will want the transaction header information when retrieving
      the transaction.  They may also want the signature to verify that
      the transaction hasn't changed since the time they submitted it.
      The return type will change from Transaction proto to Envelope proto.
      
      Change-Id: I1de238035c3b6cd00abb3ed8506c54566ee0f2b0
      Signed-off-by: default avatardenyeart <enyeart@us.ibm.com>
      9ca80f1f
  3. 26 Jan, 2017 1 commit
    • denyeart's avatar
      [FAB-1762] Refactor couchdb history db to leveldb. · 2c982629
      denyeart authored
      
      
      - Utilize leveldb for history of keys instead of couchdb
      
      - Single leveldb database serves all chains to optimize
        footprint (same structure as leveldb block index database
        and leveldb state database)
      
      - Block commit writes a history key for each key/value
        that is updated in a tran, in the form:
         ns~key~blocknum~trannum
      
      - History database is an index for history of key writes
        over time.  The key values are not included to reduce
        size of database.
      
      - GetHistoryForKey() finds all records starting with ~ns~key
        and returns the transactions that updated the key.
      
      - Subsequent changeset will lookup and return the txid
        and historic value from the block storage.  Client can
        then GetTransactionById to see the historic transactions.
      
      - Since history db size is much reduced, it is now enabled
        by default in core.yaml.
      
      - Upon crash recovery, ledger initialization will ensure
        that both state db and history db in sync with block storage
      
      Reused existing test logic, therefore the changeset
      is relatively large to ensure tests still pass.
      
      Change-Id: I79103aa39957f58d246de5b5295fb40a4b9c033b
      Signed-off-by: default avatardenyeart <enyeart@us.ibm.com>
      2c982629
  4. 20 Jan, 2017 1 commit
    • denyeart's avatar
      FAB-1585 Refactor History to enable LevelDB · 74eeb666
      denyeart authored
      
      
      The purpose of this refactor is to create a common interface and to
      isolate the db implementation, so that we can enable history on
      LevelDB. This change only refactors the existing code and tests.
      It does not add new functionality or any new tests. There will be
      future changes that add history support on LevelDB and clean up
      tests to work against LevelDB.
      
      Note: History is not enabled by default. History must be enabled in
      core.yaml historyDatabase for it's unit tests to run.
      
      Change-Id: Ia5293cd345be4499e5e3df0c2949af16f71a608d
      Signed-off-by: default avatardenyeart <enyeart@us.ibm.com>
      74eeb666
  5. 17 Jan, 2017 2 commits
    • manish's avatar
      SingleLevelDB for block index · 8cdd0f4d
      manish authored
      https://jira.hyperledger.org/browse/FAB-1664
      
      
      
      This changeset:
      - Renames package ledger/util/db to ledger/util/leveldbhelper
      - Implements a leveldb provider
        (that enables using same leveldb instance as a multiple logical dbs)
        in util package for being able to reuse across statedb, index,
        and later for historydb
      - Implements a provider as a single point of invocation
        for managing multiple block storage
      - Uses a single leveldb instance for block storage index
      - Makes the structures other than providers as private
        to their respective packages
      
      Change-Id: I5f0b3b9aa8ef3ac1ccdce4f3c6fa6d842b5318c1
      Signed-off-by: default avatarmanish <manish.sethi@gmail.com>
      8cdd0f4d
    • Xi Xue Jia's avatar
      [FAB-1390] Refactor ledger interface names · d58d51be
      Xi Xue Jia authored
      https://jira.hyperledger.org/browse/FAB-1390
      
      
      
      A few things changed according to FAB-1390 description and the latest NCAP
      1. renamed RawLedger interface to OrdererLedger
      2. added PeerLedger interface which keeps a bitmask to indicate invalid transactions as described in NCAP
      3. changed ledger API description in Next-Ledger-Architecture-Proposal
      4. refactored rawledger package to ordererledger in orderer directory
      
      Please notice that with this change set, there'll be cases of inconsistent package names with import path.
      This is a tradeoff to avoid stuttering path names
      These cases are listed as below.
      
      Package name, import path
      (ordererledger, github.com/hyperledger/fabric/orderer/ledger)
      (ramledger, github.com/hyperledger/fabric/orderer/ledger/ram)
      (fileledger, github.com/hyperledger/fabric/orderer/ledger/file)
      
      Change-Id: Ie9580678ee9012ff865d1dc507066f46e106f250
      Signed-off-by: default avatarXi Xue Jia <xixuejia@gmail.com>
      d58d51be
  6. 16 Jan, 2017 1 commit
  7. 15 Jan, 2017 1 commit
    • denyeart's avatar
      FAB-1505 and FAB-1337 Refactor CouchDB code · 9df7b0ea
      denyeart authored
      
      
      FAB-1505 refactors CouchDB code to be aligned with LevelDB structure.
      FAB-1337 applies all ledger tests written for LevelDB against CouchDB.
      
      In order to test the CouchDB refactor,
      had to implement FAB-1337 in same changeset to test.
      
      Like most refactors, this changeset touches a lot of files and code,
      since it is not feasible to do a partial refactor.
      
      Tests related to versioning and deleting have been commented out,
      these will be re-enabled in subsequent changeset when CouchDB
      has the support, in order to keep this changeset size more reasonable.
      
      Change-Id: I0d2d6cca89bd0252f6e84317457a9140b3a0d7a5
      Signed-off-by: default avatardenyeart <enyeart@us.ibm.com>
      9df7b0ea
  8. 08 Jan, 2017 1 commit
  9. 15 Dec, 2016 2 commits
  10. 14 Dec, 2016 1 commit
  11. 13 Dec, 2016 2 commits
    • Mari Wade's avatar
      FAB-1336 Add new ledger blockstorage index. · 458c5212
      Mari Wade authored
      
      
      Add a new ledger blockstorage index for History
      that will map (blocknum,trannum) to the file storage
      location for this block transaction
      
      This index will be used for the API  GetTransactionsForKey()
      for (chaincode1,key1).  It will do a key range query on chaincode1~key1
      to pick up all chaincode1~key1 records.  Results will indicate
      the set of (blocknum,trannum) transactions that updated this key.
      
      Change-Id: I81da09e5526d7e2966634c78a03d34011d514442
      Signed-off-by: default avatarMari Wade <mariwade@us.ibm.com>
      458c5212
    • Mari Wade's avatar
      FAB-1140 Ledger History Database framework · d18aa985
      Mari Wade authored
      
      
      This is an initial check-in that will
      1) create the history database in couchDB if it does not exist.
      2) add the history commit to the LEDGER Commit if history is enabled
      3) stores the history in the history database
      
      The History functionality is not enabled unless both
      couchDB and History are enabled in the config.
      
      Note that tests will be added to validate the writes
      and the actual data written to the database in future
      changes.  The tests to validate history is not possible
      until the query APIs are put in place in future changes.
      
      Change-Id: Id207007ab5faae957c1e05234e441566a116ea33
      Signed-off-by: default avatarMari Wade <mariwade@us.ibm.com>
      d18aa985
  12. 10 Dec, 2016 1 commit
  13. 08 Dec, 2016 1 commit
    • senthil's avatar
      FAB-1233 State DB recovery · c0dc54bd
      senthil authored
      
      
      We need to recover the state DB when
      (i) the peer fails during commit (partial written state).
      (ii) the database gets corrupted.
      
      We introduce a new state in DB called savepoint, and update
      it with the block height once all valid tx in the last/recent block is
      committed.
      
      Whenever peer starts (first boot up or after a failure), we compare the
      savepoint in DB and block height to see whether the state DB is in
      consistent state. If not, we execute the following steps:
      (i) retrieve all required blocks (#blocks = block height - savepoint)
      from block storage,
      (ii) compute write set for valid tx, commits these values, & update the
      savepoint.
      
      Change-Id: I769d1391de511d3cdb55c40692beb829e2cc5c2f
      Signed-off-by: default avatarsenthil <cendhu@gmail.com>
      c0dc54bd
  14. 01 Dec, 2016 1 commit
  15. 30 Nov, 2016 1 commit
    • Mari Wade's avatar
      FAB-1087 Add config option in core.yaml for history · 37314475
      Mari Wade authored
      
      
      Add config option in core.yaml for history database
      to store the transaction history.  The history database
      will be a queryable database such as CouchDB.
      
      The default will be false.  If set to false, the transaction
      history will not be persisted in the history database.
      
      Move the ledger config directory up one level to ledger
      to be the config for state and history databases as well
      as any other ledger functionality that needs a config option.
      
      Change-Id: If4607047082b686dd903d8b3805362d0c05e1c64
      Signed-off-by: default avatarMari Wade <mariwade@us.ibm.com>
      37314475
  16. 27 Nov, 2016 1 commit
    • Srinivasan Muralidharan's avatar
      FAB-1198-rm old pb.Transaction, pb.Block · 61affa05
      Srinivasan Muralidharan authored
      NOTE - Removing of old proto.Transaction is the cause for
      the large change set. It affects chaincode framework and
      all users of the framework such as endorser, system chaincodes,
      unit tests etc.
      
      Transaction2 is renamed to Transaction.
      Response2 is renamed to Response.
      Message2 is renamed to Message.
      
      The changes are fully described in
          https://jira.hyperledger.org/browse/FAB-1198
      
      
      
      Summary
      =======
         . Remove old Transaction and rename Transaction2
         . Cleanup of Chaincode protobuf message
         . Add TxID for SDK and higher layers to optionally
           set (currently errors if not set)
      
      ChaincodeMessage removes QUERY and QUERY_CHAINCODE enums.
      
      Shim interface does not enforce Query or QueryChaincode.
      
      chaincode_example02 and 05 implement Query function via
      the Invoke implementation.
      
      The "noop" system chaincode is removed
         . it was using Transaction which is not an endorser
           artifact any longer
         . there are many system chaincodes to that thoroughly
           test sys chaincode functions
      
      Change-Id: Ib77b7e5a6756eac47e888309816076580ae505e7
      Signed-off-by: default avatarSrinivasan Muralidharan <muralisr@us.ibm.com>
      61affa05
  17. 14 Nov, 2016 1 commit
    • Chris Elder's avatar
      FAB-827 Add config option in core.yaml for couchDB · ea1c53cd
      Chris Elder authored
      
      
      Add config option in core.yaml to switch state database from goleveldb to CouchDB
      
      Added the following to core.yaml
      
          # stateDatabase - options are "goleveldb", "CouchDB"
          # goleveldb - default state database stored in goleveldb.
          # CouchDB - store state database in CouchDB
          stateDatabase: goleveldb
          couchDBConfig:
             couchDBAddress: 127.0.0.1:5984
             username:
             password:
      
      Changes were also made to couchdb_txmgr to support the config option.
      
      Changes were made to couchdb_test.go and marble_example.go to support the
      config option in core.yaml and environment variables.
      
      Marbles example can now be run with couchDb with the following command:
      
      CORE_LEDGER_STATE_STATEDATABASE="CouchDB" go run marble_example.go
      
      Adding a declaration for CORE_LEDGER_STATE_STATEDATABASE will work for peer,
      couchdb_test and marbles example.
      
      Change-Id: If02ac4f7c1c234d5f4c869875f3c331e35c9a066
      Signed-off-by: default avatarChris Elder <chris.elder@us.ibm.com>
      ea1c53cd
  18. 11 Nov, 2016 1 commit
  19. 02 Nov, 2016 1 commit
    • denyeart's avatar
      FAB-828 Create couchdb database automatically · 605dcf73
      denyeart authored
      
      
      Create couchdb database automatically for main system ledger upon startup.
      If couchdb database already exists do not re-create.
      There will be one couchdb database for system ledger, and
      eventually one couchdb database for each subledger.
      
      Tests for auto-create are included, as well as some minor
      refactoring and debug enhancements that were required as
      part of the testing.
      
      Change-Id: Ia90ffd4de74346e2472d31eceaf63f24ad1bf04e
      Signed-off-by: default avatardenyeart <enyeart@us.ibm.com>
      605dcf73
  20. 29 Oct, 2016 1 commit
    • denyeart's avatar
      Truncate ledger debug trace for large values · ae7e7e1d
      denyeart authored
      
      
      Some RWSet values are large, for example default chaincode is about 5MB.
      Having 5MB of binary data dumped to debug is not helpful.  This
      change will truncate the RWSets in the debug trace after 2000 characters.
      
      Change-Id: I66de491edb396fc0b8bc95628a1e975151ea7904
      Signed-off-by: default avatardenyeart <enyeart@us.ibm.com>
      ae7e7e1d
  21. 28 Oct, 2016 1 commit
  22. 26 Oct, 2016 2 commits
    • denyeart's avatar
      Initial prototype of CouchDB state DB in ledgernext · c1529a47
      denyeart authored
      https://jira.hyperledger.org/browse/FAB-728
      
      This commit adds a transaction manager (state database) based on CouchDB,
      and a sample client to demonstrate/test it.
      
      KVLedger will continue to use file based block storage
      for the blockchain, but will use CouchDB as the state database for
      simulation and commit.
      
      This experimental feature is enabled via a feature toggle switch in
      the code (useCouchDB).
      
      CouchDB must be already installed separately.
      There is a script to start CouchDB in dev env and download a docker
      image of CouchDB if not already downloaded.  Run this command anywhere
      inside the dev env /fabric:
      "couchdb start"
      
      To switch ledger to use CouchDB,
      update kv_ledger_config.go variable useCouchDB to true.
      In kv_ledger.go NewKVLedger(), you will also need to set the CouchDB connection
      host, port, db name, id, password if using a non-local secured CouchDB.
      
      This initial commit is only a stand alone ledger prototype and not meant
      for end-to-end chaincode processing. That will come in a subsequent commit.
      
      To run the sample:
      In CouchDB http://localhost:5984/_utils create a db named marbles_app.
      You can do this from your host which has port 5984 mapped to guest 5984
      (assuming you have done vagrant up after
      https://gerrit.hyperledger.org/r/#/c/1935/
      
       went in Oct 25th, as that
      changeset open up vagrant port 5984).
      Then run the sample as follows:
      /core/ledgernext/kvledger/marble_example/main$ go run marble_example.go
      
      After running the sample, you can view the marble1 document in CouchDB.
      Be sure to delete it in CouchDB if you'd like to re-run the sample.
      
      Change-Id: Iea4f6ad498dc0e637f0254b6f749060e0298622c
      Signed-off-by: default avatardenyeart <enyeart@us.ibm.com>
      c1529a47
    • Srinivasan Muralidharan's avatar
      FAB-853 remove ledger and move ledgernext to ledger · e9ad121a
      Srinivasan Muralidharan authored
      
      
      FAB-853
      
      Ledger is removed from
         . core/peer/peer.go
         . core/rest/api.go
      
      Ledger package itself is removed. "ledgernext" is
      replaced with "ledger".
      
      Change-Id: Ie7dfbd9bf94afa0031aef759fc46929e4fb3b400
      Signed-off-by: default avatarSrinivasan Muralidharan <muralisr@us.ibm.com>
      e9ad121a
  23. 15 Sep, 2016 1 commit