This CR modifies the tranaction simulation, validation, and commit
code and delivers the end-to-end transaction flow that treats the
private data in a special manner. This CR mainly leverages the earlier
submitted independent CRs for sidedb feature for accomplishing this behavior.

This CR also allows ledger to receive the blocks and the pvt data from
another peer on the same channel (i.e., a peer catching up via state)

This CR is exceptionally large becasue of manily two reasons

1) The way currently the code (and specially the tests) is organized in
simulation/validation/commit flow, its not easy to submit such kind
of changes independently that cuase the change in the whole transaction
processing flow.

2) This CR causes a change in the existing ledger APIs which are used widely
across other packages (specially in the tests) and hence many files are included
for fixing the broken dependencies

Change-Id: Id29575176575f4c01793efd3476b68f8364cb592
Signed-off-by: manish <manish.sethi@gmail.com>

This CR enhance the read-write set builder for capturing the
additional information during simulation of a transaction

In addition to the public read-erite set for channel scoped
data, this CR allows to capture the hashed read-write set and
private read-write set for the private data.

Change-Id: I438692d7af0255d589adf8f21cda51a3f00d7418
Signed-off-by: manish <manish.sethi@gmail.com>

This CR introduces ledger storage which maintains consistency
in the underlying block storage and pvtdata store by ensuring
an atomic operaton for writing blocks (to block storage) and
writing the pvtdata (to pvtdata store)

Change-Id: Ifb21f14d401eb10233db72c1473ba56f44bc119d
Signed-off-by: manish <manish.sethi@gmail.com>

This CR implements a store for persisting the writesets produced
over the private data. From data perspective, this storage is
analogous to the block storage for the block data

Change-Id: I43b5349d3671bffa67f7975794e6f1937f99dde5
Signed-off-by: manish <manish.sethi@gmail.com>

TestLeaderYield fails sporadically.
Disabling it and will investigate offline

Change-Id: I1d59ce5d4826500998aeaf5766f1926f0727f31b
Signed-off-by: yacovm <yacovm@il.ibm.com>

The deliveryservice/client.go stores retry logic that
plays a role when the ordering service node is unreachable or malfunctioning.

It lacks logging, and thus makes it hard to analyze
postmortem and live situations.

Change-Id: I7f3a3d52c5fc67bed400f0007aebdb5e61419ec9
Signed-off-by: yacovm <yacovm@il.ibm.com>

This CR provides support for maintaining public, private, and hashed data
This includes
   - An interface for managing the three categories of data

   - A default implementation that allows the use of either
     leveldb or couchdb. The default implementation uses a
     single logical db and uses different namespaces for
     different categories of data. Alternate implementation
     based on furture need or exploration should be easy to support.
     These may include using different dbs for different categories
     such as using leveldb for hashed data and using separate dbs in
     a couch instance for public and private data

   - If the underlying db does not support storing random bytes as key
     (for example couch supports onlu valid utf-8 bytes as key), the
     key for the hashed data is encoded using base64

Change-Id: Ia8ede4f4c0ab392119e59bb7f46e9c20062a411a
Signed-off-by: manish <manish.sethi@gmail.com>

For v1, configtxgen used some hacks into the configtx core code to get a
JSON representation of the config.  This CR switches the JSON encoding
to use the same protolator helpers as the configtxlator code.

The existing method was (as already noted) hacky, and not as expressive
as the configtxlator method (which also expands other opaque fields like
MSP principals and MSP definitions.  Plus, it is unnecessary code to
maintain.

Change-Id: Ibfe79de834bae6d91e7d67a9f528564658650902
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

The configtxgen code is currently under fabric/common/configtx/tool
which is really a poor location for it.

For consistency with the other tool components, it should be moved to
common/tools/configtxgen.  This CR does that.

Change-Id: I72a6060c119a906343971fcb10a4dc53e557de85
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

* changes:
  [FAB-5552] Fix some TODOs in msgprocessor
  [FAB-5445] Consolidate filters into msgprocessor

This CR adds a transientstore to peer for temporarily holding the
private write set of a transaction. Tranisent store will be used by
endorser, committer, and gossip middleware. Currently, transientstore
exposes following five APIs.

1. Persist(txid string, endorserid string, endorsementBlkHt uint64,
        privateSimulationResults []byte)
2. GetTxPvtRWSetByTxid(txid string)
3. GetSelfSimulatedTxPvtRWSetByTxid(txid string)
4. Purge(maxBlockNumToRetain uint64)
5. GetMinEndorsementBlkHt()

Endorser: After simulating a transaction, endorsers will
collect transaction simulation results (public rwset, hashed rwset, and
private wset) from ledger and store the private write set in
transientstore using Persist().

Gossip: When gossip receives a private write set from other peers,
it will store the write set in transientstore using Persist().
When gossip receives a request from other peers for a private write set
associated with a txid, it will retrieve the write set from the transientStore
(using GetSelfSimulatedTxPvtRWSetByTxid()/GetTxPvtRWSetByTxid())

Committer: When a committer receives a block from gossip for commit,
after performing VSCC validation, committer will fetch relevant private
write sets from the transientStore (using GetSelfSimulatedTxPvtRWSet()/
GetTxPrivateRWSetByTxid()) and pass it to ledger along with the block.

In the subsequent CRs, transientstore will be integrated with
endorser and committer components. Further, endorserid needs to
be defined.

Change-Id: I957120af6c091c4d5c45c9e06b29025a2481bf49
Signed-off-by: senthil <cendhu@gmail.com>

This is a simple one line fix to expose the info message returned along
with the status code when the peer CLI makes broadcast calls (such as
for creating or updating channels).

Change-Id: I22c458b76259201615a8581ecea8a63f00cad201
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

The restructuring of the msgprocessor code has left some outstanding
TODOs.  In particular, for conditions where a CONFIG_UPDATE is rewrapped
into a containing message, like a CONFIG or an ORDERER_TRANSACTION, this
wrapping message could end up violating rules for the channel, like the
maximum message size.  This could see the message silently dropped by
the consenter (silent from a client perspective).

This CR adds a second pass of filtering in the message flow which
generates a wrapping message.

Also fixes a small comment bug from a previous CR.

Change-Id: Ic482a8195720992c68e9986a1ff7f34004d5986c
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

There is a missing array bounds check that may cause the peer to crash.
Stack trace in the JIRA item.

Change-Id: Ie018f9160aeb2eef4e9075282e15271250b25e17
Signed-off-by: yacovm <yacovm@il.ibm.com>

This commit adds an authentication layer to the chaincode service
that inspects the first REGISTER message from the shim, and based
on the passed chaincode name - authorizes the stream or rejects it.

The full details of the flow can be seen in the JIRA item.

Change-Id: I4a1b3c9b3b078d96906f3c9618520b9fe319eeb6
Signed-off-by: yacovm <yacovm@il.ibm.com>

This commit adds the needed infrastructure to implement an interceptor
that would wrap the existing chaincode service (chaincode Support struct)
and would check that the chaincode name in the REGISTER message matches
the mapping that was set at the chaincode launch.

The full details of the flow is in the JIRA item.

Change-Id: I2a8d33fc33adf845984b73e3ab1010c34914c716
Signed-off-by: yacovm <yacovm@il.ibm.com>

Having an issue instantiating chaincode which vendors
in go packages which include .h and .s files. Example:
https://github.com/golang/crypto/tree/master/curve25519



Install works fine, however when i instantiate I get the following:
2017-08-02 15:26:27.825 UTC [dockercontroller] deployImage ->
ERRO 045 Error building images: Failed to generate
platform-specific docker build: Error returned from build:
...
... testcc/vendor/golang.org/x/crypto/curve25519/mont25519_amd64.go:15:
 missing function body for "cswap"
...

Change-Id: I3adf39440a343e61718f0c519d559c3463d92cd2
Signed-off-by: gbolo <george.bolo@gmail.com>

For messages which fail signature verification, the orderer currently
returns BAD_REQUEST, as it is registered simply as an error in
processing the message.  However, this is very poor from a
serviceability perspective, and a more precise status should be returned
when possible.

This CR creates a new error type in the message processor which may be
returned for signature validation failures.

This CR also allows the broadcast error processing to use the errors
package to retrieve the cause of an error, so that additional
descriptive information beyond the basic error definition may be
included as well.

Change-Id: I7c02ca1b456bb4c052492b7979c0e75954f2b75a
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

The individual filters are currently in subpackages of the msgprocessor
package.  However, the filters are simple enough this is probably no
longer warranted.

Additionally, in order to return more sane status codes, it's necessary
to have shared error definitions, which is difficult to do without
import cycles with the sub-package structure.

This CR moves the individual filter code to msgprocessor.

Change-Id: I6617f9b053345446d4636e1b62b7daf2280dee42
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

The configtx code maintains a map of the current config, as derived from
the Config proto structure.  This map stores references to a cached
Config proto structure which is used when constructing the next Config
structure.

The problem arises when this map is used to construct a new Config to be
applied, that it mutates the cached version of of the Config.  This is
generally fine, so long as the new Config applies successfully, but in
the event of bad inputs, such as a bad certificate, the config update
fails to apply and is rolled back, but the cache has been mutated and
will not be rolled back with it.

The observed issue occurs because this Config cache is also used in
creating the new channel config template.  So, because there is a bad
certificate in the config cache, the new channel template attempts to
bootstrap using the bad key material, detects the error, and aborts.

As noted in the issue, restarting the orderer rebuilds this cache, and
channel creation can occur normally once more.

This CR fixes the code which constructs a new Config from the config map
to create a copy of the cached config in-process, rather than taint the
cache with potentially invalid data.

Note, there may be novel ways to corrupt this cache which could cause
other undesirable behavior.  However, prior to the operation which
mutates the cache, the config update has been validated to adheer to the
security constraints of the channel (including all necessary admin
signatures), so it requires in a sense, a conspiracy of channel
administrators attempting to corrupt their own channel, so the security
implications are limited or non-existant.

Change-Id: I44ef3b50c6716c3ae38df4521d860ead595e16fa
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

This CR enhances the proto message for supporting the
hashed read-write set and private read-write set for the
private data. To summarize - under a namespace (chaincodeid),
there could be one or more sub-spaces (referred to as collections)
for managing private state. The hashed read-write set is intented
be part of the block and the private read-write set is intented to be
transferred to the need-to-have peers only.

Change-Id: I2f5127f93c945ab6559ed625f677d467e67bbcda
Signed-off-by: manish <manish.sethi@gmail.com>

For hard to debug problems, actually having access to the bytes of a
message can be the only definitive way to diagnose.

This CR adds a debug configuration section, and allows for setting a
directory to log all Broadcast messages, as well as a directory to log
all Deliver messages.

Although there is no support for dynamically changing debugging
parameters at this time, the code deliberately retrieves the debug
parameters from the debug struct at every instance to allow for dynamic
control of the debugging.

Change-Id: Ib046f263dc95b374b5883af66397a9d29049ffef
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

* changes:
  [FAB-5053] Log txid for endorser txs at orderer
  [FAB-5360] Populate BroadcastResponse info field
  [FAB-5359] Add BroadcastResponse info to proto
  [FAB-5335] Move NewChannelConfig to msgprocessor