The deliver service is currently quite ambiguous about the error
condition when the channel returned by Errored() closes.  This is likely
because this message could be returned by a peer or orderer.  However,
in reality, the only time this condition ever occurs is because of a
consenter error.  In the interest of serviceability, this CR makes the
error message more specific, making a note that we should handle
generalizing it if the peer ever implements this function.

Change-Id: I5c2dfb8fa4ba32c34660b69b5facf90291ee17f3
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

This CR adds metrics for streams opened/closed, requests
received/completed, and blocks sent.

FAB-9570 #done

Change-Id: I8ffc8c1107bbe31dd0b1e7f8447e8424363ca14d
Signed-off-by: Will Lahti <wtlahti@us.ibm.com>
Signed-off-by: Matthew Sykes <sykesmat@us.ibm.com>

FAB-12610 #done

Change-Id: If71efa93555adb3445945c0324acfda4cc262974
Signed-off-by: Will Lahti <wtlahti@us.ibm.com>

Change-Id: I1f52050868f9594e5fc4f624f076cfbbf137c07c
Signed-off-by: Hui Kang <kangh@us.ibm.com>

Also update the linter script to disable the lostcancel check until we
can fix them.

Change-Id: Iadeec48b4347f19108c474a241a04a04825594ab
Signed-off-by: Matthew Sykes <sykesmat@us.ibm.com>

Presently, if in the middle of a deliver request, the server will block
indefinitely waiting for block creation or internal error before
returning.  This means that if the client requests blocks forever, then
hangs up, the client resources, including the ledger iterator and the
gRPC stack will not be released until a block is produced.

Change-Id: Ie8d9069ed4ad4dca436e388bebc23012b83eb7e9
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

Send result over chan instead of using shared, named return value.

Change-Id: I3c5edbdbd792f7787a21f6de7f2fde2df5ffd9b5
Signed-off-by: Matthew Sykes <sykesmat@us.ibm.com>

Refactor deliver to use generated mocks and demonstrate
unit testing via ginkgo.

Change-Id: Ib00fec78bc0be9b22f977748185b8c4879d77f0c
Signed-off-by: Matthew Sykes <sykesmat@us.ibm.com>
Signed-off-by: Will Lahti <wtlahti@us.ibm.com>

This commit adds a new service API to handle the deliver request to
obtain filtered blocks while still leveraging existing implementation of
the deliver service which was initially introduced inside
AtomicBroadcast RPC service. This commit takes care to preserve clear
separation of concerns by abstracting out common deliver logic making it
possible to specify delivery content while concentrating on concrete
implementation.

Change-Id: I93c49c8358ff765d50b4acc717907be2182976dd
Signed-off-by: Artem Barger <bartem@il.ibm.com>

This commit takes care to provide more clear abstraction for Deliver API
handler implementation to be able to abstract it and reuse for services
with similar sematics as AtomicBroadcast Deliver.

Change-Id: Ibd436e56934ff4503a87f38142cf6776c1beb6bf
Signed-off-by: Artem Barger <bartem@il.ibm.com>

This change set blocks expired x509 certificates from calling deliver
and also blocks them when they expire.

Change-Id: I146b9e6d1feabe66a756f73f95ab962cf39d3ec5
Signed-off-by: yacovm <yacovm@il.ibm.com>

After FAB-7521, the peer deliver service is unusable because the
BLOCKEVENT policy is not set by default. This CR uses the aclmgmt
package, which will check for the policy and, if not set, use the
default value (in this case, channel readers). It also restores the
behave tests to their previous state to ensure peer deliver remains
usable by default.

Change-Id: I46e71853881271539e28a110ce8b81d3bd248d19
Signed-off-by: Will Lahti <wtlahti@us.ibm.com>

While reusing deliverer API as a blocks event source at peer we need to
lookup for correct policy name based on the RSCC resource API definition
name. This commit adds a factory method which introduces level of
indirection to allow lookup of correct policy name based on the deliver
API initialization handler.

Change-Id: Ib896736793722549f035cca9e0b6a4c871050615
Signed-off-by: Artem Barger <bartem@il.ibm.com>

This CR adds validation checks for the timestamp (all envelopes) and
the TLSCertHash (only if mutual TLS is enabled) of incoming Deliver
envelopes.

Change-Id: I9359ee0ed5bf9057e91948951c59a2196c484afc
Signed-off-by: Will Lahti <wtlahti@us.ibm.com>

This CR updates the deliver functionality to facilitate its usage on
a peer as well as an orderer.

This required:
- modifying the signal logic for when a new block is available due to
 the difference in addition of blocks to the ledger between the orderer
 and the peer. The signal logic is now handled using the iterator
 itself, which signals when it finds a new block
- adding a policy variable to the deliver handler to ensure the peer
and orderer each can control access to deliver

Change-Id: Iebb6c25a8c5ac32d65f909eb0519f26bfde0dc31
Signed-off-by: Will Lahti <wtlahti@us.ibm.com>

This CR moves deliver from the orderer to fabric/common. This is being
done to support the future implementation of deliver on the peer.

Change-Id: If077f2c05b5a10fdeb4e6ac315111495304e4c5e
Signed-off-by: Will Lahti <wtlahti@us.ibm.com>

The orderer/common/util only contains 1 file (net.go)
that contains only 1 function (ExtractRemoteAddress).

This function can be used by any gRPC service, and therefore
its place should be in common/util.

Change-Id: Ie0fc7890a9091337ba4f85f113de9807e346da34
Signed-off-by: yacovm <yacovm@il.ibm.com>

The policy manager used to be stateful, and it was safe to retain a
reference to it.  However, this added considerable complexity and
introduced the possibility for subtle races so it was turned into an
immutable structure.

The orderer signature filter was not updated to pull a fresh reference
on each invocation, but instead still retains only a reference to the
original policy manager.  This CR updates the signature filter to fetch
a fresh reference upon each invocation.

Change-Id: Ibaf8a18cc992feaf737047c87ce8973450aa1d19
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

See JIRA for more info.

Change-Id: I4dd0d48b7548820fa096a1c4f4f72ad9dcdb48ee
Signed-off-by: Kostas Christidis <kostas@christidis.io>

This commit adds logging of client ip address to the orderer,
for enhanced user experience when reading logs.

Change-Id: I5704f3af008f7c493ca27e92d87a838bc3c8b1e2
Signed-off-by: yacovm <yacovm@il.ibm.com>

The individual filters are currently in subpackages of the msgprocessor
package.  However, the filters are simple enough this is probably no
longer warranted.

Additionally, in order to return more sane status codes, it's necessary
to have shared error definitions, which is difficult to do without
import cycles with the sub-package structure.

This CR moves the individual filter code to msgprocessor.

Change-Id: I6617f9b053345446d4636e1b62b7daf2280dee42
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

The filters package has been changed significantly over the course of
development, and especially so with the recent removal of the committers
concept.  This has left a lot of cruft which actually makes the package
less usable and less testable.

This CR removes all that cruft, resulting in a significant decrease in
lines of code, while improving the quality of the tests.

Change-Id: I3dc131b5775bde3b198b6f33438a076041a38ed0
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

The only consumer of the filters framework is now the msgprocessor
package.

This CR removes all of the filter references from multichannel, and
moves the filter code from fabric/orderer/common to
fabric/orderer/common/msgprocessor.

Change-Id: I73a562887776086c62212167c71bd72c8894b286
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

The filter committers are no longer needed, as the blockcutter no longer
returns them.

This CR moves the small bit of remaining logic from the filter
committers into chainsupport.go and then removes the entire concept of
the filter.Committer from the codebase.

Change-Id: I4471ebe4b26c17669afc53fcef30b4a393276f14
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

The orderer directory has been overdue for being organized for some time
now.

With the change series of FAB-5258 there is a significant amount of code
being rearranged already, so it makes sense to organize the directory at
the same time, as the imports etc. will already be changing.

The new structure is very simple:

fabric/orderer - Contains the startup/main code

fabric/orderer/consensus - Contains the individual consensus plugins for
the orderer - ie Kafka/Solo, and eventually SBFT

fabric/orderer/common - Contains the common code which feeds
transactions to the consensus plugin, and satisfies the orderer
atomicbroadcast API definition

Change-Id: I112044dd5ade212378dab6c176265b76eb68ca80
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

- Updated the orderer's file ledger to use it's underlying
  blockstore ledger impl's iterator APIs.
- Blockstore iterators must be explicitly closed to avoid
  leaking resources, so now orderer/ledger.Iterator must
  also be explicitly closed. Added Close() to the
  orderer/ledger.Iterator interface.

Change-Id: Id838a661a11bf5b64a0cbb57d75a27d69d251269
Signed-off-by: Luis Sanchez <sanchezl@us.ibm.com>

This changesets prepends the channel name to logging statements in the
deliver and broadcast packages in order to make log parsing when
debugging easier.

Change-Id: Iab5485d9a286830699d44a60a6d8705e32cdaecd
Signed-off-by: Kostas Christidis <kostas@christidis.io>

If a client of the Deliver interface requests to seek until max_uint64,
then this client will stay connected receiving blocks indefinitely.

If that client's access is revoked, either by removing the organization
from the channel, or the client's certificate is revoked, then this
client is not disconnected, and will continue to receive blocks
indefinitely.

This CR causes the Deliver code to check the configuration sequence
number before each block is delivered.  If the configuration sequence
has been incremented, then the client's authorization is checked once
more and ejected if the new configuration does not permit the client's
request.

Change-Id: Ie852a9a8d435917ef1e7dce2025122c791fc9248
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

The orderer multichain consenter API currently does not provide a
mechanism to provide feedback for a channel when the consenter is out of
sync or is otherwise not producing data for that channel.

This CR adds an Errored() method to the common consenter Chain
interface, and consumes that API via the Deliver common code.

It provides only a skeleton implementation for the consenter
implementatinos (Kafka and Solo).  The Kafka implementation will provide
a more robust implementation of the API in the near future.

Change-Id: I22863f2a8b37932a5ecbd5241372e2a5c512571d
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

This patch fixes several issues in error validation of deliver:
1. EOF is now caught as normal hangup of gRPC connection, hence
exit without returning error.
2. Error of inverse seek position was not caught, which results
in undesired behavior. Now it returns BAD_REQUEST status.
3. Some errors were not logged.
4. Explicit logging level checks are removed, i.e. `isEnabledFor`
This was intended to improve the performance by reducing memory
allocations caused by passing in string arguments. However, the
performance gain is not significant (~ns) and we prefer cleaner
code.

On the other hand, common logic in tests are extracted into a
reusable function.

Change-Id: Ib2ff52cd1d9ef767f0918728084ec31c075cc38a
Signed-off-by: Jay Guo <guojiannan1101@gmail.com>

Change-Id: If8fd43e256bfcf6a8d3f21a3262707ccc5e3c134
Signed-off-by: Kostas Christidis <kostas@christidis.io>

https://jira.hyperledger.org/browse/FAB-2691



The orderer replies with HTTP status codes with no error text.  Although
in some cases this is sufficient, especially in development scenarios,
the log does not provide enough information for the rejection in order
to diagnose the root cause of the bad status.

This CR enhances the logging of both Deliver and Broadcast to always log
a verbose error if logging is enabled at debug for why a client request
is being rejected.

Change-Id: I4150a5c43a101d0a3d727cee1bde1aeeb8b8ada2
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

https://jira.hyperledger.org/browse/FAB-2615



Now that the hierarchical config and policies are in place, the old
style of specifying ingress and egress policies is wrong.

This CR switches the orderer to use the ChannelReaders and
ChannelWriters policies instead of the Ingress/Egress policy names and
removes the references to those now removed config policies.

Change-Id: Ie818bfc37bb5d2b4eb55addaf76ac510a2fdfd4b
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

https://jira.hyperledger.org/browse/FAB-2581



A set of if blocks end with return statements, meaning there's no reason
for a corresponding else block. This changeset addresses that.

Change-Id: I8a97bb3c232f6c174241bcc206e070fd5a9c2697
Signed-off-by: Kostas Christidis <kostas@christidis.io>

https://jira.hyperledger.org/browse/FAB-2579



This changeset:

1. Removes the stuttering that was prevalent in filenames and package
names throughout the package
2. Updates function definitions where appropriate
3. Makes the layout of "ram" and "json" implementation consistent with
the "file" one by moving the relevant functions to `factory.go`
4. Switches all panic calls to their logger-equivalents. (We will be
doing this for all packages inside `orderer`.)
5. Per Binh's request: creates paths via the `filepath.Join` call

Change-Id: I918519cf66b3287864560d1fa2e6734b6953d657
Signed-off-by: Kostas Christidis <kostas@christidis.io>

This change set ensures that the protobuf representation for headers be in
bytes. This makes sure that if ever protobuf marshalling is non-deterministic,
we do not have problems because whenever we hash a header, we do that over
the serialized version we receive.

Change-Id: I838e0d5dec2f79f88fab63d92bdfb51d92c2f069
Signed-off-by: Alessandro Sorniotti <ale.linux@sopit.net>

https://jira.hyperledger.org/browse/FAB-2321



The channel config values handling has been moved around a lot, this is
the first in a series of CRs designed to cleanup the style of the config
packages.

Change-Id: I6456516fe182e1e37bcd9585ebff64f314941a63
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

https://jira.hyperledger.org/browse/FAB-2312



There used to be a single type for configuration, 'ConfigItem', but that
has bifurcated into 'ConfigValue', 'ConfigGroup', and 'ConfigPolicy'.
In order to handle the 'ConfigValue' and the 'ConfigPolicy' in similar
ways, the 'ConfigValue' handling needs to be pushed out into its own
package just like policy handling is.  This CR accomplishes that.

This is in preparation for treating the PolicyHandler as more of a first
class object and to eliminate the need for hacky importing.

Change-Id: I690a33e09843eb4c29611319f2e3942baae2dfdb
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

https://jira.hyperledger.org/browse/FAB-2033



The user facing documentation refers to channels, but the vast majority
of the fabric code refers to chains.  This CR switches the protos to
refer to channels as they are generally the interface between the user
and fabric code.

Change-Id: I869e6d68c1228e638a99e2460274889f18f2543d
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>

https://jira.hyperledger.org/browse/FAB-2030

According to the proto style guide, fields should be lower_underscored
which generally converts nicely to golang as LowerUnderscored.  However,
for certain cases, like chaincode_id, this converts to ChaincodeId
rather than as desired to ChaincodeID.  This change updates the
problematic fields to use the correct proto style, and updates the
golang to expect the appropriate (although odd) output.

Note, per the proto devs, https://github.com/golang/protobuf/issues/53
there is no intent to modify the protoc generation to attempt to produce
upper case ID API, etc.

Still, the odd golang results seem worth following the style guide.

Note, this CR also changes chainID to channel_id as these lines were
already being affected and a community apparently came to a decision
that the world channel is preferable to the word chain.

https://jira.hyperledger.org/browse/FAB-2033



Change-Id: I9c5ebab88fc8d7f45335c904a56676c55a467e5d
Signed-off-by: Jason Yellick <jyellick@us.ibm.com>