Rename CAP-0083 + add meta example (#1940)

* Rename CAP-0083 + add meta example

This change renames `STELLAR_VALUE_SKIP` to `STELLAR_VALUE_EMPTY_TX_SET`
based on feedback on the name "skip" being misleading. It also adds
example `LedgerCloseData` meta.

* Fix a few missed "skip" usages that copilot found

Author: bboston7

core/README.md

@@ -95,7 +95,7 @@
 | [CAP-0080](cap-0080.md) | 26 | Host functions for efficient ZK BN254 use cases | Siddharth Suresh | Implemented |
 | [CAP-0081](cap-0081.md) | TBD | TTL-Ordered Eviction | Garand Tyson | Accepted |
 | [CAP-0082](cap-0082.md) | 26 | Checked 256-bit integer arithmetic host functions | Jay Geng | Implemented |
-| [CAP-0083](cap-0083.md) | TBD | Allow validators to vote to skip the current ledger | Brett Boston | Awaiting Decision |
+| [CAP-0083](cap-0083.md) | TBD | Allow validators to vote to drop the transaction set from the current ledger | Brett Boston | Awaiting Decision |
 
 ### Draft Proposals
 | Number | Title | Author | Status |

core/cap-0083.md

@@ -2,7 +2,7 @@
 
 ```
 CAP: 0083
-Title: Allow validators to vote to skip the current ledger
+Title: Allow validators to vote to drop the transaction set from the current ledger
 Working Group:
     Owner: Brett Boston <@bboston7>
     Authors: Brett Boston <@bboston7>
@@ -16,26 +16,26 @@ Protocol version: TBD
 ## Simple Summary
 
 This CAP introduces a new `StellarValue` type to allow validators to explicitly
-skip the ledger that is being voted on. It also relaxes validation criteria on
-PREPARE messages.
+drop the transaction set from the ledger that is being voted on. It also relaxes
+validation criteria on PREPARE messages.
 
 ## Working Group
 
 As specified in the preamble.
 
 ## Motivation
 
-Providing a mechanism for validators to explicitly vote to skip ledgers allows
-them to conduct some SCP voting prior to receiving a transaction set. This will
-improve SCP performance, as transaction set dissemination is costly and
-validators can safely make initial progress on consensus while waiting for
+Providing a mechanism for validators to explicitly vote to drop transaction sets
+allows them to conduct some SCP voting prior to receiving a transaction set.
+This will improve SCP performance, as transaction set dissemination is costly
+and validators can safely make initial progress on consensus while waiting for
 transaction sets.
 
 If, during the PREPARE phase of balloting, a validator has not received the
 transaction set being voted on within a reasonable amount of time, or the
-transaction set is invalid, they may vote to skip the current ledger. This
-prevents consensus from getting stuck waiting for a transaction set that may
-never arrive, as a valid transaction set is required before moving on to the
+transaction set is invalid, they may vote to drop it from the current ledger.
+This prevents consensus from getting stuck waiting for a transaction set that
+may never arrive, as a valid transaction set is required before moving on to the
 CONFIRM phase of balloting.
 
 Relaxing validity checks on PREPARE messages is necessary to enable early
@@ -50,11 +50,12 @@ This CAP is aligned with the following Stellar Network Goal:
 
 ## Abstract
 
-This CAP introduces a new `StellarValue` extension type `STELLAR_VALUE_SKIP`.
-With this change, validators may switch their ballot to a `STELLAR_VALUE_SKIP`
-during the PREPARE phase of balloting prior to setting `c`[^1].
-`STELLAR_VALUE_SKIP` includes information about the value that a validator was
-considering before switching to voting to skip the ledger.
+This CAP introduces a new `StellarValue` extension type
+`STELLAR_VALUE_EMPTY_TX_SET`.  With this change, validators may switch their
+ballot to a `STELLAR_VALUE_EMPTY_TX_SET` during the PREPARE phase of balloting
+prior to setting `c`[^1].  `STELLAR_VALUE_EMPTY_TX_SET` includes information
+about the value that a validator was considering before switching to voting to
+drop the transaction set.
 
 This CAP also relaxes the validation of PREPARE messages to permit invalid
 transaction sets. This change ensures the network does not drop ballots from
@@ -65,7 +66,8 @@ early balloting stages.
 
 ### XDR changes
 
-To enable ledger skipping, this CAP proposes the following change to
+To enable explicitly empty transaction sets, this CAP proposes the following
+change to
 `Stellar-ledger.x`:
 
 ```diff mddiffcheck.base=0a621ec7811db000a60efae5b35f78dee3aa2533
@@ -79,15 +81,15 @@ index a17036b..667f8e7 100644
      STELLAR_VALUE_BASIC = 0,
 -    STELLAR_VALUE_SIGNED = 1
 +    STELLAR_VALUE_SIGNED = 1,
-+    STELLAR_VALUE_SKIP = 2
++    STELLAR_VALUE_EMPTY_TX_SET = 2
  };
 
  struct LedgerCloseValueSignature
 @@ -43,6 +44,14 @@ struct StellarValue
          void;
      case STELLAR_VALUE_SIGNED:
          LedgerCloseValueSignature lcValueSignature;
-+    case STELLAR_VALUE_SKIP:
++    case STELLAR_VALUE_EMPTY_TX_SET:
 +        struct
 +        {
 +            Hash txSetHash;
@@ -124,7 +126,7 @@ struct StellarValue
         void;
     case STELLAR_VALUE_SIGNED:
         LedgerCloseValueSignature lcValueSignature;
-    case STELLAR_VALUE_SKIP:
+    case STELLAR_VALUE_EMPTY_TX_SET:
         struct
         {
             Hash txSetHash;
@@ -139,29 +141,112 @@ struct StellarValue
 
 
 #### XDR Example
-TBD
+The stellar-core repo contains [an example `LedgerCloseData` meta file demonstrating the application of a `STELLAR_VALUE_EMPTY_TX_SET` ledger](https://github.com/stellar/stellar-core/blob/master/test-lcm/HerderTests/370dd5e0af5cfa8d.xdr).
+For readability, this CAP provides a JSON-ified version of this XDR below.
+```json
+[
+    {
+        "v": 2,
+        "v2": {
+            "ext": {
+                "v": 0
+            },
+            "ledgerHeader": {
+                "hash": "797ed2ec897cb6faac1b103cc6f027b5af5075ddf39039e973555d71613a1851",
+                "header": {
+                    "ledgerVersion": 27,
+                    "previousLedgerHash": "2585cbeb05e12e8112f89e78d91e6c8c5a75fc08b91d7539773a48a8b6a35b3a",
+                    "scpValue": {
+                        "txSetHash": "0000000000000000000000000000000000000000000000000000000000000000",
+                        "closeTime": 15,
+                        "upgrades": [],
+                        "ext": {
+                            "v": "STELLAR_VALUE_EMPTY_TX_SET",
+                            "proposedValue": {
+                                "txSetHash": "827145925ccb74c257c14372b8325ccb9f4af0099e57025e777a6e2cb365acaf",
+                                "previousLedgerHash": "2585cbeb05e12e8112f89e78d91e6c8c5a75fc08b91d7539773a48a8b6a35b3a",
+                                "previousLedgerVersion": 27,
+                                "lcValueSignature": {
+                                    "nodeID": "GAXSR3PPLX3TDFUXXQ2GGOCQISJKADKKFAJ72PEEYKA3DV6MJ6FCG4O6",
+                                    "signature": "05aadb43376fa6544048fb646212b987f9aec110a65c2e9d02806c9dca51d7ed9286d71e68f68a13b0af7efd5b95b6b928d0d8f4c7ed03a327786fbd1a287c03"
+                                }
+                            }
+                        }
+                    },
+                    "txSetResultHash": "df3f619804a92fdb4057192dc43dd748ea778adc52bc498ce80524c014b81119",
+                    "bucketListHash": "ff0815329f0269bcfdb3385fb751f1db076c88da3028f9733d302d564565b551",
+                    "ledgerSeq": 5,
+                    "totalCoins": 1000000000000000000,
+                    "feePool": 0,
+                    "inflationSeq": 0,
+                    "idPool": 0,
+                    "baseFee": 100,
+                    "baseReserve": 100000000,
+                    "maxTxSetSize": 50,
+                    "skipList": [
+                        "0000000000000000000000000000000000000000000000000000000000000000",
+                        "0000000000000000000000000000000000000000000000000000000000000000",
+                        "0000000000000000000000000000000000000000000000000000000000000000",
+                        "0000000000000000000000000000000000000000000000000000000000000000"
+                    ],
+                    "ext": {
+                        "v": 0
+                    }
+                },
+                "ext": {
+                    "v": 0
+                }
+            },
+            "txSet": {
+                "v": 1,
+                "v1TxSet": {
+                    "previousLedgerHash": "2585cbeb05e12e8112f89e78d91e6c8c5a75fc08b91d7539773a48a8b6a35b3a",
+                    "phases": [
+                        {
+                            "v": 0,
+                            "v0Components": []
+                        },
+                        {
+                            "v": 1,
+                            "parallelTxsComponent": {
+                                "baseFee": null,
+                                "executionStages": []
+                            }
+                        }
+                    ]
+                }
+            },
+            "txProcessing": [],
+            "upgradesProcessing": [],
+            "scpInfo": [],
+            "totalByteSizeOfLiveSorobanState": 0,
+            "evictedKeys": []
+        }
+    }
+]
+```
 
-### Skip Value Semantics
+### Empty Transaction Set Value Semantics
 
-#### Skip Value Representation
+#### Empty Transaction Set Value Representation
 
-When the network votes to skip the ledger, validators will externalize a `StellarValue` with fields set as follows:
+When the network votes to drop a transaction set, validators will externalize a `StellarValue` with fields set as follows:
 
 * `txSetHash` : Set to `0x0`
 * `closeTime` : Set as usual
 * `upgrades` : Set as usual
-* `ext` : Set to `STELLAR_VALUE_SKIP`
-    * `proposedValue` : Details about the original `StellarValue` that a quorum of validators decided to skip.
+* `ext` : Set to `STELLAR_VALUE_EMPTY_TX_SET`
+    * `proposedValue` : Details about the original `StellarValue` that a quorum of validators decided to drop the transaction set from.
         * `proposedValue.txSetHash` : The hash of the transaction set that was insufficiently disseminated or invalid.
-        * `proposedValue.previousLedgerHash` : The hash of the ledger prior to this one. Used to construct an empty transaction set corresponding to this skip value.
-        * `proposedValue.previousLedgerVersion` : The ledger version of the previous ledger. Used to construct an empty transaction set corresponding to this skip value.
+        * `proposedValue.previousLedgerHash` : The hash of the ledger prior to this one. Used to construct an empty transaction set corresponding to this empty-tx-set value.
+        * `proposedValue.previousLedgerVersion` : The ledger version of the previous ledger. Used to construct an empty transaction set corresponding to this empty-tx-set value.
         * `proposedValue.lcValueSignature` : The signature from the original `StellarValue`. Can be used to determine which validator proposed the original `StellarValue`.
 
-#### Skip Value Application
+#### Empty Transaction Set Value Application
 
-At apply time, ledgers containing a skip value should be treated as ledgers with
-empty transaction sets. In every other way, they should be treated as normal
-`STELLAR_VALUE_SIGNED` ledgers.
+At apply time, ledgers containing a `STELLAR_VALUE_EMPTY_TX_SET` value should be
+treated as any other ledger with an empty transaction set. In every other way,
+they should be treated as normal `STELLAR_VALUE_SIGNED` ledgers.
 
 ### PREPARE Message Validation
 
@@ -177,20 +262,20 @@ ballots for `STELLAR_VALUE_SIGNED` values with invalid transaction sets.
 
 We considered a few alternative methods to achieve parallel downloading of transaction sets before landing on this protocol change:
 
-* Rather than skipping a ledger on failure to download a transaction set, we looked into rolling back SCP to an earlier phase. However, this would be a large change to SCP, which was not designed to ever step backwards. We decided it was cleaner, and easier to verify the change if validators instead voted to skip the current ledger.
-* We analyzed how far along consensus a validator could safely proceed without a transaction set if there was no skip or rollback mechanism, and decided that the lack of such a mechanism was too limiting. Without some method for validators to agree that a transaction set is unavailable, it is only safe to proceed with early nomination voting.
+* Rather than dropping a transaction set on failure to download a transaction set, we looked into rolling back SCP to an earlier phase. However, this would be a large change to SCP, which was not designed to ever step backwards. We decided it was cleaner, and easier to verify the change if validators instead voted to drop the transaction set from the current ledger.
+* We analyzed how far along consensus a validator could safely proceed without a transaction set if there was no drop or rollback mechanism, and decided that the lack of such a mechanism was too limiting. Without some method for validators to agree that a transaction set is unavailable, it is only safe to proceed with early nomination voting.
 
 ## Protocol Upgrade Transition
 
-`STELLAR_VALUE_SKIP` values will become valid on the network at the protocol boundary. However, validators will not generate them unless they have enabled parallel transaction set downloading.  Parallel transaction set downloading will initially be disabled by default, and we would like to gradually enable it across the validator network.
+`STELLAR_VALUE_EMPTY_TX_SET` values will become valid on the network at the protocol boundary. However, validators will not generate them unless they have enabled parallel transaction set downloading.  Parallel transaction set downloading will initially be disabled by default, and we would like to gradually enable it across the validator network.
 
 Validation of PREPARE messages will be relaxed at the protocol boundary.
 
 ### Downstream Impact
 
-Downstream consumers of `StellarValue`s may require modification. Specifically, those that reason about `txSetHash` will need to handle the new `0x0` value, and those that reason about `ext` will need to handle the new `STELLAR_VALUE_SKIP` extension.
+Downstream consumers of `StellarValue`s may require modification. Specifically, those that reason about `txSetHash` will need to handle the new `0x0` value, and those that reason about `ext` will need to handle the new `STELLAR_VALUE_EMPTY_TX_SET` extension.
 
-To help determine if you depend on something that contains a `StellarValue`, transitive dependencies on `StellarValue` are as follows:
+To help determine if you depend on something that contains a `StellarValue`, transitive dependencies on `StellarValue` (outside of the SCP messages already discussed in this CAP) are as follows:
 ```
 StellarValue
     │
@@ -209,7 +294,7 @@ StellarValue
     └──► StoredDebugTransactionSet.scpValue
 ```
 
-Downstream consumers of `StellarValue`s should treat skipped ledgers as empty ledgers.
+Downstream consumers of `StellarValue`s should treat ledgers containing `STELLAR_VALUE_EMPTY_TX_SET` as empty ledgers.
 
 ### Backwards Incompatibilities
 As detailed in the previous section, downstream consumers of `StellarValue`s may require modification.
@@ -218,15 +303,17 @@ As detailed in the previous section, downstream consumers of `StellarValue`s may
 We expect this change to increase throughput on validators by expediting consensus. Simulation has shown this to be manageable. However, we will rollout the parallel downloading portion of this change slowly to ensure that real-world performance data matches simulation.
 
 ## Security Concerns
-This CAP introduces a new method by which malicious validators could negatively impact the network. A malicious validator could introduce bad transaction set hashes, which then force the network to skip a ledger after some delay. However, this attack is mitigated by the following factors:
+This CAP introduces a new method by which malicious validators could negatively impact the network. A malicious validator could introduce bad transaction set hashes, which then force the network to externalize an empty ledger after some delay. However, this attack is mitigated by the following factors:
 
 * The attacker would have to win leader election to propose the bad value. This limits the frequency with which this attack can be pulled off.
-* By design, we include the original value that was skipped in the `STELLAR_VALUE_SKIP` struct, including the signature for the value. This enables node operators to easily determine which validators are misbehaving and remove them from their quorum sets.
+* By design, we include the original value that was dropped in the `STELLAR_VALUE_EMPTY_TX_SET` struct, including the signature for the value. This enables node operators to easily determine which validators are misbehaving and remove them from their quorum sets.
 
 ## Test Cases
-TBD
+The [pull request that implements this feature](https://github.com/stellar/stellar-core/pull/5209) contains many tests for it.  Of particular interest to CAP readers might be the test "network externalizes empty-tx-set on missing value", which generates `LedgerCloseData` meta for `STELLAR_VALUE_EMPTY_TX_SET` ledgers when run with the `--capture-lcm` flag.
+
+Future pull requests will introduce an automated test harness that stresses a large number of interleavings for transaction set arrival and SCP messages.
 
 ## Implementation
-Draft implementation: https://github.com/stellar/stellar-core/pull/5209
+Implementation: https://github.com/stellar/stellar-core/pull/5209
 
 [^1]: See the [SCP IETF draft](https://www.ietf.org/archive/id/draft-mazieres-dinrg-scp-05.txt) for a complete semantics of `c`