Fork identifier for chain compatibility checks

2019-05-03T00:00:00+00:00

Simple Summary</h2>
Currently nodes in the Ethereum network try to find each other by establishing random connections to remote machines "looking" like an Ethereum node (public networks, private networks, test networks, etc), hoping that they found a useful peer (same genesis, same forks). This wastes time and resources, especially for smaller networks.
To avoid this overhead, Ethereum needs a mechanism that can precisely identify whether a node will be useful, as early as possible. Such a mechanism requires a way to summarize chain configurations, as well as a way to disseminate said summaries in the network.
This proposal focuses only on the definition of said summary - a generally useful fork identifier - and it's validation rules, allowing it to be embedded into arbitrary network protocols (e.g. discovery ENRs</a> or `eth/6x</code> handshakes).`

Abstract</h2> There are many public and private Ethereum networks, but the discovery protocol doesn't differentiate between them. The only way to check if a peer is good or bad (same chain or not), is to establish a TCP/IP connection, wrap it with RLPx cryptography, then execute an eth</code> handshake. This is an extreme cost to bear if it turns out that the remote peer is on a different network and it's not even precise enough to differentiate Ethereum and Ethereum Classic. This cost is magnified for small networks, where a lot more trial and errors are needed to find good nodes. Even if the peer is on the same chain, during non-controversial consensus upgrades, not everybody updates their nodes in time (developer nodes, leftovers, etc). These stale nodes put a meaningless burden on the peer-to-peer network, since they just latch on to good nodes, but don't accept upgraded blocks. This causes valuable peer slots and bandwidth to be lost until the stale nodes finally update. This is a serious issue for test networks, where leftovers can linger for months. This EIP proposes a new identity scheme to both precisely and concisely summarize the chain's current status (genesis and all applied forks). The conciseness is particularly important to make the identity useful across datagram protocols too. The EIP solves a number of issues: If two nodes are on different networks, they should never even consider connecting.</li> If a hard fork passes, upgraded nodes should reject non-upgraded ones, but NOT before.</li> If two chains share the same genesis, but not forks (ETH / ETC), they should reject each other.</li> </ul> This EIP does not attempt to solve the clean separation of 3-way-forks! If at the same future block number, the network splits into three (non-fork, fork-A and fork-B), separating the forkers from each another will need case-by-case special handling. Not handling this keeps the proposal pragmatic, simple and also avoids making it too easy to fork off mainnet. To keep the scope limited, this EIP only defines the identity scheme and validation rules. The same scheme and algorithm can be embedded into various networking protocols, allowing both the eth/6x</code> handshake to be more precise (Ethereum vs. Ethereum Classic); as well as the discovery to be more useful (reject surely peers without ever connecting). Motivation</h2> Peer-to-peer networking is messy and hard due to firewalls and network address translation (NAT). Generally only a small fraction of nodes have publicly routed addresses and P2P networks rely mainly on these for forwarding data for everyone else. The best way to maximize the utility of the public nodes is to ensure their resources aren't wasted on tasks that are worthless to the network. By aggressively cutting off incompatible nodes from each other we can extract a lot more value from the public nodes, making the entire P2P network much more robust and reliable. Supporting this network partitioning at a discovery layer can further enhance performance as we avoid the costly crypto and latency/bandwidth hit associated with establishing a stream connection in the first place. Specification</h2> Each node maintains the following values: FORK_HASH</code>: IEEE CRC32 checksum ([4]byte</code>) of the genesis hash and fork blocks numbers that already passed. The fork block numbers are fed into the CRC32 checksum in ascending order.</li> If multiple forks are applied at the same block, the block number is checksummed only once.</li> Block numbers are regarded as uint64</code> integers, encoded in big endian format when checksumming.</li> If a chain is configured to start with a non-Frontier ruleset already in its genesis, that is NOT considered a fork.</li> </ul> </li> FORK_NEXT</code>: Block number (uint64</code>) of the next upcoming fork, or 0</code> if no next fork is known.</li> </ul> E.g. FORK_HASH</code> for mainnet would be: forkhash₀ = 0xfc64ec04</code> (Genesis) = CRC32(<genesis-hash>)</code></li> forkhash₁ = 0x97c2c34c</code> (Homestead) = CRC32(<genesis-hash> || uint64(1150000))</code></li> forkhash₂ = 0x91d1f948</code> (DAO fork) = CRC32(<genesis-hash> || uint64(1150000) || uint64(1920000))</code></li> </ul> The fork identifier is defined as RLP([FORK_HASH, FORK_NEXT])</code>. This forkid</code> is cross validated (NOT naively compared) to assess a remote chain's compatibility. Irrespective of fork state, both parties must come to the same conclusion to avoid indefinite reconnect attempts from one side. Validation rules</h4> If local and remote FORK_HASH</code> matches, compare local head to FORK_NEXT</code>.</li> </ol> The two nodes are in the same fork state currently. They might know of differing future forks, but that's not relevant until the fork triggers (might be postponed, nodes might be updated to match). 1a) A remotely announced but remotely not passed block is already passed locally, disconnect, since the chains are incompatible.</li> 1b) No remotely announced fork; or not yet passed locally, connect.</li> </ul> </li> </ul> </li> If the remote FORK_HASH</code> is a subset of the local past forks and the remote FORK_NEXT</code> matches with the locally following fork block number, connect.</li> </ol> Remote node is currently syncing. It might eventually diverge from us, but at this current point in time we don't have enough information.</li> </ul> </li> If the remote FORK_HASH</code> is a superset of the local past forks and can be completed with locally known future forks, connect.</li> </ol> Local node is currently syncing. It might eventually diverge from the remote, but at this current point in time we don't have enough information.</li> </ul> </li> Reject in all other cases.</li> </ol> </li> </ul> Stale software examples</h4> The examples below try to exhaust the fork combination possibilities that arise when nodes do not run matching software versions, but otherwise follow the same chain (mainnet nodes, testnet nodes, etc). Past forks</th> Future forks</th> Head</th> Remote FORK_HASH</code></th> Remote FORK_NEXT</code></th> Connect</th> Reason</th></tr></thead> A</td> </td> </td> A</td> </td> Yes (1b)</td> Same forks, same sync state.</td></tr> A</td> </td> < B</td> A</td> B</td> Yes (1b)</td> Remote is advertising a future fork, but that is uncertain.</td></tr> A</td> </td> >= B</td> A</td> B</td> No (1a)</td> Remote is advertising a future fork that passed locally.</td></tr> A</td> B</td> </td> A</td> </td> Yes (1b)</td> Local knows about a future fork, but that is uncertain.</td></tr> A</td> B</td> </td> A</td> B</td> Yes (1b)</td> Both know about a future fork, but that is uncertain.</td></tr> A</td> B1</td> < B2</td> A</td> B2</td> Yes (1b)</td> Both know about differing future forks, but those are uncertain.</td></tr> A</td> B1</td> >= B2</td> A</td> B2</td> No (1a)</td> Both know about differing future forks, but the remote one passed locally.</td></tr> [A,B]</td> </td> </td> A</td> B</td> Yes (2)</td> Remote out of sync.</td></tr> [A,B,C]</td> </td> </td> A</td> B</td> Yes¹ (2)</td> Remote out of sync. Remote will need a software update, but we don't know it yet.</td></tr> A</td> B</td> </td> A ⊕ B</td> </td> Yes (3)</td> Local out of sync.</td></tr> A</td> B,C</td> </td> A ⊕ B</td> </td> Yes (3)</td> Local out of sync. Local also knows about a future fork, but that is uncertain yet.</td></tr> A</td> </td> </td> A ⊕ B</td> </td> No (4)</td> Local needs software update.</td></tr> A</td> B</td> </td> A ⊕ B ⊕ C</td> </td> No² (4)</td> Local needs software update.</td></tr> [A,B]</td> </td> </td> A</td> </td> No (4)</td> Remote needs software update.</td></tr> </tbody></table> Note, there's one asymmetry in the table, marked with ¹ and ². Since we don't have access to a remote node's future fork list (just the next one), we can't detect that it's software is stale until it syncs up. This is acceptable as 1) the remote node will disconnect from us anyway, and 2) this is a temporary fluke during sync, not permanent with a leftover node. Rationale</h2> Why flatten FORK_HASH</code> into 4 bytes? Why not share the entire genesis and fork list?</h5> Whilst the eth</code> devp2p protocol permits arbitrarily much data to be transmitted, the discovery protocol's total space allowance for all ENR entries is 300 bytes. Reducing the FORK_HASH</code> into a 4 bytes checksum ensures that we leave ample room in the ENR for future extensions; and 4 bytes is more than enough for arbitrarily many Ethereum networks from a (practical) collision perspective. Why use IEEE CRC32 as the checksum instead of Keccak256?</h5> We need a mechanism that can flatten arbitrary data into 4 bytes, without ignoring any of the input. Any other checksum or hashing algorithm would work, but since nodes can lie at any time, there's no value in cryptographic hash functions. Instead of just taking the first 4 bytes of a Keccak256 hash (seems odd) or XOR-ing all the 4-byte groups (messy), CRC32 is a better alternative, as this is exactly what it was designed for. IEEE CRC32 is also used by ethernet, gzip, zip, png, etc, so every programming language support should not be a problem. We're not using FORK_NEXT</code> for much, can't we get rid of it somehow?</h5> We need to be able to differentiate whether a remote node is out of sync or whether its software is stale. Sharing only the past forks cannot tell us if the node is legitimately behind or stuck. Why advertise only one next fork, instead of "hashing" all known future ones like the FORK_HASH</code>?</h5> Opposed to past forks that have already passed (for us locally) and can be considered immutable, we don't know anything about future ones. Maybe we're out of sync or maybe the fork didn't pass yet. If it didn't pass yet, it might be postponed, so enforcing it would split the network apart. It could also happen that we're not yet aware of all future forks (haven't updated our software in a while). Backwards Compatibility</h2> This EIP only defines an identity scheme, it does not define functional changes. Test Cases</h2> Here's a full suite of tests for all possible fork IDs that Mainnet, Ropsten, Rinkeby and Görli can advertise given the Petersburg fork cap (time of writing). type testcase struct { head uint64 want ID } tests := []struct { config *params.ChainConfig genesis common.Hash cases []testcase }{ // Mainnet test cases { params.MainnetChainConfig, params.MainnetGenesisHash, []testcase{ {0, ID{Hash: 0xfc64ec04, Next: 1150000}}, // Unsynced {1149999, ID{Hash: 0xfc64ec04, Next: 1150000}}, // Last Frontier block {1150000, ID{Hash: 0x97c2c34c, Next: 1920000}}, // First Homestead block {1919999, ID{Hash: 0x97c2c34c, Next: 1920000}}, // Last Homestead block {1920000, ID{Hash: 0x91d1f948, Next: 2463000}}, // First DAO block {2462999, ID{Hash: 0x91d1f948, Next: 2463000}}, // Last DAO block {2463000, ID{Hash: 0x7a64da13, Next: 2675000}}, // First Tangerine block {2674999, ID{Hash: 0x7a64da13, Next: 2675000}}, // Last Tangerine block {2675000, ID{Hash: 0x3edd5b10, Next: 4370000}}, // First Spurious block {4369999, ID{Hash: 0x3edd5b10, Next: 4370000}}, // Last Spurious block {4370000, ID{Hash: 0xa00bc324, Next: 7280000}}, // First Byzantium block {7279999, ID{Hash: 0xa00bc324, Next: 7280000}}, // Last Byzantium block {7280000, ID{Hash: 0x668db0af, Next: 0}}, // First and last Constantinople, first Petersburg block {7987396, ID{Hash: 0x668db0af, Next: 0}}, // Today Petersburg block }, }, // Ropsten test cases { params.TestnetChainConfig, params.TestnetGenesisHash, []testcase{ {0, ID{Hash: 0x30c7ddbc, Next: 10}}, // Unsynced, last Frontier, Homestead and first Tangerine block {9, ID{Hash: 0x30c7ddbc, Next: 10}}, // Last Tangerine block {10, ID{Hash: 0x63760190, Next: 1700000}}, // First Spurious block {1699999, ID{Hash: 0x63760190, Next: 1700000}}, // Last Spurious block {1700000, ID{Hash: 0x3ea159c7, Next: 4230000}}, // First Byzantium block {4229999, ID{Hash: 0x3ea159c7, Next: 4230000}}, // Last Byzantium block {4230000, ID{Hash: 0x97b544f3, Next: 4939394}}, // First Constantinople block {4939393, ID{Hash: 0x97b544f3, Next: 4939394}}, // Last Constantinople block {4939394, ID{Hash: 0xd6e2149b, Next: 6485846}}, // First Petersburg block {6485845, ID{Hash: 0xd6e2149b, Next: 6485846}}, // Last Petersburg block {6485846, ID{Hash: 0x4bc66396, Next: 0}}, // First Istanbul block {7500000, ID{Hash: 0x4bc66396, Next: 0}}, // Future Istanbul block }, }, // Rinkeby test cases { params.RinkebyChainConfig, params.RinkebyGenesisHash, []testcase{ {0, ID{Hash: 0x3b8e0691, Next: 1}}, // Unsynced, last Frontier block {1, ID{Hash: 0x60949295, Next: 2}}, // First and last Homestead block {2, ID{Hash: 0x8bde40dd, Next: 3}}, // First and last Tangerine block {3, ID{Hash: 0xcb3a64bb, Next: 1035301}}, // First Spurious block {1035300, ID{Hash: 0xcb3a64bb, Next: 1035301}}, // Last Spurious block {1035301, ID{Hash: 0x8d748b57, Next: 3660663}}, // First Byzantium block {3660662, ID{Hash: 0x8d748b57, Next: 3660663}}, // Last Byzantium block {3660663, ID{Hash: 0xe49cab14, Next: 4321234}}, // First Constantinople block {4321233, ID{Hash: 0xe49cab14, Next: 4321234}}, // Last Constantinople block {4321234, ID{Hash: 0xafec6b27, Next: 5435345}}, // First Petersburg block {5435344, ID{Hash: 0xafec6b27, Next: 5435345}}, // Last Petersburg block {5435345, ID{Hash: 0xcbdb8838, Next: 0}}, // First Istanbul block {6000000, ID{Hash: 0xcbdb8838, Next: 0}}, // Future Istanbul block }, }, // Goerli test cases { params.GoerliChainConfig, params.GoerliGenesisHash, []testcase{ {0, ID{Hash: 0xa3f5ab08, Next: 1561651}}, // Unsynced, last Frontier, Homestead, Tangerine, Spurious, Byzantium, Constantinople and first Petersburg block {1561650, ID{Hash: 0xa3f5ab08, Next: 1561651}}, // Last Petersburg block {1561651, ID{Hash: 0xc25efa5c, Next: 0}}, // First Istanbul block {2000000, ID{Hash: 0xc25efa5c, Next: 0}}, // Future Istanbul block }, }, }</code></pre> Here's a suite of tests of the different states a Mainnet node might be in and the different remote fork identifiers it might be required to validate and decide to accept or reject: tests := []struct { head uint64 id ID err error }{ // Local is mainnet Petersburg, remote announces the same. No future fork is announced. {7987396, ID{Hash: 0x668db0af, Next: 0}, nil}, // Local is mainnet Petersburg, remote announces the same. Remote also announces a next fork // at block 0xffffffff, but that is uncertain. {7987396, ID{Hash: 0x668db0af, Next: math.MaxUint64}, nil}, // Local is mainnet currently in Byzantium only (so it's aware of Petersburg), remote announces // also Byzantium, but it's not yet aware of Petersburg (e.g. non updated node before the fork). // In this case we don't know if Petersburg passed yet or not. {7279999, ID{Hash: 0xa00bc324, Next: 0}, nil}, // Local is mainnet currently in Byzantium only (so it's aware of Petersburg), remote announces // also Byzantium, and it's also aware of Petersburg (e.g. updated node before the fork). We // don't know if Petersburg passed yet (will pass) or not. {7279999, ID{Hash: 0xa00bc324, Next: 7280000}, nil}, // Local is mainnet currently in Byzantium only (so it's aware of Petersburg), remote announces // also Byzantium, and it's also aware of some random fork (e.g. misconfigured Petersburg). As // neither forks passed at neither nodes, they may mismatch, but we still connect for now. {7279999, ID{Hash: 0xa00bc324, Next: math.MaxUint64}, nil}, // Local is mainnet Petersburg, remote announces Byzantium + knowledge about Petersburg. Remote // is simply out of sync, accept. {7987396, ID{Hash: 0xa00bc324, Next: 7280000}, nil}, // Local is mainnet Petersburg, remote announces Spurious + knowledge about Byzantium. Remote // is definitely out of sync. It may or may not need the Petersburg update, we don't know yet. {7987396, ID{Hash: 0x3edd5b10, Next: 4370000}, nil}, // Local is mainnet Byzantium, remote announces Petersburg. Local is out of sync, accept. {7279999, ID{Hash: 0x668db0af, Next: 0}, nil}, // Local is mainnet Spurious, remote announces Byzantium, but is not aware of Petersburg. Local // out of sync. Local also knows about a future fork, but that is uncertain yet. {4369999, ID{Hash: 0xa00bc324, Next: 0}, nil}, // Local is mainnet Petersburg. remote announces Byzantium but is not aware of further forks. // Remote needs software update. {7987396, ID{Hash: 0xa00bc324, Next: 0}, ErrRemoteStale}, // Local is mainnet Petersburg, and isn't aware of more forks. Remote announces Petersburg + // 0xffffffff. Local needs software update, reject. {7987396, ID{Hash: 0x5cddc0e1, Next: 0}, ErrLocalIncompatibleOrStale}, // Local is mainnet Byzantium, and is aware of Petersburg. Remote announces Petersburg + // 0xffffffff. Local needs software update, reject. {7279999, ID{Hash: 0x5cddc0e1, Next: 0}, ErrLocalIncompatibleOrStale}, // Local is mainnet Petersburg, remote is Rinkeby Petersburg. {7987396, ID{Hash: 0xafec6b27, Next: 0}, ErrLocalIncompatibleOrStale}, // Local is mainnet Petersburg, far in the future. Remote announces Gopherium (non existing fork) // at some future block 88888888, for itself, but past block for local. Local is incompatible. // // This case detects non-upgraded nodes with majority hash power (typical Ropsten mess). {88888888, ID{Hash: 0x668db0af, Next: 88888888}, ErrLocalIncompatibleOrStale}, // Local is mainnet Byzantium. Remote is also in Byzantium, but announces Gopherium (non existing // fork) at block 7279999, before Petersburg. Local is incompatible. {7279999, ID{Hash: 0xa00bc324, Next: 7279999}, ErrLocalIncompatibleOrStale}, }</code></pre> Here's a couple of tests to verify the proper RLP encoding (since FORK_HASH</code> is a 4 byte binary but FORK_NEXT</code> is an 8 byte quantity): tests := []struct { id ID want []byte }{ { ID{Hash: 0, Next: 0}, common.Hex2Bytes("c6840000000080"), }, { ID{Hash: 0xdeadbeef, Next: 0xBADDCAFE}, common.Hex2Bytes("ca84deadbeef84baddcafe"), }, { ID{Hash: math.MaxUint32, Next: math.MaxUint64}, common.Hex2Bytes("ce84ffffffff88ffffffffffffffff"), }, }</code></pre>Implementation</h2> Geth: https://github.com/ethereum/go-ethereum/tree/master/core/forkid Copyright</h2> Copyright and related rights waived via CC0</a>. Node Discovery via DNS 2018-09-26T00:00:00+00:00 Abstract</h2> This document describes a scheme for authenticated, updateable Ethereum node lists retrievable via DNS. Motivation</h2> Many Ethereum clients contain hard-coded bootstrap node lists. Updating those lists requires a software update. The current lists are small, giving the client little choice of initial entry point into the Ethereum network. We would like to maintain larger node lists containing hundreds of nodes, and update them regularly. The scheme described here is a replacement for client bootstrap node lists with equivalent security and many additional benefits. Large lists populated by traversing the node discovery DHT can serve as a fallback option for nodes which can't join the DHT due to restrictive network policy. DNS-based node lists may also be useful to Ethereum peering providers because their customers can configure the client to use the provider's list. Specification</h2> A 'node list' is a list of 'node records' as defined by EIP-778</a> of arbitrary length. Lists may refer to other lists using links. The entire list is signed using a secp256k1 private key. The corresponding public key must be known to the client in order to verify the list. To refer to a DNS node list, clients use a URL with 'enrtree' scheme. The URL contains the DNS name on which the list can be found as well as the public key that signed the list. The public key is contained in the username part of the URL and is the base32 encoding (RFC-4648) of the compressed 32-byte binary public key. Example: enrtree://AM5FCQLWIZX2QFPNJAP7VUERCCRNGRHWZG3YYHIUV7BVDQ5FDPRT2@nodes.example.org</code></pre> This URL refers to a node list at the DNS name 'nodes.example.org' and is signed by the public key 0x049f88229042fef9200246f49f94d9b77c4e954721442714e85850cb6d9e5daf2d880ea0e53cb3ac1a75f9923c2726a4f941f7d326781baa6380754a360de5c2b6</code> DNS Record Structure</h3> The nodes in a list are encoded as a merkle tree for distribution via the DNS protocol. Entries of the merkle tree are contained in DNS TXT records. The root of the tree is a TXT record with the following content: enrtree-root:v1 e=<enr-root> l=<link-root> seq=<sequence-number> sig=<signature></code></pre> where enr-root</code> and link-root</code> refer to the root hashes of subtrees containing nodes and links subtrees.</li> sequence-number</code> is the tree's update sequence number, a decimal integer.</li> signature</code> is a 65-byte secp256k1 EC signature over the keccak256 hash of the record content, excluding the sig=</code> part, encoded as URL-safe base64 (RFC-4648).</li> </ul> Further TXT records on subdomains map hashes to one of three entry types. The subdomain name of any entry is the base32 encoding of the (abbreviated) keccak256 hash of its text content. enrtree-branch:<h₁>,<h₂>,...,<hₙ></code> is an intermediate tree entry containing hashes of subtree entries.</li> enrtree://<key>@<fqdn></code> is a leaf pointing to a different list located at another fully qualified domain name. Note that this format matches the URL encoding. This type of entry may only appear in the subtree pointed to by link-root</code>.</li> enr:<node-record></code> is a leaf containing a node record. The node record is encoded as a URL-safe base64 string. Note that this type of entry matches the canonical ENR text encoding. It may only appear in the enr-root</code> subtree.</li> </ul> No particular ordering or structure is defined for the tree. Whenever the tree is updated, its sequence number should increase. The content of any TXT record should be small enough to fit into the 512 byte limit imposed on UDP DNS packets. This limits the number of hashes that can be placed into an enrtree-branch</code> entry. Example in zone file format: ; name ttl class type content @ 60 IN TXT enrtree-root:v1 e=JWXYDBPXYWG6FX3GMDIBFA6CJ4 l=C7HRFPF3BLGF3YR4DY5KX3SMBE seq=1 sig=o908WmNp7LibOfPsr4btQwatZJ5URBr2ZAuxvK4UWHlsB9sUOTJQaGAlLPVAhM__XJesCHxLISo94z5Z2a463gA C7HRFPF3BLGF3YR4DY5KX3SMBE 86900 IN TXT enrtree://AM5FCQLWIZX2QFPNJAP7VUERCCRNGRHWZG3YYHIUV7BVDQ5FDPRT2@morenodes.example.org JWXYDBPXYWG6FX3GMDIBFA6CJ4 86900 IN TXT enrtree-branch:2XS2367YHAXJFGLZHVAWLQD4ZY,H4FHT4B454P6UXFD7JCYQ5PWDY,MHTDO6TMUBRIA2XWG5LUDACK24 2XS2367YHAXJFGLZHVAWLQD4ZY 86900 IN TXT enr:-HW4QOFzoVLaFJnNhbgMoDXPnOvcdVuj7pDpqRvh6BRDO68aVi5ZcjB3vzQRZH2IcLBGHzo8uUN3snqmgTiE56CH3AMBgmlkgnY0iXNlY3AyNTZrMaECC2_24YYkYHEgdzxlSNKQEnHhuNAbNlMlWJxrJxbAFvA H4FHT4B454P6UXFD7JCYQ5PWDY 86900 IN TXT enr:-HW4QAggRauloj2SDLtIHN1XBkvhFZ1vtf1raYQp9TBW2RD5EEawDzbtSmlXUfnaHcvwOizhVYLtr7e6vw7NAf6mTuoCgmlkgnY0iXNlY3AyNTZrMaECjrXI8TLNXU0f8cthpAMxEshUyQlK-AM0PW2wfrnacNI MHTDO6TMUBRIA2XWG5LUDACK24 86900 IN TXT enr:-HW4QLAYqmrwllBEnzWWs7I5Ev2IAs7x_dZlbYdRdMUx5EyKHDXp7AV5CkuPGUPdvbv1_Ms1CPfhcGCvSElSosZmyoqAgmlkgnY0iXNlY3AyNTZrMaECriawHKWdDRk2xeZkrOXBQ0dfMFLHY4eENZwdufn1S1o</code></pre>Client Protocol</h3> To find nodes at a given DNS name, say "mynodes.org": Resolve the TXT record of the name and check whether it contains a valid "enrtree-root=v1" entry. Let's say the enr-root</code> hash contained in the entry is "CFZUWDU7JNQR4VTCZVOJZ5ROV4".</li> Verify the signature on the root against the known public key and check whether the sequence number is larger than or equal to any previous number seen for that name.</li> Resolve the TXT record of the hash subdomain, e.g. "CFZUWDU7JNQR4VTCZVOJZ5ROV4.mynodes.org" and verify whether the content matches the hash.</li> The next step depends on the entry type found: for enrtree-branch</code>: parse the list of hashes and continue resolving them (step 3).</li> for enr</code>: decode, verify the node record and import it to local node storage.</li> </ul> </li> </ol> During traversal, the client must track hashes and domains which are already resolved to avoid going into an infinite loop. It's in the client's best interest to traverse the tree in random order. Client implementations should avoid downloading the entire tree at once during normal operation. It's much better to request entries via DNS when-needed, i.e. at the time when the client is looking for peers. Rationale</h2> Why DNS?</h3> We have chosen DNS as the distribution medium because it is always available, even under restrictive network conditions. The protocol provides low latency and answers to DNS queries can be cached by intermediate resolvers. No custom server software is needed. Node lists can be deployed to any DNS provider such as CloudFlare DNS, dnsimple, Amazon Route 53 using their respective client libraries. Why is this a merkle tree?</h3> Being a merkle tree, any node list can be authenticated by a single signature on the root. Hash subdomains protect the integrity of the list. At worst intermediate resolvers can block access to the list or disallow updates to it, but cannot corrupt its content. The sequence number prevents replacing the root with an older version. Synchronizing updates on the client side can be done incrementally, which matters for large lists. Individual entries of the tree are small enough to fit into a single UDP packet, ensuring compatibility with environments where only basic UDP DNS is available. The tree format also works well with caching resolvers: only the root of the tree needs a short TTL. Intermediate entries and leaves can be cached for days. Why does the link subtree exist?</h3> Links between lists enable federation and web-of-trust functionality. The operator of a large list can delegate maintenance to other list providers. If two node lists link to each other, users can use either list and get nodes from both. The link subtree is separate from the tree containing ENRs. This is done to enable client implementations to sync these trees independently. A client wanting to get as many nodes as possible will sync the link tree first and add all linked names to the sync horizon. Security Considerations</h2> Discovery via DNS is less secure than via DHT, because it relies on a trusted party to publish the records regularly. The actor could easily eclipse bootstrapping nodes by only publishing node records that it controls. Copyright</h2> Copyright and related rights waived via CC0</a>. Node Discovery v4 ENR Extension 2018-02-02T00:00:00+00:00 Abstract</h1> This EIP defines an extension to Node Discovery Protocol v4 to enable authoritative resolution of Ethereum Node Records (ENR)</a>. Motivation</h1> To bridge current and future discovery networks and to aid the implementation of other relay mechanisms for ENR such as DNS, we need a way to request the most up-to-date version of a node record. This EIP provides a way to request it using the existing discovery protocol. Specification</h1> Implementations of Node Discovery Protocol v4 should support two new packet types, a request and reply of the node record. The existing ping and pong packets are extended with a new field containing the sequence number of the ENR. Ping Packet (0x01)</h3> packet-data = [version, from, to, expiration, enr-seq]</code></pre> enr-seq</code> is the current sequence number of the sending node's record. All other fields retain their existing meaning. Pong Packet (0x02)</h3> packet-data = [to, ping-hash, expiration, enr-seq]</code></pre> enr-seq</code> is the current sequence number of the sending node's record. All other fields retain their existing meaning. ENRRequest Packet (0x05)</h3> packet-data = [ expiration ]</code></pre> When a packet of this type is received, the node should reply with an ENRResponse packet containing the current version of its record. To guard against amplification attacks, the sender of ENRRequest should have replied to a ping packet recently (just like for FindNode). The expiration</code> field, a UNIX timestamp, should be handled as for all other existing packets i.e. no reply should be sent if it refers to a time in the past. ENRResponse Packet (0x06)</h3> packet-data = [ request-hash, ENR ]</code></pre> This packet is the response to ENRRequest. request-hash</code> is the hash of the entire ENRRequest packet being replied to.</li> ENR</code> is the node record.</li> </ul> The recipient of the packet should verify that the node record is signed by node who sent ENRResponse. Resolving Records</h2> To resolve the current record of a node public key, perform a recursive Kademlia lookup using the FindNode, Neighbors packets. When the node is found, send ENRRequest to it and return the record from the response. Copyright</h1> Copyright and related rights waived via CC0</a>. Ethereum Node Records (ENR) 2017-11-23T00:00:00+00:00 Abstract</h1> This EIP defines Ethereum Node Records, an open format for p2p connectivity information. Motivation</h1> Ethereum nodes discover each other through the node discovery protocol. The purpose of that protocol is relaying node identity public keys (on the secp256k1 curve), their IP address and two port numbers. No other information can be relayed. This specification seeks to lift the restrictions of the discovery v4 protocol by defining a flexible format, the node record, for connectivity-related information. Node records can be relayed through a future version of the node discovery protocol. They can also be relayed through arbitrary other mechanisms such as DNS, ENS, a devp2p subprotocol, etc. Node records improve cryptographic agility and handling of protocol upgrades. A record can contain information about arbitrary transport protocols and public key material associated with them. Another goal of the new format is to provide authoritative updates of connectivity information. If a node changes its endpoint and publishes a new record, other nodes should be able to determine which record is newer. Specification</h1> The components of a node record are: signature</code>: cryptographic signature of record contents</li> seq</code>: The sequence number, a 64-bit unsigned integer. Nodes should increase the number whenever the record changes and republish the record.</li> The remainder of the record consists of arbitrary key/value pairs</li> </ul> A record's signature is made and validated according to an identity scheme. The identity scheme is also responsible for deriving a node's address in the DHT. The key/value pairs must be sorted by key and must be unique, i.e. any key may be present only once. The keys can technically be any byte sequence, but ASCII text is preferred. Key names in the table below have pre-defined meaning. Key</th> Value</th></tr></thead> id</code></td> name of identity scheme, e.g. "v4"</td></tr> secp256k1</code></td> compressed secp256k1 public key, 33 bytes</td></tr> ip</code></td> IPv4 address, 4 bytes</td></tr> tcp</code></td> TCP port, big endian integer</td></tr> udp</code></td> UDP port, big endian integer</td></tr> ip6</code></td> IPv6 address, 16 bytes</td></tr> tcp6</code></td> IPv6-specific TCP port, big endian integer</td></tr> udp6</code></td> IPv6-specific UDP port, big endian integer</td></tr> </tbody></table> All keys except id</code> are optional, including IP addresses and ports. A record without endpoint information is still valid as long as its signature is valid. If no tcp6</code> / udp6</code> port is provided, the tcp</code> / udp</code> port applies to both IP addresses. Declaring the same port number in both tcp</code>, tcp6</code> or udp</code>, udp6</code> should be avoided but doesn't render the record invalid. RLP Encoding</h3> The canonical encoding of a node record is an RLP list of [signature, seq, k, v, ...]</code>. The maximum encoded size of a node record is 300 bytes. Implementations should reject records larger than this size. Records are signed and encoded as follows: content = [seq, k, v, ...] signature = sign(content) record = [signature, seq, k, v, ...]</code></pre>Text Encoding</h3> The textual form of a node record is the base64 encoding of its RLP representation, prefixed by enr:</code>. Implementations should use the URL-safe base64 alphabet</a> and omit padding characters. "v4" Identity Scheme</h3> This specification defines a single identity scheme to be used as the default until other schemes are defined by further EIPs. The "v4" scheme is backwards-compatible with the cryptosystem used by Node Discovery v4. To sign record content</code> with this scheme, apply the keccak256 hash function (as used by the EVM) to content</code>, then create a signature of the hash. The resulting 64-byte signature is encoded as the concatenation of the r</code> and s</code> signature values (the recovery ID v</code> is omitted).</li> To verify a record, check that the signature was made by the public key in the "secp256k1" key/value pair of the record.</li> To derive a node address, take the keccak256 hash of the uncompressed public key.</li> </ul> Rationale</h1> The format is meant to suit future needs in two ways: Adding new key/value pairs: This is always possible and doesn't require implementation consensus. Existing clients will accept any key/value pairs regardless of whether they can interpret their content.</li> Adding identity schemes: these need implementation consensus because the network won't accept the signature otherwise. To introduce a new identity scheme, propose an EIP and get it implemented. The scheme can be used as soon as most clients accept it.</li> </ul> The size of a record is limited because records are relayed frequently and may be included in size-constrained protocols such as DNS. A record containing a IPv4 address, when signed using the "v4" scheme occupies roughly 120 bytes, leaving plenty of room for additional metadata. You might wonder about the need for so many pre-defined keys related to IP addresses and ports. This need arises because residential and mobile network setups often put IPv4 behind NAT while IPv6 traffic—if supported—is directly routed to the same host. Declaring both address types ensures a node is reachable from IPv4-only locations and those supporting both protocols. Test Vectors</h1> This is an example record containing the IPv4 address 127.0.0.1</code> and UDP port 30303</code>. The node ID is a448f24c6d18e575453db13171562b71999873db5b286df957af199ec94617f7</code>. enr:-IS4QHCYrYZbAKWCBRlAy5zzaDZXJBGkcnh4MHcBFZntXNFrdvJjX04jRzjzCBOonrkTfj499SZuOh8R33Ls8RRcy5wBgmlkgnY0gmlwhH8AAAGJc2VjcDI1NmsxoQPKY0yuDUmstAHYpMa2_oxVtw0RW_QAdpzBQA8yWM0xOIN1ZHCCdl8</code></pre> The record is signed using the "v4" identity scheme using sequence number 1</code> and this private key: b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291</code></pre> The RLP structure of the record is: [ 7098ad865b00a582051940cb9cf36836572411a47278783077011599ed5cd16b76f2635f4e234738f30813a89eb9137e3e3df5266e3a1f11df72ecf1145ccb9c, 01, "id", "v4", "ip", 7f000001, "secp256k1", 03ca634cae0d49acb401d8a4c6b6fe8c55b70d115bf400769cc1400f3258cd3138, "udp", 765f, ]</code></pre>Copyright</h1> Copyright and related rights waived via CC0</a>. DEVp2p snappy compression 2017-09-07T00:00:00+00:00 Abstract</h2> The base networking protocol (DEVp2p) used by Ethereum currently does not employ any form of compression. This results in a massive amount of bandwidth wasted in the entire network, making both initial sync as well as normal operation slower and laggier. This EIP proposes a tiny extension to the DEVp2p protocol to enable Snappy compression</a> on all message payloads after the initial handshake. After extensive benchmarks, results show that data traffic is decreased by 60-80% for initial sync. You can find exact numbers below. Motivation</h2> Synchronizing the Ethereum main network (block 4,248,000) in Geth using fast sync currently consumes 1.01GB upload and 33.59GB download bandwidth. On the Rinkeby test network (block 852,000) it's 55.89MB upload and 2.51GB download. However, most of this data (blocks, transactions) are heavily compressible. By enabling compression at the message payload level, we can reduce the previous numbers to 1.01GB upload / 13.46GB download on the main network, and 46.21MB upload / 463.65MB download on the test network. The motivation behind doing this at the DEVp2p level (opposed to eth for example) is that it would enable compression for all sub-protocols (eth, les, bzz) seamlessly, reducing any complexity those protocols might incur in trying to individually optimize for data traffic. Specification</h2> Bump the advertised DEVp2p version number from 4</code> to 5</code>. If during handshake, the remote side advertises support only for version 4</code>, run the exact same protocol as until now. If the remote side advertises a DEVp2p version >= 5</code>, inject a Snappy compression step right before encrypting the DEVp2p message during sending: A message consists of {Code, Size, Payload}</code></li> Compress the original payload with Snappy and store it in the same field.</li> Update the message size to the length of the compressed payload.</li> Encrypt and send the message as before, oblivious to compression.</li> </ul> Similarly to message sending, when receiving a DEVp2p v5 message from a remote node, insert a Snappy decompression step right after decrypting the DEVp2p message: A message consists of {Code, Size, Payload}</code></li> Decrypt the message payload as before, oblivious to compression.</li> Decompress the payload with Snappy and store it in the same field.</li> Update the message size to the length of the decompressed payload.</li> </ul> Important caveats: The handshake message is never compressed, since it is needed to negotiate the common version.</li> Snappy framing is not used, since the DEVp2p protocol already message oriented.</li> </ul> Note: Snappy supports uncompressed binary literals (up to 4GB) too, leaving room for fine-tuned future optimisations for already compressed or encrypted data that would have no gain of compression (Snappy usually detects this case automatically). Avoiding DOS attacks</h3> Currently a DEVp2p message length is limited to 24 bits, amounting to a maximum size of 16MB. With the introduction of Snappy compression, care must be taken not to blindly decompress messages, since they may get significantly larger than 16MB. However, Snappy is capable of calculating the decompressed size of an input message without inflating it in memory (the stream starts with the uncompressed length up to a maximum of 2^32 - 1</code> stored as a little-endian varint</a>). This can be used to discard any messages which decompress above some threshold. The proposal is to use the same limit (16MB) as the threshold for decompressed messages. This retains the same guarantees that the current DEVp2p protocol does, so there won't be surprises in application level protocols. Alternatives (discarded)</h2> Alternative solutions to data compression that have been brought up and discarded are: Extend protocol xyz</code> to support compressed messages versus doing it at DEVp2p level: Pro: Can be better optimized when to compress and when not to.</li> Con: Mixes in transport layer encoding into application layer logic.</li> Con: Makes the individual message specs more convoluted with compression details.</li> Con: Requires cross client coordination on every single protocol, making the effort much harder and repeated (eth, les, shh, bzz).</li> </ul> Introduce seamless variations of protocol such as xyz</code> expanded with xyz-compressed</code>: Pro: Can be done (hacked in) without cross client coordination.</li> Con: Litters the network with client specific protocol announces.</li> Con: Needs to be specced in an EIP for cross interoperability anyway.</li> </ul> Other ideas that have been discussed and discarded: Don't explicitly limit the decompressed message size, only the compressed one: Pro: Allows larger messages to traverse through DEVp2p.</li> Con: Upper layer protocols need to check and discard large messages.</li> Con: Needs lazy decompression to allow size limitations without DOS.</li> </ul> Backwards Compatibility</h2> This proposal is fully backward compatible. Clients upgrading to the proposed DEVp2p protocol version 5</code> should still support skipping the compression step for connections that only advertise version 4</code> of the DEVp2p protocol. Implementation</h2> You can find a reference implementation of this EIP in https://github.com/ethereum/go-ethereum/pull/15106. Test vectors</h2> There is more than one valid encoding of any given input, and there is more than one good internal compression algorithm within Snappy when trading off throughput for output size. As such, different implementations might produce slight variations in the compressed form, but all should be cross compatible between each other. As an example, take hex encoded RLP of block #272621 from the Rinkeby test network: block.rlp (~3MB)</a>. Encoding the raw RLP via Go's Snappy library</a> yields: block.go.snappy (~70KB)</a>.</li> Encoding the raw RLP via Python's Snappy library</a> yields: block.py.snappy (~70KB)</a>.</li> </ul> You can verify that an encoded binary can be decoded into the proper plaintext using the following snippets: Go</h3> $ go get https://github.com/golang/snappy</code></pre>package main import ( "bytes" "encoding/hex" "fmt" "io/ioutil" "log" "os" "github.com/golang/snappy" ) func main() { // Read and decode the decompressed file plainhex, err := ioutil.ReadFile(os.Args[1]) if err != nil { log.Fatalf("Failed to read decompressed file %s: %v", os.Args[1], err) } plain, err := hex.DecodeString(string(plainhex)) if err != nil { log.Fatalf("Failed to decode decompressed file: %v", err) } // Read and decode the compressed file comphex, err := ioutil.ReadFile(os.Args[2]) if err != nil { log.Fatalf("Failed to read compressed file %s: %v", os.Args[2], err) } comp, err := hex.DecodeString(string(comphex)) if err != nil { log.Fatalf("Failed to decode compressed file: %v", err) } // Make sure they match decomp, err := snappy.Decode(nil, comp) if err != nil { log.Fatalf("Failed to decompress compressed file: %v", err) } if !bytes.Equal(plain, decomp) { fmt.Println("Booo, decompressed file does not match provided plain text!") return } fmt.Println("Yay, decompressed data matched provided plain text!") }</code></pre>$ go run main.go block.rlp block.go.snappy Yay, decompressed data matched provided plain text! $ go run main.go block.rlp block.py.snappy Yay, decompressed data matched provided plain text!</code></pre>Python</h3> $ pip install python-snappy</code></pre>import snappy import sys # Read and decode the decompressed file with open(sys.argv[1], 'rb') as file: plainhex = file.read() plain = plainhex.decode("hex") # Read and decode the compressed file with open(sys.argv[2], 'rb') as file: comphex = file.read() comp = comphex.decode("hex") # Make sure they match decomp = snappy.uncompress(comp) if plain != decomp: print "Booo, decompressed file does not match provided plain text!" else: print "Yay, decompressed data matched provided plain text!"</code></pre>$ python main.py block.rlp block.go.snappy Yay, decompressed data matched provided plain text! $ python main.py block.rlp block.py.snappy Yay, decompressed data matched provided plain text!</code></pre>References</h2> Snappy website: https://google.github.io/snappy/</li> Snappy specification: https://github.com/google/snappy/blob/master/format_description.txt</li> </ul> Copyright</h2> Copyright and related rights waived via CC0</a>. Whisper Specification 2017-05-05T00:00:00+00:00 Abstract</h2> This EIP describes the format of Whisper messages within the ÐΞVp2p Wire Protocol. This EIP should substitute the existing specification</a>. More detailed documentation on Whisper could be found here</a>. Motivation</h2> It is necessary to specify the standard for Whisper messages in order to ensure forward compatibility of different Whisper clients. Specification</h2> All Whisper messages sent as ÐΞVp2p Wire Protocol packets should be RLP-encoded arrays of data containing two objects: integer packet code followed by another object (whose type depends on the packet code). If Whisper node does not support a particular packet code, it should just ignore the packet without generating any error. Packet Codes</h3> The message codes reserved for Whisper protocol: 0 - 127. Messages with unknown codes must be ignored, for forward compatibility of future versions. The Whisper sub-protocol should support the following packet codes: EIP</th> Name</th> Int Value</th></tr></thead> </td> Status</td> 0</td></tr> </td> Messages</td> 1</td></tr> </td> PoW Requirement</td> 2</td></tr> </td> Bloom Filter</td> 3</td></tr> -------</td> ----------------------------</td> -----------</td></tr> </tbody></table> The following message codes are optional, but they are reserved for specific purpose. EIP</th> Name</th> Int Value</th></tr></thead> </td> P2P Request</td> 126</td></tr> </td> P2P Message</td> 127</td></tr> -------</td> ----------------------------</td> -----------</td></tr> </tbody></table> Packet Format and Usage</h3> Status [0</code>] This packet contains two objects: integer message code (0x00) followed by a list of values: integer version, float PoW requirement, and bloom filter, in this order. The bloom filter parameter is optional; if it is missing or nil, the node is considered to be full node (i.e. accepts all messages). The format of PoW and bloom filter please see below (message codes 2 and 3). Status message should be sent after the initial handshake and prior to any other messages. Messages [1</code>, whisper_envelopes</code>] This packet contains two objects: integer message code (0x01) followed by a list (possibly empty) of Whisper Envelopes. This packet is used for sending the standard Whisper envelopes. PoW Requirement [2</code>, PoW</code>] This packet contains two objects: integer message code (0x02) followed by a single floating point value of PoW. This value is the IEEE 754 binary representation of 64-bit floating point number. Values of qNAN, sNAN, INF and -INF are not allowed. Negative values are also not allowed. This packet is used by Whisper nodes for dynamic adjustment of their individual PoW requirements. Recipient of this message should no longer deliver the sender messages with PoW lower than specified in this message. PoW is defined as average number of iterations, required to find the current BestBit (the number of leading zero bits in the hash), divided by message size and TTL: PoW = (2**BestBit) / (size * TTL)</code></pre> PoW calculation: fn short_rlp(envelope) = rlp of envelope, excluding env_nonce field. fn pow_hash(envelope, env_nonce) = sha3(short_rlp(envelope) ++ env_nonce) fn pow(pow_hash, size, ttl) = 2**leading_zeros(pow_hash) / (size * ttl)</code></pre> where size is the size of the full RLP-encoded envelope. Bloom Filter [3</code>, bytes</code>] This packet contains two objects: integer message code (0x03) followed by a byte array of arbitrary size. This packet is used by Whisper nodes for sharing their interest in messages with specific topics. The Bloom filter is used to identify a number of topics to a peer without compromising (too much) privacy over precisely what topics are of interest. Precise control over the information content (and thus efficiency of the filter) may be maintained through the addition of bits. Blooms are formed by the bitwise OR operation on a number of bloomed topics. The bloom function takes the topic and projects them onto a 512-bit slice. At most, three bits are marked for each bloomed topic. The projection function is defined as a mapping from a 4-byte slice S to a 512-bit slice D; for ease of explanation, S will dereference to bytes, whereas D will dereference to bits. LET D[*] = 0 FOREACH i IN { 0, 1, 2 } DO LET n = S[i] IF S[3] & (2 ** i) THEN n += 256 D[n] = 1 END FOR</code></pre> OPTIONAL P2P Request [126</code>, whisper_envelope</code>] This packet contains two objects: integer message code (0x7E) followed by a single Whisper Envelope. This packet is used for sending Dapp-level peer-to-peer requests, e.g. Whisper Mail Client requesting old messages from the Whisper Mail Server. P2P Message [127</code>, whisper_envelope</code>] This packet contains two objects: integer message code (0x7F) followed by a single Whisper Envelope. This packet is used for sending the peer-to-peer messages, which are not supposed to be forwarded any further. E.g. it might be used by the Whisper Mail Server for delivery of old (expired) messages, which is otherwise not allowed. Whisper Envelope</h3> Envelopes are RLP-encoded structures of the following format: [ Expiry, TTL, Topic, Data, Nonce ]</code></pre> Expiry</code>: 4 bytes (UNIX time in seconds). TTL</code>: 4 bytes (time-to-live in seconds). Topic</code>: 4 bytes of arbitrary data. Data</code>: byte array of arbitrary size (contains encrypted message). Nonce</code>: 8 bytes of arbitrary data (used for PoW calculation). Contents of Data Field of the Message (Optional)</h3> This section outlines the optional description of Data Field to set up an example. Later it may be moved to a separate EIP. It is only relevant if you want to decrypt the incoming message, but if you only want to send a message, any other format would be perfectly valid and must be forwarded to the peers. Data field contains encrypted message of the Envelope. In case of symmetric encryption, it also contains appended Salt (a.k.a. AES Nonce, 12 bytes). Plaintext (unencrypted) payload consists of the following concatenated fields: flags, auxiliary field, payload, padding and signature (in this sequence). flags: 1 byte; first two bits contain the size of auxiliary field, third bit indicates whether the signature is present. auxiliary field: up to 4 bytes; contains the size of payload. payload: byte array of arbitrary size (may be zero). padding: byte array of arbitrary size (may be zero). signature: 65 bytes, if present. salt: 12 bytes, if present (in case of symmetric encryption).</code></pre> Those unable to decrypt the message data are also unable to access the signature. The signature, if provided, is the ECDSA signature of the Keccak-256 hash of the unencrypted data using the secret key of the originator identity. The signature is serialised as the concatenation of the R</code>, S</code> and V</code> parameters of the SECP-256k1 ECDSA signature, in that order. R</code> and S</code> are both big-endian encoded, fixed-width 256-bit unsigned. V</code> is an 8-bit big-endian encoded, non-normalised and should be either 27 or 28. The padding field was introduced in order to align the message size, since message size alone might reveal important metainformation. Padding can be arbitrary size. However, it is recommended that the size of Data Field (excluding the Salt) before encryption (i.e. plain text) should be factor of 256 bytes. Payload Encryption</h3> Asymmetric encryption uses the standard Elliptic Curve Integrated Encryption Scheme with SECP-256k1 public key. Symmetric encryption uses AES GCM algorithm with random 96-bit nonce. Rationale</h2> Packet codes 0x00 and 0x01 are already used in all Whisper versions. Packet code 0x02 will be necessary for the future development of Whisper. It will provide possibility to adjust the PoW requirement in real time. It is better to allow the network to govern itself, rather than hardcode any specific value for minimal PoW requirement. Packet code 0x03 will be necessary for scalability of the network. In case of too much traffic, the nodes will be able to request and receive only the messages they are interested in. Packet codes 0x7E and 0x7F may be used to implement Whisper Mail Server and Client. Without P2P messages it would be impossible to deliver the old messages, since they will be recognized as expired, and the peer will be disconnected for violating the Whisper protocol. They might be useful for other purposes when it is not possible to spend time on PoW, e.g. if a stock exchange will want to provide live feed about the latest trades. Backwards Compatibility</h2> This EIP is compatible with Whisper version 6. Any client which does not implement certain packet codes should gracefully ignore the packets with those codes. This will ensure the forward compatibility. Implementation</h2> The golang implementation of Whisper (v.6) already uses packet codes 0x00 - 0x03. Parity's implementation of v.6 will also use codes 0x00 - 0x03. Codes 0x7E and 0x7F are reserved, but still unused and left for custom implementation of Whisper Mail Server. Copyright</h2> Copyright and related rights waived via CC0</a>. devp2p Forward Compatibility Requirements for Homestead 2015-12-18T00:00:00+00:00 Abstract</h3> This EIP introduces new forward-compatibility requirements for implementations of the devp2p Wire Protocol, the RLPx Discovery Protocol and the RLPx TCP Transport Protocol. Clients which implement EIP-8 behave according to Postel's Law: Be conservative in what you do, be liberal in what you accept from others. </blockquote> Specification</h3> Implementations of the devp2p Wire Protocol should ignore the version number of hello packets. When sending the hello packet, the version element should be set to the highest devp2p version supported. Implementations should also ignore any additional list elements at the end of the hello packet. Similarly, implementations of the RLPx Discovery Protocol should not validate the version number of the ping packet, ignore any additional list elements in any packet, and ignore any data after the first RLP value in any packet. Discovery packets with unknown packet type should be discarded silently. The maximum size of any discovery packet is still 1280 bytes. Finally, implementations of the RLPx TCP Transport protocol should accept a new encoding for the encrypted key establishment handshake packets. If an EIP-8 style RLPx auth-packet</code> is received, the corresponding ack-packet</code> should be sent using the rules below. Decoding the RLP data in auth-body</code> and ack-body</code> should ignore mismatches of auth-vsn</code> and ack-vsn</code>, any additional list elements and any trailing data after the list. During the transitioning period (i.e. until the old format has been retired), implementations should pad auth-body</code> with at least 100 bytes of junk data. Adding a random amount in range [100, 300] is recommended to vary the size of the packet. auth-vsn = 4 auth-size = size of enc-auth-body, encoded as a big-endian 16-bit integer auth-body = rlp.list(sig, initiator-pubk, initiator-nonce, auth-vsn) enc-auth-body = ecies.encrypt(recipient-pubk, auth-body, auth-size) auth-packet = auth-size || enc-auth-body ack-vsn = 4 ack-size = size of enc-ack-body, encoded as a big-endian 16-bit integer ack-body = rlp.list(recipient-ephemeral-pubk, recipient-nonce, ack-vsn) enc-ack-body = ecies.encrypt(initiator-pubk, ack-body, ack-size) ack-packet = ack-size || enc-ack-body where X || Y denotes concatenation of X and Y. X[:N] denotes an N-byte prefix of X. rlp.list(X, Y, Z, ...) denotes recursive encoding of [X, Y, Z, ...] as an RLP list. sha3(MESSAGE) is the Keccak256 hash function as used by Ethereum. ecies.encrypt(PUBKEY, MESSAGE, AUTHDATA) is the asymmetric authenticated encryption function as used by RLPx. AUTHDATA is authenticated data which is not part of the resulting ciphertext, but written to HMAC-256 before generating the message tag.</code></pre>Motivation</h3> Changes to the devp2p protocols are hard to deploy because clients running an older version will refuse communication if the version number or structure of the hello (discovery ping, RLPx handshake) packet does not match local expectations. Introducing forward-compatibility requirements as part of the Homestead consensus upgrade will ensure that all client software in use on the Ethereum network can cope with future network protocol upgrades (as long as backwards-compatibility is maintained). Rationale</h3> The proposed changes address forward compatibility by applying Postel's Law (also known as the Robustness Principle) throughout the protocol stack. The merit and applicability of this approach has been studied repeatedly since its original application in RFC 761. For a recent perspective, see "The Robustness Principle Reconsidered" (Eric Allman, 2011)</a>. Changes to the devp2p Wire Protocol</h4> All clients currently contain statements such as the following: # pydevp2p/p2p_protocol.py if data['version'] != proto.version: log.debug('incompatible network protocols', peer=proto.peer, expected=proto.version, received=data['version']) return proto.send_disconnect(reason=reasons.incompatible_p2p_version)</code></pre> These checks make it impossible to change the version or structure of the hello packet. Dropping them enables switching to a newer protocol version: Clients implementing a newer version simply send a packet with higher version and possibly additional list elements. If such a packet is received by a node with lower version, it will blindly assume that the remote end is backwards-compatible and respond with the old handshake.</li> If the packet is received by a node with equal version, new features of the protocol can be used.</li> If the packet is received by a node with higher version, it can enable backwards-compatibility logic or drop the connection.</li> </ul> Changes to the RLPx Discovery Protocol</h4> The relaxation of discovery packet decoding rules largely codifies current practice. Most existing implementations do not care about the number of list elements (an exception being go-ethereum) and do not reject nodes with mismatching version. This behaviour is not guaranteed by the spec, though. If adopted, the change makes it possible to deploy protocol changes in a similar manner to the devp2p hello change: simply bump the version and send additional information. Older clients will ignore the additional elements and can continue to operate even when the majority of the network has moved on to a newer protocol. Changes to the RLPx TCP Handshake</h4> Discussions of the RLPx v5 changes (chunked packets, change to key derivation) have faltered in part because the v4 handshake encoding provides only one in-band way to add a version number: shortening the random portion of the nonce. Even if the RLPx v5 handshake proposal were accepted, future upgrades are hard because the handshake packet is a fixed size ECIES ciphertext with known layout. I propose the following changes to the handshake packets: Adding the length of the ciphertext as a plaintext header.</li> Encoding the body of the handshake as RLP.</li> Adding a version number to both packets in place of the token flag (unused).</li> Removing the hash of the ephemeral public key (it is redundant).</li> </ul> These changes make it possible to upgrade the RLPx TCP transport protocol in the same manner as described for the other protocols, i.e. by adding list elements and bumping the version. Since this is the first change to the RLPx handshake packet, we can seize the opportunity to remove all currently unused fields. Additional data is permitted (and in fact required) after the RLP list because the handshake packet needs to grow in order to be distinguishable from the old format. Clients can employ logic such as the following pseudocode to handle both formats simultaneously. packet = read(307, connection) if decrypt(packet) { // process as old format } else { size = unpack_16bit_big_endian(packet) packet += read(size - 307 + 2, connection) if !decrypt(packet) { // error } // process as new format }</code></pre> The plain text size prefix is perhaps the most controversial aspect of this document. It has been argued that the prefix aids adversaries that seek to filter and identify RLPx connections on the network level. This is largely a question of how much effort the adversary is willing to expense. If the recommendation to randomise the lengths is followed, pure pattern-based packet recognition is unlikely to succeed. For typical firewall operators, blocking all connections whose first two bytes form an integer in range [300,600] is probably too invasive. Port-based blocking would be a more effective measure to filter most RLPx traffic.</li> For an attacker who can afford to correlate many criteria, the size prefix would ease recognition because it adds to the indicator set. However, such an attacker could also be expected to read or participate in RLPx Discovery traffic, which would be sufficient to enable blocking of RLPx TCP connections whatever their format is.</li> </ul> Backwards Compatibility</h3> This EIP is backwards-compatible, all valid version 4 packets are still accepted. Implementation</h3> go-ethereum</a> libweb3core</a> pydevp2p</a> Test Vectors</h3> devp2p Base Protocol</h4> devp2p hello packet advertising version 22 and containing a few additional list elements: f87137916b6e6574682f76302e39312f706c616e39cdc5836574683dc6846d6f726b1682270fb840 fda1cff674c90c9a197539fe3dfb53086ace64f83ed7c6eabec741f7f381cc803e52ab2cd55d5569 bce4347107a310dfd5f88a010cd2ffd1005ca406f1842877c883666f6f836261720304</code></pre>RLPx Discovery Protocol</h4> Implementations should accept the following encoded discovery packets as valid. The packets are signed using the secp256k1 node key b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291</code></pre> ping packet with version 4, additional list elements: e9614ccfd9fc3e74360018522d30e1419a143407ffcce748de3e22116b7e8dc92ff74788c0b6663a aa3d67d641936511c8f8d6ad8698b820a7cf9e1be7155e9a241f556658c55428ec0563514365799a 4be2be5a685a80971ddcfa80cb422cdd0101ec04cb847f000001820cfa8215a8d790000000000000 000000000000000000018208ae820d058443b9a3550102</code></pre> ping packet with version 555, additional list elements and additional random data: 577be4349c4dd26768081f58de4c6f375a7a22f3f7adda654d1428637412c3d7fe917cadc56d4e5e 7ffae1dbe3efffb9849feb71b262de37977e7c7a44e677295680e9e38ab26bee2fcbae207fba3ff3 d74069a50b902a82c9903ed37cc993c50001f83e82022bd79020010db83c4d001500000000abcdef 12820cfa8215a8d79020010db885a308d313198a2e037073488208ae82823a8443b9a355c5010203 040531b9019afde696e582a78fa8d95ea13ce3297d4afb8ba6433e4154caa5ac6431af1b80ba7602 3fa4090c408f6b4bc3701562c031041d4702971d102c9ab7fa5eed4cd6bab8f7af956f7d565ee191 7084a95398b6a21eac920fe3dd1345ec0a7ef39367ee69ddf092cbfe5b93e5e568ebc491983c09c7 6d922dc3</code></pre> pong packet with additional list elements and additional random data: 09b2428d83348d27cdf7064ad9024f526cebc19e4958f0fdad87c15eb598dd61d08423e0bf66b206 9869e1724125f820d851c136684082774f870e614d95a2855d000f05d1648b2d5945470bc187c2d2 216fbe870f43ed0909009882e176a46b0102f846d79020010db885a308d313198a2e037073488208 ae82823aa0fbc914b16819237dcd8801d7e53f69e9719adecb3cc0e790c57e91ca4461c9548443b9 a355c6010203c2040506a0c969a58f6f9095004c0177a6b47f451530cab38966a25cca5cb58f0555 42124e</code></pre> findnode packet with additional list elements and additional random data: c7c44041b9f7c7e41934417ebac9a8e1a4c6298f74553f2fcfdcae6ed6fe53163eb3d2b52e39fe91 831b8a927bf4fc222c3902202027e5e9eb812195f95d20061ef5cd31d502e47ecb61183f74a504fe 04c51e73df81f25c4d506b26db4517490103f84eb840ca634cae0d49acb401d8a4c6b6fe8c55b70d 115bf400769cc1400f3258cd31387574077f301b421bc84df7266c44e9e6d569fc56be0081290476 7bf5ccd1fc7f8443b9a35582999983999999280dc62cc8255c73471e0a61da0c89acdc0e035e260a dd7fc0c04ad9ebf3919644c91cb247affc82b69bd2ca235c71eab8e49737c937a2c396</code></pre> neighbours packet with additional list elements and additional random data: c679fc8fe0b8b12f06577f2e802d34f6fa257e6137a995f6f4cbfc9ee50ed3710faf6e66f932c4c8 d81d64343f429651328758b47d3dbc02c4042f0fff6946a50f4a49037a72bb550f3a7872363a83e1 b9ee6469856c24eb4ef80b7535bcf99c0004f9015bf90150f84d846321163782115c82115db84031 55e1427f85f10a5c9a7755877748041af1bcd8d474ec065eb33df57a97babf54bfd2103575fa8291 15d224c523596b401065a97f74010610fce76382c0bf32f84984010203040101b840312c55512422 cf9b8a4097e9a6ad79402e87a15ae909a4bfefa22398f03d20951933beea1e4dfa6f968212385e82 9f04c2d314fc2d4e255e0d3bc08792b069dbf8599020010db83c4d001500000000abcdef12820d05 820d05b84038643200b172dcfef857492156971f0e6aa2c538d8b74010f8e140811d53b98c765dd2 d96126051913f44582e8c199ad7c6d6819e9a56483f637feaac9448aacf8599020010db885a308d3 13198a2e037073488203e78203e8b8408dcab8618c3253b558d459da53bd8fa68935a719aff8b811 197101a4b2b47dd2d47295286fc00cc081bb542d760717d1bdd6bec2c37cd72eca367d6dd3b9df73 8443b9a355010203b525a138aa34383fec3d2719a0</code></pre>RLPx Handshake</h4> In these test vectors, node A initiates a connection with node B. The values contained in all packets are given below: Static Key A: 49a7b37aa6f6645917e7b807e9d1c00d4fa71f18343b0d4122a4d2df64dd6fee Static Key B: b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291 Ephemeral Key A: 869d6ecf5211f1cc60418a13b9d870b22959d0c16f02bec714c960dd2298a32d Ephemeral Key B: e238eb8e04fee6511ab04c6dd3c89ce097b11f25d584863ac2b6d5b35b1847e4 Nonce A: 7e968bba13b6c50e2c4cd7f241cc0d64d1ac25c7f5952df231ac6a2bda8ee5d6 Nonce B: 559aead08264d5795d3909718cdd05abd49572e84fe55590eef31a88a08fdffd</code></pre> (Auth₁) RLPx v4 format (sent from A to B): 048ca79ad18e4b0659fab4853fe5bc58eb83992980f4c9cc147d2aa31532efd29a3d3dc6a3d89eaf 913150cfc777ce0ce4af2758bf4810235f6e6ceccfee1acc6b22c005e9e3a49d6448610a58e98744 ba3ac0399e82692d67c1f58849050b3024e21a52c9d3b01d871ff5f210817912773e610443a9ef14 2e91cdba0bd77b5fdf0769b05671fc35f83d83e4d3b0b000c6b2a1b1bba89e0fc51bf4e460df3105 c444f14be226458940d6061c296350937ffd5e3acaceeaaefd3c6f74be8e23e0f45163cc7ebd7622 0f0128410fd05250273156d548a414444ae2f7dea4dfca2d43c057adb701a715bf59f6fb66b2d1d2 0f2c703f851cbf5ac47396d9ca65b6260bd141ac4d53e2de585a73d1750780db4c9ee4cd4d225173 a4592ee77e2bd94d0be3691f3b406f9bba9b591fc63facc016bfa8</code></pre> (Auth₂) EIP-8 format with version 4 and no additional list elements (sent from A to B): 01b304ab7578555167be8154d5cc456f567d5ba302662433674222360f08d5f1534499d3678b513b 0fca474f3a514b18e75683032eb63fccb16c156dc6eb2c0b1593f0d84ac74f6e475f1b8d56116b84 9634a8c458705bf83a626ea0384d4d7341aae591fae42ce6bd5c850bfe0b999a694a49bbbaf3ef6c da61110601d3b4c02ab6c30437257a6e0117792631a4b47c1d52fc0f8f89caadeb7d02770bf999cc 147d2df3b62e1ffb2c9d8c125a3984865356266bca11ce7d3a688663a51d82defaa8aad69da39ab6 d5470e81ec5f2a7a47fb865ff7cca21516f9299a07b1bc63ba56c7a1a892112841ca44b6e0034dee 70c9adabc15d76a54f443593fafdc3b27af8059703f88928e199cb122362a4b35f62386da7caad09 c001edaeb5f8a06d2b26fb6cb93c52a9fca51853b68193916982358fe1e5369e249875bb8d0d0ec3 6f917bc5e1eafd5896d46bd61ff23f1a863a8a8dcd54c7b109b771c8e61ec9c8908c733c0263440e 2aa067241aaa433f0bb053c7b31a838504b148f570c0ad62837129e547678c5190341e4f1693956c 3bf7678318e2d5b5340c9e488eefea198576344afbdf66db5f51204a6961a63ce072c8926c</code></pre> (Auth₃) EIP-8 format with version 56 and 3 additional list elements (sent from A to B): 01b8044c6c312173685d1edd268aa95e1d495474c6959bcdd10067ba4c9013df9e40ff45f5bfd6f7 2471f93a91b493f8e00abc4b80f682973de715d77ba3a005a242eb859f9a211d93a347fa64b597bf 280a6b88e26299cf263b01b8dfdb712278464fd1c25840b995e84d367d743f66c0e54a586725b7bb f12acca27170ae3283c1073adda4b6d79f27656993aefccf16e0d0409fe07db2dc398a1b7e8ee93b cd181485fd332f381d6a050fba4c7641a5112ac1b0b61168d20f01b479e19adf7fdbfa0905f63352 bfc7e23cf3357657455119d879c78d3cf8c8c06375f3f7d4861aa02a122467e069acaf513025ff19 6641f6d2810ce493f51bee9c966b15c5043505350392b57645385a18c78f14669cc4d960446c1757 1b7c5d725021babbcd786957f3d17089c084907bda22c2b2675b4378b114c601d858802a55345a15 116bc61da4193996187ed70d16730e9ae6b3bb8787ebcaea1871d850997ddc08b4f4ea668fbf3740 7ac044b55be0908ecb94d4ed172ece66fd31bfdadf2b97a8bc690163ee11f5b575a4b44e36e2bfb2 f0fce91676fd64c7773bac6a003f481fddd0bae0a1f31aa27504e2a533af4cef3b623f4791b2cca6 d490</code></pre> (Ack₁) RLPx v4 format (sent from B to A): 049f8abcfa9c0dc65b982e98af921bc0ba6e4243169348a236abe9df5f93aa69d99cadddaa387662 b0ff2c08e9006d5a11a278b1b3331e5aaabf0a32f01281b6f4ede0e09a2d5f585b26513cb794d963 5a57563921c04a9090b4f14ee42be1a5461049af4ea7a7f49bf4c97a352d39c8d02ee4acc416388c 1c66cec761d2bc1c72da6ba143477f049c9d2dde846c252c111b904f630ac98e51609b3b1f58168d dca6505b7196532e5f85b259a20c45e1979491683fee108e9660edbf38f3add489ae73e3dda2c71b d1497113d5c755e942d1</code></pre> (Ack₂) EIP-8 format with version 4 and no additional list elements (sent from B to A): 01ea0451958701280a56482929d3b0757da8f7fbe5286784beead59d95089c217c9b917788989470 b0e330cc6e4fb383c0340ed85fab836ec9fb8a49672712aeabbdfd1e837c1ff4cace34311cd7f4de 05d59279e3524ab26ef753a0095637ac88f2b499b9914b5f64e143eae548a1066e14cd2f4bd7f814 c4652f11b254f8a2d0191e2f5546fae6055694aed14d906df79ad3b407d94692694e259191cde171 ad542fc588fa2b7333313d82a9f887332f1dfc36cea03f831cb9a23fea05b33deb999e85489e645f 6aab1872475d488d7bd6c7c120caf28dbfc5d6833888155ed69d34dbdc39c1f299be1057810f34fb e754d021bfca14dc989753d61c413d261934e1a9c67ee060a25eefb54e81a4d14baff922180c395d 3f998d70f46f6b58306f969627ae364497e73fc27f6d17ae45a413d322cb8814276be6ddd13b885b 201b943213656cde498fa0e9ddc8e0b8f8a53824fbd82254f3e2c17e8eaea009c38b4aa0a3f306e8 797db43c25d68e86f262e564086f59a2fc60511c42abfb3057c247a8a8fe4fb3ccbadde17514b7ac 8000cdb6a912778426260c47f38919a91f25f4b5ffb455d6aaaf150f7e5529c100ce62d6d92826a7 1778d809bdf60232ae21ce8a437eca8223f45ac37f6487452ce626f549b3b5fdee26afd2072e4bc7 5833c2464c805246155289f4</code></pre> (Ack₃) EIP-8 format with version 57 and 3 additional list elements (sent from B to A): 01f004076e58aae772bb101ab1a8e64e01ee96e64857ce82b1113817c6cdd52c09d26f7b90981cd7 ae835aeac72e1573b8a0225dd56d157a010846d888dac7464baf53f2ad4e3d584531fa203658fab0 3a06c9fd5e35737e417bc28c1cbf5e5dfc666de7090f69c3b29754725f84f75382891c561040ea1d dc0d8f381ed1b9d0d4ad2a0ec021421d847820d6fa0ba66eaf58175f1b235e851c7e2124069fbc20 2888ddb3ac4d56bcbd1b9b7eab59e78f2e2d400905050f4a92dec1c4bdf797b3fc9b2f8e84a482f3 d800386186712dae00d5c386ec9387a5e9c9a1aca5a573ca91082c7d68421f388e79127a5177d4f8 590237364fd348c9611fa39f78dcdceee3f390f07991b7b47e1daa3ebcb6ccc9607811cb17ce51f1 c8c2c5098dbdd28fca547b3f58c01a424ac05f869f49c6a34672ea2cbbc558428aa1fe48bbfd6115 8b1b735a65d99f21e70dbc020bfdface9f724a0d1fb5895db971cc81aa7608baa0920abb0a565c9c 436e2fd13323428296c86385f2384e408a31e104670df0791d93e743a3a5194ee6b076fb6323ca59 3011b7348c16cf58f66b9633906ba54a2ee803187344b394f75dd2e663a57b956cb830dd7a908d4f 39a2336a61ef9fda549180d4ccde21514d117b6c6fd07a9102b5efe710a32af4eeacae2cb3b1dec0 35b9593b48b9d3ca4c13d245d5f04169b0b1</code></pre> Node B derives the connection secrets for (Auth₂, Ack₂) as follows: aes-secret = 80e8632c05fed6fc2a13b0f8d31a3cf645366239170ea067065aba8e28bac487 mac-secret = 2ea74ec5dae199227dff1af715362700e989d889d7a493cb0639691efb8e5f98</code></pre> Running B's ingress-mac</code> keccak state on the string "foo" yields the hash ingress-mac("foo") = 0c7ec6340062cc46f5e9f1e3cf86f8c8c403c5a0964f5df0ebd34a75ddc86db5</code></pre>Copyright</h3> Copyright and related rights waived via CC0</a>.

eth/66 request identifier

2020-01-17T00:00:00+00:00

Abstract</h2>
The `eth</code> protocol defines various request and response commands that are used to exchange data between Ethereum nodes. For example, to ask a peer node for a specific set of headers, a node sends it the` `GetBlockHeaders</code></a> command.`
`Citing from the GetBlockHeaders</code> spec definition</a>:`
[block: {P, B_32}, maxHeaders: P, skip: P, reverse: P in {0, 1}]</code> </blockquote> Require peer to return a BlockHeaders</code> message. Reply must contain a number of block headers, of rising number when reverse</code> is 0</code>, falling when 1</code>, skip</code> blocks apart, beginning at block block</code> (denoted by either number or hash) in the canonical chain, and with at most maxHeaders</code> items. </blockquote> The node that receives the GetBlockHeaders</code> command should answer it with the BlockHeaders</code></a> response command accordingly. Citing from the BlockHeaders</code> spec definition</a>: [blockHeader_0, blockHeader_1, ...]</code> </blockquote> Reply to GetBlockHeaders</code>. The items in the list (following the message ID) are block headers in the format described in the main Ethereum specification, previously asked for in a GetBlockHeaders message. This may validly contain no block headers if none of the requested block headers were found. The number of headers that can be requested in a single message may be subject to implementation-defined limits. </blockquote> Let's consider a client making many simultaneous requests for GetBlockHeaders</code> to one of its peers. By nature it can not be guaranteed that the expected responses arrive in the same order as they were sent. For the client to associate the incoming responses to the correct requests it has to loop through all pending requests trying to match it with the incoming response based on its contents. This can be particular tricky for responses that are ambiguous such as empty responses. This EIP proposes to change the GetBlockHeaders</code> and the BlockHeaders</code> command to include a request_id</code>. The request_id</code> is a 64-bit integer set by the client when it makes the request. On the responding side, the exact same request_id</code> from the incoming request is put back into the response object. This change allows the requesting client to match incoming responses directly back to their pending requests without going through all of the pending requests to check if they might match based on the response data. The selected request/response pair serves as an example for many similar request/response pairs in the eth</code> networking protocol. Motivation</h2> The lack of request identifiers in the request / response paris of the eth</code> protocol puts unnecessary burden of code complexity into every Ethereum client. It also makes the communication slightly less efficient. Another argument can be made that the addition of request identifiers makes the protocol more aligned with the les</code> protocol which does already defines request identifiers for each request / response pair. Specification</h2> Change the following message types in the eth</code> protocol: GetBlockHeaders (0x03)</code> Current (eth/65): [block: {P, B_32}, maxHeaders: P, skip: P, reverse: P in {0, 1}]</code></li> Then (eth/66): [request_id: P, [block: {P, B_32}, maxHeaders: P, skip: P, reverse: P in {0, 1}]]</code></li> </ul> </li> BlockHeaders (0x04)</code> Current (eth/65): [blockHeader_0, blockHeader_1, ...]</code></li> Then (eth/66): [request_id: P, [blockHeader_0, blockHeader_1, ...]]</code></li> </ul> </li> GetBlockBodies (0x05)</code> Current (eth/65): [hash_0: B_32, hash_1: B_32, ...]</code></li> Then (eth/66): [request_id: P, [hash_0: B_32, hash_1: B_32, ...]]</code></li> </ul> </li> GetPooledTransactions (0x09)</code>: Current (eth/65)[hash_0: B_32, hash_1: B_32, ...]</code></li> Then (eth/66)[request_id: P, [hash_0: B_32, hash_1: B_32, ...]]</code></li> </ul> </li> PooledTransactions (0x0a)</code>: Current (eth/65)[[nonce: P, receivingAddress: B_20, value: P, ...], ...]</code></li> Then (eth/66)[request_id: P, [[nonce: P, receivingAddress: B_20, value: P, ...], ...]]</code></li> </ul> </li> BlockBodies (0x06)</code> Current (eth/65): [hash_0: B_32, hash_1: B_32, ...]</code></li> Then (eth/66): [request_id: P, [hash_0: B_32, hash_1: B_32, ...]]</code></li> </ul> </li> GetNodeData (0x0d)</code> Current (eth/65): [hash_0: B_32, hash_1: B_32, ...]</code></li> Then (eth/66): [request_id: P, [hash_0: B_32, hash_1: B_32, ...]]</code></li> </ul> </li> NodeData (0x0e)</code> Current (eth/65): [value_0: B, value_1: B, ...]</code></li> Then (eth/66): [request_id: P, [value_0: B, value_1: B, ...]]</code></li> </ul> </li> GetReceipts (0x0f)</code> Current (eth/65): [blockHash_0: B_32, blockHash_1: B_32, ...]</code></li> Then (eth/66): [request_id: P, [blockHash_0: B_32, blockHash_1: B_32, ...]]</code></li> </ul> </li> Receipts (0x10)</code> Current (eth/65): [[receipt_0, receipt_1], ...]</code></li> Then (eth/66): [request_id: P, [[receipt_0, receipt_1], ...]]</code></li> </ul> </li> </ul> To elaborate, each command is altered in the following way: Create a list with the request_id</code> being the first element.</li> Make the second element the list that defines the whole command in the current scheme.</li> </ol> The request_id</code> has the following characteristics: 64 bit integer</li> Doesn't need to be sequential (can be random)</li> Does allow duplicates</li> </ul> Rationale</h2> Q: The efficiency gains might encourage clients to flood their peers with too many simultaneous requests Peers can always throttle or disconnect if they don't feel treated well. This is the same as today. Q: If les</code> already defines the commands like this, why not just use the les</code> protocol? In practice, peers that serve the les</code> protocol are much harder to find in the network. The reasons for this are varied but might boil down to client defaults, immature implementations or missing incentives. Q: Networking works today, isn't this just adding bloat? This is adding a single integer per command while at the same time reducing code complexity and improving networking efficiency. The addition seems justified. Q: Why not demand request ids to be sequential? Assuming request ids start always to count from 0</code> upon connection, things will become messy when connections are lost and clients reconnect and start over with the same request ids that they had used in the previous session. Q: Why allow duplicate request ids? The main benefit is flexibility and simplicity on the implementation side. Clients could decide to share the same ids across multiple different request types since they are naturally separated anyway. Clients could even decide to not rely on request ids at all, therefore using the same constant request id across all requests. Q: Why choose a 64-bit integer for the request ids 64-bit integer were chosen to keep compatibility with the les</code> protocol. Backwards Compatibility</h2> This EIP extends the eth</code> protocol in a backwards incompatible way and requires rolling out a new version, eth/66</code>. However, devp2p</code> supports running multiple versions of the same wire protocol side-by-side, so rolling out eth/66</code> does not require client coordination, since non-updated clients can keep using eth/65</code>. This EIP does not change the consensus engine, thus does not require a hard fork. Test Cases</h2> These testcases cover RLP-encoding of all the redefined messages types, where the rlp</code> portion is the rlp-encoding of the message defined in the data</code> portion. { "type": "GetBlockHeadersPacket66", "rlp": "0xe8820457e4a000000000000000000000000000000000000000000000000000000000deadc0de050580", "data": { "RequestId": 1111, "Origin": { "Hash": "0x00000000000000000000000000000000000000000000000000000000deadc0de", "Number": 0 }, "Amount": 5, "Skip": 5, "Reverse": false } }</code></pre>{ "type": "GetBlockHeadersPacket66", "rlp": "0xca820457c682270f050580", "data": { "RequestId": 1111, "Origin": { "Hash": "0x0000000000000000000000000000000000000000000000000000000000000000", "Number": 9999 }, "Amount": 5, "Skip": 5, "Reverse": false } }</code></pre>{ "type": "BlockHeadersPacket66", "rlp": "0xf90202820457f901fcf901f9a00000000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000000940000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000000b90100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008208ae820d0582115c8215b3821a0a827788a00000000000000000000000000000000000000000000000000000000000000000880000000000000000", "data": { "RequestId": 1111, "BlockHeadersPacket": [ { "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000", "sha3Uncles": "0x0000000000000000000000000000000000000000000000000000000000000000", "miner": "0x0000000000000000000000000000000000000000", "stateRoot": "0x0000000000000000000000000000000000000000000000000000000000000000", "transactionsRoot": "0x0000000000000000000000000000000000000000000000000000000000000000", "receiptsRoot": "0x0000000000000000000000000000000000000000000000000000000000000000", "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", "difficulty": "0x8ae", "number": "0xd05", "gasLimit": "0x115c", "gasUsed": "0x15b3", "timestamp": "0x1a0a", "extraData": "0x7788", "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000", "nonce": "0x0000000000000000", "hash": "0x8c2f2af15b7b563b6ab1e09bed0e9caade7ed730aec98b70a993597a797579a9" } ] } }</code></pre>{ "type": "GetBlockBodiesPacket66", "rlp": "0xf847820457f842a000000000000000000000000000000000000000000000000000000000deadc0dea000000000000000000000000000000000000000000000000000000000feedbeef", "data": { "RequestId": 1111, "GetBlockBodiesPacket": [ "0x00000000000000000000000000000000000000000000000000000000deadc0de", "0x00000000000000000000000000000000000000000000000000000000feedbeef" ] } }</code></pre>{ "type": "BlockBodiesPacket66", "rlp": "0xf902dc820457f902d6f902d3f8d2f867088504a817c8088302e2489435353535353535353535353535353535353535358202008025a064b1702d9298fee62dfeccc57d322a463ad55ca201256d01f62b45b2e1c21c12a064b1702d9298fee62dfeccc57d322a463ad55ca201256d01f62b45b2e1c21c10f867098504a817c809830334509435353535353535353535353535353535353535358202d98025a052f8f61201b2b11a78d6e866abc9c3db2ae8631fa656bfe5cb53668255367afba052f8f61201b2b11a78d6e866abc9c3db2ae8631fa656bfe5cb53668255367afbf901fcf901f9a00000000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000000940000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000000b90100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008208ae820d0582115c8215b3821a0a827788a00000000000000000000000000000000000000000000000000000000000000000880000000000000000", "data": { "RequestId": 1111, "BlockBodiesPacket": [ { "Transactions": [ { "nonce": "0x8", "gasPrice": "0x4a817c808", "gas": "0x2e248", "to": "0x3535353535353535353535353535353535353535", "value": "0x200", "input": "0x", "v": "0x25", "r": "0x64b1702d9298fee62dfeccc57d322a463ad55ca201256d01f62b45b2e1c21c12", "s": "0x64b1702d9298fee62dfeccc57d322a463ad55ca201256d01f62b45b2e1c21c10", "hash": "0x588df025c4c2d757d3e314bd3dfbfe352687324e6b8557ad1731585e96928aed" }, { "nonce": "0x9", "gasPrice": "0x4a817c809", "gas": "0x33450", "to": "0x3535353535353535353535353535353535353535", "value": "0x2d9", "input": "0x", "v": "0x25", "r": "0x52f8f61201b2b11a78d6e866abc9c3db2ae8631fa656bfe5cb53668255367afb", "s": "0x52f8f61201b2b11a78d6e866abc9c3db2ae8631fa656bfe5cb53668255367afb", "hash": "0xf39c7dac06a9f3abf09faf5e30439a349d3717611b3ed337cd52b0d192bc72da" } ], "Uncles": [ { "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000", "sha3Uncles": "0x0000000000000000000000000000000000000000000000000000000000000000", "miner": "0x0000000000000000000000000000000000000000", "stateRoot": "0x0000000000000000000000000000000000000000000000000000000000000000", "transactionsRoot": "0x0000000000000000000000000000000000000000000000000000000000000000", "receiptsRoot": "0x0000000000000000000000000000000000000000000000000000000000000000", "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", "difficulty": "0x8ae", "number": "0xd05", "gasLimit": "0x115c", "gasUsed": "0x15b3", "timestamp": "0x1a0a", "extraData": "0x7788", "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000", "nonce": "0x0000000000000000", "hash": "0x8c2f2af15b7b563b6ab1e09bed0e9caade7ed730aec98b70a993597a797579a9" } ] } ] } }</code></pre>{ "type": "GetNodeDataPacket66", "rlp": "0xf847820457f842a000000000000000000000000000000000000000000000000000000000deadc0dea000000000000000000000000000000000000000000000000000000000feedbeef", "data": { "RequestId": 1111, "GetNodeDataPacket": [ "0x00000000000000000000000000000000000000000000000000000000deadc0de", "0x00000000000000000000000000000000000000000000000000000000feedbeef" ] } }</code></pre>{ "type": "NodeDataPacket66", "rlp": "0xce820457ca84deadc0de84feedbeef", "data": { "RequestId": 1111, "NodeDataPacket": [ "0xdeadc0de", "0xfeedbeef" ] } }</code></pre>{ "type": "GetReceiptsPacket66", "rlp": "0xf847820457f842a000000000000000000000000000000000000000000000000000000000deadc0dea000000000000000000000000000000000000000000000000000000000feedbeef", "data": { "RequestId": 1111, "GetReceiptsPacket": [ "0x00000000000000000000000000000000000000000000000000000000deadc0de", "0x00000000000000000000000000000000000000000000000000000000feedbeef" ] } }</code></pre>{ "type": "ReceiptsPacket66", "rlp": "0xf90172820457f9016cf90169f901668001b9010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000f85ff85d940000000000000000000000000000000000000011f842a0000000000000000000000000000000000000000000000000000000000000deada0000000000000000000000000000000000000000000000000000000000000beef830100ff", "data": { "RequestId": 1111, "ReceiptsPacket": [ [ { "root": "0x", "status": "0x0", "cumulativeGasUsed": "0x1", "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", "logs": [ { "address": "0x0000000000000000000000000000000000000011", "topics": [ "0x000000000000000000000000000000000000000000000000000000000000dead", "0x000000000000000000000000000000000000000000000000000000000000beef" ], "data": "0x0100ff", "blockNumber": "0x0", "transactionHash": "0x0000000000000000000000000000000000000000000000000000000000000000", "transactionIndex": "0x0", "blockHash": "0x0000000000000000000000000000000000000000000000000000000000000000", "logIndex": "0x0", "removed": false } ], "transactionHash": "0x00000000000000000000000000000000000000000000000000000000deadc0de", "contractAddress": "0x0000000000000000000000000000000000011111", "gasUsed": "0x1b207", "blockHash": "0x0000000000000000000000000000000000000000000000000000000000000000", "transactionIndex": "0x0" } ] ] } }</code></pre>{ "type": "GetPooledTransactionsPacket66", "rlp": "0xf847820457f842a000000000000000000000000000000000000000000000000000000000deadc0dea000000000000000000000000000000000000000000000000000000000feedbeef", "data": { "RequestId": 1111, "GetPooledTransactionsPacket": [ "0x00000000000000000000000000000000000000000000000000000000deadc0de", "0x00000000000000000000000000000000000000000000000000000000feedbeef" ] } }</code></pre>{ "type": "PooledTransactionsPacket66", "rlp": "0xf8d7820457f8d2f867088504a817c8088302e2489435353535353535353535353535353535353535358202008025a064b1702d9298fee62dfeccc57d322a463ad55ca201256d01f62b45b2e1c21c12a064b1702d9298fee62dfeccc57d322a463ad55ca201256d01f62b45b2e1c21c10f867098504a817c809830334509435353535353535353535353535353535353535358202d98025a052f8f61201b2b11a78d6e866abc9c3db2ae8631fa656bfe5cb53668255367afba052f8f61201b2b11a78d6e866abc9c3db2ae8631fa656bfe5cb53668255367afb", "data": { "RequestId": 1111, "PooledTransactionsPacket": [ { "nonce": "0x8", "gasPrice": "0x4a817c808", "gas": "0x2e248", "to": "0x3535353535353535353535353535353535353535", "value": "0x200", "input": "0x", "v": "0x25", "r": "0x64b1702d9298fee62dfeccc57d322a463ad55ca201256d01f62b45b2e1c21c12", "s": "0x64b1702d9298fee62dfeccc57d322a463ad55ca201256d01f62b45b2e1c21c10", "hash": "0x588df025c4c2d757d3e314bd3dfbfe352687324e6b8557ad1731585e96928aed" }, { "nonce": "0x9", "gasPrice": "0x4a817c809", "gas": "0x33450", "to": "0x3535353535353535353535353535353535353535", "value": "0x2d9", "input": "0x", "v": "0x25", "r": "0x52f8f61201b2b11a78d6e866abc9c3db2ae8631fa656bfe5cb53668255367afb", "s": "0x52f8f61201b2b11a78d6e866abc9c3db2ae8631fa656bfe5cb53668255367afb", "hash": "0xf39c7dac06a9f3abf09faf5e30439a349d3717611b3ed337cd52b0d192bc72da" } ] } }</code></pre>Security Considerations</h2> None Copyright</h2> Copyright and related rights waived via CC0</a>.

Ethereum Improvement Proposals - Networking

eth/vhash - Blob-Aware Mempool

2025-11-29T00:00:00+00:00

Abstract</h2>
This EIP eliminates the need to redistribute blob content in the mempool if only the metadata (fees) of a transaction are updated, making RBF (replace-by-fee) more efficient and cheaper for the network. It achieves this modifying the devp2p ‘eth’ protocol to address blobs in type 3 transaction sidecars by content (vhash).

Motivation</h2>
In the current version of devp2p eth/69, when a transaction is replaced, it must be redistributed in the mempool like any new transaction. Even if the actual content is largely the same, protocol participants have no means to figure this out before getting the full content, making a replacement use the same amount of network resources as a new transaction would.
What is especially problematic is that RBF is used most in periods of fee volatility, and a network overload is the typical case of such a situation. Thus, when there is already high demand, we make the situation worse by adding extra traffic redistributing blob content that was already distributed.

Specification</h2>

Transactions (0x02) changes</h3>
Type 3 transaction should be sent without sidecar

PooledTransactions (0x0a) changes</h3>
Type 3 transaction should be sent without sidecar

GetPooledBlobs (msg code to be assigned <-- TODO -->)</h3>
[request-id: P, [vhash₁: B_32, vhash₂: B_32, ...]]
This message requests blobs from the recipient's transaction pool by vhash.

PooledBlobs (msg code to be assigned <-- TODO -->)</h3>

[request-id: P, [blob₁, blob₂...]]

This is the response to GetPooledBlobs, returning the requested blobs. The items in the list are blobs in the format described in the main Ethereum specification.

Note: optionally, we might prefix the blob format with the blob version number

Note: optionally, we might decide to improve the blob format allowing nodes to reconstruct RS encoding instead of using extra bandwidth, by sending the following fields per blob:

blob version</li>
blob content, excluding the erasure coding extension in case of version 1</li>
blob commitment</li>

blob proof(s), including cell proofs of the erasure coded piece in case of version 1</li> </ul>
It is important to include all cell proofs to keep reconstruction CPU-efficient. </blockquote>
The blobs must be in the same order as in the request, but it is OK to skip blobs which are not available. Since the recipient have to check that transmitted blob hashes correspond to the requested vhashes anyway, we can avoid sending the list of vhashes as part of this message.

Note: Optionally, we could extend this message with a bitmap of sent/unsent blobs from the request, or with the list of vhashes sent. This information is redundant, but it can simplify processing on the receiver side. </blockquote>
Other spec changes</h3>
EIP-4844 introduced the following:
Nodes MUST NOT automatically broadcast blob transactions to their peers. Instead, those transactions are only announced using NewPooledTransactionHashes messages, and can then be manually requested via GetPooledTransactions.</code></pre> The above should be changed as follows: Nodes MUST send (broadcast or send in NewPooledTransactionHashes) blob transaction **without** sidecars to their peers. Peers can then request blob content using `GetPooledBlobs` messages. Nodes MUST NOT forward blob transactions before receiving and validating all blobs".</code></pre>Rationale</h2> A typical blob transaction RBF changes the fees only, while the sidecar (blob content) remains the same. If a node that has the previous version would know this, it could avoid pulling the sidecar, largely reducing bandwidth consumption. However, this is not possible with the current messaging. To make this happen, we have to expose blob (or at least sidecar) identifiers in mempool messaging. Should we use blob identifiers or a sidecar identifier?</h3> We can either use vhashes, or a sidecar level hash. The latter has the slight advantage of being a single element, thus simplifying message format, but it has several disadvantages: It would be a new identifier, while the blob level vhash is already well established (just not in devp2p)</li> It would not allow restructuring the message, sending e.g. less blobs under a fee surge</li> </ul> Thus, we use vhashes. Implementation options</h3> There are several options to bring vhashes to devp2p messaging: Option 1</h4> Extend announcements with vhashes</li> Allow nodes to request transaction with/without sidecar content, or even selecting which parts are needed (bitmap or vhash list)</li> </ul> Option 2</h4> Extend announcements with nonce (see EIP-8077)</li> If hash differs from what we have, request with/without sidecar content based on whether we already have the sidecars for the same nonce, assuming this is a simple replacement</li> Request again with sidecar if vhashes differ</li> </ul> Option 3 (selected)</h4> Push (or announce and then pull) type 3 transaction without sidecar</li> Allow to request sidecar separately (new message type)</li> </ul> At first it might seem that Option 3 is slowing down distribution, adding one more RTT latency per hop. However, since most type 3 transactions are small without a sidecar, we could change the protocol behaviour to allow pushing these transactions without the sidecar, leaving it to the receiver of the push to ask for the blobs if needed. Forwarding of type 3 transactions without sidecar should be prohibited until sidecars are fetched and the content can be verified. After considering the above options, we chose to propose Option 3, introducing a new message type. Relation to other EIPs in draft state</h3> EIP-8077 (announce source and nonce): the changes can be simply combined.</li> EIP-8070 (Sparse blobpool): both EIPs change how blob transactions are propagated over the network. The goal of the two EIPs are different. EIP-8070 is about a proportional bandwidth reduction in the normal case without dealing with specifics of RBF. This EIP is about enabling RBF without using extra bandwidth. The two EIPs can be combined, but the combination depends on the order of introduction, hence we leave this for later <-- TODO -->.</li> </ul> Backwards Compatibility</h2> This EIP changes the eth protocol and requires rolling out a new version. Supporting multiple versions of a wire protocol is routine practice. Rolling out this new version does not break older clients, since they can keep using the previous protocol version. This EIP does not change consensus rules of the EVM and does not require a hard fork. Security Considerations</h2> Copyright</h2> Copyright and related rights waived via CC0.

Specification</h2>

NewPooledTransactionHashes message changes</h3>

Modify the NewPooledTransactionHashes (0x08) message as follows:

(eth/69): [txtypes: B, [txsize₁: P, txsize₂: P, ...], [txhash₁: B_32, txhash₂: B_32, ...]]

(eth/XX): [txtypes: B, [txsize₁: P, txsize₂: P, ...], [txhash₁: B_32, txhash₂: B_32, ...], [txsource₁: B_20, txsource₂: B_20, ...], [txnonce₁: P, txnonce₂: P, ...]]

Changes to message handling</h3>

Changes on the sender side of NewPooledTransactionHashes</code> messages are trivial. Only the transmitted data changes. Since the size of announcements is increased, implementations MAY revisit the condition (typically transaction size) to select between eager push and announcement.

At the receiver side of NewPooledTransactionHashes</code> messages the extra information can be used to improve scheduling choices, however these are not mandated by the protocol and thus we leave it to implementations.

Ethereum Improvement Proposals - Networking

eth/vhash - Blob-Aware Mempool

Specification</h2>

Transactions (0x02) changes</h3> Type 3 transaction should be sent without sidecar</p>

PooledTransactions (0x0a) changes</h3> Type 3 transaction should be sent without sidecar</p>

GetPooledBlobs (msg code to be assigned <-- TODO -->)</h3> [request-id: P, [vhash₁: B_32, vhash₂: B_32, ...]]</p> This message requests blobs from the recipient's transaction pool by vhash.</p>

Implementation options</h3> There are several options to bring vhashes to devp2p messaging:</p>

Security Considerations</h2>

Copyright</h2> Copyright and related rights waived via CC0.</p>

eth/XX - announce transactions with nonce

Abstract</h2> This EIP improves mempool propagation, extending the devp2p 'eth' protocol's NewPooledTransactionHashes</code> message to also announce each transaction's source address and nonce together with the already announced hash, type, and size.</p>

Specification</h2>

Overhead</h3> The modification adds a significant overhead to announcements by adding a B_20 and a variable size filed to the current B_32, size, and type fields. We think this overhead is worth the additional gain in protocol efficiency. Details TBD <-- TODO --> .</p>

Copyright</h2> Copyright and related rights waived via CC0.</p>

Sparse Blobpool

Rationale</h2>

Security Considerations</h2>

eth/70 - partial block receipt lists

Abstract</h2> This EIP modifies the 'eth' p2p protocol to allow requesting partial block receipt lists.</p>

Motivation</h2> As Ethereum moves toward a higher block gas limit on mainnet, the worst-case total size of a block receipts list also becomes larger, and may eventually exceed the 10MiB message size limit commonly applied in clients. This can lead to sync failures.</p>

Specification</h2> Modify the encoding for receipts in the Receipts (0x10)</code> message as follows:</p>

etha - Sharded Blocks Subprotocol

Specification</h2> Subprotocol Handshake</h3> Introduce a new subprotocol named etha</code>.</li>

Subprotocol Handshake</h3> Introduce a new subprotocol named etha</code>.</li>

Supported Messages</h3> The etha</code> subprotocol MUST</strong> include support for the following messages from the eth/69</code> protocol to facilitate historical data serving:</p>

Copyright</h2> This document is CC0-licensed; rights are waived through CC0</a>.</p>

eth/69 - history expiry and simpler receipts

Specification</h2>

Status message changes</h3> Modify the Status (0x00)</code> message as follows:</p>

Security Considerations</h2> None</p> Copyright</h2> Copyright and related rights waived via CC0</a>.</p>

Copyright</h2> Copyright and related rights waived via CC0</a>.</p>

Extension of EIP-778 for "client" ENR Entry

Specification</h2> The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 and RFC 8174.</p>

Rationale</h2> One key was chosen over using many keys to make efficient use of space. The use of one string, however, does not align with other EIPs of similar purpose and as such the RLP list was decided as the best encoding.</p>

Backwards Compatibility</h2> This EIP is fully backwards compatible as it extends the ENR specification by adding an optional entry. Existing implementations that do not recognize the "client" entry will ignore it without any adverse effects on ENR processing or network behavior.</p>

Test Cases</h2> A node running Geth version 1.10.0 on the mainnet might have an ENR client</code> entry like:</p>

Copyright</h2> Copyright and related rights waived via CC0</a>.</p>

eth/70 - Cease serving history before PoS

Abstract</h2> Execution layer clients will no longer request or respond to p2p queries about block data before the Paris upgrade.</p>

Specification</h2> Add a new eth</code> protocol capability with version 70</code>.</p> Clients connected on this version must not make or respond to p2p queries about block bodies or receipts before block 15537393.</p> The affected protocol messages are:</p>

Rationale</h2>

Backwards Compatibility</h2> After this EIP is activated, nodes will no longer be able to full sync from the devp2p network. To continue doing so, they must retrieve the data out-of-band.</p>

Security Considerations</h2> TODO</p>

eth/70 - available-blocks-extended protocol

Specification</h2> Advertise a new eth</code> protocol capability (version) at eth/70</code>.</p>

Security Considerations</h2> This change is not a standardization of not storing and serving historical blocks before the implementation of alternative historical blocks storage solutions.</p>

Copyright</h2> Copyright and related rights waived via CC0</a>.</p>

Forkid checks based on timestamps

Specification</h2> Each node maintains the following values:</p>

Rationale</h2> Shanghai will be scheduled by timestamp thus the forkid calculations need to be updated to work with timestamps and blocks. Since all block number based forks are before time based forks, nodes need to check the block based forks before the time based forks.</p>

Security Considerations</h2> No known security risks</p>

Copyright</h2> Copyright and related rights waived via CC0</a>.</p>

eth/68 - Add tx type to tx announcement

"eth/67 - Removal of GetNodeData"

Specification</h2> Remove the following message types from the eth</code> protocol:</p> GetNodeData (0x0d)</code> (eth/66)</strong>: [request_id: P, [hash_0: B_32, hash_1: B_32, ...]]</code></li> </ul> </li>

Bound Historical Data in Execution Clients

Abstract</h2> Clients must stop serving historical headers, bodies, and receipts older than one year on the p2p layer. Clients may locally prune this historical data.</p>

Rationale</h2> This proposal forces clients to stop serving old historical data over p2p. We make this explicit to force clients to seek historical data from other sources, instead of relying on the optional behavior of some clients which would result in quality degradation.</p>

Why a year?</h3> This proposal sets HISTORY_PRUNE_EPOCHS</code> to 82125 epochs (one earth year). This constant is large enough to provide sufficient room for the Weak Subjectivity Period to grow, and it's also small enough so as to not occupy too much disk space.</p>

Transactions (0x02) changes</h3>
Type 3 transaction should be sent without sidecar</p>

PooledTransactions (0x0a) changes</h3>
Type 3 transaction should be sent without sidecar</p>

GetPooledBlobs (msg code to be assigned <-- TODO -->)</h3>
[request-id: P, [vhash₁: B_32, vhash₂: B_32, ...]]</p>
This message requests blobs from the recipient's transaction pool by vhash.</p>

Implementation options</h3>
There are several options to bring vhashes to devp2p messaging:</p>

Copyright</h2>
Copyright and related rights waived via CC0.</p>

Abstract</h2>
This EIP improves mempool propagation, extending the devp2p 'eth' protocol's `NewPooledTransactionHashes</code> message to also announce each transaction's source address and nonce together with the already announced hash, type, and size.</p>`

Overhead</h3>
The modification adds a significant overhead to announcements by adding a B_20 and a variable size filed to the current B_32, size, and type fields. We think this overhead is worth the additional gain in protocol efficiency. Details TBD <-- TODO --> .</p>

Copyright</h2>
Copyright and related rights waived via CC0.</p>

Abstract</h2>
This EIP modifies the 'eth' p2p protocol to allow requesting partial block receipt lists.</p>

Motivation</h2>
As Ethereum moves toward a higher block gas limit on mainnet, the worst-case total size of a block receipts list also becomes larger, and may eventually exceed the 10MiB message size limit commonly applied in clients. This can lead to sync failures.</p>

Copyright</h2>
This document is CC0-licensed; rights are waived through CC0</a>.</p>

`Security Considerations</h2> None</p> Copyright</h2> Copyright and related rights waived via CC0</a>.</p>`

Copyright</h2>
Copyright and related rights waived via CC0</a>.</p>

Rationale</h2>
One key was chosen over using many keys to make efficient use of space. The use of one string, however, does not align with other EIPs of similar purpose and as such the RLP list was decided as the best encoding.</p>

Backwards Compatibility</h2>
This EIP is fully backwards compatible as it extends the ENR specification by adding an optional entry. Existing implementations that do not recognize the "client" entry will ignore it without any adverse effects on ENR processing or network behavior.</p>

Copyright</h2>
Copyright and related rights waived via CC0</a>.</p>

Abstract</h2>
Execution layer clients will no longer request or respond to p2p queries about block data before the Paris upgrade.</p>

Backwards Compatibility</h2>
After this EIP is activated, nodes will no longer be able to full sync from the devp2p network. To continue doing so, they must retrieve the data out-of-band.</p>

Security Considerations</h2>
TODO</p>

Security Considerations</h2>
This change is not a standardization of not storing and serving historical blocks before the implementation of alternative historical blocks storage solutions.</p>

Copyright</h2>
Copyright and related rights waived via CC0</a>.</p>

Rationale</h2>
Shanghai will be scheduled by timestamp thus the forkid calculations need to be updated to work with timestamps and blocks. Since all block number based forks are before time based forks, nodes need to check the block based forks before the time based forks.</p>

Security Considerations</h2>
No known security risks</p>

Copyright</h2>
Copyright and related rights waived via CC0</a>.</p>

Abstract</h2>
Clients must stop serving historical headers, bodies, and receipts older than one year on the p2p layer. Clients may locally prune this historical data.</p>

Rationale</h2>
This proposal forces clients to stop serving old historical data over p2p. We make this explicit to force clients to seek historical data from other sources, instead of relying on the optional behavior of some clients which would result in quality degradation.</p>

Why a year?</h3>
This proposal sets `HISTORY_PRUNE_EPOCHS</code> to 82125 epochs (one earth year). This constant is large enough to provide sufficient room for the Weak Subjectivity Period to grow, and it's also small enough so as to not occupy too much disk space.</p>`