feat(rpc): simplify SyncChainMmr endpoint

[kkovaacs]

Miden client uses the SyncChainMmr endpoint exclusively to sync up to the committed tip of the chain. This PR simplifies the request so that it no longer supports arbitrary block ranges. The client is now expected to include the block number of its latest known block in the request and the response always contains the delta up to the committed tip.

The new interface looks like this:

// Chain MMR synchronization request.
message SyncChainMmrRequest {
    // Block number from which to synchronize. Set this to the last block
    // already present in the caller's MMR so the delta begins at the next block.
    fixed32 current_block_height = 1;
}

// Represents the result of syncing chain MMR.
message SyncChainMmrResponse {
    // Block header for the committed chain tip.
    blockchain.BlockHeader latest_committed_header = 1;

    // Block number of the last proven block.
    primitives.BlockNumber latest_proven_block_num = 2;

    // Data needed to update the partial MMR from `request.current_block_height + 1` to
    // the committed chain tip.
    primitives.MmrDelta latest_committed_mmr_delta = 3;

    // Merkle path to verify the committed chain tip's inclusion in the MMR.
    primitives.MerklePath latest_committed_mmr_path = 4;
}

Extra fields added are:

• latest_proven_block_num: now we always include the block number of the proven block
• latest_committed_mmr_path is the MMR path of the committed tip

Other changes:

• latest_committed_mmr_delta and latest_committed_mmr_path are only present in the response if the delta is non-empty (ie. current_block_height != latest_committed_header)

Closes #2044

[bobbinth]

Without having looked at the code, a couple of questions/comments:

• I imagine in the future we'd introduce at least one more "levels" of finality. For example, once a block is posted to L1, we could say that the block has been "finalized" (or something similar). This would require adding a new field - something like latest_finalized_block_num - right?
• Hopefully, at some point in the future (though, probably a couple of years out), we'll make proving fast enough where the difference between committed and proven would disappear (e.g., we'd effectively merge committed and proven). How would we update this endpoint in this case?
• If a client wants to execute transactions against the proven (or, in the future, finalized) block - how would they accomplish this? SyncChainMmr would give them the last committed block header - but how would they get the last proven one (together with the relevant MMR info)?

I guess the alternative shape of this endpoint could be:

message SyncChainMmrRequest {
    fixed32 current_block_height = 1;
    FinalityLevel finality_level = 2;
}

enum FinalityLevel {
    FINALITY_UNSPECIFIED = 0;
    FINALITY_COMMITTED = 1;
    FINALITY_PROVEN = 2;
}

This is not as big of a change compared to the current endpoint - and so, maybe it doesn't simplify things too much. But it does feel a bit more flexible and easy to extend in the future.

[kkovaacs]

This is not as big of a change compared to the current endpoint - and so, maybe it doesn't simplify things too much. But it does feel a bit more flexible and easy to extend in the future.

I think this would still be a simplification (removing the arbitrary block number target). I'm fine with this one too, we should just finalize this.

That would leave us with the following:

enum FinalityLevel {
    FINALITY_UNSPECIFIED = 0;
    FINALITY_COMMITTED = 1;
    FINALITY_PROVEN = 2;
}

message SyncChainMmrRequest {
    fixed32 current_block_height = 1;
    FinalityLevel finality_level = 2;
}

message SyncChainMmrResponse {
    BlockRange block_range = 1;
    primitives.MmrDelta mmr_delta = 2;
    blockchain.BlockHeader block_header = 3;
}

FinalityLevel is effectively a rename of our current ChainTip, and the rest is pretty much what we have now except for the arbitrary block sync target in the request.

[Mirko-von-Leipzig]

Catching up on this.. iiuc

1. @bobbinth proposal: specify a specific block height to sync to + include that block header. A downside is if a client needs different finalities for different purposes, then they need separate mmr's.
2. @kkovaacs proposal: always sync to latest committed, include other finality block heights, caller can query additional block header as needed, e.g. proven block header.

Hopefully, at some point in the future (though, probably a couple of years out), we'll make proving fast enough where the difference between committed and proven would disappear (e.g., we'd effectively merge committed and proven). How would we update this endpoint in this case?

They would still be called the same thing, just coincidentally be identical. Or we could update the docs to state they're identical and have it be official. No API change required.

---

tbh I would have separated/normalized by removing the block header etc information completely. SyncMmr always (and only) returns the full mmr delta up to the chain tip. Client then additionally requests the block header they desire e.g.

enum HeaderRequest {
    BlockNumber(u32),
    CommittedTip,
    ProvenTip,
    L1FinalizedTip,
}

But I know preference so far has been fewer but broader requests..

---

--edit-- Okay so this only enables a PartialMmr which means that one cannot select an arbitrary block - only the specific block returned by the query. In which case option (1) makes the most sense.

[kkovaacs]

OK, let's converge on option 1 then. I'll update the proposal in the original bug.

[bobbinth]

@bobbinth proposal: specify a specific block height to sync to + include that block header.

I don't think the user would be asking for specific block height (and maybe that's what you ment in the edit.

A downside is if a client needs different finalities for different purposes, then they need separate mmr's.

Only on MMR will be sufficient, but the client will need to make 2 requests:

1. Sync to the proven tip. This gets the block number of the latest proven block.
2. Sync to the committed tip using the block number we got from the first step.

Both requests update the same MMR on the client, but now the MMR contains enough info to execute transactions against either proven or committed blocks.

FinalityLevel is effectively a rename of our current ChainTip

Right - and i'm fine keeping it as ChainTip. I just thought FinalityLevel was a bit more descriptive.

---

Separately, following my comment from https://github.com/0xMiden/miden-client/pull/1887/changes#r3236867263, I think we may need to include the block signature in the response as well (it is not a part of the block header AFAICT). So, the response my look more like:

message SyncChainMmrResponse {
    BlockRange block_range = 1;
    primitives.MmrDelta mmr_delta = 2;
    blockchain.BlockHeader block_header = 3;
    blockchain.BlockSignature validator_signature = 4;
}

[kkovaacs]

Based on the discussion here I'm closing this in favor of #2075.