feat: add signature aggregation using python bindings #238

anshalshukla · 2025-12-18T09:39:19Z

🗒️ Description

Supports aggregation as per our discussion for devnet2.

The current implementation of aggregation bindings don't actually aggregate the signatures and instead return a single 0 byte, same for the verification API. This will be updated when we have aggregation support for current signature encoding scheme on TEST_CONFIG.

✅ Checklist

Ran tox checks to avoid unnecessary CI fails:
```
uvx tox
```
Considered adding appropriate tests for the changes.
Considered updating the online docs in the ./docs/ directory.

src/lean_spec/types/byte_arrays.py

src/lean_spec/subspecs/xmss/aggregation.py

src/lean_spec/subspecs/forkchoice/store.py

…ations

src/lean_spec/subspecs/containers/block/block.py

src/lean_spec/subspecs/containers/state/state.py

anshalshukla · 2025-12-22T11:43:07Z

In my bindings of leanMultisig I've added a is_test flag which when set return a single 0 byte, similarly in verification API with the is_test flag set it returns true if the payload is single 0 byte.

I've commented out testcases related to invalid signatures as current the aggregation & verification is in test mode.

Once we have working devnet-2 branch on leanMultisig for TEST_CONFIG signatures, it will be a basic upgrade to remove the is_test flag from the aggregation and verification APIs in aggregation.py along with other updates on the bindings repo. I think given the minimal changes we will require here once we have the aggregation stuff starts working we can merge this PR. Client teams can start working on this and they should be able to aggregate as sigs as well as devnet-2 branch works fine for PROD_CONFIG.

src/lean_spec/subspecs/forkchoice/store.py

packages/testing/src/consensus_testing/test_fixtures/fork_choice.py

src/lean_spec/subspecs/containers/block/block.py

src/lean_spec/subspecs/containers/state/state.py

src/lean_spec/subspecs/containers/block/block.py

unnawut · 2025-12-26T09:20:38Z

I wonder for the process of build_block(), do we need to manipulate aggregated attestations at all?

If I understand these parts correctly (and I have low confidence I do 😅 ), I think these are all to do with re-using/merging/splitting/deduplicating aggregated attestations that are the complex ones:

But can we say that:

Aggregated attestations that are already included in a block is final. Once it's processed, it becomes part of state.justifications and so it never needs to be revisited/reused. It can be dropped after processing into the state. The only need for storing it is for slashing detection (which we don't consider soon).
New individual attestations can be aggregated without concerning about existing aggregations. Because 1) leanMultisig can merge duplicate attestations, 2) Duplicate attestations don't affect the state.
Each block can already have multiple aggregated attestations, so late aggregation and on-time aggregation don't compete for a block. So multiple aggregations can keep flowing in anyway without the merging/splitting.
Eventually we'll have a separate aggregated_attestation gossip topic. We can do the merging by listening & responding to gossips than being part of block building? This would allow for a separate aggregator role in the future too.

If this all makes sense then maybe we can drop all the aggregation merging and splitting from the specs and only need 2 features around aggregation?

Build block by aggregating individual attestations per unique attestation data. There may be chances a block has multiple aggregations due to different attestation data
Ability to process multiple aggregated attestations from a block.body.attestations

tcoratger · 2025-12-26T11:07:55Z

packages/testing/src/consensus_testing/test_fixtures/fork_choice.py

+        else:
+            signature = Signature(
+                path=HashTreeOpening(siblings=HashDigestList(data=[])),
+                rho=Randomness(data=[Fp(0) for _ in range(PROD_CONFIG.RAND_LEN_FE)]),


I don't think that we should have any PROD_CONFIG in the codebase, we should use the env variable right?

tcoratger · 2025-12-26T11:16:01Z

packages/testing/src/consensus_testing/keys.py

+        For each aggregated attestation, collect the participating validators' public keys and
+        signatures, then produce a single leanVM aggregated signature proof blob using
+        `xmss_aggregate_signatures` (via `aggregate_signatures`).


These implementation details should change in the future (especially when we will not rely on the bindings anylire) so no need to add them here in the doc.

Suggested change

For each aggregated attestation, collect the participating validators' public keys and

signatures, then produce a single leanVM aggregated signature proof blob using

`xmss_aggregate_signatures` (via `aggregate_signatures`).

For each aggregated attestation, collect the participating validators' public keys and

signatures, then produce a single leanVM aggregated signature proof.

src/lean_spec/subspecs/containers/block/block.py

tcoratger · 2025-12-26T11:29:11Z

src/lean_spec/subspecs/containers/block/types.py

-                return True
-            seen.add(attestation.data)
-        return False
+    def each_duplicate_attestation_has_unique_participant(self) -> bool:


Here maybe we could have a less verbose name such as validate_unique_contributions ? Or maybe another idea?

tcoratger · 2025-12-26T11:33:43Z

src/lean_spec/subspecs/containers/block/types.py

+        groups: dict[bytes, list[AggregationBits]] = defaultdict(list)
+
+        for att in self:
+            groups[att.data.data_root_bytes()].append(att.aggregation_bits)
+
+        for bits_list in groups.values():
+            if len(bits_list) <= 1:
+                continue
+
+            counts: Counter[int] = Counter()
+
+            # Pass 1: count participants across the group
+            for bits in bits_list:
+                for i, bit in enumerate(bits.data):
+                    if bit:
+                        counts[i] += 1
+
+            # Pass 2: each attestation must have a participant that appears exactly once
+            for bits in bits_list:
+                unique = False
+                for i, bit in enumerate(bits.data):
+                    if bit and counts[i] == 1:
+                        unique = True
+                        break
+                if not unique:
+                    return False
+
+        return True


Something like this should be more pythonic and compact no?

groups = defaultdict(list) for att in self: groups[att.data.data_root_bytes()].append(att.aggregation_bits) for bits_list in groups.values(): if len(bits_list) <= 1: continue # 1. Convert bitfields to sets of active indices sets = [{i for i, bit in enumerate(bits.data) if bit} for bits in bits_list] # 2. Count index occurrences across the entire group counts = Counter(idx for s in sets for idx in s) # 3. Verify EVERY attestation has ANY index that appears EXACTLY once if not all(any(counts[i] == 1 for i in s) for s in sets): return False return True

yes looks simple to read

tcoratger · 2025-12-26T12:50:20Z

tests/lean_spec/subspecs/containers/test_state_aggregation.py

+def make_attestation(validator_index: int, data: AttestationData) -> Attestation:
+    """Create an attestation for the provided validator."""
+    return Attestation(validator_id=Uint64(validator_index), data=data)


Same I think that is useless

tcoratger · 2025-12-26T12:52:34Z

tests/lean_spec/subspecs/containers/test_state_aggregation.py

+# ============================================================================
+# Additional edge case tests for _aggregate_signatures_from_gossip
+# ============================================================================


No need for this we can directly put the other tests without this comment.

Suggested change

# ============================================================================

# Additional edge case tests for _aggregate_signatures_from_gossip

# ============================================================================

tcoratger · 2025-12-26T12:52:52Z

tests/lean_spec/subspecs/containers/test_state_aggregation.py

+# ============================================================================
+# Additional edge case tests for _aggregate_signatures_from_block_payload
+# ============================================================================


Same

Suggested change

# ============================================================================

# Additional edge case tests for _aggregate_signatures_from_block_payload

# ============================================================================

tcoratger · 2025-12-26T12:53:22Z

tests/lean_spec/subspecs/containers/test_state_aggregation.py

+# ============================================================================
+# Additional edge case tests for split_aggregated_attestations
+# ============================================================================


Suggested change

# ============================================================================

# Additional edge case tests for split_aggregated_attestations

# ============================================================================

tcoratger · 2025-12-26T12:53:39Z

tests/lean_spec/subspecs/containers/test_state_aggregation.py

+# ============================================================================
+# Additional edge case tests for compute_aggregated_signatures
+# ============================================================================


Suggested change

# ============================================================================

# Additional edge case tests for compute_aggregated_signatures

# ============================================================================

g11tech · 2025-12-27T13:24:06Z

src/lean_spec/subspecs/containers/state/state.py

+                    aggregated_payloads and aggregated_payloads.get(attestation_key)
+                )
+
+                if has_gossip_sig or has_block_payload:


this condition should always be true, we just need to track if we have gossip sig for it or not

src/lean_spec/subspecs/containers/state/state.py

unnawut

I want to spend a bit more time to understand compute_aggregated_signatures() and split_aggregated_attestations() and the tests, but like Thomas said these can go separate conversations.

anshalshukla · 2025-12-29T06:55:49Z

I've removed compute_aggregated_signatures() and simplified split_aggregated_attestaations(). I've also removed validate_unique_participants().

In the updated logic there is a possibility that there are some attestations that are do not contribute with any unique validator, this is fine as in the following devnet, we will recursively aggregate the attestations with same data_root so having a little bit of inefficiency for clarity is okay I believe. In the current form, implementing recursive aggregation will be easier as well.

Other changes are nicely described here

feat: add signature aggregation using python bindings

b27fd5a

anshalshukla marked this pull request as draft December 18, 2025 09:39