Skip to content

Comments

feat(message): Add FlatBuffers zero-copy message support alongside Protobuf#54

Draft
Copilot wants to merge 5 commits intofastdds-tagfrom
copilot/replace-protobuf-with-flatbuffers
Draft

feat(message): Add FlatBuffers zero-copy message support alongside Protobuf#54
Copilot wants to merge 5 commits intofastdds-tagfrom
copilot/replace-protobuf-with-flatbuffers

Conversation

Copy link
Contributor

Copilot AI commented Feb 23, 2026

Introduces FlatBufferMessage — a zero-copy message type for Cyber RT's transport layer — and a FlatBuffers-based Chatter talker/listener pair as a reference implementation. FlatBufferMessage is fully integrated into the Cyber RT message traits system, enabling it to work with the SHM transmitter/receiver and transport layer out of the box. Existing Protobuf infrastructure is untouched.

Core additions

  • cyber/message/flatbuffers_message.hFlatBufferMessage class implementing the standard Cyber RT message interface. The key primitive is GetRoot<T>(), which returns a typed pointer directly into the internal buffer via flatbuffers::GetRoot<T>(buf) — zero copies, zero allocations on the receive path. Includes SetTypeName() for HasSetType trait detection (allows the transport layer's ParseFromHC to propagate type info) and set_type_name() for conventional C++ usage.
void MessageCallback(const std::shared_ptr<FlatBufferMessage>& msg) {
  // Zero-copy: no deserialization step, just pointer arithmetic into the SHM buffer
  const Chatter* chatter = msg->GetRoot<Chatter>();
  AINFO << "seq->" << chatter->seq() << " content->" << chatter->content()->str();
}
  • cyber/message/flatbuffers_traits.h — Inline overloads for ByteSize, SerializeToArray, and ParseFromArray for FlatBufferMessage, following the same pattern as raw_message_traits.h and py_message_traits.h. The ByteSize overload is essential for the SHM transmitter (message::ByteSize(msg) would otherwise return -1 for types without ByteSizeLong()).

  • cyber/message/message_traits.h — Now includes flatbuffers_traits.h alongside the other traits headers, so FlatBufferMessage works everywhere message traits are used.

  • cyber/message/flatbuffers_message_test.cc — Unit tests for construction, zero-copy GetRoot, serialize/parse round-trips (array + string), edge cases, and copy semantics.

  • cyber/message/message_traits_test.ccFlatBufferMessage coverage added to all existing test cases: type_trait, byte_size, serialize_to_array, serialize_to_string, parse_from_array, parse_from_string, message_type, and descriptor.

Schema & generated code

  • cyber/examples/proto/examples.fbs — FlatBuffers schema mirroring examples.proto (Chatter table with identical fields).
  • cyber/examples/proto/examples_generated.h — Pre-generated FlatBuffers C++ header committed directly; no build-time flatc invocation required.

Build system

  • bazel/flatbuffers.bzl — Module extension creating a @flatbuffers local repository that symlinks the system-installed headers (/usr/include/flatbuffers). FlatBuffers C++ runtime is header-only; no linking needed.
  • MODULE.bazel — Registers flatbuffers_extension; exposes @flatbuffers//:flatbuffers to all targets.
  • cyber/message/BUILD — Adds flatbuffers_message, flatbuffers_traits, and their test targets; flatbuffers_traits is wired into message_traits dependencies.
  • cyber/examples/proto/BUILD — Adds examples_fbs cc_library wrapping the generated header.
  • cyber/examples/BUILD — Adds talker_fb and listener_fb binary targets.
Original prompt

This section details on the original issue you should resolve

<issue_title>Zero copy</issue_title>
<issue_description>Use flatbuffer instead of protobuf</issue_description>

<agent_instructions>## Task: Replace Protobuf with FlatBuffers in Apollo Cyber RT


Stage 1 — Deep Analysis

Copilot Prompt:

🔍 Analyze the underlying architecture of Apollo Cyber RT, focusing on the cyber/message and cyber/transport modules.

  1. Identify all key classes that depend on Protobuf-generated code (e.g., MessageHeader, message traits, factories).
  2. Analyze the coupling between RawMessage, ProtobufFactory, and Protobuf serialization.
  3. Assess the compatibility of FlatBuffers’ zero-copy features with Cyber RT’s shared memory (SHM) transport mechanisms.
  4. Produce a list of files that require modification and document potential performance bottlenecks or architectural constraints.

Deliverables:

  • Dependency map of Protobuf usage in Cyber
  • Coupling risk analysis report
  • Preliminary performance impact notes

📋 Stage 2 — Implementation Plan

Copilot Prompt:

🛠 Based on the analysis results, create a step-by-step migration plan from Protobuf to FlatBuffers:

  1. Schema Definition: Detail how to convert existing .proto files into .fbs schemas while maintaining backward-compatible field semantics.
  2. Adapter Layer: Design an abstract message wrapper interface that allows Cyber RT to support both Protobuf and FlatBuffers formats simultaneously for a smooth transition.
  3. Transport Layer Changes: Update the transport/transmitter and receiver logic to handle FlatBuffers root pointers and buffer lifetimes correctly.
  4. Build System Update: Modify Bazel BUILD files and rules to integrate FlatBuffers compiler (flatc) and generation workflows.
  5. Provide rough timeline/effort estimates per step.

Deliverables:

  • Migration plan README
  • Schema conversion guidelines
  • Adapter interface spec

✅ Stage 3 — Testing and Validation

Copilot Prompt:

🧪 Create a comprehensive testing strategy for the FlatBuffers migration:

  1. Unit Tests: Write tests for the new FlatBufferMessage class to verify serialization/deserialization correctness and field integrity.
  2. Benchmark Tests: Compare Protobuf vs FlatBuffers across multiple payload sizes (e.g., 1 KB, 1 MB, 10 MB) with respect to CPU usage, memory footprint, and latency.
  3. Integration Tests: Simulate a Talker/Listener pattern using shared memory transport and verify that FlatBuffers buffers are correctly zero-copy read and propagated.
  4. Regression Tests: Confirm that unmodified Protobuf-based components still function correctly until full migration is complete.

Deliverables:

  • Unit test suite
  • Benchmark results and graphs
  • Integration test results with pass/fail logs

🧠 Summary for Copilot

High-Level Task:
▶️ Replace Protobuf with FlatBuffers in Cyber RT
Objectives:
✔ Reduce serialization overhead
✔ Enable zero-copy shared memory transport
✔ Maintain backward compatibility during migration
</agent_instructions>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Co-authored-by: daohu527 <10419854+daohu527@users.noreply.github.com>
Copilot AI changed the title [WIP] Use flatbuffer instead of protobuf in Apollo Cyber RT feat(message): Add FlatBuffers zero-copy message support alongside Protobuf Feb 23, 2026
Copilot AI requested a review from daohu527 February 23, 2026 02:52
@daohu527
Copy link
Collaborator

daohu527 commented Feb 23, 2026

@copilot continue

…ystem

Co-authored-by: daohu527 <10419854+daohu527@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Feb 23, 2026

@copilot continue

Completed in commit 09b74f3. The following pieces were added to finish the integration:

  • cyber/message/flatbuffers_traits.h — inline overloads for ByteSize, SerializeToArray, and ParseFromArray for FlatBufferMessage, following the exact same pattern as raw_message_traits.h and py_message_traits.h. The ByteSize overload is the key fix: without it the SHM transmitter's message::ByteSize(msg) would return -1 (since FlatBufferMessage has no ByteSizeLong() method, which is what the generic SFINAE template looks for).

  • SetTypeName() added to FlatBufferMessage — satisfies the HasSetType trait so ParseFromHC can propagate the type identifier when deserializing a wire buffer with a MessageHeader.

  • message_traits.h now #includes flatbuffers_traits.h alongside the other traits headers, so FlatBufferMessage works everywhere message traits are used (SHM transmitter/receiver, etc.).

  • message_traits_test.cc — added FlatBufferMessage coverage to all 7 existing test cases (type_trait, byte_size, serialize_to_array, serialize_to_string, parse_from_array, parse_from_string, message_type, descriptor).

@daohu527 daohu527 changed the base branch from main to fastdds-tag February 23, 2026 08:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Zero copy

2 participants