|
12 | 12 | # SPDX-License-Identifier: Apache-2.0 |
13 | 13 | # ******************************************************************************* |
14 | 14 |
|
15 | | -binary config loader |
16 | | -==================== |
| 15 | +FlatBuffers-Library |
| 16 | +=================== |
17 | 17 |
|
18 | | -.. document:: binary config loader |
19 | | - :id: doc__binary_config_loader |
20 | | - :status: draft |
| 18 | +.. document:: FlatBuffers-Library |
| 19 | + :id: doc__flatbuffers |
| 20 | + :status: valid |
21 | 21 | :safety: ASIL_B |
| 22 | + :security: YES |
22 | 23 | :tags: component_request |
| 24 | + :realizes: wp__cmpt_request |
| 25 | + |
| 26 | + |
| 27 | +.. toctree:: |
| 28 | + :hidden: |
| 29 | + |
| 30 | + requirements/index.rst |
| 31 | + architecture/index.rst |
23 | 32 |
|
24 | 33 |
|
25 | 34 | Abstract |
26 | | --------- |
| 35 | +======== |
| 36 | + |
| 37 | +This component request proposes the integration of Google FlatBuffers [#flatbuffers]_ providing |
| 38 | +serialization, zero-copy read access, and structural verification of FlatBuffers data, as well as code |
| 39 | +generation via the ``flatc`` compiler. |
| 40 | + |
| 41 | +Additionally, the library introduces FlatBuffers binary configuration format to replace JSON-based |
| 42 | +module configuration. FlatBuffers provides zero-copy access, schema validation, and access code |
| 43 | +generation for C++, Rust, and further languages. The approach eliminates the need for runtime |
| 44 | +parsing and accelerates module startup times. Safety certification covers ``flatc`` tool |
| 45 | +qualification, runtime library verification, and module-level testing of generated code. |
27 | 46 |
|
28 | | -This feature introduces FlatBuffers [#flatbuffers]_ binary configuration format to replace JSON-based module configuration. |
29 | | -FlatBuffers provides zero-copy access, schema validation, and access code generation for C++, Rust and further languages. |
30 | | -The approach eliminates the need for runtime parsing and accelerates module startup times. |
31 | | -Safety certification covers flatc tool qualification, runtime library verification and module-level testing of generated code. |
32 | 47 |
|
33 | 48 | Motivation |
34 | | ----------- |
| 49 | +========== |
35 | 50 |
|
36 | | -Module specific configuration is a cross-cutting concern that impacts system startup time and development efficiency. |
37 | | -For read-only configuration scenarios, runtime parsing approaches can limit startup performance in time-critical applications. |
| 51 | +Module specific configuration is a cross-cutting concern that impacts system startup time and |
| 52 | +development efficiency. For read-only configuration scenarios, runtime parsing approaches can |
| 53 | +limit startup performance in time-critical applications. |
38 | 54 |
|
39 | | -FlatBuffers [#flatbuffers]_ binary configuration approach addresses these engineering challenges by: |
| 55 | +The FlatBuffers binary configuration approach addresses these engineering challenges by: |
40 | 56 | - Eliminating the need for runtime parsing to meet aggressive startup time requirements |
41 | 57 | - Providing compile-time type safety through generated access code |
42 | 58 | - Reducing development effort through automated access code generation |
43 | 59 | - Ensuring schema validation at build time |
44 | 60 |
|
45 | 61 | Given its cross-cutting nature, this architectural decision shall be done early in the project. |
46 | 62 |
|
| 63 | + |
47 | 64 | Rationale |
48 | | ---------- |
| 65 | +========= |
| 66 | + |
| 67 | +Real-world experience with complex modules (e.g. diagnostics, SOME/IP) demonstrates that read-only |
| 68 | +configuration scenarios benefit significantly from zero-copy access patterns. For these use cases, |
| 69 | +FlatBuffers is ideal as it allows zero-copy data access. The schema-driven code generation further |
| 70 | +accelerates development by providing type-safe access patterns, reducing both implementation effort |
| 71 | +and the potential for configuration-related runtime errors. |
| 72 | + |
49 | 73 |
|
50 | | -Real-world experience with complex modules (e.g. diagnostics, SOME/IP) demonstrates that read-only configuration |
51 | | -scenarios benefit significantly from zero-copy access patterns. For these use cases, FlatBuffers is ideal as it allows |
52 | | -zero-copy data access. The schema-driven code generation further accelerates development by providing |
53 | | -type-safe access patterns, reducing both implementation effort and the potential for configuration-related runtime errors. |
| 74 | +Basic Functionality |
| 75 | +=================== |
54 | 76 |
|
55 | | -Specification |
56 | | -------------- |
| 77 | +The ``flatc`` compiler shall generate code for serializing, accessing, and verifying |
| 78 | +FlatBuffers binary data. Code generation shall be provided for C++ and Rust at ASIL-B level |
| 79 | +and other languages as needed at QM level (e.g. Python). |
57 | 80 |
|
58 | | -Module configuration shall use FlatBuffers [#flatbuffers]_ binary format for read-only configuration scenarios to achieve |
59 | | -aggressive start-up time requirements. |
| 81 | +The FlatBuffers-Library shall provide services for serialization, zero-copy access of |
| 82 | +FlatBuffers binary data, and structural verification of buffers. The library shall also support |
| 83 | +loading FlatBuffers binary files and optionally provide common buffer identification and version |
| 84 | +checking mechanisms. |
| 85 | + |
| 86 | +Note: FlatBuffers verification mechanism validates structural well-formedness only (e.g. offsets, vtables, |
| 87 | +field boundaries), not payload data integrity. |
| 88 | + |
| 89 | +Module configuration shall use FlatBuffers binary format for read-only configuration scenarios to |
| 90 | +achieve aggressive start-up time requirements. |
60 | 91 |
|
61 | 92 | .. _flatbuffers_overview: |
62 | 93 |
|
63 | 94 | Implementation |
64 | | -^^^^^^^^^^^^^^ |
| 95 | +-------------- |
65 | 96 |
|
66 | 97 | .. figure:: _assets/flatbuffers_overview.drawio.svg |
67 | 98 | :alt: FlatBuffers overview |
68 | 99 | :align: center |
69 | 100 | :width: 80% |
70 | 101 |
|
71 | 102 | | FlatBuffers schema files (``config.fbs``) define configuration structure using Interface Definition Language (IDL). |
72 | | -| The flatc compiler generates C++ or Rust access code from these schemas (``config.fbs``). |
73 | | -| The flatc compiler generates cross-platform data binary from schema (``config.fbs``) and JSON (``config.json``) input. |
| 103 | +| The ``flatc`` compiler generates C++ or Rust access code from these schemas (``config.fbs``). |
| 104 | +| The ``flatc`` compiler generates cross-platform data binary from schema (``config.fbs``) and JSON (``config.json``) input. |
74 | 105 | | Runtime access operates directly on binary config data (``config.bin``) without parsing (lazy loading). |
75 | | -| The flatc compiler can convert binary config data (``config.bin``) back to JSON using the schema (``config.fbs``) for debugging purposes. |
| 106 | +| The ``flatc`` compiler can convert binary config data (``config.bin``) back to JSON using the schema (``config.fbs``) for debugging purposes. |
76 | 107 |
|
77 | 108 | Schema Evolution |
78 | | -"""""""""""""""" |
| 109 | +^^^^^^^^^^^^^^^^ |
79 | 110 |
|
80 | 111 | Backward compatibility through: |
81 | 112 | - Optional fields for new parameters |
82 | 113 | - Default values for missing fields |
83 | 114 | - Controlled field deprecation |
84 | 115 |
|
85 | 116 | Schema Versioning |
86 | | -""""""""""""""""" |
| 117 | +^^^^^^^^^^^^^^^^^ |
87 | 118 |
|
88 | 119 | FlatBuffers binary files do not contain embedded schema information. Schema identification requires: |
89 | 120 | - Embedded version fields in the schema root table |
90 | 121 | - File naming conventions (e.g., config_v1.2.bin) |
91 | 122 |
|
92 | 123 | Build Integration |
93 | | -^^^^^^^^^^^^^^^^^ |
| 124 | +----------------- |
94 | 125 |
|
95 | 126 | Build system integration shall provide reusable rules for: |
96 | 127 | - Binary configuration file generation from module specific schema and user-provided JSON data |
97 | 128 | - Reverse conversion from binary to JSON for debugging purposes |
98 | 129 |
|
99 | | -Base libs |
100 | | -^^^^^^^^^ |
| 130 | +Binary Config Loading |
| 131 | +--------------------- |
101 | 132 |
|
102 | | -Binary config loader provides unified interface supporting: |
103 | | - - Binary data file loading (production, see :ref:`FlatBuffers overview <flatbuffers_overview>`) |
104 | | - - Runtime binary data creation from schema and JSON, allowing on-target JSON manipulation (development mode, QM) |
| 133 | +The FlatBuffers-Library provides a unified interface for binary data file loading |
| 134 | +(see :ref:`FlatBuffers overview <flatbuffers_overview>`). |
105 | 135 |
|
106 | | -Mode selection between development and production can be transparent to calling modules. |
107 | | - |
108 | | -.. image:: _assets/flatbuffers_dev_mode.drawio.svg |
109 | | - :alt: FlatBuffers development mode (QM) |
110 | | - :align: center |
111 | | - :width: 80% |
112 | 136 |
|
113 | 137 | Backwards Compatibility |
114 | | ------------------------ |
| 138 | +======================= |
115 | 139 |
|
116 | 140 | Switching from JSON to FlatBuffers for module configuration is not backwards compatible. |
117 | 141 |
|
| 142 | + |
118 | 143 | Security Impact |
119 | | ---------------- |
| 144 | +=============== |
120 | 145 |
|
121 | 146 | No change expected when compared to JSON based configuration approach. |
122 | 147 |
|
| 148 | + |
123 | 149 | Safety Impact |
124 | | -------------- |
| 150 | +============= |
125 | 151 |
|
126 | | -**Tool Qualification**: flatc compiler qualification shall be limited to binary creation use case. |
| 152 | +**Tool Qualification**: ``flatc`` compiler qualification shall be limited to buffer serialization use case. |
127 | 153 | Brief qualification is supplemented by module-specific validation. |
128 | 154 |
|
129 | 155 | **Verification Runtime Library**: Footprint when excluding verifier/builder classes |
130 | 156 | - C++: 12 headers, ~250 LOC (incl. comments), standard library only |
131 | 157 | - Rust: 11 files, ~150 LOC (incl. comments), core/alloc only (assumes std/serialize features disabled) |
132 | 158 |
|
133 | 159 | **Verification Generated Code**: Module-level verification is equivalent to handwritten access code verification. |
134 | | -Module testing contributes to flatc tool validation for specific schemas. |
| 160 | +Module testing contributes to ``flatc`` tool validation for specific schemas. |
135 | 161 | Test from configuration data (JSON) to value verification in access APIs. |
136 | 162 |
|
| 163 | + |
137 | 164 | License Impact |
138 | | --------------- |
| 165 | +============== |
139 | 166 |
|
140 | 167 | None. FlatBuffers is licensed under the Apache License Version 2.0. |
141 | 168 |
|
| 169 | + |
142 | 170 | How to Teach This |
143 | | ------------------ |
| 171 | +================= |
144 | 172 |
|
145 | 173 | Developer adoption requires practical examples and reusable patterns. |
146 | | -The module template or baselibs should provide examples for reference implementations. |
| 174 | +The FlatBuffers-Library should provide examples for reference implementations. |
| 175 | + |
147 | 176 |
|
148 | 177 | Rejected Ideas |
149 | | --------------- |
| 178 | +============== |
150 | 179 |
|
151 | 180 | **Protocol Buffers**: Requires runtime parsing and memory allocation, defeating startup time objectives. |
152 | 181 |
|
153 | 182 | **Custom binary formats**: Higher development and maintenance overhead compared to proven FlatBuffers ecosystem. |
154 | 183 |
|
| 184 | + |
155 | 185 | Open Issues |
156 | | ------------ |
| 186 | +=========== |
157 | 187 |
|
158 | 188 | No open issues identified yet. |
159 | 189 |
|
| 190 | + |
| 191 | +Future Extensions |
| 192 | +================= |
| 193 | + |
| 194 | + |
160 | 195 | Footnotes |
161 | | ---------- |
| 196 | +========= |
162 | 197 |
|
163 | 198 | .. [#flatbuffers] https://google.github.io/flatbuffers/ |
0 commit comments