Skip to content

Commit 9b89f56

Browse files
al1imgal1img
committed
design: add runtime deps design document
Signed-off-by: Oleksandr Grytsov <oleksandr_grytsov@epam.com>
1 parent da9a6e0 commit 9b89f56

1 file changed

Lines changed: 267 additions & 0 deletions

File tree

‎doc/design/runtime_deps.md‎

Lines changed: 267 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,267 @@
1+
# Runtime dependencies
2+
3+
Requirements (https://kb.epam.com/spaces/EPMPAOS/pages/2460226723/FOTA+SOTA+unification) "Specifying runtime
4+
dependencies for update items".
5+
6+
## Architecture
7+
8+
* implement `common::instancestatus` module to provide all instances statuses across all nodes;
9+
* use same `common::instancestatus` on CM and SM side;
10+
* perform runtime dependencies normalization and cyclic/missing dependencies check in `cm::launcher`;
11+
* start instances according to runtime dependencies in `sm::launcher` based on instance statuses provided by
12+
`instancestatus` module.
13+
14+
### Handling instance statuses
15+
16+
1. Replace common module `instancestatusprovider` with `instancestatus` that will be located in CM and SM.
17+
18+
It implements the following interfaces:
19+
20+
* [aos::instancestatus::ProviderItf](itf/provider.hpp) - provides instance statuses for other modules;
21+
* [aos::instancestatus::HandlerItf](itf/handler.hpp) - handles instance status updates from other modules.
22+
23+
```mermaid
24+
classDiagram
25+
class InstanceStatusItf ["aos::instancestatus::InstanceStatusItf"] {
26+
<<interface>>
27+
}
28+
29+
class ListenerItf ["aos::instancestatus::Listener"] {
30+
<<interface>>
31+
OnInstanceStatusChanged(status)
32+
}
33+
34+
class ProviderItf ["aos::instancestatus::ProviderItf"] {
35+
<<interface>>
36+
GetAllInstancesStatuses(out:statuses)
37+
SubscribeListener(listener)
38+
UnsubscribeListener(listener)
39+
}
40+
41+
class HandlerItf ["aos::instancestatus::HandlerItf"] {
42+
<<interface>>
43+
SetAllInstancesStatuses(statuses)
44+
SetInstanceStatus(status)
45+
}
46+
47+
InstanceStatusItf ..|> ProviderItf
48+
InstanceStatusItf ..|> HandlerItf
49+
50+
InstanceStatus ..|> InstanceStatusItf
51+
ProviderItf ..> ListenerItf
52+
```
53+
54+
2. Rename `cm::launcher::InstanceRunnerItf` to `cm::launcher::InstanceHandlerItf` and add the following methods:
55+
56+
```mermaid
57+
classDiagram
58+
class InstanceHandlerItf ["cm::launcher::InstanceHandlerItf"] {
59+
<<interface>>
60+
Error UpdateInstances(node, instances)
61+
Error SendInstanceStatus(node, status)
62+
Error SendAllInstancesStatuses(node, statuses)
63+
}
64+
```
65+
66+
* `SendAllInstancesStatuses` is called for all connected node after receiving `OnNodeInstanceStatusesReceived` message
67+
from all node SM's;
68+
* `SendInstanceStatus` is called after receiving `OnInstanceStatusReceived` for all connected node SM's except node that
69+
status is received from.
70+
71+
3. On CM side:
72+
73+
`launcher` uses `instancestatus::HandlerItf` to set instance statuses received from SM. `smcontroller` updates node
74+
instance statuses and then `launcher` update `instancestatus` module with new statuses. `instancestatus` uses
75+
`smcontroller` as `SenderItf` to send instance status updates to other nodes.
76+
77+
```mermaid
78+
sequenceDiagram
79+
participant smcontroller
80+
participant launcher
81+
participant instancestatus
82+
participant statuslistener
83+
84+
Note over launcher: Initialization
85+
86+
launcher ->> instancestatus: SetAllInstancesStatuses(statuses)
87+
88+
loop All instances
89+
instancestatus ->> statuslistener: OnInstanceStatusChanged(status)
90+
end
91+
92+
loop All instances
93+
smcontroller ->> launcher: OnInstanceStatusReceived(status)
94+
launcher ->> instancestatus: SetInstanceStatus(status)
95+
instancestatus ->> statuslistener: OnInstanceStatusChanged
96+
launcher ->> smcontroller: SendInstanceStatus(status)
97+
end
98+
99+
Note over launcher: Run instances
100+
101+
loop All nodes
102+
launcher ->> smcontroller: UpdateInstances(node, instances)
103+
104+
loop All node instances
105+
smcontroller -->> launcher: OnInstanceStatusReceived(status)
106+
launcher ->> instancestatus: SetInstanceStatus(status)
107+
instancestatus ->> statuslistener: OnInstanceStatusChanged(status)
108+
launcher ->> smcontroller: SendInstanceStatus(status)
109+
end
110+
111+
smcontroller ->> launcher: OnNodeInstanceStatusesReceived(node, instances)
112+
end
113+
114+
launcher ->> instancestatus: SetAllInstancesStatuses(statuses)
115+
loop All instances
116+
instancestatus ->> statuslistener: OnInstanceStatusChanged(status)
117+
end
118+
launcher ->> smcontroller: SendAllInstancesStatuses(statuses)
119+
```
120+
121+
4. On SM side:
122+
123+
```mermaid
124+
sequenceDiagram
125+
participant smclient
126+
participant launcher
127+
participant instancestatus
128+
participant statuslistener
129+
130+
Note over launcher: Initialization
131+
132+
loop All node instances
133+
launcher ->> instancestatus: SetInstanceStatus(status)
134+
instancestatus ->> statuslistener: OnInstanceStatusChanged(status)
135+
launcher ->> smclient: SendInstanceStatus(status)
136+
end
137+
138+
launcher ->> smclient: SendNodeInstancesStatuses(statuses)
139+
140+
Note over launcher: Update instances
141+
142+
loop All node instances
143+
launcher ->> instancestatus: SetInstanceStatus(status)
144+
instancestatus ->> statuslistener: OnInstanceStatusChanged(status)
145+
launcher ->> smclient: SendInstanceStatus(status)
146+
end
147+
148+
launcher ->> smclient: SendNodeInstancesStatuses(statuses)
149+
150+
smclient ->> instancestatus: SetAllInstancesStatuses(statuses)
151+
loop All instances
152+
instancestatus ->> statuslistener: OnInstanceStatusChanged(status)
153+
end
154+
```
155+
156+
5. SM CM communication:
157+
158+
```mermaid
159+
sequenceDiagram
160+
participant cm_instancestatus as CM instancestatus
161+
participant smcontroller
162+
participant smclient
163+
participant sm_instancestatus as SM instancestatus
164+
165+
Note over smcontroller, smclient: Connect
166+
167+
smcontroller ->>+ cm_instancestatus: GetAllInstancesStatuses
168+
cm_instancestatus -->>- smcontroller: statuses
169+
smcontroller ->> smclient: SendAllInstancesStatuses(statuses)
170+
smclient ->>+ sm_instancestatus: SetAllInstancesStatuses(statuses)
171+
```
172+
173+
6. Update instance state:
174+
175+
* eScheduled - instance is scheduled to run but not running yet;
176+
* eDependency - instance is waiting for dependencies to be satisfied;
177+
* eActivating - instance is activating;
178+
* eActive - instance is active;
179+
* eInactive - instance is inactive;
180+
* eFailed - instance failed to activate.
181+
* eRemoved - instance is removed.
182+
183+
### Dependency handling on CM side
184+
185+
* `cm::launcher` should normalize dependencies:
186+
* `sm::launcher` can't handle before deps, though `cm::launcher` should convert before deps to after deps and add them
187+
to instance dependencies;
188+
* `sm::launcher` doesn't know total num of deps instances, therefore `cm::launcher` should add total num of deps
189+
instances to instance dependencies;
190+
* check cyclic or missing dependencies before running instances. If cyclic dependencies are detected, CM should not
191+
schedule these instances and set their state to eFailed with corresponding error message.
192+
193+
### Dependency handling on SM side
194+
195+
* once `UpdateInstance` command is received, `sm::launcher` gets all instances statuses and sets `eDependency` state for
196+
all instances that have unsatisfied dependencies;
197+
* `sm::launcher` subscribes to instance status updates from `instancestatus` module and check if dependencies are
198+
satisfied for each instance. That instances that have all dependencies satisfied are scheduled to run;
199+
* instances should be grouped by priorities: if `sm::launcher` ready to start some instances with different priorities,
200+
it should start instances with higher priority first. Wait for all instances with higher priority to be actually
201+
started and then start instances with lower priority.
202+
203+
## TODO
204+
205+
### Cloud protocol
206+
207+
* add runtime dependencies to update item config;
208+
209+
### API
210+
211+
* `UpdateInstances` message should contain only scheduled instances instead of start/stop instances;
212+
* `InstanceInfo` message should contain instance dependencies and total num of deps instances;
213+
* `update_instances_status` should be renamed to `instance_status` and contain only one instance status;
214+
* add `all_instances_statuses` message to send all instances statuses in one message.
215+
216+
### common::instancestatus
217+
218+
* implement `common::instancestatus` module;
219+
* initialize it and add to CM and SM.
220+
221+
### cm::launcher
222+
223+
* add runtime deps to aos::InstanceInfo struct;
224+
* update instance states at initialization - should be set to eScheduled;
225+
* modify and use `cm::launcher::InstanceRunnerItf`;
226+
* add handling of `instancestatus::HandlerItf` to update instance statuses received from SM;
227+
* send update instances with desired instances to SM on `UpdateInstances` command;
228+
* implement runtime deps normalization on run instances;
229+
* implement cyclic and missing dependencies check on run instances.
230+
231+
### cm::database
232+
233+
* add instance runtime dependencies and total num of deps instances to database for `launcher::StorageItf`.
234+
235+
### cm: smcontroller
236+
237+
* update `cm::launcher::InstanceHandlerItf` implementation;
238+
* send all instances statuses to SM on SM connection.
239+
240+
### sm::launcher
241+
242+
* add handling of `instancestatus::HandlerItf` to update instance statuses;
243+
* handle `UpdateInstances` command with desired instances only;
244+
* update `SenderItf`: rename `SendUpdateInstancesStatuses` to `SendInstanceStatus` with one instance status;
245+
* implement runtime deps handling on update instances;
246+
* implement instances grouping by priorities and starting instances with higher priority first.
247+
248+
### sm::database
249+
250+
* add instance runtime dependencies and total num of deps instances to database for `launcher::StorageItf`.
251+
252+
### sm::smclient
253+
254+
* use `instancestatus::HandlerItf` to update instance status and all instances statuses received from CM;
255+
* handle `InstanceInfo` struct with instance dependencies and total num of deps instances.
256+
257+
### sm::monitoring
258+
259+
* filter instances state by node: add only current node instances to monitoring.
260+
261+
### Discuss with cloud
262+
263+
1. New instance states
264+
265+
### Negotiate requirements needed
266+
267+
1. Priority and runtimes deps: how to handle priority and runtime deps together?

0 commit comments

Comments
 (0)