Frontend Data Layer Refactor

[DXCanas]

I’ve been looking through the lit docs and thinking about some if the problems laid out in #173 and #170 regarding reactivity.

The points around legibility and maintainability are valid. And I think a lot of the trouble with following along is inconsistency with the patterns; through what means data flows in (or, maybe one day, out), what the scope/responsibility of a component is, how we’re rendering that component, etc.

We’ve got dishes in the oven and cutlery in the fridge, so to speak.

lit does offer tools and opinions in this regard. It’s just not strict about enforcing them. I think standardizing the handful of components here would make the Frontend

1. more performant
2. less intimidating/easier to build on to of
3. easier to port/add to (state, upgrades, even other libraries dish the line)
   If we follow rules, it’s easy to know what goes where if we ever switch to something else

I think @Abasz has touched on many of the same points. But I wanted to bring up the refactor as a “project” for the next release, so that we’re all aware of each other’s construction zones😊

after laying this groundwork, I think it’d be a cinch to move on to planned workouts, etc.

Pretty excited about the idea. If you’ll have me, of course.

[Abasz]

This is the point exactly. I have not spent enough time with the docs yet to make up my mind, but its clear that index.js and app.js need a full revamp (I mean ripping it out and redoing it kind of). Also the draw logic of the dashboard elements needs rethinking (currently we make a decision to redraw on every incoming message instead of targeting the component that changed (since dashboard layout only changes via setting or refresh).

The issue is (and that may be my lack of understanding of lit) is that since we use a top-down approach we need to update the full tree we cannot targettedly aim for a specific component.

So this does not work (unless you call requestUpdate()):

this._appState.metrics.handleForcesCurv = newHnadleForcesCurve

however, if you call reqeustUpdate() in the parent it will flag shouldChange for all child components essentially rerendering (calling render() on) every element. So this is equivalent to replacing the whole appState so the change detection flows down but that will trigger all elements as all reference are invalidated.

I am pretty sure that signals() will remedy this where data is not passed from top but rather injected via a service from the data layer. This way the component will know if it really has changed or not

[DXCanas]

its clear that index.js and app.js need a full revamp (I mean ripping it out and redoing it kind of)

though I have some lighter-touch ideas regarding index and app.js.

As an incremental step, what do you think of this idea I had last night:

Leverage light dom/slots

Ensures each component is only handling the data it needs.

The gist of it would be to flatten out the components to stop having the PerformanceDashboard act as a "parent" handing down the metrics data to "children". PerformanceDashboard doesn't need any of that; the state that it needs is config, and that's mostly for the settings modal.

So pseudo-ish-code:

socket = new WebSocket(`ws://${location.host}/websocket`)

// The other stuff going on in app.js
// ...

export class App extends LitElement {
    @state()
       // Potential to flatten each out into its own piece of state, but for brevity:
        appState = {
            blePeripheralMode: '',
            hrmPeripheralMode: '',
            antPeripheralMode: '',
            uploadEnabled: false,
            shutdownEnabled: false,
            guiConfigs: {
                dashboardMetrics: ['distance', 'timer', 'pace', 'power', 'stkRate', 'totalStk', 'calories', 'actions'],
                showIcons: true,
                maxNumberOfTiles: 8
            }
        }
    // other things, including methods and event handlers
    // ...
    render () {
      return html`
        <toolbar 
            // Whatever literals it needs
            .blePeripheralMode=${this.appState.blePeripheralMode}
            .hrmPeripheralMode=${this.appState.hrmPeripheralMode}
            .antPeripheralMode=${this.appState.antPeripheralMode}
            .uploadEnabled=${this.appState.uploadEnabled}
            .shutdownEnabled=${this.appState.shutdownEnabled}
            // react to user input, update accordingly
            @onUserInput=${this.handleEvent(event)}
        ></toolbar>

        <settings-modal
            // Or whatever it needs
            .appState=${this.appState}
            // react to user input, update accordingly
            @onUserInput=${this.handleEvent(event)}
        ></settings-modal>

        // Only needed to set the grid dimensions
        <performance-dashboard maxNumberOfTiles=${this.appState.guiConfigs.maxNumberOfTiles}>
          ${this.guiconfigs.dashboardMetrics.map((metric) => {
            if (metric === 'forceCurve') {
                return html`
                  <dashboard-force-curve 
                    .metric=${metric}
                    .value=${this.socket.metrics.driveHandleForceCurve}
                    .updateForceCurve=${this.socket.metrics.driveHandleForceCurve}
                  ></dashboard-force-curve>
                `
            }
            return html`<dashboard-metric
              // Just literals, when possible. update `onChange` otherwise.
              // Use (builtin) converter methods instead of currentformatNumber/distance/seconds calls.
              .value=${socket.metrics[metric]}
              .displayName=${this.metricDisplayName(metric)}
              .metric=${metric}
            ></dashboard-metric>`
          })}
        </performance-dashboard>
      `
    }
}

I get there are flaws here, but I hope it communicates the gist.

[Abasz]

While the general idea of having a bigger component would work from the data update perspective this will be a bit more complicated than this. Which makes me thinking if this is the right approach. Though admittedly I am not too convinced that our original approach was such a good idea but any way.

Originally we used a factory to create the tiles dynamically. The idea behind it was so anyone can add a new tile without really needing to understand Lit and front end and the whole client code (https://github.com/JaapvanEkris/openrowingmonitor/blob/main/app/client/store/dashboardMetrics.js). One place you put the staff in and the rest of the code takes care of everything.

Then the template property started getting logic to determine how to display certain things (like calories, or countdown in distance and all that) and got more complex.

Anyway so now its not just a simple iteration and one "if" branch but the idea of flattening the component list could actually work ! Because when we process the config with the factory actually we pass the metric and the config but then each template only passes the property it needs as input. So (though this needs to be tested) we are not passing global state and each component should be able to detect changes to the property it uses.

Do you know actually why Lit is not able to detect changes on child components even though the property they use as input has changed (with strict equality) but not the main object that holds them? Because was my root issue when I was playing around and I could not figure this out.

[JaapvanEkris]

Then the template property started getting logic to determine how to display certain things (like calories, or countdown in distance and all that) and got more complex.

I've been thinking about that. I think a better place would be in the webserver (i.e. server side). In WebServer.js logic is already present WHEN to update the clients, perhaps that is also the place WHAT to update and HOW to present it (i.e. decide if time is based on time rowed or remaining time). The pattern is actually quite similar how the recorders now developped, where these also decide when to record and what it should record at that moment, also doing some pre-processing (for example the fitrecorder managing active and resting splits).

Do you know actually why Lit is not able to detect changes on child components even though the property they use as input has changed (with strict equality) but not the main object that holds them? Because was my root issue when I was playing around and I could not figure this out.

Hmm, sounds like a very similar approach to JavaScript's copy behaviour. JavaScript uses "Shallow copy" by default for complex objects, which is tricky when more complex objects (say workoutSegments) are part of your structure as it essentially treats them as references. Had some issues with that one in the recorders.

[DXCanas]

So (though this needs to be tested) we are not passing global state and each component should be able to detect changes to the property it uses.

This was essentially my idea, yeah!

Do you know actually why Lit is not able to detect changes on child components even though the property they use as input has changed (with strict equality) but not the main object that holds them? Because was my root issue when I was playing around and I could not figure this out.

I had to re-read this a few times, so I'm not sure if I'm understanding correctly. What I'm imagining is a scenario like this:

<parent-element .someObject=${appState.metrics}>
  <child-element .property=${someObject.literalValue}></child-element>
</parent-element>

Then appState.metrics.literalValue changes, but appState.metrics reference stays the same.

Is that right?

If so, my first thought is that the parent component doesn't call render() to evaluate the template expression. And that might be because the property reference, .someObject, hasn't changed -- so even if hasChanged() was called, appState.metrics===appState.metrics is still true.

So then I would guess the child's hasChanged() isn't even triggered; literals are pass-by-value, so the template needs to evaluate in order for the child to trigger its hasChanged() -> onUpdate() -> render() lifecycle.

[DXCanas]

@JaapvanEkris’s point could explain it too, from another angle I think.

Generally all the object copy and pass-by-reference vs pass-by-value stuff is a trip. And the first thing I suspect when stuff like that happens.

[Abasz]

Please find below the high level plan for changing the data layer and other parts of the front end for 0.9.8:

Change plan

1. Move to Typescript and Vite
2. Start using rxjs for streaming websocket data: https://rxjs.dev/api/webSocket/webSocket
3. Create a web-socket.service singleton that will handle the web-socket connection it selfe and will have a send and receive method
4. Create metrics-data.service singleton (that uses web-socket service as dependency) that filters the incoming messages and routes them to the appropriate property (e.g. rxjs allows us to create different observables from one stream and filter them based on a property. this would allow the observable to only emit a new value when we want. Specifically this means that we would have a forceCurveData observable that only emits when we want (new value comes in, and this is controlled by the context metrics).
5. Create settings.service singleton (that relies on web-socket.service) that manages the config related incoming data. It uses rxjs filter for the incoming "config" message. This will also mange the GUI settings. This can be injected into the components of the GUI directly
6. Create command.service (that uses web-socket service as dependency) that will manage sending commands back to the server. This will avoid having to raise global events at subcomponent level that are cought in index.js and handled (and are a mess to extend/comprehend as there is no visible direct link...).
7. Inject the metrics-data.service int to the components that need to use (this will avoid having to pass down a giant object and manage component data more directly). (still we could keep the performance monitor component but like @DXCanas suggested we can have a performance-dashboard component but it will not need an input property.
8. Inject the settings service into the components that need to use
9. Inject command.service into those components that will need to use it (for instance we can avoid this in any dashboard metric)
10. All component could use "Signal" (where the set() will indicate change to the changedetectoin system as far as I understand) but I see that it is still experimental so I have to do further digging whether it is worth to jump on that wagon. Otherwise we can use rxjs in the template (I found a good article that implements a custom async "pipe" - this is a concept in Angular - that would help with auto subscribing and unsubscribing - last time I was experimenting with this I have not thought about doing something like this - or these ReactiveController and AsyncDirective did not exist cant remember).

RXjs and LIt: https://medium.com/netanelbasal/lit-rxjs-made-for-each-other-ace3d13f3c24 This provides a very good interop handler example for the Rxjs observables and Lit component templates. It manages subscription/unsubscriptoin automatically and with these rxjs usage in the template becomes much simpler.

note: I am still digesting the docs and the biggest question for me currently is how efficient the injection of the dependency services directly into the component but based on this article and this video on context specifically on lit context and dependency injection it seems doable, though I am not 100% sure if best practice is that this should be used for values only (if this is the case we can inject the specific getter on the data services but I prefer injecting the whole class) even though the docs states that this is for service as well

While I understand that this is seemingly a big change (code wise I think this is not that big apart from typescript migration) but the above structure would have a lot of benefit on the long term:

• It massively separates different concepts and concerns from the index.js and the app.js.
• It helps long term extensibility and maintenance by having the responsibility clearly communicated by the design.
• It helps testing component and app behaviour via dependency injection
• Data stream handling via RxJs is very efficient and clear (though it has a learning curve to it) as it was made for async stream specifically
• Essentially we can avoid prop drilling (and potential unnecessary updates and forcing potentially inefficient immutability) and tighten the scopes of data that is received by each component.

If someone would raise that this may be an overkill I would argue that we do not know what the future of this GUI is on the long term. Where it may develop. The point (and hence the aim) of the above that would serve as solid foundation for any potential extention of the GUI. I understand that there are way to do incremental steps but it will just abstract the main issue that is at the root of the current GUI architecture which is how data is being handled (updated/read etc.).

But I do think there are smaller changes partways through.

Please feel free to detail them here.

Rational behind build tool migration

Benefits and rational on build tool migration (I am rooting for this so cons must be raised by someone else :D:D:D ):

Vite/Vitest (Overall: everything is in one place one config)

• Faster builds and instant dev server (no need to manual refreshing client page, potential hot reload etc.)
• Vite uses native ES modules and esbuild (significantly more efficient builds and feedback loop)
• Simpler configuration surface as no need for multiple tools (Rollup config, Babel config, polyfill strategy becomes a single Vite config).
• Easier long-term maintenance as less custom plumbing means fewer breaking changes to chase across multiple tools.
• Vitest integration with Vite (Shared config, Shared module resolution etc.)
• More feature rich testing framework with included mocking capabilities
• Much faster test execution via Vitest
• Lower cognitive overhead by one mental model for build + test, instead of keeping uvu-specific patterns separate from your build pipeline.

TypeScript migration

• Stronger API contracts for Web Components since Lit 3.x leans heavily on typed decorators, reactive properties, and controllers.
• TypeScript gives you "compile"-time guarantees (earlier feedback, fewer runtime errors on wrong property types, missing fields, invalid imports)
• Better refactorability in the future
• IDE and DX improvements (autocomplete, go-to-definition, and inline documentation dramatically improve productivity):
• Most modern Lit examples, libraries, and community tooling assume TypeScript-first usage.

I am happy to receive feedback (any kind):D