Relationship of Component YAML vs. external implementation

[natevw]

I'm confused by why in the SPI example component there are code snippets like on_begin: enable_spi(self) and configure: self->SPI->CDIV = calculate_clock(clock_speed);.

Or perhaps at a higher level my question is/are/s/he:

• if the purpose of Component YAML is "simply" to describe an external C/JS/Lua/Rust/Go/OCaml/Erlang/Pascal/CommonLisp/Befunge-93/Verilog implementation in terms of a Grand Unified Theory why doesn't it just look like bindings metadata?
• if the purpose of Component YAML is "simply" to model state and available transitions thereupon, then/why isn't it just a new (or existing!) programming language that we write all our code in instead of C/JS/Lua/Rust/etc?

[natevw]

I'll hazard a guess, maybe this will illuminate my (mis)understanding or at least give another context within which to actually answer.

Answer?

One way to enable libraries/modules/components written one language to be used by another is to force them to interact with each other via a common-denominator language. Probably C.

But C is hard, and especially when trying to wrap a language that isn't already compiled to a traditional "object file" (say wrapping JS or Lua, vs. wrapping C/Go/Rust) the wrapper is pretty much just tons and tons of ugly boilerplate (e.g. "extern C" functions wrapping C++ method calls manipulating V8 API representations of state/logic in the actual JS context).

So rather than making Lua/Lisp/Verilog programmers also learn to write C, and that in the most painful of circumstances, we will have everyone write Component YAML instead. And then Fractal will generate the underlying common-denominator wrapper implementation for us. Doing this "only as simple as possible" results in the apparent complexity of the situation described above, where some snippets of the native/wrapped language must necessarily leak out into the meta-binding artifact.

---

Is that the situation? Could a different model avoid the "leakiness" of the current approach?

[kevinmehall]

Your guess is about right. The overarching framework is language-agnostic, but you want to bind it to code written in C (substitute any other language in this paragraph), you eventually want to call C code. Well, there's this existing syntax for making C calls, and a tool that already knows how to compile it, known as C and a C compiler, so just embed snippets of C, and YAML's block quotes make it look somewhat reasonable.

The intention of YAML was that it's not "yet another language to learn". Not super common in JS circles (JSON would be ugly here), but they've probably heard of it, and rubyists are more likely to know it. However, once we go this far putting semantics on top of YAML, it almost is a new language, and one with suboptimal syntax, at that. So I've been coming up with alternatives. One is to be able to do extern "<language>" {} in Signalspec, and embed code there, or we could put annotations in the source files of each language.

Rust has strong support for macros, including procedural macros, which are compiler plugins of with Rust code that transforms the token tree. These macros could define an interface, with Rust code embedded, all in a .rs file. C could get to almost the same point with a bunch of empty macros, and a separate parser that parses just the macros from the same file, kind of like Qt MOC does. But languages that don't traditionally have a compilation step are tougher.

To use a component in JS we'd want to leverage the JS-native APIs like EventEmitter, DuplexStreams, and plain callbacks as much as possible, but defining components in JS needs something declaring the component interfaces that can be processed at compile time without executing the JS in order to generate the C-callable function stubs for the other components to link to.

Going beyond just macros, an academic project that did similar componentization invented a dialect of C called nesC for this purpose. That scares some people off, and since we want to support many languages, it's even tougher, because we really don't want to also have and maintain compilers for nesJS, nesRust, and nesLua too.

[kevinmehall]

Note that the definition of the interface and the implementation are mixed together. I've thought about splitting that -- "Here's the interface any SPI controller offers in abstract form; here's a C implementation of that for the LPC1830 SPI peripheral; here's a Rust implementation for SAMD21". One problem there is when we don't actually care if an implementation covers the whole interface; just the parts that are used. If we come across a bad SPI controller that doesn't support SPI mode 2, that should only be an error if you stack a component on top that uses that mode. If we can check all this at compile time, it's effectively compile-time duck typing.