Thoughts on providing a way to customize type-system?

[timocov]

Hi all,

I wanted to ask your opinion/vision on what do you think about providing a way to extend default types.

Problem

For example, imagine your API has a field that is suppose to be a date (the format doesn't matter, but we can say it is ISO-based date i.e. YYYY-MM-DD). At the moment, you have 2 options to define it in your model:

1. By using typestamps:

@timestampFormat("date-time")
timestamp Date

structure Person {
  dob: Date
}

The issues with this approach:

• the type in in Java type system will be Instant. It is not particularly bad, but the type is too wide and it doesn't allow you to be more precise in your intention (and yes, I do understand that not all languages have "Date" class support e.g. javascript)
• if this type is being used as an input, it would let users to enter values with a time part, which sounds more dangerous but it is more like a smithy issue than smithy-java I guess?

2. By using a custom type with extra validation attached:

@pattern("^\\d\\d\\d\\d-\\d\\d-\\d\\d$")
string Date

structure Person {
  dob: Date
}

The issues with this approach:

• the target type in all codegens will be a string, which doesn't imply any validation whatsoever (unless you validated it before, but still it is a string)
• you can provide invalid date e.g. 2000-31-99. even if you tweak the pattern, it might before unmaintainable and harder to debug

Potential solution

The smithy syntax is very flexible, and what I was thinking was to define either a custom trait that I can apply that will change the type definition or relying on a custom type defined in a common library that will do the same (but limits consumers of defining different date types). For instance, I could use a trait from Alloy library https://github.com/disneystreaming/alloy/blob/v0.3.36/modules/docs/misc/constraints.md#alloydateformat or even define my own in the same way and handle it. I started looking into smithy-java codegen for this purpose and realized that it doesn't look possible. Options I explored:

• provide a custom GetterSection interceptor and for such members write a getter that transforms return type to whatever type I need (e.g. LocalDate). The problem is that such a conversion will happen each time you call the getter and it might have performance penalties.
• I was looking into hooks and interceptors to apply modifications to the structure class/its builder, but it gets even more complicated and I don't think it is even possible without re-writing the whole codegen myself.

So I want to ask your opinion on what a solution for this problem should be based on your experience and expertise.

To me, I was thinking that it would be good to have a way to configure the codegen so that customers could modify types they want to use for specific shapes and their serialization modifications e.g. a plugin, that can modify a symbol being used for a shape (this is kinda possible already if customers create their own codegen plugin by replicating what java-client-codegen does but providing custom SymbolProvider (e.g. by extending JavaSymbolProvider and overriding stringShape to return LocalDate for strings with dateFormat trait attached to them). But this won't change the (de)serialization and the code simply won't compile because of type incompatibility in the generated (de)serializer code.

Another option that I was considering was if code interceptors could affect the context of the codegen, not just "all or nothing" - e.g. for a given piece of code the interceptor could override context values such as member to be something else. This could in theory allow to override certain codegen parts without re-writing everything, but it doesn't look flexible and exposes implementation details for context keys.

Thanks in advance!

[timocov]

Another slightly related problem is errors codegen (for the server in particular) - if someone wants/needs to follow things like https://docs.spring.io/spring-framework/reference/web/webmvc/mvc-ann-rest-exceptions.html to generate errors, there is no way to tell the codegen what the base class/interface for an exception should be. In theory this might be solved by having a custom middleware/filter that would transform things accordingly, but it is worth mentioning. Ignore this one please, no need in this, #1032 would be a much better fit I think.

[timocov]

I had a play with the codebase, posting some results here.

Considering the following model (simplified version):

use alloy#dateFormat

@dateFormat
string Date

structure Obj {
    date: Date
}

In order to make it so that:

• Obj.getDate() returns LocalDate
• date is serialized as a string and deserialized correctly from a string

I had to change the following components:

• in JavaSymbolProvider#stringShape method add

   if (stringShape.hasTrait("alloy#dateFormat")) {
       return CodegenUtils.fromClass(LocalDate.class);
   }

  note that this part can be customized via custom codegen (I can create a class that extends JavaSymbolProvider and override it there)

• in SerializerMemberGenerator#stringShape method add

   if (stringShape.hasTrait("alloy#dateFormat")) {
       writer.write("serializer.writeString(${schema:L}, ${state:L}.toString())");
       return null;
   }

  this cannot be customized easily because the class is final and it is being used directly in other generators

• in DeserializerGenerator#stringShape method add

   if (stringShape.hasTrait("alloy#dateFormat")) {
       writer.write("SharedSerde.deserializeDate(${schemaName:L}, ${deserializer:L})");
       return null;
   }

  this cannot be customized easily because the class is final and it is being used directly in other generators

• create DateGenerator class that writes to sharedSerde the following:

   var template = """
         static ${localDate:T} deserializeDate(${schema:T} schema, ${shapeDeserializer:T} deserializer) {
             return ${localDate:T}.parse(deserializer.readString(schema));
         }
   """;
   writer.putContext("schema", Schema.class);
   writer.putContext("localDate", LocalDate.class);
   writer.putContext("shapeDeserializer", ShapeDeserializer.class);
   writer.write(template);

  and then invoke it somewhere in DirectedJavaClientCodegen (e.g. in customizeBeforeIntegrations method). This can be customzed via custom codegen.

Not sure if it is helpful, but at least to provide some hints on what maybe be adjusted in order to make it more flexible I guess?

[timocov]

I guess my main question is it something the team is interesting in adding to smithy-java? And if not, then what would be your recommendation for me to do (e.g. creating completely custom codegen without relying on smithy-java; or maybe forking and adding minimal possible custom changes to avoid issues whilst merging with upstream; something else?). Thanks!