plugin: new framework to implement plugins

[FiloSottile]

No description provided.

[AnomalRoil]

I'm not entirely done with my tests but I can confirm both the standard and the interactive mode worked for me.

[AnomalRoil]

Okay, a first set of issues when trying to use plugin when using age as a library willing to support arbitrary age plugin recipients:

• the currently exposed age.ParseRecipient and age.ParseIdentity are both only returning ones of type *X25519Identity only.
• there exist a parseIdentities function supporting plugin ones, but it seems the only way to use it is to rely on EncryptedIdentities and there's no such things for recipients.

The rest of the library code is fairly agnostic of whether this is a plugin recipient or identity, so having an easy way of parsing plugin identities would be nice.

Currently #518 is providing a way to detect "regular" identities from plugin identities through error handling, which could work as well but isn't as convenient I guess, although it's more in line with the rest of the currently exposed parsing functions (one is supposed to parse SSH vs age recipients and identities using the specific exported functions for each currently, meaning everyone ends up re-implementing this).

In other notes:

• the plugin.NewIdentity and plugin.NewRecipient are both expecting a plugin.ClientUI but there isn't any exported "default" ClientUI. This might be a decision to force people to implement their own?
• The existing plugin.ClientUI interface does not expect any context to be passed to any of the prompts, whereas a library would be typically carrying these around, especially for such prompts that we might want to timeout on. It might be good to refactor the ClientUI interface to expect contexts in each of the function calls?

[olastor]

It works great in my tests so far 🎉

One thing I noticed though is that the age client that communicates with the plugins currently still seems to spawn separate processes. For example, age -d -i identities.txt -o test somefile.enc (two identities in identities.txt) will do something like this (the binary I used is v1.2.0, but I think it would be the same case with this branch):

-> add-identity AGE-PLUGIN-SSS-1R79SSQQQQQQQQQ8LQNQVZZKZXQX
-> grease-1331f5cec305b37b

-> recipient-stanza 0 X25519 DsW/n
xxxx
-> done

-> msg
SGFuZGxlZCBpZGVudGl0eSBpbiAxNzEzNQ
age: sss plugin: Handled identity in 17135
-> ok

-> grease-7cada51a126db013 78
gBr1Q7wtAl4/a8vPQe/OLdZyc0z5hVVuf06OuIxigh7vNBlTn07r8PTPCqAE0bN3
ZaljVNGkPNxFyYgaIwrejyID6rk
-> unsupported

-> done

-> add-identity AGE-PLUGIN-SSS-1R79SSQQQQQQQQQ8L4FTV5NPF26

-> grease-236eb6a331bae717

-> recipient-stanza 0 X25519 DsW
xxx
-> done

-> msg
SGFuZGxlZCBpZGVudGl0eSBpbiAxNzE0MQ
age: sss plugin: Handled identity in 17141
-> ok

-> done

age: error: no identity matched any of the recipients

The debug messages I added show that each identity is handled in a different process/pid (each also gets the same stanza separately), so I think the recipients/identities arrays might currently not get longer than 1. It works, but sorting identities won't be possible like this. This is basically #526 , which I closed because it seemed to be intentional at the time. I think having only one plugin process with all the data could open more opportunities (more advance logic, ordering, maybe batching encrypts/decrypts in one request/operation, caching pins).

[AnomalRoil]

After further testing in gopass, I can confirm that for a library trying to support Plugin Recipients and Identities, the current interfaces are not enough yet:

• I had to copy the parseIdentity and parseIdentities functions to support plugin identities from cmd/age/parse.go
• I had to copy the func identitiesToRecipients(ids []age.Identity) ([]age.Recipient, error) from cmd/age/age.go
• I had to create my own wrappedIdentity in order to have a way to output the encoding of an identity, it might be best to provide a way of serializing identities easily? (Maybe implement the Stringer interface for both plugin.Identity and plugin.Recipient?)
• While plugin.Identity has a func (i *Identity) Recipient() *Recipient, it sadly can easily contains secret key material, since it basically contains the plugin.Identity, it would be nice to have a mapping from plugin.Identity to plugin.Recipient that is "safe", but this probably requires changing the plugin spec and interface to add a new type of interaction, so might not be ideal.

On the plus side:

• implementing my own terminalUI was very easy with the current ClientUI, my previous comment about passing it contextes still holds since I had to put context.Background() everywhere in mine for now.
• just passing the plugin identities to age.Decrypt does work nicely.

One last comment for now, it might be good in age.Decrypt, when ranging over stanzas/identities to first sort them to prioritize native identities over plugin ones?

[Enzime]

I'm not sure if it is relevant to the scope of this PR, but one thing I think would be useful would be plugin support for age-keygen, primarily age-keygen -y, this would simplify the CLIs for plugins and make it more consistent

[AnomalRoil]

@FiloSottile Any chance of getting this merged soon?

Or have the security fix related to plugin in it? We're using it in Gopass master atm and are planning a release soon ^^'
Edit: nevermind, we went back to using the latest version since we weren't actually using the new APIs directly.

I think all of the "bumps on the road" I had using this framework can be solved by additive improvements that could come later on, such as:

• exporting parseIdentity and parseIdentities
• exporting identitiesToRecipients
• adding Context to the UI

[AnomalRoil]

I'd argue that a common pattern in a CLI is to say "Please provide XYZ or press enter to use the default value", so expecting a Stanza with a non-0 length body is not always practical.

At least it did bite me in my testing:

-> request-public
cGxlYXNlIHByb3ZpZGUgdGhlIGNoYWluaGFzaCBvZiB0aGUgbmV0d29yayB5b3Ug
d2FudCB0byB3b3JrIHdpdGggKGFuIGVtcHR5IHZhbHVlIHdpbGwgdXNlIHRoZSBk
ZWZhdWx0IG9uZSk
please provide the chainhash of the network you want to work with (an empty value will use the default one)

this felt safe, but isn't supported with the current implementation.

[cedws]

I'm looking to implement an age plugin but a bit confused about what state the support is in, the docs on pkg.dev don't show really how to implement one.

Am I correct in saying this PR needs to be merged for support for implementing age plugins in Go?

[cedws]

When I call RequestValue with secret set to false my input is still masked, is this intended?

Also, when I press enter without entering anything I get this:

age: amnesia plugin: ok stanza has 0 bytes of body, want >0
age: error: amnesia plugin: failed to read line: EOF
age: report unexpected or unhelpful errors at https://filippo.io/age/report

[FiloSottile]

Apologies for letting this sit for so long.

• the plugin.NewIdentity and plugin.NewRecipient are both expecting a plugin.ClientUI but there isn't any exported "default" ClientUI

See #591 and #611.

• The existing plugin.ClientUI interface does not expect any context to be passed to any of the prompts, whereas a library would be typically carrying these around, especially for such prompts that we might want to timeout on. It might be good to refactor the ClientUI interface to expect contexts in each of the function calls?

That would have been a good idea, but ClientUI is already on main, and this does not feel worth a break.

One thing I noticed though is that the age client that communicates with the plugins currently still seems to spawn separate processes.

This is intentional, to give the user more control over ordering. It's a tricky tradeoff which I am open to reconsidering. See str4d/rage#414 (comment) and maybe open an issue with a concrete use case description?

• I had to copy the parseIdentity and parseIdentities functions to support plugin identities from cmd/age/parse.go

Opened #665 to track pluggable recipient/identity parsing, which is orthogonal to this PR.

• I had to copy the func identitiesToRecipients(ids []age.Identity) ([]age.Recipient, error) from cmd/age/age.go

I can't think of a good API to expose this, without depending on every package that implements an Identity, which anyway would be incomplete as there can be third-party ones. Somewhat related to #665.

• be implement the Stringer interface for both plugin.Identity and plugin.Recipient?

Done, but Recipient.String returns <identity-based recipient> if it's an identity-derived recipient.

it might be good in age.Decrypt, when ranging over stanzas/identities to first sort them to prioritize native identities over plugin ones?

Good idea. This is not plugin-specific, we can sort identities we know require no user interaction and can't fail (the core native ones) before the rest.

Done in a separate commit.

• While plugin.Identity has a func (i *Identity) Recipient() *Recipient, it sadly can easily contains secret key material

I'm not sure if it is relevant to the scope of this PR, but one thing I think would be useful would be plugin support for age-keygen, primarily age-keygen -y, this would simplify the CLIs for plugins and make it more consistent

Plugin key generation can be extremely complex and involve hardware interaction, so we should leave it to the plugin.

Adding an interface for age-keygen -y might be interesting, and would also make it possible to (try to) actually convert an identity string to a recipient string in Identity.Recipient. Opened #666.

When I call RequestValue with secret set to false my input is still masked, is this intended?

Fixed by #519.

[FiloSottile]

• the plugin.NewIdentity and plugin.NewRecipient are both expecting a plugin.ClientUI but there isn't any exported "default" ClientUI

See #591 and #611.

Implemented plugin.NewClientUI.