Introduce LspPlugin API

[rchl]

"Introduce new plugin API" is a little over the top. I just wanted to jump-start it with one new method to allow for progressive enhancement and get early feedback.

The new handle_update_or_installation_async method combines needs_update_or_installation and install_or_update. Those were split because the code sets installing... status text if needs_update_or_installation returns true but that meant that various if checks had to be duplicated across those two and generally made the logic harder to read. With this one method, there is now a callable function passed that can be used to start the progress (perhaps unnecessary since user could use configuration.set_view_status but then that would likely lead to inconsistent status messages across packages).

The new method takes a dict with params. This is to allow adding new params without breaking API compatibility.

Most class methods are passed PluginContext now with configuration instance now to allow packages to get resolved configuration. Some packages currently read settings with sublime.load_settings but that has the problem of not considering project overrides.

Resolves #2039
Resolves #2491

[rchl]

Actually just dumped some old code that I had in stash.

[rchl]

I've been thinking about case that tends to come up often - people want to use non-managed server binary.

Overriding command is one way to do it and it would work for simple cases but in more complex cases like LSP-clangd where there is a lot of arguments and some are even programmatically adjusted during start, it wouldn't work.

So my thought is that, given we use a variable like $server_binary, user should be able to override just the variables from the config file. It would need to override variables added through the additional_variables API.

Just a little concerned that it's not the simplest to explain to the user. Or at least will require a bunch of extra doc in every package but that's the best I can think of.

[jwortmann]

@predragnikolic Do you maybe see anything else for the LspPlugin class in api.py that could be improved? Like the method names, or the parameter names and types, or is anything else missing?

I think it looks mostly good from my side now, but we should try to get it "perfect" now, so that we don't need any breaking changes or awkward additions later.

[predragnikolic]

I will have some free time at the begging of next week. (Mon, Tue)

I'll try to find time to go through the changes and try to migrate some plugins to try out the new API (I will do it only locally).

Maybe I will return with some feedback.
Else if I cross that timeline I would suggest to not wait on me.
And maybe transition one the most complicated plugins first just to see if something seems like it needs additional tweaking.

[rchl]

I'm toying with the idea of making ClientConfig fully typed but discussing it here could get messy so created discussion for it - #2860

Feel free to chime in with opinions or better ideas.

[rchl]

I think I should also add a method for extending PATH to support this functionality from lsp_utils.

[predragnikolic]

Currently some LSP-* plugins rely on lsp_utils and LSP,
It would be nice if the new API completely replaced lsp_utils, and made the plugins only rely on LSP.

Resasons:

• lsp_utils is sometimes requires to be in sync with the latest changes in LSP, and PC sometimes can causes issues because it doesn't install dependecies when we expect it.
• Doing local development of dependencies is hard to setup in ST for some reason. (I have some steps in some discord thread on what I have to do in order to do local development for lsp_utils and not let it be overriden by PC, and also I have not really followed the thread on how to publish lsp_utils now...) To much steps. But I see having that code in LSP would ease maintenance.

This might meant that we move code from lsp_utils in LSP, but not sure how other LSP maintainers would feel about it. (but as said it would ease maintenance of lsp_utils, with the cost of moving it into LSP and shipping unusued code if people did not use some of those plugins)

[predragnikolic]

If we add file pattern support in the future
It would be nice to think in advance and see if the current API design would make it easy to add file patterns sometimes in the future.

For example, instead of currently only supporting selector, think if the current API is nice, and will allow adding on_uri: NotRequired[list[str]] (If specified the server will only start for the given uri. and will shutdown as soon as the last view that matches the uri is closed.) (https://github.com/zed-industries/package-version-server should only be started in package.json files)

and workspace_contains: NotRequired[list[str]] in a nice way. (lsp-ast-grep should only start if the server if the workspace contains a configuration file, tailwind css should only start in a workspace that has tailwindcss configured)

[predragnikolic]

There are some fine improvements in this PR.
This is just a first look at the PR, will try to take a look once more, tonight and tomorrow.

I did leave some wishes that I would like to see in the next version of LspPlugin.

Left some ideas just so you can consider them. Feel free to steal them if you like some of them, else feel free to discard them.

This is just an initial first review from my side.

[rchl]

It would be nice if the new API completely replaced lsp_utils, and made the plugins only rely on LSP.

Moving api decorators to LSP is a step in that direction. Also I guess a bit better and easier to use API through LspPlugin.

But I think it would be hard to convince everyone (maybe even myself) that things like NodeRuntime or UvVenvManager should be in LSP.

If we add file pattern support in the future
It would be nice to think in advance and see if the current API design would make it easy to add file patterns sometimes in the future.

I guess the only relevant method is is_applicable. I don't think we'd need to change anything in the API for that. Lets say we add opt-in activation_events in ClientConfig. I don't think that would affect the API in any way.

[predragnikolic]

Let say that we want to override project settings for a LSP-rust-analyzer

Currently we need to specify

{
    // folders: [
    //   ...
    // ]
    "settings": {
        "LSP": {
            "rust-analyzer": {
                "settings": {
                    //Setting-here
                }
            }
        }
    }
}

In the old API the key in project overrides, I think, Was the config name.
With the new API in this PR,
the key value that we need to specify in project overrides is still the name property? (Not the PackageName) Is that right?

[rchl]

With the new plugin API it would the package-name so it's a breaking change. It was discussed in #2739 (comment) but kinda left without final conclusion.

[rchl]

We might want to consider (re)adding on_settings_changed?

LSP-json wants to notify server using json/schemaAssociations whenever settings change. Currently it can react to workspace/didChangeConfiguration but that's not future proof. If the server would start supporting workspace/configuration then it wouldn't be called on changes but we would still want to notify server immediately when settings change of updated schema associations.