mp.input: allow async completions and refactor complete handling

[CogentRedTester]

Previously, it was impossible to generate a list of completions for an input.get() request asynchronously, as the values had to be returned by the callback function.

This PR provides a way to send completions asynchronously by passing a second argument to the complete callback; a function which can be passed the return values to respond to the complete event.

This PR also refactors how console.lua handles completions so as to avoid presenting the user with outdated completions, or allowing the user to select them. The old code was fundamentally broken, something that went undetected because commands.lua, the only script using this API, responded so quickly that no-one ever noticed the inconsistencies. To prevent visual flickering, old completions are dimmed to indicate to the user they are disabled.

Motivation

Some scripts may not want or be able to generate the completion results synchronously. For example, they may want to do asynchronous IO for performance reasons, or the design/code of the script may simply not allow certain information to be collected synchronously.

My file-browser script is a good example. I would like to be able to use the file-browser code to read the contents of directories so that I can complete paths automatically in input boxes. However, file-browser reads directories asynchronously for performance reasons, and it is now so embedded into the code architecture that it is not viable to change or disable it.

Examples

Here is a basic example of a script responding asynchronously with an exaggerated delay:

mp.add_key_binding('KP8', nil, function()
    input.get({
        prompt = 'Lua: >',
        complete = function(line, response)
            mp.add_timeout(1, function()
                response({'value1', 'value2', 'value3'}, 1)
            end)
        end,
    })
end)

And here is a more complex example that uses ls to complete file paths to send to loadfile:

local function command_async(args)
    local co = coroutine.running()

    mp.command_native_async({
            name = "subprocess",
            playback_only = false,
            capture_stdout = true,
            capture_stderr = false,
            args = args
        }, function(...)
            coroutine.resume(co, ...)
        end)
    local success, res = coroutine.yield()
    if not success then error(res) end
    return res.stdout
end

local function process_ls(directory, cb)
    local args = {"ls", "-1", "-p", "-A", "-N", "--zero", "-L", directory}
    local files = command_async(args)
    if not files then return cb({}) end

    local list = {}
    for str in files:gmatch("%Z+") do
        table.insert(list, str)
    end

    return cb(list)
end

mp.add_key_binding('KP1', nil, function()
    input.get({
        prompt = 'Open file:',
        complete = function(line, response)
            local dir = utils.split_path(line)

            coroutine.wrap(process_ls)(dir, function(contents)
                response(contents, #dir+1)
            end)
        end,
        submit = function(line) mp.commandv('loadfile', line) end,
    })
end)

[github-actions]

Download the artifacts for this pull request:

Windows

• mpv-v0.41.0-dev-gc14c9cf6c-pr17305-22274405123-aarch64-pc-windows-msvc
• mpv-v0.41.0-dev-gc14c9cf6c-pr17305-22274405123-aarch64-pc-windows-msvc-pdb
• mpv-v0.41.0-dev-gc14c9cf6c-pr17305-22274405123-i686-w64-mingw32
• mpv-v0.41.0-dev-gc14c9cf6c-pr17305-22274405123-x86_64-pc-windows-msvc
• mpv-v0.41.0-dev-gc14c9cf6c-pr17305-22274405123-x86_64-pc-windows-msvc-pdb
• mpv-v0.41.0-dev-gc14c9cf6c-pr17305-22274405123-x86_64-w64-mingw32

macOS

• mpv-v0.41.0-dev-gc14c9cf6c-pr17305-22274405123-macos-14-arm
• mpv-v0.41.0-dev-gc14c9cf6c-pr17305-22274405123-macos-15-arm
• mpv-v0.41.0-dev-gc14c9cf6c-pr17305-22274405123-macos-15-intel
• mpv-v0.41.0-dev-gc14c9cf6c-pr17305-22274405123-macos-26-arm

[guidocella]

It's simpler to just make complete() the only documented way to complete and deprecate the return values. You can keep trying to use the return values in input.lua internally for one release but the only scripts using them are commands.lua and my lua-repl.lua anyway. It's fine to remove them from defaults.js. I verified this using my script to clone all scripts in the wiki:

#!/bin/sh

cd /tmp
> failed.log
mkdir mpv-scripts 2>/dev/null
cd mpv-scripts

# skip huge and/or non-script repositories
for url in $(curl https://raw.githubusercontent.com/wiki/mpv-player/mpv/User-Scripts.md | grep -Po 'https://[^)]*(?=\)\*)' | grep -Ev 'CNN|CuNNy|mpv-prescalers|rikai-mpv|mpv-composition-guides|Anime4K|Upscale-Hub|bShaders'); do
    [ -e ${url##*/} ] && continue

    case $url in
        https://github.com/*/blob/*) curl -O $(echo $url | sed s~/blob/~/raw/~) ;;
        https://gist.github.com/*/*) curl -o ${url##*/} $(echo $url | sed s~github~githubusercontent~)/raw ;;
        https://raw.githubusercontent.com/*) curl -O $url ;;
        https://sr.ht/*) git clone --depth=1 $(echo $url | sed s~sr.ht~git.sr.ht~) ;;
        https://git.sr.ht/*/tree/*) curl -O $(echo $url | sed s~/tree/~/blob/~) ;;
        https://www.npmjs.com/package/*) npm i ${url#*/package/} ;;
        https://gist.github.com/igv) ;; # unsupported, it's a list of gists
        https://www.npmjs.com/search?q=keywords:mpv-script) ;; # not a script
        *)
            url=${url%\#*}
            # converts e.g. https://github.com/paradox460/mpv-scripts/tree/master/writeedits
            # to https://github.com/paradox460/mpv-scripts
            case $url in
                https://github.com/*/*/*|https://codeberg.org/*/*/*) url=$(echo $url | cut -d / -f 1-5) ;;
            esac
            # multiple users have a mpv-scripts repository, so include the
            # username in the directory
            dir=$(echo $url | cut -d / -f 4)_$(echo $url | cut -d / -f 5)
            [ -e $dir ] && continue
            git clone --depth=1 $url $dir || echo $url >> /tmp/failed.log ;;
    esac
done

and then rg 'complete(:| =)' ${(f)"$(rg -l mp\.input)"}.

[CogentRedTester]

It's simpler to just make complete() the only documented way to complete and deprecate the return values. You can keep trying to use the return values in input.lua internally for one release but the only scripts using them are commands.lua and my lua-repl.lua anyway.

I have updated the documentation to only mention the function method. Do you think we should add a deprecation notice? (in the documentation and/or a printed warning when done in the code)

[guidocella]

I guess the check to call complete() only once can also be removed at this point?

I think adding a mention in interface-changes is enough since nobody uses this.

[CogentRedTester]

Upon further reflection, I believe that the removal of the counter means that we need to also send console.lua the old line and line position, as the completion_old_line values in console.lua may not be associated with the same complete event anymore. Technically, I think this was a problem before as well, but the completions are typically returned so quickly that the race condition is not triggered. But if we're allowing and encouraging asynchronous operations then we should ensure this cannot happen. Or have I misunderstood how the completion code in console.lua works?

[guidocella]

In theory you could change the input line back and forth until completions arrive, but even if you revert to the line of when completions were requested then the completions for that old line will apply so what's the matter?

[CogentRedTester]

Maybe I have misunderstood how the completions are supposed to work. My interpretation was the following:

1. The user types something into the console or moves the cursor.
2. The complete event is sent to the client with the contents of the line before the cursor.
3. The client receives the complete event and responds with the completion options that should be presented based on the line received (the text before the cursor).
4. console.lua receives the completions and discards them if the text before the cursor has since changed.
5. The completions are presented to the user.

However, in the current code, step four above does not happen (at least not accurately). This is because the code that tests for this scenario:

    if message.client_name ~= input_caller or message.handler_id ~= input_caller_handler
       or line ~= completion_old_line or cursor ~= completion_old_cursor then
        return
    end

does a comparison with the completion_old_line and completion_old_cursor values, but these values are stored locally in console.lua and are overridden any time a new complete event is sent to a client. This means that if a second complete event is sent to a client before the response to the first complete event is received, then the first set of completions, when received, will not be discarded.

Normally, this would be difficult to detect, as the responses occur so quickly that the incorrect completions are never actually drawn on the screen, but if the delay is larger (which may happen if we're doing asynchronous I/O) it will be very much noticeable. You can test this on master right now by adding a busy-loop to commands.lua:

local function complete(before_cur)
    ...
    ...
    ...
    local t = mp.get_time()
    while (mp.get_time() - t < 1.5) do end
    return completions or {}, completion_pos, completion_append
end

Then, in commands.lua, just type run ${ wait a second, and erase the ${. You will see property completions shown on the screen for a second before they are replaced with the correct completions for run.

---

Assuming I have understood all of this correctly, we probably want to either ensure that a client cannot send old completion events (the counter approach I originally proposed), or we have the clients send the line value back to console.lua to compare with the current line (which I believe has the behaviour we have been discussing).

[CogentRedTester]

I have added a commit that has the clients send the original line to console.lua for comparison. It does work, and the code changes are quite minimal. However, I have noticed while testing that completions that have already been received are not actually wiped until a new set of completions has been received. Perhaps it would be better to wipe the completion buffer immediately upon making a change to the text before the cursor? (not sure the best place in the code for that, perhaps in the complete() function?)

[guidocella]

Huh you're right my old line logic was completely non-functional lol.

Clearing the completion buffer on cursor move would indeed be better. I think you can

• call completion_buffer = {} unconditionally in handle_cursor_move
• in get-input replace completion_buffer = {} and

if line ~= "" then
    complete()
elseif open then
    -- This is needed to update the prompt if a new request is
    -- received while another is still active.
    render()
end

with handle_cursor_move() to deduplicate the logic

[guidocella]

Actually nevermind that looks terrible if completions are replaced immediately e.g. in commands.lua because they quickly disappear and reappear over and over.

[CogentRedTester]

I will experiment with some different options to clear/deactivate the completions and see if there is a way to prevent incorrect completions without visual regressions.

[CogentRedTester]

In the latest commit, instead of clearing the completion buffer, I am setting a completions_pending flag that will be cleared when an up-to-date completion is received. When this flag is set, the completions are not removed, but instead have their opacity reduced, and the completion cycle key is disabled. Because the text does not change, the worst of the flickering is removed, and I think the dimmed completions get the idea across when the completions are delayed.

The only time when the completion flickering is more noticeable is when the completion options do not change while typing (for example typing the second half of playlist in commands.lua), otherwise the changes in the results tend to mask the opacity flickering. Even in this scenario, I don't personally find the flickering anywhere near as bad as when clearing the buffer. While it's not perfect, I think this may be an okay compromise?

For terminal output mode I am just not drawing the completions, I did not notice any visible flicker on my terminal.

[guidocella]

While the flicker is small I still don't think it's worth adding when commands.lua completions is all 99% of users will ever use.

I checked what neovim does and until new completions arrive, it filters through the existing completions based on the new input line. You can try to do that if you want to.

EDIT: well that may still look bad in our case because of the grid completions instead of column...
Another option is to fade completions only after a small timeout.

[CogentRedTester]

I have added a short timeout before the completions are faded. On my device there is no visible flicker when using commands.lua. This should mean that clients which return completions almost immediately will show no flicker, and clients which are delayed will show dimmed completions. Theoretically, there may be some clients which take longer to generate completions than console.lua, but which still respond in less than a second, for which there will still be a visible flicker. However, given that such clients don't seem to currently exist, I think we can consider this a reasonable compromise.

I have also added a sentence to the documentation clarifying that the response function should be called with an empty table if there are no completions to display. Without doing so the old, dimmed completions would be displayed until the input is closed.