Build with Rust 1.93 failing

[jeffcharles]

See this GitHub Action workflow run. It's reporting an out of memory error from Rust.

I'm able to reproduce the failure when running locally on MacOS. Downgrading from Rust 1.93.0 to Rust 1.92.0 seems to resolve the issue. It also looks like this only affects WASI preview 2 and not WASI preview 1.

Looks like the failing test is this one.

[jeffcharles]

I've determined the error isn't related to Javy.

I see the exact same error when I run this Rust program with WASI p2 with Rust 1.93.0:

fn main() {
    for _ in 0..125 {
        std::fs::read("empty.txt").unwrap();
    }
}

Reducing the 125 to 124 causes the error to go away. If I use 125 and WASI p1 the error goes away. If I use 125 and WASI p2 with Rust 1.92.0 the error goes away.

I suspect this is manifesting in our CI because we open 5 files per 262 test case. I'll need to test with C to see if this is a bug with WASI libc.

[jeffcharles]

Reducing the problem further:

use std::os::fd::AsRawFd;

fn main() {
    for _ in 0..125 {
        let f = std::fs::File::open("empty.txt").unwrap();
        eprintln!("{}", f.as_raw_fd());
    }
}

With Rust 1.93.0, this results in an incrementing number until it hits 127 and then crashes. The WASI adapter imposes a limit of 128 open file descriptors and return an ENOMEM when that limit is hit. Rust 1.92.0 and WASI preview 1 return 4 as the file descriptor number for each pass through the loop. So for whatever reason, this isn't re-using a presumably previously closed file descriptor in each pass of the loop.

I've also confirmed that this C program compiled with WASI SDK version 29 does not exhibit the same problem:

#include <stdio.h>
#include <stdlib.h>

int main() {
    for (int i = 0; i < 125; i++) {
        FILE *file = fopen("empty.txt", "r");
        if (file == NULL) {
            perror("Failed to open file");
            return 1;
        }
        fclose(file);
    }

    return 0;
}

[jeffcharles]

My current hypothesis is this change which would have been pulled into Rust 1.93 (uses WASI SDK 29) and wasn't present in Rust 1.92 (uses WASI SDK 27) resulted in fd_close no longer being called in the adapter so path_open would always allocate a new descriptor entry.

For additional context, for Rust and WASI p1 and p2, Rust uses WASI preview 1's path_open to create all file descriptor entries. In p2, path_open goes through an adapter to use p2 APIs. The WASI fd wraps a Rust fd. The Rust fd implements drop and calls libc::close for the drop and ignores any returned errors. So the call to open a file goes through an adapter with its own descriptor list but the call to close bypasses the adapter.

[jeffcharles]

Looks like this has been fixed in Rust nightly. So presumably we just need to wait for a newer Rust version to be released.