fix: build and run in the browser with GOOS=js GOARCH=wasm

[paralin]

Fixes build + run in the web browser with GOOS=js GOARCH=wasm.

Testable with:

https://github.com/agnivade/wasmbrowsertest/

[ncruces]

Doesn't it make more sense to just do:

func (vfsOS) FullPathname(path string) (string, error) {
	if runtime.GOOS == "js" {
		return filepath.Clean(path), nil
	}

	link, err := evalSymlinks(path)
	// ...
}

[paralin]

@ncruces Ack I fixed that now. I had to do some other changes to get the tests to pass, (and they all do now), but Im going and reviewing what can be undone now to reduce the commit size.

[ncruces]

OK. I'll be frank.

I tried to run the tests. A bunch failed. If the needed changes are big, it's very likely this will regress unless tested.

And I'm not sure I want to commit to supporting js as an OS. I'm not sure I want to add a bunch of code for it. I'm not sure I want to spend the CI "effort" on it.

So, first of all, why? Why do you want to open files with js? What does that even mean? Cause if you're not going to open files, if you're going to work with in memory DBs (tests on those seem to already pass) why do you need any of this?

[paralin]

@ncruces I understand you don't want to commit to supporting it. I excluded some of the test cases with go:build flags in the most recent commit. If you want to close this out I'm happy to just fork and work on it there instead.

But to answer your questions:

Why do you want to open files with js? What does that even mean?

The Origin Private File System: https://developer.mozilla.org/en-US/docs/Web/API/File_System_API/Origin_private_file_system

It's already possible to open and use sqlite databases with this through sqlite.wasm, however, the thought is to do it from Go via your library (to avoid a separate .wasm binary).

See: https://sqlite.org/wasm/doc/trunk/persistence.md#opfs

[ncruces]

OK, sure.

But isn't that better resolved by implementing a VFS from scratch for just that use case?

Is the Go os package - as used by the default VFS - really the best approach here? What about file locking and concurrency?

I've often wondered if the default VFS should be os, or if it should be possible to build SQLite without the default VFS (or rather, with a different one).

It seems entirely non-obvious to me that my default VFS would be useful in a browser. SQLite doesn't ship its Unix VFS to browsers either.

[paralin]

That's valid. I'll retract this for now and think it through further. Thanks.

[paralin]

@ncruces I would like to add that I actually did get this working just fine:

cjs@flame sqlite % GOOS=js GOARCH=wasm go test -v
=== RUN   TestSQLite
--- PASS: TestSQLite (0.17s)
=== RUN   TestSQLiteWithMode
--- PASS: TestSQLiteWithMode (0.01s)
=== RUN   TestSQLiteIterator
    sqlite_test.go:173: Found key: prefix:key1, value: value1
    sqlite_test.go:173: Found key: prefix:key2, value: value2
    sqlite_test.go:173: Found key: prefix:key3, value: value3
--- PASS: TestSQLiteIterator (0.01s)
PASS
ok      /store/kvtx/sqlite     1.375s

package wasm

import (
	"context"
	"database/sql"
	"os"
	"path/filepath"
	"runtime"

	sqlite "github.com/ncruces/go-sqlite3"
	_ "github.com/ncruces/go-sqlite3/driver"
	_ "github.com/ncruces/go-sqlite3/vfs/memdb"
)

// SqliteWasmConfig implements the SQLiteDriverConfig interface for pure Go SQLite driver.
// This uses github.com/ncruces/go-sqlite3 which is sqlite -> wasm -> go.
type SqliteWasmConfig struct{}

// DriverName returns the driver name for pure Go SQLite.
func (c SqliteWasmConfig) DriverName() string {
	return "sqlite3"
}

// OpenDSN returns the DSN to use with sql.Open().
func (c SqliteWasmConfig) OpenDSN(path string) string {
	if runtime.GOOS == "js" {
		return "file:" + filepath.ToSlash(path) +
			"?vfs=memdb&_pragma=journal_mode(WAL)&_pragma=synchronous(NORMAL)&_pragma=busy_timeout(5000)"
	}
	return "file:" + filepath.ToSlash(path) +
		"?_pragma=journal_mode(WAL)&_pragma=synchronous(NORMAL)&_pragma=busy_timeout(5000)"
}

// Description returns a description for pure Go SQLite.
func (c SqliteWasmConfig) Description() string {
	return "SQLite database key-value store using SQLite to wasm to pure Go driver"
}

// IsBusyError checks if the error is a SQLITE_BUSY error for pure Go driver.
func (c SqliteWasmConfig) IsBusyError(err error) bool {
	if sqliteErr, ok := err.(*sqlite.Error); ok {
		return sqliteErr.Code() == sqlite.BUSY
	}
	return false
}

// IsNestedTxError checks if the error is a nested transaction error for pure Go driver.
// This occurs when BeginTx is called on a connection that already has an active transaction.
func (c SqliteWasmConfig) IsNestedTxError(err error) bool {
	if sqliteErr, ok := err.(*sqlite.Error); ok {
		// SQLITE_ERROR (1) with message containing "cannot start a transaction within a transaction"
		return sqliteErr.Code() == sqlite.ERROR
	}
	return false
}

// Store is a SQLite database key-value store using pure Go SQLite driver.
type Store = common.Store[SqliteWasmConfig]

// NewStore constructs a new key-value store from a SQLite database.
func NewStore(db *sql.DB, table string) (*Store, error) {
	return common.NewStore(db, table, SqliteWasmConfig{})
}

// Open opens a SQLite database store using pure Go driver.
func Open(ctx context.Context, path string, table string) (*Store, error) {
	return common.Open(ctx, path, table, SqliteWasmConfig{})
}

// OpenWithMode opens a SQLite database store with file mode.
func OpenWithMode(ctx context.Context, path string, mode os.FileMode, table string) (*Store, error) {
	if runtime.GOOS == "js" {
		return common.Open(ctx, path, table, SqliteWasmConfig{})
	}
	return common.OpenWithMode(ctx, path, mode, table, SqliteWasmConfig{})
}

// _ is a type assertion
var _ kvtx.Store = ((*Store)(nil))

[ncruces]

So I had more errors because I was using an older version of wasmbrowsertest. Compiling it from head seems to fix a bunch of them.

In the end, I think I ended up where you did: testable examples don't work with wasmbrowsertest. And there are a few errors regarding symlinks not working and the wrong error being returned in some places.

I still don't think the standard file based os VFS is very useful. I imagine one purpose built for OPFS would be better.

And the problem remains that if I don't add something like this to CI, it'll become instantly broken. So, I'm not sure.

I could not reproduce the crashes in #370.

[paralin]

I'm running gotip which might have something to do with it. My PR to bring wasmbrowsertest up to date with latest deps was just approved and hopefully merged soon; that fixed a lot of random errors. I agree that deeper thought is required on how to make this work properly in your library. It's promising though - I was able to get SQL working! As for CI it's possible to run wasmbrowsertest in GitHub actions. Anyway, right now I'm pushing forward on binding to the upstream sqlite-wasm for browser as it has some complex logic for syncing between multiple writers with SharedArrayBuffer that I don't want to think through replicating in Go just yet, but maybe soon I will circle back and try to tackle having feature parity with that here too, since being able to run sqlite without the message passing latency is definitely a win.

One of the main issues that I ran into is that currently you can't access the synchronous file system apis within a SharedWorker, which is where I'm running go currently. So that kind of makes any performance wins from this approach less promising, however, the asynchronous apis are available and theoretically could work. From what I've read though, they are a lot slower than the synchronous version.

[ncruces]

I just committed some of the test skips. No point in excluding examples, as that's easier done with:

go test -short -skip Example ./...

Some time back I had to remove running tests with wasmtime as it is too slow. I'm considering my options, maybe I run this instead.