Handling `JLArrays.jl` inputs

[lkdvos]

I wanted to open a quick discussion on how to handle JLArrays.jl.
I'm struggling a bit to come up with the correct way, because of the following clashing things:

1. JLArrays is mostly for testing and checking that no GPU-incompatible paths are taken. However, our GPU paths are totally separate from the CPU ones, and we don't want to ban scalar indexing on the CPU side, and pulling a JLArray through the CPU codepaths does not actually test that our GPU codepaths are clear of scalar indexing.
2. On the flip side, it is very convenient to be able to test higher-level codebases (thinking of MPS libraries etc) and have the factorizations "just work", to test the code around without having to set up complicated special cases.

So, I think to make things convenient the best strategy might be to just "make it work", ie write a package extension that tries to do some overloads basically by inserting @allow_scalar more or less everywhere, and going from there.
I'd be open to suggestions and discussions though!

Tagging the following people to include in the discussion:

@kshyatt
@Jutho
@mtfishman

[kshyatt]

Yeah the annoying thing is of course JLArrays doesn't "natively" support most factorizations. Maybe we could create a dummy driver for them that converts back to a regular array "at the last moment" before the LAPACK call?

[lkdvos]

Actually just using the LAPACK driver seems to work fine since the pointer conversion logic works, it's the surrounding code that breaks, e.g. the positive = true in the QR:

MatrixAlgebraKit.jl/src/implementations/qr.jl

Lines 178 to 190 in e6e844f

	if driver === LAPACK()
	@inbounds for j in 1:minmn
	s = sign_safe(A[j, j])
	@simd for i in 1:m
	Q[i, j] *= s
	end
	end
	else
	# guaranteed τ exists and no longer needed
	τ .= sign_safe.(diagview(A))
	Qf = view(Q, 1:m, 1:minmn) # first minmn columns of Q
	Qf .= Qf .* transpose(τ)
	end

[mtfishman]

At least for our purposes, an extension that overloads CPU functions that make use of scalar indexing and uses @allow_scalar or converts back and forth to a plain Array to avoid the scalar indexing would be sufficient. We have a lot of tests using JLArrays and MatrixAlgebraKit which were working fine until the change to QR that makes it positive by default, which hit the scalar indexing codepath @lkdvos pointed out. So I think a lot of code actually works quite well as-is for JLArray, it may not require many overloads. I would maybe bias towards converting back and forth to Array rather than @allow_scalar since maybe it would be better for efficiency (I'm not sure if JLArrays have fast scalar indexing?) but probably that doesn't matter since they are just designed to be used for testing.

I realize you want might want to avoid making things more complicated, but I see in that QR example, driver === LAPACK() is used as a "trait" for whether or not there is fast scalar indexing, would it make sense to have a devoted trait for that which uses driver === LAPACK() as a fallback?