sparse matrix as input

[caroheymes]

Hello everyone,

I am completely new to the subject.
I have a very large dataset with a strong class imbalance (Class 1 < 1%)
After one hot encoding, I have 15,000 geographical location categories.
Naturally, the kernel crashes with the following code:

xpl = SmartExplainer(
preprocessing=enc_columntransfo,
model=model,
backend='acv',
data=X_train.toarray(),
)

I am unsure how to deal with csr matrix neither how to implement shap.kmeans.

Thank you in advance for your help.

[ThomasBouche]

Hi,
Have you tested with shap instead of acv?
if you remove the backend parameter, by default shapash compiles shap.

Using the preprocessing argument, the geographic column should be on one column.
However, there is no data parameter in the SmartExplainer.

You can try this:

xpl = SmartExplainer(
preprocessing=enc_columntransfo,
model=model,
)

xpl.compile(x=X_train.toarray())

app = xpl.run_app(title_story='Title', port=8020)

app.kill()

If you still have an error, can you send the error message?