question about the drop_condition for categorical embeddings

[qym7]

Hi,

Thanks for your work and your codes !
I am trying to understand the logic for condition dropout and, I might be mistaken, and I found that for categorical embedding, there is no condition dropout enforced during training with the following codes:

    labels = labels.long().view(-1)
    use_dropout = self.dropout_prob > 0
    if (train and use_dropout) or (force_drop_ids is not None):
        labels = self.token_drop(labels, force_drop_ids)

Which is because that force_drop_ids is never None so we only drop conditions based on force_drop_ids.

Can you help me understand this? Thanks a lot for your time.

Best regards,

[liugangcode]

Thanks for the issue. I have revised the code to solve the issue. Let me know if you have further questions.

[qym7]

Thanks for your reply.
My concern is that after changing this line, the performance on categorical data will actually drop (tested on HIV condition). Do you have similar observations?