Skip to content

Incorrect edit_functions values for some instances #24

@jatinganhotra

Description

@jatinganhotra

Hi Loc-Bench creators,

We have our own implementation of extracting the modified nodes from a given gold patch and we were converting our format to map to Loc-Bench edit_functions format.

We have hit an issue for these 3 instances, where the edit_functions values are incorrect for the Loc-Bench czlll/Loc-Bench_V1 dataset on Huggingface. For these 3 instances, these files and functions/modules are missing from edit_functions

Mismatch instance_id: AzureAD__microsoft-authentication-library-for-python-454
  missing_from_ours (0): []
  extra_in_ours   (3): ['msal/application.py', 'msal/oauth2cli/assertion.py', 'msal/oauth2cli/oauth2.py']
================================================================================
Mismatch instance_id: NVIDIA__NeMo-9100
  missing_from_ours (0): []
  extra_in_ours   (1): ['nemo/collections/asr/parts/submodules/ctc_greedy_decoding.py']
================================================================================
Mismatch instance_id: matrix-org__synapse-8744
  missing_from_ours (0): []
  extra_in_ours   (1): ['synapse/handlers/receipts.py']

I noticed another open issue on GH here: #20 which points out the same issue.

Could you please take a look? Please let me know if you need additional information.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions