Skip to content

fix: line-count score configured null-language extensions instead of skipping them#1275

Open
MkDev11 wants to merge 1 commit into
entrius:testfrom
MkDev11:fix/issue-1274-null-language-line-count-scoring
Open

fix: line-count score configured null-language extensions instead of skipping them#1275
MkDev11 wants to merge 1 commit into
entrius:testfrom
MkDev11:fix/issue-1274-null-language-line-count-scoring

Conversation

@MkDev11
Copy link
Copy Markdown
Contributor

@MkDev11 MkDev11 commented May 14, 2026

Summary

Extensions configured in programming_languages.json with no language field receive score=0.0 with scoring_method='skipped-unsupported'. The scorer only routed a file to line-count scoring when its extension appeared in the NON_CODE_EXTENSIONS constant — it never consulted the JSON config. The 11 affected extensions are: env, gitattributes, gitignore, gql, graphql, ipynb, move, nim, pde, puml, tex.

Root cause: calculate_token_score_from_file_changes gated line-count scoring on ext in NON_CODE_EXTENSIONS. A configured null-language extension not in that list fell through to skipped-unsupported. Adding a weight to the JSON had no effect unless NON_CODE_EXTENSIONS was also updated — a silent maintenance trap.

Fix:

  • Add TokenConfig.is_line_count_extension() — returns True when the extension is in NON_CODE_EXTENSIONS or is configured in programming_languages.json with language=None.
  • Replace the ext in NON_CODE_EXTENSIONS gate with weights.is_line_count_extension(ext).
  • Remove the now-unused NON_CODE_EXTENSIONS import from tree_sitter_scoring.py.

This is the null-language complement of #986, which fixed extensionless configured files that do have a tree-sitter language.

Related Issues

Closes #1274

Type of Change

  • Bug fix

Testing

  • Tests added/updated

TestNullLanguageLineCountScoring in tests/validator/test_token_scoring_integration.py:

  • test_graphql_scores_line_count — primary reproduction case; asserts scoring_method='line-count' and score > 0
  • test_gitignore_scores_line_count — dotfile extension path
  • test_unknown_extension_stays_skipped_unsupported — negative control; confirms fix is bounded to configured extensions
  • test_graphql_score_uses_configured_weight — pins exact value: graphql weight 1.0 × 1 line = 1.0
  • test_all_configured_null_language_extensions_are_line_count_reachable — config-coverage guard; iterates every null-language entry in the JSON and fails if any future addition silently produces skipped-unsupported

Checklist

  • Code follows project style guidelines
  • Self-review completed
  • Changes are documented (if applicable)

…skipping them

Extensions in programming_languages.json with no language field (no
tree-sitter parser) were silently scored as skipped-unsupported because
the routing gate in calculate_token_score_from_file_changes only checked
the NON_CODE_EXTENSIONS constant, not the JSON config.

Affected extensions: env, gitattributes, gitignore, gql, graphql, ipynb,
move, nim, pde, puml, tex.

Fix:
- Add TokenConfig.is_line_count_extension() that returns True when an
  extension is in NON_CODE_EXTENSIONS OR is configured in
  programming_languages.json with language=None.
- Replace the ext in NON_CODE_EXTENSIONS gate with
  weights.is_line_count_extension(ext).
- Remove the now-unused NON_CODE_EXTENSIONS import from tree_sitter_scoring.

Tests:
- Add TestNullLanguageLineCountScoring with regressions for graphql and
  gitignore (positive), unknown extension (negative control), graphql
  weight-accuracy pin, and a config-coverage guard that iterates every
  null-language entry in programming_languages.json and asserts it reaches
  line-count — catching future config/code drift before it zeroes miner scores.
@xiao-xiao-mao xiao-xiao-mao Bot added the bug Something isn't working label May 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] configured null-language extensions score 0 (skipped-unsupported) instead of line-count

1 participant