codeintel: faster symbol search via LSIF

[chrismwendt]

The biggest complaint related to symbols today is that the search-based hover popover is slow to appear on big repos. This is slow because we have to re-process each commit separately, and big repos tend to have high commit volume, which means caching is not very effective.

Let's autoindex document symbols using existing LSIF indexers or a new lsif-tree-sitter so that we get symbols data pushed through diffs for recent commits (what most people a viewing).

Tasks:

• Add documentSymbolResult to lsif-* (example implementation Spike adding deprecations to document symbols lsif-go#141)
• Write a new lsif-tree-sitter (or lsif-ctags) which emits only symbols
• Autoindex all repos that don't have symbol data with lsif-tree-sitter
• Add a new GraphQL endpoint under lsif for symbols
• Prefer symbol search via LSIF if available rather than ctags+SQLite

Tangentially related RFC Improving symbols search

[github-actions]

Heads up @macraig - the "team/code-intelligence" label was applied to this issue.

[scalabilitysolved]

Prefer symbol search via LSIF if available rather than ctags+SQLite

Don't we also store symbols produced from ctags in Zoekt in some cases @chrismwendt ?

[chrismwendt]

Prefer symbol search via LSIF if available rather than ctags+SQLite

Don't we also store symbols produced from ctags in Zoekt in some cases @chrismwendt ?

Only the default branch and possibly a few others if configured have symbols stored in Zoekt. The main use case to cover here is when viewing PRs, and you can't configure those branch names in Zoekt ahead of time because you don't know what they will be.

[chrismwendt]

I'm focusing attention on a quicker/easier solution first: https://github.com/sourcegraph/sourcegraph/issues/27611

[varungandhi-src]

Closing as duplicate in favor of https://github.com/sourcegraph/sourcegraph/issues/58727