efritz's comments

> On the face of it, it seems to me like tree-sitter could server as a perfectly fine building block for generating LSIF or whatever from a code file.

It does seem this way. Another reply [1] this post makes the same point with a nice proof-of-concept as well.

[1]: https://news.ycombinator.com/item?id=26230900

I'm an engineer on the code intelligence team at Sourcegraph.

We've been busy building out true precise code intelligence/navigation support, but we also have a mode for zero-configuration code navigation based on text search, universal-ctags, and hand-rolled regular expressions (which works surprisingly well!). Tree-sitter would definitely give better results than our current ctags-based approach. It's been catching our attention more and more lately, and we have plans to use it to upgrade our out-of-the-box, instant code navigation experience.

It's not the exact right fit for our primary goals though, since it's designed around being extremely fast while editing and robust against errors. Sourcegraph is only used for navigating committed code, so we're leveraging formats like LSIF to generate complete semantic graphs of codebases and their entire dependency tree. That'll enable a lot of features that are out of reach for tree-sitter, but is a lot harder to get working out of the box and it's a much bigger technical investment.

It's very interesting to see the topological space that houses these solutions fill out. Every tool has its own set of unique trade-offs and fall somewhere on these spectrums:

- fast vs slow

- precise vs imprecise

- zero-configuration vs configuration required

We've visited a few islands in this space but still very curious to see what other islands can be discovered. We're especially excited about tools and formats like tree-sitter and LSIF around which a large and supportive community can grow so that all the products we love and rely on as developers can all make forward progress.

I'm the author. Sorry for lack of context for people not familiar with Sourcegraph.

This blog post outlines changes to the backend services that enable precise code intelligence over the last year. Code intelligence in this context includes things like hover text, definitions, and references for a position in a source file, compiler/linting diagnostics for a file or directory, etc. To see it in action, try hovering over identifiers in this repository: https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-...

Sourcegraph has search-based code intelligence, which works with over 30 languages without configuration, but is heuristic and not always accurate (for example, jump-to-definition may jump to a definition with the same name, but not the correct one). We had support for running language servers to provide _precise_ code intelligence, but we moved away from that strategy due to speed and resource concerns. Since then we've moved to pre-indexing source code to provide precise code intelligence, which is where we're at today (and what the blog post is about). The original announcement of these features can be found at https://about.sourcegraph.com/blog/code-intelligence-with-ls....