Steve Yegge's recent blog posts claim that SourceGraph are getting a pretty good result by using embeddings created from their knowledge graph of the code structure. That's still the usual [create embeddings, search against embedding of query, retrieve results and use them as prompt] schlep, so yeah, it isn't really understanding architecture well yet.
I too have a job where almost every question is about structural understanding and improvement of a large existing codebase. I'd love to have AI help, but I think it's going to take another iteration or three of model architecture to get there.
I also want this and was excited by that blog post too
So far the SourceGraph product ("Cody") is rather underwhelming, it doesn't seem to make deep use of the project context, where the blog post seemed to say that SourceGraph's special sauce would make that possible
The results seem quite similar with Copilot Chat - in both cases they seem to basically stuff the currently focused file as context to your prompt, and the results are no better than if you did the same with ChatGPT, and looks worse because it's cramped into a VS Code sidebar.
Hey, I'm the Sourcegraph CTO. Appreciate the critical feedback here. I suspect that Cody is using "keyword context", which is our fallback when we don't have an index on Sourcegraph that Cody can use (we need to do a better job of conveying when this happens to users). Would you mind sending me a screenshot of Cody not doing a good job of answering a question / fetching the wrong context? You can email me at beyang@sourcegraph.com or DM me on Twitter (https://twitter.com/beyang).
I was hoping it would just tap into VS Code's knowledge of my project structure and index what it needed to automatically
From what I saw on the Discord after getting my invite, people were making requests in the chat for specific github repos to get indexed... is that how it works? So for projects with dependencies which have been indexed I might get better results? And I need to get my own project indexed too?
I too have a job where almost every question is about structural understanding and improvement of a large existing codebase. I'd love to have AI help, but I think it's going to take another iteration or three of model architecture to get there.