GitHub Copilot Workspace: Technical Preview(github.blog)
github.blog
GitHub Copilot Workspace: Technical Preview
https://github.blog/2024-04-29-github-copilot-workspace/
323 コメント
While I've not used this product, I've created somewhat similar setup using open source LLMs that runs locally. After having used it for about three months, I can say that debugging LLM prompts was far more annoying than debugging code. Ultimately, I ended up abandoning my setup and going in favor of writing code the good old fashioned way. YMMV
Student here: I legitimately cannot understand how senior developers can dismiss these LLM tools when they've gone from barely stringing together a TODO app to structuring and executing large-scale changes in entire repositories in 3 years. I'm not a singulatarian, but this seems like a brutal S-curve we're heading into. I also have a hard time believing that there is enough software need to make such an extreme productivity multiplier not be catastrophic to labor demand.
Are there any arguments that could seriously motivate me to continue with this career outside of just blind hope that it will be okay? I'm not a total doomer, currently 'hopium' works and I'm making progress, but I wish my hopes could at least be founded.
Are there any arguments that could seriously motivate me to continue with this career outside of just blind hope that it will be okay? I'm not a total doomer, currently 'hopium' works and I'm making progress, but I wish my hopes could at least be founded.
I think Microsoft is going the wrong direction with Copilot (though it's a reasonable direction given their incentives).
Right now Copilot is terrible at large changes to complex codebases; the larger and more complex, the worse. But it's great at suggesting very short snippets that guess exactly what you were in the middle of writing and write it for you.
I wish Copilot focused more on the user experience at the small scale: faster and smaller completions, using the semantic info provided by the IDE, new affordances besides "press tab to complete" (I'd love a way for Copilot to tell me "your cursor should jump to this line next"), etc. This whole focus on making the AI do the entire end-to-end process for you seems like a dead end to me.
Right now Copilot is terrible at large changes to complex codebases; the larger and more complex, the worse. But it's great at suggesting very short snippets that guess exactly what you were in the middle of writing and write it for you.
I wish Copilot focused more on the user experience at the small scale: faster and smaller completions, using the semantic info provided by the IDE, new affordances besides "press tab to complete" (I'd love a way for Copilot to tell me "your cursor should jump to this line next"), etc. This whole focus on making the AI do the entire end-to-end process for you seems like a dead end to me.
This should've been done in VSCode. Most developers don't want to write code in a browser. Once VSCode extension is built they could generalize it to other editors.
I'm sure they are thinking of VSCode integration and I am hopping for that to be available soon
I'm sure they are thinking of VSCode integration and I am hopping for that to be available soon
Interesting. I don't think the AI code generation will live up to developer expectations, but I do see the value in "project management for developers". The value I see with workspace is not the code generation, but the ability to help developers organize their thoughts.
Watching the examples and other people's demos, I get the sense that this product completely ignores what makes good software. It has no "big picture" plan or contextual awareness required to make those nuanced yet critical decisions in the far ends of applications. Those cases where you badly need domain expertise, understanding of how the user interfaces with the product, awareness of existing technical debt, and so on.
The scope it can assist in is too limited. It is exactly the kind of thing I'd use to generate self-destructing spaghetti. The thing is, people with less experience will eat this up because it can do some tasks with ease which might not be familiar to them yet. Over time, the code they produce will be a mine field of incongruent patterns and solutions.
It appears to support the hypothesis that these tools may actually create more work, but I worry that the work will be much like that created by outsourced developers shipping duct-taped nonsense back to North America throughout the last 20 years or so.
The scope it can assist in is too limited. It is exactly the kind of thing I'd use to generate self-destructing spaghetti. The thing is, people with less experience will eat this up because it can do some tasks with ease which might not be familiar to them yet. Over time, the code they produce will be a mine field of incongruent patterns and solutions.
It appears to support the hypothesis that these tools may actually create more work, but I worry that the work will be much like that created by outsourced developers shipping duct-taped nonsense back to North America throughout the last 20 years or so.
I'm not sure why they insist on code spaces instead of running this inside vs code?
Setups/working looks pretty similar to aider [1] that I've been using and liking, Aider is smaller steps than plandex, but plandex went into some kind of loops a couple of times so I stoped using it for now.
[1] https://github.com/paul-gauthier/aider
Setups/working looks pretty similar to aider [1] that I've been using and liking, Aider is smaller steps than plandex, but plandex went into some kind of loops a couple of times so I stoped using it for now.
[1] https://github.com/paul-gauthier/aider
For anyone who might be interested in an open source, terminal-based approach to using AI on larger tasks and real-world projects, I'm building Plandex: https://github.com/plandex-ai/plandex
I've tried to create a very tight feedback loop between the developer and the LLM. I wanted something that feels similar to git.
Apart from the planning and code generation itself, Plandex is also strongly focused on version control. Every interaction with the model is versioned so that it's easy to try out different strategies and backtrack/revise when needed.
I just released a new update (literally a few minutes ago) that includes some major improvements to reliability, as well as support for all kinds of models (previously it had been OpenAI-only). It's been fun trying out Claude Opus and Mixtral especially. I'd love to hear people's thoughts!
I've tried to create a very tight feedback loop between the developer and the LLM. I wanted something that feels similar to git.
Apart from the planning and code generation itself, Plandex is also strongly focused on version control. Every interaction with the model is versioned so that it's easy to try out different strategies and backtrack/revise when needed.
I just released a new update (literally a few minutes ago) that includes some major improvements to reliability, as well as support for all kinds of models (previously it had been OpenAI-only). It's been fun trying out Claude Opus and Mixtral especially. I'd love to hear people's thoughts!
I wish there were fixes to devcontainers before doing adding copilot. I really want declarative, repeatable builds that are easily used in both Codespaces AND Actions. I know all of the functionality is theoretically there in devcontainers.json, it is so manual to configure and confusing, that anytime I've done it, I use it for 2 weeks and then just go back to developing on local because I don't have time to keep that up. ESPECIALLY if you're deploying to AWS cloud and also, want to use alternative package managers like poetry, uv, yarn, jsr, etc.
If folks want to see what Copilot Workspace looks like, here’s a sample session where I addressed a feature request in an OSS VS Code extension I maintain: https://copilot-workspace.githubnext.com/lostintangent/gitdo....
You can see the originating issue and the resulting PR from there. And note that while the initial spec/plan/code was mostly good, I iterated on a couple parts of the plan, and then made a minor tweak to the code manually (everything in CW is editable). Which is a key part of our goal with CW: to help bootstrap you with a task (or think out loud with AI), and then provide the iteration primitives to explore further.
You can see the originating issue and the resulting PR from there. And note that while the initial spec/plan/code was mostly good, I iterated on a couple parts of the plan, and then made a minor tweak to the code manually (everything in CW is editable). Which is a key part of our goal with CW: to help bootstrap you with a task (or think out loud with AI), and then provide the iteration primitives to explore further.
Hi All! GitHub Next here, happy to answer questions about Copilot Workspace and Next in general <3
I don't use Copilot in particular but in the software space, everything is constantly changing, and the models are trained infrequently so they are never familiar with the things I ask about. Reality is outrunning the models for now but that may change.
I just wrote up my experiences of using it for a week: https://austinhenley.com/blog/copilotworkspace.html
TL;DR: Copilot Workspace is a great concept. But the UX and framing is entirely wrong and sets the wrong expectation for users. The current prototype is very slow (5+ minutes for one-line code changes) and the generated code is often buggy or has nothing to do with the specification. It doesn’t help me understand the code. The generated plan is usually good. ChatGPT does a much better job in my head-to-head comparisons (assuming I already know exactly what code is relevant). I'm still optimistic of where it can go from here.
I recommend everyone sign up to try it and give the team feedback.
TL;DR: Copilot Workspace is a great concept. But the UX and framing is entirely wrong and sets the wrong expectation for users. The current prototype is very slow (5+ minutes for one-line code changes) and the generated code is often buggy or has nothing to do with the specification. It doesn’t help me understand the code. The generated plan is usually good. ChatGPT does a much better job in my head-to-head comparisons (assuming I already know exactly what code is relevant). I'm still optimistic of where it can go from here.
I recommend everyone sign up to try it and give the team feedback.
> and is expressly designed to deliver–not replace–developer creativity
I'm not fooled by such corp speak.
I'm not fooled by such corp speak.
I think these types of tools will become really effective when they stop trying to target every single programming language and platform ever created and focus on one specific version of an integrated platform.
Most businesses don't actually need software to be in whatever flavor of the month the dev team happens to be interested in. They just want to be able customize their software.
By restricting the scope somewhat, you make it much more feasible to make sure the model has the training and/or knowledge ready for retrieval to fulfill tasks using a specific stack.
So I see this type of thing as quickly evolving into a tool for non-developers. And within a few years, these tools will cut into software engineering jobs. It will become part of the evolution of no-code and low-code.
Most businesses don't actually need software to be in whatever flavor of the month the dev team happens to be interested in. They just want to be able customize their software.
By restricting the scope somewhat, you make it much more feasible to make sure the model has the training and/or knowledge ready for retrieval to fulfill tasks using a specific stack.
So I see this type of thing as quickly evolving into a tool for non-developers. And within a few years, these tools will cut into software engineering jobs. It will become part of the evolution of no-code and low-code.
This is quite similar to my idea and prototype I applied to YC with just last week. Uh-oh! Haha. I did call out GitHub as a primary competitor so I did see this coming - and I guess on second looks I'm still quite differentiated. All these new tools will overlap some but also have their own pros/cons since there are so many different ways to approach building an AI code tool!
Just yesterday I put up a cheap $5 linode running a prototype but not ready for signups yet.
https://codeplusequalsai.com/
I think I have some good ideas about how to get the the LLM to modify code - specifically working with ASTs.
I wonder how GitHub prompts the LLM to get usable code modifications out of it. How often does it encounter errors?
Just yesterday I put up a cheap $5 linode running a prototype but not ready for signups yet.
https://codeplusequalsai.com/
I think I have some good ideas about how to get the the LLM to modify code - specifically working with ASTs.
I wonder how GitHub prompts the LLM to get usable code modifications out of it. How often does it encounter errors?
You can see my earlier comment somewhere here, but I feel obliged to remind everyone that the claim that Github Copilot made developers "55% more productive" came from a study where they asked 100 developers to implement an HTTP server in Javascript, split the group up roughly 50/50, and gave one group Github Copilot. The Copilot group did it in an hour and 11 minutes, whereas the control group got it done 2 hours and 41 minutes.
https://github.blog/2022-09-07-research-quantifying-github-c...
That's where the 55% number is coming from. It's coming from this experiment, and only this experiment.
So yeah, if you're wondering why you aren't somehow 50% more productive when using Github Copilot, it's probably because you're not implementing a simple task that's been done to death in countless tutorials.
https://github.blog/2022-09-07-research-quantifying-github-c...
That's where the 55% number is coming from. It's coming from this experiment, and only this experiment.
So yeah, if you're wondering why you aren't somehow 50% more productive when using Github Copilot, it's probably because you're not implementing a simple task that's been done to death in countless tutorials.
I'm a fan of AI-empowered coding, but I'm not a fan of a single SaaS vendor dominating the entire software lifecycle. That's an incredibly dangerous proposition.
I'd love to see anyone but Microsoft push ahead, and ideally, lots of players.
I'd love to see anyone but Microsoft push ahead, and ideally, lots of players.
One side effect that I don't think I like of these tools is the inevitable push for developers to just use the most popular languages. Javascript and python are the main LLM trained languages, so it just self reenforces that it is best to use those for everything. There is something a bit sad about that. I guess the dream is that they are just intermediatory data anyway, like a chatty bytecode layer.
Cool idea! Agree on the end vision. The alpha is in figuring out how to get there and making it work really well along the way.
Also, why isn’t anybody connecting the Copilot Workspace announcement to Devin? Biggest company in the world announces a release of a competitor to the most widely seen dev product announcement of the year? Only saw one incidental mention of the connection.
Also, why isn’t anybody connecting the Copilot Workspace announcement to Devin? Biggest company in the world announces a release of a competitor to the most widely seen dev product announcement of the year? Only saw one incidental mention of the connection.
Curious to see how it goes.
For 20 years I have been working with Vim and recently added the Co-pilot plugin which is awesome.
I can't however see myself using a different editor/IDE than the good'ol vim.
It's not that I haven't tried, but all those IDEs tend to be bloated and consume tons of memory.
I use Copilot in VS code for work and it is great for auto complete, but the chat experience is very poor compared to tools like Cursor. The inline edits are even worse.
Long story short, is I don’t have a lot of confidence in the product right now.
That being said, I am very optimistic on the product long term and I generally like the vision.
Long story short, is I don’t have a lot of confidence in the product right now.
That being said, I am very optimistic on the product long term and I generally like the vision.
A little late to the party here, but we (https://www.bismuthos.com) offer something already to scratch this itch. We provide a workspace to build Python backends. Chat on the left, code and visual editors on the right. However, we also handle deployments, data storage (we have a blob store), serving (we built a home grown function runtime) and logging. The experience is tightly integrated with the copilot and the idea is to get ideas off the ground as quickly as possible. We're early days and you can try it for free right now.
Hello! GitHub Next here, happy to answer questions and unpack how we think about AI tools for developers (spoiler: it's less about codegen and more about helping with the rest of the dev cycle — building an understanding of how the system works, clearly specifying how it should change, etc)
> At last count, Copilot had over 1.8 million paying individual and 50,000 enterprise customers.
> Copilot loses an average of $20 a month per user, according to a Wall Street Journal report, with some customers costing GitHub as much as $80 a month.
Presumably at some point they need to actually make money on this? That is a $432 million / yr loss just on individual users.
> Copilot loses an average of $20 a month per user, according to a Wall Street Journal report, with some customers costing GitHub as much as $80 a month.
Presumably at some point they need to actually make money on this? That is a $432 million / yr loss just on individual users.
was able to get around waitlist: copilot-workspace.githubnext.com /<owner>/<repo>?task=<description>
You can try out GitHub Copilot Workspace now in read-only mode:
https://copilot-workspace.githubnext.com/AnandChowdhary/anan...
https://copilot-workspace.githubnext.com/AnandChowdhary/anan...
[deleted]
RIP Devin