Show HN: Python Tools for Visual Studio(pytools.codeplex.com)
pytools.codeplex.com
Show HN: Python Tools for Visual Studio
http://pytools.codeplex.com
14 comments
I'd like to check it out, but was not able find anything on the Codeplex site about this - does it work with the Express edition of Visual Studio?
Express is not extensible, so no. This is a business decision not a technical issue. I personally disagree with it, but then again I don't make these decisions.
It works with the free VS-Shell runtimes, so in theory it should work with Express editions, but YMMV.
toyg is correct. You can use PTVS w the the free "Integrated Shell" which gives you "Python Express". Please see the installation page on http://pytools.codeplex.com. thx.
Thank you for the info -- I'll give it a whirl.
[deleted]
This looks fantastic. Do you know if there's any kind of similar implementation for Ruby? Would it be terribly complicated to do something similar for Ruby or a different dynamic language?
I attended a Microsoft talk given 4-6 years ago where they were talking about Iron Ruby development. A quick search shows that they have Visual Studio tools available: http://www.ironruby.net/tools/
I love PTVS, but I keep running into a problem when I debug my code. One time it'll properly highlight in my IDE where an exception (or breakpoint) occurred; the next time it won't, and I have to look in the black Python console window and inspect the stack trace . Using PTVS 1.1.50530.0 with VS2010 Ultimate.
It looks freaking awesome. I will definitely try it out! Thanks!
One of the few times I've felt jealous as a Linux user :)
Really cool, I love the
Thank you, it's pretty inspirational in regards to how programming in Python could look like
Really cool, I love the
a={2:42}
a.values()[0]. <int completion>
Thank you, it's pretty inspirational in regards to how programming in Python could look like
Roughly the same example (type inference from literals and type tracking through containers) in KDevelop, KDE's IDE, obviously available for Linux: http://simplest-image-hosting.net/png-0-i12983
KDevelop's Python plugin is currently in early beta stage, but already offers one of the most sophisticated implementations of interactive static analysis for Python currently available. Meanwhile, if you like C++, it's already rock solid and second to none :). It's noteworthy that two languages that different can be catered to with best-in-class semantic support by the same underlying language plugin framework.
In regards to how this is done, KDevelop works purely statically. One of the smartest Python IDEs on the market, Wing IDE (also available for Linux) implements a hybrid approach of static analysis and gathering additional information by instrumenting programs at runtime. However, KDevelop's static analysis figures out a lot more than Wing's does.
ETA: Since Ruby support was asked about elsewhere in the thread, I thought I'd add that a Ruby support in KDevelop is also in the works in earnest as of this summer, but currently not yet as far along as Python support. The other production-ready language plugin next to C++ (which is KDE's bread and butter as a community) is the PHP one, which supports everything up to PHP 5.4 (i.e. including namespaces).
KDevelop's Python plugin is currently in early beta stage, but already offers one of the most sophisticated implementations of interactive static analysis for Python currently available. Meanwhile, if you like C++, it's already rock solid and second to none :). It's noteworthy that two languages that different can be catered to with best-in-class semantic support by the same underlying language plugin framework.
In regards to how this is done, KDevelop works purely statically. One of the smartest Python IDEs on the market, Wing IDE (also available for Linux) implements a hybrid approach of static analysis and gathering additional information by instrumenting programs at runtime. However, KDevelop's static analysis figures out a lot more than Wing's does.
ETA: Since Ruby support was asked about elsewhere in the thread, I thought I'd add that a Ruby support in KDevelop is also in the works in earnest as of this summer, but currently not yet as far along as Python support. The other production-ready language plugin next to C++ (which is KDE's bread and butter as a community) is the PHP one, which supports everything up to PHP 5.4 (i.e. including namespaces).
We don't actually execute any code to figure this out, it's all based upon analysis of the code.
Basically how it works is that we do an abstract interpretation of the code. To take this specific example first we analyze the a = {2:42} line, and the dictionary literal produces a new unique value in the system, and obviously we know it's key/value types. From there we'll analyze the "a.values" which will produce a new value in the system for the bound "values" method. Then we do an abstract call on that, and it says that it returns a list (and goes off to the original dictionary value, and produces a list which contains the int objects). And then we do the indexing, and finally we get back the original 42 value that the dictionary was created with. So basically at each step along the way we are just propagating sets of abstract values. A naive implementation of a system like this would iterate until it hits a fixed point - we're a little more sophisticated in that we do a bunch of dependency tracking to know what needs to be analyzed based upon changes in the system. And then we also need to deal with a user editing the program in real time which makes things even more complicated as we need to discard old stale information which has been propagated from edited files. The actual implementation of the basic Python semantics is actually the easy part, it's getting this to perform on large programs and working correctly with live editing which is the difficult part.
This can go wrong in a couple of ways. First off we don't model control flow, so you could have something like:
if False: a = 42 else: a = 'bar'
And we'd think that a was an int or a bar. In the future we might model control flow, but it's a big feature. Mainly I just need an excuse to implement it :). Another way it could go wrong is if the user is using exec/eval which we don't have any insight into. And finally there can be things that we simply don't understand. So while we know about the primitive types and the methods on them, we don't model all of Python's built-in types. So for example if you're using deque in collections you won't get the same results.
All of this is open source (under the Apache license) and the parser and analysis engine are entirely stand alone components, so they could even be re-used by other IDEs. You can check out the source code over here: http://pytools.codeplex.com/SourceControl/changeset/view/3b8...
Basically how it works is that we do an abstract interpretation of the code. To take this specific example first we analyze the a = {2:42} line, and the dictionary literal produces a new unique value in the system, and obviously we know it's key/value types. From there we'll analyze the "a.values" which will produce a new value in the system for the bound "values" method. Then we do an abstract call on that, and it says that it returns a list (and goes off to the original dictionary value, and produces a list which contains the int objects). And then we do the indexing, and finally we get back the original 42 value that the dictionary was created with. So basically at each step along the way we are just propagating sets of abstract values. A naive implementation of a system like this would iterate until it hits a fixed point - we're a little more sophisticated in that we do a bunch of dependency tracking to know what needs to be analyzed based upon changes in the system. And then we also need to deal with a user editing the program in real time which makes things even more complicated as we need to discard old stale information which has been propagated from edited files. The actual implementation of the basic Python semantics is actually the easy part, it's getting this to perform on large programs and working correctly with live editing which is the difficult part.
This can go wrong in a couple of ways. First off we don't model control flow, so you could have something like:
if False: a = 42 else: a = 'bar'
And we'd think that a was an int or a bar. In the future we might model control flow, but it's a big feature. Mainly I just need an excuse to implement it :). Another way it could go wrong is if the user is using exec/eval which we don't have any insight into. And finally there can be things that we simply don't understand. So while we know about the primitive types and the methods on them, we don't model all of Python's built-in types. So for example if you're using deque in collections you won't get the same results.
All of this is open source (under the Apache license) and the parser and analysis engine are entirely stand alone components, so they could even be re-used by other IDEs. You can check out the source code over here: http://pytools.codeplex.com/SourceControl/changeset/view/3b8...
That's a nice example and good explanation. We're about in the same spot over at KDevelop and call a an "unsure (int, string)" there: http://simplest-image-hosting.net/png-0-n12983
Things like incremental reparsing on edits are handled by KDevelop's DUChain framework, which is shared among the language plugins. The Python language plugin actually reuses the original CPython parser to do the job, and then plugs CPython's AST into KDevelop's DUChain representation.
And just to make me feel less like a thread hijacker, let me add that we've been quite impressed by the work the PTVS team has been doing. You're doing great things for the Python community over there :).
Things like incremental reparsing on edits are handled by KDevelop's DUChain framework, which is shared among the language plugins. The Python language plugin actually reuses the original CPython parser to do the job, and then plugs CPython's AST into KDevelop's DUChain representation.
And just to make me feel less like a thread hijacker, let me add that we've been quite impressed by the work the PTVS team has been doing. You're doing great things for the Python community over there :).
Thanks, you're making me want to check out KDevelop to compare now. So far I thought we had the best Python intellisense hands down, but it sounds like you're a pretty good competitor :)
PTVS in VS2012 paired up with ViEmu is probably my favorite setup, although I still use plain-old-vim fairly often.
I've spent a lot of time in Visual Studio over the years and have come to really like it when I'm doing C/C++/C#, so when this first came out I jumped all over it. The recently added feature of creating a project from existing code is pretty sharp, and removes the need for a third-party "explorer" plugin.
I've spent a lot of time in Visual Studio over the years and have come to really like it when I'm doing C/C++/C#, so when this first came out I jumped all over it. The recently added feature of creating a project from existing code is pretty sharp, and removes the need for a third-party "explorer" plugin.
"Send to Interactive"[0]
In general, is there a particular reason we don't see this more often in tooling for other (REPL-friendly) languages, outside of the Lisps? Admittedly I haven't spent a significant amount of time in the Python community lately (so not sure if this is common in Python tooling or not), but after becoming addicted to SLIME via Clojure, I'd have killed to be able to dump Ruby blocks from my editor straight to the REPL.
[0] http://pytools.codeplex.com/wikipage?title=Features%20Intera...
In general, is there a particular reason we don't see this more often in tooling for other (REPL-friendly) languages, outside of the Lisps? Admittedly I haven't spent a significant amount of time in the Python community lately (so not sure if this is common in Python tooling or not), but after becoming addicted to SLIME via Clojure, I'd have killed to be able to dump Ruby blocks from my editor straight to the REPL.
[0] http://pytools.codeplex.com/wikipage?title=Features%20Intera...
I'm currently struggling to find a satisfactory IDE for python for this exact reason. This seemingly obvious feature has to be hacked on to just about every popular coding tool. The best things I've found so far are the ipython web notebook (the only problem being that it stores my code as JSON and has poor/no keybindings) and emacs python-mode (which is buggy as all hell).
NB: The only tool I've ever found that works in the manner I want it to as far as REPL languages go is RStudio. If I had an RStudio that was language agnostic so it could work with python, ruby, lisps, or what have you, I would be a happy, happy man.
edit: if anyone has any suggestions in this regard, please, leave a comment!
NB: The only tool I've ever found that works in the manner I want it to as far as REPL languages go is RStudio. If I had an RStudio that was language agnostic so it could work with python, ruby, lisps, or what have you, I would be a happy, happy man.
edit: if anyone has any suggestions in this regard, please, leave a comment!
I use WingIDE's[1] professional version (which includes the editor source code) and it has the ability to highlight a section and evaluate it either in the REPL or at the current breakpoint in a debug probe.
It also has decent vi and emacs keybinding support, really nice interactive debugger (including template debugging for Django) and great intellisense. It's the best Python IDE I have found, the support is great and I have been a very happy customer for three years now. (I was using emacs for everything before).
[1]http://wingware.com/
It also has decent vi and emacs keybinding support, really nice interactive debugger (including template debugging for Django) and great intellisense. It's the best Python IDE I have found, the support is great and I have been a very happy customer for three years now. (I was using emacs for everything before).
[1]http://wingware.com/
Do you know if it supports ipython? I'm on the scientific computing side of the python world, so django is less important to me . . .
It does not unfortunately. You can do interactive matplotlib plotting but it doesn't behave in the same "shell + REPL" way that ipython does.
I hope the dev team at Jetbrains are looking at these features for future integration into PyCharm :)
Good job PTVS team :) You're making me envious!
Good job PTVS team :) You're making me envious!
Thanks! PyCharm, Wing, KDevelop, IPython, Komodo, ... are all pretty darn good, each with its own strengths. It's a good time to be a Python developer!
Why is the photo hosted at imgur ? http://i.imgur.com/tyvjN.png
We want the link to be clickable so it opens full screen in your browser. Doing that from CodePlex is impossible AFAIK. For example if you copy an image location for some of our other images you get an url like: http://download-codeplex.sec.s-msft.com/Download?ProjectName.... If you put that in as a link, or just try and open it from your browser window, you get a download prompt instead of seeing the image. My guess this is because the content type on the downloaded file is not being set to be an image, so imgur lets us work around that.
Out of interest, why does hosting it at imgur matter?
You know, I don't have a good answer... I was going to host it on another page on codeplex, but my teammate insisted on something w a direct link & no misc. stuff around it. I'll use your comment & tell him "Seeee!" :).
I would have said "imgur is great" except it deletes images. Imgur is simple for the consumer, and you don't pay the bills for bandwidth usage but http://imgur.com/faq#long says if it doesn't get at least 1 view in 6 months, it can deleted.
http://meta.askubuntu.com/questions/1523/imgur-com-is-deleti...
http://meta.askubuntu.com/questions/1523/imgur-com-is-deleti...
StackOverflow also hosts images on imgur. Is using something that happens to be used by Reddit inherently wrong, or did I miss your point?
Given the very sad state of the extensibility of Visual Studio (I wrote about it earlier this year http://evain.net/blog/articles/2012/02/27/visual-studio-your...), it turns out that not only are the Python Tools an excellent plugin to write Python code in VS, it's also one of the best example of how to implement support for a new language or a debugger in Visual Studio >= 2010.
Some things we do in my product would not have been possible without having access to the source of the Python Tools (which again say a lot about the sad story of extending VS).
Shameless plug, if that's still too much work for you, you can contact my company to integrate your language in VS.
Some things we do in my product would not have been possible without having access to the source of the Python Tools (which again say a lot about the sad story of extending VS).
Shameless plug, if that's still too much work for you, you can contact my company to integrate your language in VS.
jb - Thanks! Glad that PTVS has been useful in that regard. & yes, VS Extensibility can be a PITA compared to Eclipse which was built from ground up with that in mind. They're working on improving it.
Really really impressive! Django development on windows is a PITA (compared to linux/mac), and all other (free) IDEs that I've taken a look at haven't really given me too much. I'll definitely give this a go for a test application to see what it does for me.
Thanks! Would love to get your feedback. Note that you can use PTVS for Django development/debugging & then push to a cloud of your choice (ie, not Azure) as well. See this link for some info: https://www.windowsazure.com/en-us/develop/python/tutorials/...
What are the clouds of your choice other than Azure? I don't see them mentioned in the tutorial. I'm on Windows but my Python projects are on a Ubuntu VM and I push to AWS, Racksace Coud, dedicated servers, etc. My project can also interact with the shell, phantomjs, node js, Redis, etc. How can I benefit from the VS IDE for my python projects? I've seen mentions of Linux clients but not sure what they mean. thanks. I'm dearly looking for a good IDE with debugging support.
Sorry i should have been more clear. I meant that you can take advantage of PTVS's editor/intellisense/template debugging facilities for Django, but then take further steps to push to another cloud yourself. Some folks have used PTVS this way for their MPI debugging w Linux for example. Also, Amazon has a good add-in for VS to interface with their cloud which maybe of use in your scenario: http://aws.amazon.com/visualstudio/
Forgot to answer part of your question - the Linux/MacOS Client Libraries are a set of Python bindings for the services that Azure provides. They're hosted on Github here: https://github.com/WindowsAzure/azure-sdk-for-python ; and you can see a video overview here: http://www.youtube.com/watch?v=nDvqNP_Ja5s&hd=1
Thanks for the answers! Very helpful.
Looks great! I don't use Python, but I do use Visual Studio every day and this looks like a really complete implementation.
Did you find that Visual Studio easy to integrate with, or did you have to fight it to do what you wanted?
Did you find that Visual Studio easy to integrate with, or did you have to fight it to do what you wanted?
Integrating with VS is definitely a non-trivial task. But I work at Microsoft and have access to all of the VS source code, so that's a big help. And when that fails I can usually track down someone who owns that feature. But usually it's only an issue when going off an implementing some brand new integration point for the 1st time, most of the time is spent working on our own bits.
As Dino said it's not trivial, but one of the reasons we fought hard for open sourcing it was to make PTVS a Real World sample in case someone else wants to do a plug-in for their language. We know of at least two major companies that are checking it out for that purpose.
Earlier releases emphasized NumPy/SciPy for IronPython. Is that effort dead? There were major holes, like no support for matplotlib, and I had kind of hoped that these would be filled over time, but now it looks as if the whole thing is moving into other directions?
David, our group funded that effort (development by Enthought) and the code is still out there and mostly functional, but it really never got traction. The 1st phase involved major surgery on the NumPy/SciPy libraries which they needed anyway, so some good came out of it. We'd love to see some kind volunteers finish the remaining "10" percent. It would be very useful to other .Net users as well... yeah, I shed a tear :(.
Looks pretty neat!
Does it integrate with virtualenv/pip?
Does it integrate with virtualenv/pip?
There's nothing built-in yet, but we do have a feature tracking it that I'd encourage you to vote for: http://pytools.codeplex.com/workitem/691
The priority of this has gone up for us after PEP 405 but we weren't able to get to it for 1.5. You can go to Tools->Options->Python Tools->Interpreter Options and setup a custom interpreter which points to your virtualenv, but we need to make it easier than doing that.
The priority of this has gone up for us after PEP 405 but we weren't able to get to it for 1.5. You can go to Tools->Options->Python Tools->Interpreter Options and setup a custom interpreter which points to your virtualenv, but we need to make it easier than doing that.
That kind of reminds me of the IPython notebook. I really like the variable watching too. Are there any issues using virtualenv with it?
Yup! We're big fans of IPython. So much so that we collaborated with them on two front2: IPython on Azure (w a Linux or Windows backend): https://www.windowsazure.com/en-us/develop/python/tutorials/... and directly incorporating its REPL into PTVS (right window): http://i.imgur.com/tyvjN.png
With the IPython notebook on Azure you can basically use Linux/Mac with Chrome, FF, Safari, ... and have a nice browser based Python IDE and completely cut Windows & IE out of the picture :).
With the IPython notebook on Azure you can basically use Linux/Mac with Chrome, FF, Safari, ... and have a nice browser based Python IDE and completely cut Windows & IE out of the picture :).
Cool! Pardon my ignorance, but can this be used as a decent Python IDE? Or does it have a more specific/narrow use case?
It really depends on your definition of decent :). It's built on on VS, so if you're looking for something extremely lightweight (eg editor++) this isn't it. OTOH, it has many advanced features such as really good intellisense, MPI debugging, inline graphics, integrated IPython, Profiling, etc etc. I'd take a look at the videos on the page & decide for yourself. This should give u a basic idea: http://www.youtube.com/watch?v=7CoGsSlrxKk&hd=1 ; more videos on the project home page at http://pytools.codeplex.com
edit: missing link.
edit: missing link.
Thanks that was great. Looks very promising. Will try it out.
I think it can. This is the python IDE I'm waiting for. Can't wait to find excuses to use this on real project :D
Disclaimer: PTVS is built by few Python/OSS enthusiasts at msft & I'll be around in case people have any questions!
Cheers.