Emacs setup for Python development(caisah.info)
caisah.info
Emacs setup for Python development
http://caisah.info/emacs-for-python/
55 comments
Yep, Rope in emacs is the way to go. I remember having trouble getting Pymacs set up the first time around, but if you are working on an actual python project (not just some loosely couple source files) I have not found a better environment anywhere.
On a related note, emacs support for ipython notebook with ein (http://tkf.github.io/emacs-ipython-notebook/) is marvelous. If you are comfortable with emacs, ein is far superior for interacting with your ipython notebook compared to the browser experience.
Seriously. I spend a large fraction of my day in ein. Great for exploratory data analysis.
I have found EmacsRedux informative for learning GNUemacs. The key for me was understanding that Emacs is command driven and that shortcuts are merely shortcuts to commands - i.e. M-x <command> invokes the <command> of which emacs has hundreds (+ lisp expressions). Unlike shortcuts, commands can be accessed with auto-complete.
http://emacsredux.com/
http://emacsredux.com/
The Emacs tutorial is a great place to start.
Using Emacs is a bit of a paradigm shift in text editing, so you should expect a bit more work up front for great results later on. Going through the tutorial and learning the basic vocabulary and commands takes care of most of that up-front cost and should not take long. I think it took me less than two weeks of fairly casual use.
Part of the beauty is that most of the things in the blog are not limited to Python. All the general text-editing stuff trivially transfers to other languages you may want to learn. And Emacs supports any language you do want to learn to one extent or the other. For some languages, Emacs is even the only option!
So you should consider going through at least the beginning of all this even if you're not interested in Python per se.
Also, check out some of the Emacs Rocks![1] videos. I keep on recommending them because they are awesome. They really show off why Emacs is truly magical.
[1]: http://emacsrocks.com/
Using Emacs is a bit of a paradigm shift in text editing, so you should expect a bit more work up front for great results later on. Going through the tutorial and learning the basic vocabulary and commands takes care of most of that up-front cost and should not take long. I think it took me less than two weeks of fairly casual use.
Part of the beauty is that most of the things in the blog are not limited to Python. All the general text-editing stuff trivially transfers to other languages you may want to learn. And Emacs supports any language you do want to learn to one extent or the other. For some languages, Emacs is even the only option!
So you should consider going through at least the beginning of all this even if you're not interested in Python per se.
Also, check out some of the Emacs Rocks![1] videos. I keep on recommending them because they are awesome. They really show off why Emacs is truly magical.
[1]: http://emacsrocks.com/
>> For some languages, Emacs is even the only option
Which ones would those be? Common Lisp using SLIME?
Which ones would those be? Common Lisp using SLIME?
Erlang. Even though some IDE's are starting to support it now. Erlang is the original reason I got into using Emacs actually.
Not really a programming mode.
But OrgMode is probably one of the most powerful and unique emacs modes. There is nothing like it anywhere else.
Once you go OrgMode, there is no going back.
But OrgMode is probably one of the most powerful and unique emacs modes. There is nothing like it anywhere else.
Once you go OrgMode, there is no going back.
I was mostly thinking of Agda[1] and, to a lesser extent, similar languages like Coq[2] with Proof General[3].
You could write Common Lisp in Notepad if you really wanted to. I do not think using Agda would be at all practical without Emacs.
[1]: http://wiki.portal.chalmers.se/agda/pmwiki.php
[2]: http://coq.inria.fr/
[3]: http://proofgeneral.inf.ed.ac.uk/
You could write Common Lisp in Notepad if you really wanted to. I do not think using Agda would be at all practical without Emacs.
[1]: http://wiki.portal.chalmers.se/agda/pmwiki.php
[2]: http://coq.inria.fr/
[3]: http://proofgeneral.inf.ed.ac.uk/
There's decent Common Lisp support in other editors. Lots of people use Vim with Slimv. Clojure is still supported better (especially now that we have tpope's fireplace.vim), but CL is decent to edit.
I was thinking maybe gp was referring to (at least) elisp.
Emacs Lisp
I have a similar emacs setup with cedet, autocomplete, and whatnot, but I have one problem -- things like "evaluate this buffer" don't work when your python environment is on a remote server (due to work constraints I am obliged to do this). Files open fine over tramp, but a lot of things (python-mode, jedi, etc.) don't understand that they are interacting with a remote python shell. Is there any easy way that someone here has found to get around that?
If you have a remote file open with Tramp, you can also get a shell open there with M-x shell. Try doing that, running python in that shell and renaming the shell buffer to Python. It might work :P.
If that doesn't work, you might have to find the variable which specifies which buffer has the current running Python process. You should then be able to just change that to a remote shell as above.
You can also automate all of this with a bit of elisp. After you've played around with it a bit, ask around on #emacs--I'm sure there are people there both willing and able to help you out.
If that doesn't work, you might have to find the variable which specifies which buffer has the current running Python process. You should then be able to just change that to a remote shell as above.
You can also automate all of this with a bit of elisp. After you've played around with it a bit, ask around on #emacs--I'm sure there are people there both willing and able to help you out.
Nice trick to hoodwink emacs :) It didn't work though.
I understand what you're saying in the rest of your comment though -- basically I will have to learn some elisp. Thanks for the help.
I understand what you're saying in the rest of your comment though -- basically I will have to learn some elisp. Thanks for the help.
The way I get around that is running emacs on the remote server in screen/tmux. If there's a better way I'd love to know it, but I haven't found it to be too much of a problem.
Well, you can put emacs into server mode and then connect with emacsclient, provided you tunnel X. I got this working once but I didn't like using the X Windows version emacs over the native desktop version. Sorry I don't remember the details, but I don't recall it being tricky.
My goal was to connect a local emacsclient to a remote emacs server over TCP... if this were possible, I would be fairly ecstatic.
My goal was to connect a local emacsclient to a remote emacs server over TCP... if this were possible, I would be fairly ecstatic.
I have tried this, and it works fine...almost :) The only problem I have is that doesn't give you some of the GUI goodness. For example, I find CEDET's code folding incredibly useful when I want to get a bird's eye view of my modules, and the CLI version doesn't seem to support drawing the expandoes.
Thanks for the tip though.
Thanks for the tip though.
True, but "emacs in screen/tmux" has a few other advantages for me - namely that I can leave all my buffers open when I leave the office and easily pick up from home. It's also more pleasant over a slow vpn connection.
I'm happy with using elpy for python development in Emacs. The defaults are nice. Also backends like jedi/rope are supported.
https://github.com/jorgenschaefer/elpy
The author seems to imply that sublime does not qualify as a "serious" editor, and offers a link to quora as explanation, but the article there doesn't address what Sublime is missing at all.
>doesn't address what Sublime is missing at all.
Being a fully programmable work environment.
Edit: I used ST2 for about a month. The Python was neat but it's going to fade away eventually.
Being a fully programmable work environment.
Edit: I used ST2 for about a month. The Python was neat but it's going to fade away eventually.
Vim and Emacs have had decades to get to that point. ST is a comparative baby. I'd love to see a modern editor that isn't designed around a terminal join those ranks.
Pretty hard to "get to that point" when your editor isn't programmable.
Which goes back to my original point.
Seen lots of IDEs and editors come and go. Nothing new here, just more polish than the last guy.
Which goes back to my original point.
Seen lots of IDEs and editors come and go. Nothing new here, just more polish than the last guy.
There is Acme for Plan9 [1] and it has inspired a few other editors. I don't use them myself because they don't have syntax highlighting.
[1]: demo at http://www.youtube.com/watch?v=dP1xVpMPn8M
[1]: demo at http://www.youtube.com/watch?v=dP1xVpMPn8M
That looks awesome. Having used the WMII window manager it looks quite familiar. Even then the learning curve seems steep.
What can't you do with ST2 if you write a plugin? Or is that just a relative pain compared to emacs & vim?
I know that one thing you can’t do is have custom tabstops for text. The plugin https://github.com/SublimeText/ElasticTabstops is unable to completely implement elastic tabstops, and its README says that “Limitations in Sublime Text's API make it virtually impossible to use elastic tabstops with spaces.”
Though neither Vim nor Emacs currently provide all the APIs necessary for elastic tabstops plugins, either. They only have the possible advantage that since they’re open source, such support is a little more likely to be added.
Though neither Vim nor Emacs currently provide all the APIs necessary for elastic tabstops plugins, either. They only have the possible advantage that since they’re open source, such support is a little more likely to be added.
Friction + capability + turtles all the way down.
Can you give a specific example of how being "fully programmable" improves your workflow?
When I decide something needs to be added to my workflow, it is easy to add. The difference is whether your tool drives your workflow or you workflow drives your tool.
The Java world is a great example of tool-driving-workflow. IDEs are massive, and you are generally constrained to a code-build process as thought out by the vendor. Generally speaking, the tools have good "average" efficiency. Anybody who has used the tool could sit in front of anybody else's and be as efficient.
Emacs is obviously the other end. My workflow influences the tool in so many ways that it would be nearly impossible for somebody to sit in front of my Emacs and use it efficiently. However, I am far more efficient than within any IDE, because I have bent the tool to my will.
Something like Sublime doesn't necessarily prohibit you from building a workflow and tools around it. It would just have include the command line as the programmable part.
EDIT: Specifics. A simple example: I was editing a bunch of wiki pages, do a combination of creating, formatting and moving content. I built a small set of tools to handle the repetitive parts. It is the aggregation of all the small things like this that makes the tool so much more valuable.
The Java world is a great example of tool-driving-workflow. IDEs are massive, and you are generally constrained to a code-build process as thought out by the vendor. Generally speaking, the tools have good "average" efficiency. Anybody who has used the tool could sit in front of anybody else's and be as efficient.
Emacs is obviously the other end. My workflow influences the tool in so many ways that it would be nearly impossible for somebody to sit in front of my Emacs and use it efficiently. However, I am far more efficient than within any IDE, because I have bent the tool to my will.
Something like Sublime doesn't necessarily prohibit you from building a workflow and tools around it. It would just have include the command line as the programmable part.
EDIT: Specifics. A simple example: I was editing a bunch of wiki pages, do a combination of creating, formatting and moving content. I built a small set of tools to handle the repetitive parts. It is the aggregation of all the small things like this that makes the tool so much more valuable.
I understand that customizing can improve productivity. What I haven't seen is someone name a particular task, and show how much easier it is to customize in their favorite editor vs emacs/vim/sublime/editor of the day. Sort of like a benchmarksgame.alioth.debian.org or rosettacode.org but for editors.
I'm not sure how that would work, because the very nature of the beast is exceedingly personal. As I mentioned in the example I added to my original comment, it is about small things that, cumulatively, add up to big things.
I think the kinds of things that would show up on a rosettacode-type site are most likely already exposed in terms of plugins of some sort (however the editor defines "plugin"). It is the murky area between keyboard macros and plugins that the programmable editor makes its mark, and those things are highly individualized.
It would be fun to have something like seewhatididinmyeditor.com, that just let you post snippets of these kinds of things in your editor's native tongue.
Here's another example: I was converting some javascript to coffeescript. Coffeeescript's conventions are snake_case, and I wanted to follow those conventions, so I added this little thing:
Again, not a big deal, but it made the workflow for converting those js files just that much easier.
I think the kinds of things that would show up on a rosettacode-type site are most likely already exposed in terms of plugins of some sort (however the editor defines "plugin"). It is the murky area between keyboard macros and plugins that the programmable editor makes its mark, and those things are highly individualized.
It would be fun to have something like seewhatididinmyeditor.com, that just let you post snippets of these kinds of things in your editor's native tongue.
Here's another example: I was converting some javascript to coffeescript. Coffeeescript's conventions are snake_case, and I wanted to follow those conventions, so I added this little thing:
(defun my-uncamelize (s &optional sep start)
"Convert CamelCase string S to lower case with word separator SEP.
Default for SEP is an underscore \"_\".
If third argument START is non-nil, convert words after that
index in STRING."
(let ((case-fold-search nil))
(while (string-match "[A-Z]" s (or start 1))
(setq s (replace-match (concat (or sep "_")
(downcase (match-string 0 s)))
t nil s)))
(downcase s)))
Again, not a big deal, but it made the workflow for converting those js files just that much easier.
Here’s an example of powerful customization. I recently wrote a macro in Vim that, when run in a particular Visual Basic file, would search for a certain part of the file with an SQL string, delete everything in the file but the SQL string (just temporarily), and reformat the SQL string for easy viewing. I needed a macro because that Visual Basic file was compiled from another source, so it kept getting rebuilt to the full version but with different SQL, and I had to keep on using my macro to view the new SQL.
The main reason this was easier in Vim than it would be in Sublime Text is that I could edit the macro after the fact – the macro was just text. All macros are stored in a register, and registers are basically like multiple clipboards, so to edit the macro, I just had to paste the contents, change the text, and copy it back to the same register. Because of that, I could develop the macro bit by bit. At first, I just made the macro delete the parts of the file other than the string with SQL. Then, when I realized that I was going to be viewing the SQL in this file many more times, I edited the macro’s register. I was able to develop the macro bit by bit. And I could even save the macro in a text file for me to load later. Basically, I used Vim macros as a substitute for a scripting language – Vim has convenient text-manipulation functions built-in, and text manipulation and viewing were my goals.
Another way Vim made writing the macro easier is that it has more powerful macro primitives. It is easier to, for instance, select all the text in the next quoted string on this line, because Vim’s text-manipulation primitives are more granular. I think Sublime Text can select the cursor’s current “block”, but not a double-quoted string specifically.
Here’s the final macro (^M carriage return characters were turned into actual newlines, and ^[ and ^V characters are invisible):
The main reason this was easier in Vim than it would be in Sublime Text is that I could edit the macro after the fact – the macro was just text. All macros are stored in a register, and registers are basically like multiple clipboards, so to edit the macro, I just had to paste the contents, change the text, and copy it back to the same register. Because of that, I could develop the macro bit by bit. At first, I just made the macro delete the parts of the file other than the string with SQL. Then, when I realized that I was going to be viewing the SQL in this file many more times, I edited the macro’s register. I was able to develop the macro bit by bit. And I could even save the macro in a text file for me to load later. Basically, I used Vim macros as a substitute for a scripting language – Vim has convenient text-manipulation functions built-in, and text manipulation and viewing were my goals.
Another way Vim made writing the macro easier is that it has more powerful macro primitives. It is easier to, for instance, select all the text in the next quoted string on this line, because Vim’s text-manipulation primitives are more granular. I think Sublime Text can select the cursor’s current “block”, but not a double-quoted string specifically.
Here’s the final macro (^M carriage return characters were turned into actual newlines, and ^[ and ^V characters are invisible):
/History Section goes here
f"vi":s/\( \+\)\( \+\)/\r\2/g
gv--dgg/\.replace(
hhi
--
dG:setf sql
gg/"Select
a
-V-I--
.I can give a few, but I also think it's something that needs to be experienced to really "get it".
At work I have to use a proprietary version control and build system (don't ask), and it was relatively easy to hook it into Emacs.
We also have an internal website with a cross referenced, hyperlinked version of the code. I have a function to launch a browser and go to the cross reference site for the symbol under the cursor.
One of our coding conventions is that the .h file for a class named ClassName is ClassName.h. When I use a class I haven't used before, I have a keyboard shortcut to add the #include for me. If I miss one, the compile error jumps right to the class name and I use the keyboard shortcut - it takes about 2 seconds.
I have a function that will take whatever text I've highlighted and launch a web browser doing a search in DuckDuckGo for that text.
I have a function to run the selected text through Python and replace the selection with the result. If I type 4+5 I can select it, press Ctrl + Alt + P and it gets replaced with 9.
By default Emacs opens *.h files in C mode. I have a function that checks whether there's more .cpp files or .c files in the same directory and sets C++ or C mode appropriately. I replaced the default behavior with my version, so it happens automatically.
I could go on all day.
Having a fully programmable editor means that if I ever find myself doing the same thing more than a few times, I can automate it. It means that if I don't like the way Emacs does something I can change it. Customization in most other editors is limited by the editor's plugin API. There's no such limitation in Emacs.
At work I have to use a proprietary version control and build system (don't ask), and it was relatively easy to hook it into Emacs.
We also have an internal website with a cross referenced, hyperlinked version of the code. I have a function to launch a browser and go to the cross reference site for the symbol under the cursor.
One of our coding conventions is that the .h file for a class named ClassName is ClassName.h. When I use a class I haven't used before, I have a keyboard shortcut to add the #include for me. If I miss one, the compile error jumps right to the class name and I use the keyboard shortcut - it takes about 2 seconds.
I have a function that will take whatever text I've highlighted and launch a web browser doing a search in DuckDuckGo for that text.
I have a function to run the selected text through Python and replace the selection with the result. If I type 4+5 I can select it, press Ctrl + Alt + P and it gets replaced with 9.
By default Emacs opens *.h files in C mode. I have a function that checks whether there's more .cpp files or .c files in the same directory and sets C++ or C mode appropriately. I replaced the default behavior with my version, so it happens automatically.
I could go on all day.
Having a fully programmable editor means that if I ever find myself doing the same thing more than a few times, I can automate it. It means that if I don't like the way Emacs does something I can change it. Customization in most other editors is limited by the editor's plugin API. There's no such limitation in Emacs.
In the end, personal taste is what counts. All of these you mentioned can be done in Sublime Text as well.
I used emacs for 2 years, than switched to Sublime Text 2. All of the shortcuts I use is available via emacs package, and due to it's extensive API, I can add the functionality I want, with python, my favourite language.
Emacs is much more extensible than Sublime Text for sure. On the other hand, Sublime Text's extensibility is enough for my needs.
I used emacs for 2 years, than switched to Sublime Text 2. All of the shortcuts I use is available via emacs package, and due to it's extensive API, I can add the functionality I want, with python, my favourite language.
Emacs is much more extensible than Sublime Text for sure. On the other hand, Sublime Text's extensibility is enough for my needs.
Mind sharing your config?
I'm impressed with Sublime Text after trying it out with some dedication — it has a lot to offer, and I'd recommend it over Emacs to someone just starting out, unless they think they might want to learn Lisp. But I think the author was right in saying that Emacs is more serious right now, even though I think Sublime Text will get there.
It's hip to hate Sublime because it can't do X that Emacs or Vim can.
There's a lot of criticism of ST by people who never actually used it or used it for a couple of minutes. Pathetic really.
I haven't seen a lot of hate in that direction. As someone who uses ST almost primarily, my biggest complaint is that it's not open source and free, which is going to stunt it's growth and keep it from being a real competitor with Emacs or Vim.
I use sublime for WordPress because it doesn't use PEAR coding standards and that's what my emacs phone is setup for (also you don't have to set up multiple major modes).
Everything else is done in emacs because it is the better editor. Just wish it was pretty.
Everything else is done in emacs because it is the better editor. Just wish it was pretty.
I've attempted something similar but without installing any external package and by documenting the config inline so that one could pick only what needed:
http://giuliofidente.com/2013/04/getting-to-know-and-use-ema...
In there I tried to address also other issues like saving the backup files out of the working dir, show the line numbers, highlight long lines.
http://giuliofidente.com/2013/04/getting-to-know-and-use-ema...
In there I tried to address also other issues like saving the backup files out of the working dir, show the line numbers, highlight long lines.
Can someone link a vim setup for python, for comparison puposes, perhaps with a short feature comparison? I'm currently a vim user with a fairly basic setup (I tried st 2 but couldn't figure it out for managing my work flow) , but I find the idea of emacs attractive.
Conque shell can 'run a terminal' inside of vim, but it's a bit buggy. It's not really fair to compare vim and emacs, because vim is _just_ a text editor. Vim + screen or vim + tmux (my preference) is much closer to the same ... category of thing that emacs is.
You can split tmux tabs so that you can have vim up in one pane on a 'tab' and python running in another pane in the same 'tab'. Both tmux and screen have decent copy/paste functionality that mimics vim's movement behaviour to some degree. (Tmux also has scriptable sessions, if you like different workflows for different types of projects)
If you want a nice vim setup SPF-13 is a decent place to start. There are tmux configs everywhere. Also Ipython does a lot for you as far as completion, looking at documentation, and the like.
I think that's kind of what you're after, sorry if it's not.
You can split tmux tabs so that you can have vim up in one pane on a 'tab' and python running in another pane in the same 'tab'. Both tmux and screen have decent copy/paste functionality that mimics vim's movement behaviour to some degree. (Tmux also has scriptable sessions, if you like different workflows for different types of projects)
If you want a nice vim setup SPF-13 is a decent place to start. There are tmux configs everywhere. Also Ipython does a lot for you as far as completion, looking at documentation, and the like.
I think that's kind of what you're after, sorry if it's not.
Tmuxinator especially makes doing the vim + tmux thing so much better because it has nice session support (just type "mux <name>" to bring up a session). I've been using it for about six months now, and my current setup can do anything an IDE can in a fraction of a time.
What do you use for automatic refactoring? Are there good command line tools for renaming methods across an entire source tree for example?
sed can do in place recursive changes for search patterns. if you use vim search and replace across a file it's pretty much the same.
I was thinking of something more context aware than a simple find/replace. If you're changing MyClass.getFoo() to MyClass.getBar(), you only want to do that on instances of MyClass.
If there is a plugin for intelligent refactoring in vim, I have yet to see it. You can use tabdo or buffdo for multiple files, but the class/object parsing isn't there.
You could run emacs with evil[1]. It's got a lot of the familiar keystrokes and modes. I would install it using el-get.
[1] http://emacswiki.org/emacs/Evil
[1] http://emacswiki.org/emacs/Evil
I've got a vim setup hosted on github: https://github.com/srathbun/vim-plugins
It has Klen's python-mode plugin as a submodule: https://github.com/klen/python-mode
Python-mode provides awesome sause :) To get the repo, clone it recursively, or you won't have all the submodules. `git clone --recursive https://github.com/srathbun/vim-plugins.git ~/.vim`
It has Klen's python-mode plugin as a submodule: https://github.com/klen/python-mode
Python-mode provides awesome sause :) To get the repo, clone it recursively, or you won't have all the submodules. `git clone --recursive https://github.com/srathbun/vim-plugins.git ~/.vim`
My vimrc is at https://github.com/Julian/dotfiles/blob/master/.vimrc but it isn't really divided up into Python specific stuff.
(It's also a bit in flux as I migrate to tpope/vim-dispatch)
I can try elaborating feature-wise when I get home.
(It's also a bit in flux as I migrate to tpope/vim-dispatch)
I can try elaborating feature-wise when I get home.
[deleted]
iPython Notebook is really cool and the emacs client is extremely well done. I highly recommend checking it out.
http://tkf.github.io/emacs-ipython-notebook/
http://tkf.github.io/emacs-ipython-notebook/
Take a look here at my dotfiles to see how to use Rope/Ropemode/Ropemacs to get a Python-aware Emacs environment with the goodies that entails:
https://github.com/bitemyapp/dotfiles/
Edit: Try to ignore or not be daunted by the other directories, my Emacs setup is well maintained but very old and has over 400,000 LOC.