The emacs 30 Day Challenge: Using 'gnus' to read mail(mostlymaths.net)
mostlymaths.net
The emacs 30 Day Challenge: Using 'gnus' to read mail
http://www.mostlymaths.net/2010/12/emacs-30-day-challenge-using-gnus-to.html
8 comments
I, too, have a Maildir Spotlight importer on my list of projects. I have made no more or no less progress on it than you have, apparently :) It's really too bad that Apple didn't choose Maildir for Mail.app's message storage; my understanding is that its format is very similar.
I recently did some research on integrating Address Book.app with Emacs, and decided to use external-abook.el rather than the eudc-* stuff. It works great, so far. Did you know about it when you chose eudc-mab, and is there any particular reason you chose the latter instead?
At the moment I'm using external-abook.el along with bbdb, mostly because of bbdb's snarfing. I have to use C-c TAB to expand addresses with external-abook.el, which is a bit of a pain, because my muscle memory is wired to use plain old tab ala bbdb. I should figure out what I need to do to swap the expand keys.
edit: the latest version of external-abook.el that I can find is here:
http://www.emacswiki.org/emacs/external-abook.el
There's a git repo on Github somewhere, too, but the version on Emacswiki has some fixes that the Github one doesn't. Many developers of small, but useful, Emacs extensions have an unfortunate tendency to maintain their code on Emacswiki, rather than in a proper VCS. Oh well.
*edit2: typo fix in edit.
I recently did some research on integrating Address Book.app with Emacs, and decided to use external-abook.el rather than the eudc-* stuff. It works great, so far. Did you know about it when you chose eudc-mab, and is there any particular reason you chose the latter instead?
At the moment I'm using external-abook.el along with bbdb, mostly because of bbdb's snarfing. I have to use C-c TAB to expand addresses with external-abook.el, which is a bit of a pain, because my muscle memory is wired to use plain old tab ala bbdb. I should figure out what I need to do to swap the expand keys.
edit: the latest version of external-abook.el that I can find is here:
http://www.emacswiki.org/emacs/external-abook.el
There's a git repo on Github somewhere, too, but the version on Emacswiki has some fixes that the Github one doesn't. Many developers of small, but useful, Emacs extensions have an unfortunate tendency to maintain their code on Emacswiki, rather than in a proper VCS. Oh well.
*edit2: typo fix in edit.
Didn't know about external-abook; I'll check it out, thanks!
Interesting. I'm considering running a local imap server (to speed up search), but so far bbdb looks fine for contact search. I'll look up some way to get contact autocompletion, I'll take a look at eudc (I found a small page about it in the emacswiki). Thanks for making me aware of it!
I'm a newb with the formatting here, but after installing the eudc-mab library, I do the following:
http://gnufoo.org/contacts/
(eval-after-load
"message"
'(define-key message-mode-map [(tab)] 'eudc-expand-inline))
(eval-after-load
"sendmail"
'(define-key mail-mode-map [(tab)] 'eudc-expand-inline))
(require 'eudcb-mab nil t)
(setq eudc-server "localhost"
eudc-protocol 'mab)
(eudc-protocol-set 'eudc-inline-query-format
'((name)
(email))
'mab)
(eudc-protocol-set 'eudc-inline-expansion-format
'("%s <%s>" name email)
'mab)
http://gnufoo.org/contacts/
I just tried eudc-bbdb, and it somehow works (oddly enough, it expands one contact but not another... I'll have to look at the configuration options). The eudc-mab solution looks better for my Mac, although I think bbdb can work better in the long run: I use a Macbook at home and a Linux netbook at work: I have my bbdb file in the Dropbox folder, thus I can have a nice cross-system contact file.
Yeah, you really want bi-directional syncing with the bbdb, because bbdb does nice autosnarfing and whatnot. That's a bit of a sticky wicket (understatement of the morning). It doesn't help that the bbdb code is largely incomprehensible.
I'm curious what gnus offers over MH-E/nmh. If you could elaborate, I'd appreciate it.
IMAP support. Really, that's it. I loved mh back in the day, but it's never going to be able to cope with a centralized mail store.
The MH-E manual states "Most people use nmh, but you may be interested in trying out GNU mailutils MH, which supports IMAP."
The mailutils home page states "the utilities provided by Mailutils include imap4d and pop3d mail servers, mail reporting utility comsatd, general-purpose mail delivery agent maidag, mail filtering program sieve, and an implementation of MH message handling system."
I've always wanted to try this, but never found the time.
The mailutils home page states "the utilities provided by Mailutils include imap4d and pop3d mail servers, mail reporting utility comsatd, general-purpose mail delivery agent maidag, mail filtering program sieve, and an implementation of MH message handling system."
I've always wanted to try this, but never found the time.
By the way, if you start your project, please, let me know! Spotlight is perfect for searching... If I can interact get it with emacs and my gmail accounts it is a win-win situation
Do you feel a tinge of jealousy when you see 'normal' people simply use Gmail?
Only for searching. This feeling is mitigated a bit by the recent improvements in nnir IMAP search in the development version of Gnus, but it's still not as nice as Gmail search.
On the other hand, I use Gmail to archive some mailing lists (precisely because the search is so good -- almost always better than whatever archiving system the list maintainers are using), and my god is it slow! Slow to switch folders, slow to search.... So in that regard, I don't understand how people use Gmail as their primary MUA.
On occasion I hear a story about how someone complains that Gmail is slow, often in a public place like Twitter, and then some magical Gmail fairy moves their records to a less-contended shard/tablet/whatever, which improves things. Maybe I'm just unlucky.
On the other hand, I use Gmail to archive some mailing lists (precisely because the search is so good -- almost always better than whatever archiving system the list maintainers are using), and my god is it slow! Slow to switch folders, slow to search.... So in that regard, I don't understand how people use Gmail as their primary MUA.
On occasion I hear a story about how someone complains that Gmail is slow, often in a public place like Twitter, and then some magical Gmail fairy moves their records to a less-contended shard/tablet/whatever, which improves things. Maybe I'm just unlucky.
I have installed the latest cvs gnus... And search looks broken: I made a search (that I knew worked) and it keeps on fetching and reordering the buffer of results forever. I'll try to check what is going on, or waiting a little longer... :/
Edited: Holy sh*t! Installed the latest GIT gnus... Blazingly fast!! I'll write about it this week, this is something that needs to be installed... I just can say WOW.
Edited: Holy sh*t! Installed the latest GIT gnus... Blazingly fast!! I'll write about it this week, this is something that needs to be installed... I just can say WOW.
You can use notmuch-search with Gnus: http://www.emacswiki.org/emacs/NotMuch
Now there's nothing at all you could possibly miss from Gmail ;)
Not really. My habits are far too ingrained at this point. I have taken on the INBOX/everything else structure at this point, though, and no longer maintain the vast array of twisty sorting rules of a decade ago.
Too, there's no google in my life, and every other webmail interface I've seen is utter, utter pants.
Too, there's no google in my life, and every other webmail interface I've seen is utter, utter pants.
The "reading mail in emacs" option he didn't mention is notmuch (http://notmuchmail.org/). It's a mail indexer with full-text search and a few front-ends, the most-widely used being for emacs. It's search is really fast, and it was fast enough to make me forget folders once and for all.
I didn't mention because I have not tried yet, some people suggested it in Reddit. I'll try it some of these days, the problem is that configuring it in Mac OS will take me more time (probably!) than I have this week. But it is something I have in my list, thanks for reminding me of it ;)
For what it's worth, MacPorts has all the dependencies. Worked for me out of the box after doing:
$ sudo port install valgrind gmime talloc xapian-core glib2Thanks, I'll try it tomorrow then... It is well worth the try to get fast search
There is one thing that Gnus was awesome at: dynamic scoring.
It would monitor what you read and assign different weights to articles based on your habits: author of a post gets +10 points, keywords in a subject +2 points each, etc...
Then it reordered the newsgroup articles based on these weights.
After a week or so, you realize that when you enter a newsgroup, you will typically read the first 5-10 discussions that Gnus has put at the top of the list and just ignore the rest.
It was magical and I don't think any other newsreader has ever come close to that.
It would monitor what you read and assign different weights to articles based on your habits: author of a post gets +10 points, keywords in a subject +2 points each, etc...
Then it reordered the newsgroup articles based on these weights.
After a week or so, you realize that when you enter a newsgroup, you will typically read the first 5-10 discussions that Gnus has put at the top of the list and just ignore the rest.
It was magical and I don't think any other newsreader has ever come close to that.
the three features that made me move into gnus:
1 - scoring - which you just mentioned
2 - group levels - you can assign levels to each of the mail groups/folders and it'll only download groups in each level at a time, it's very good for organization.
3 - integration with org-mode - you can add an annotated link to an email with a single keypress, it's a lot more productive than copy-pasting text because you know where the text came from and you can reply to the email on the spot, with another key press.
I can't help myself. I have to ask, even it's annoying to say, "What is the interest in finding new ways to use lame, 30 year old interfaces?"
And yes, this sounds trollish but I honestly am interested in what applications promise new, interesting advances to computer interfaces. You can do do anything in EMACS, FORTH or MASM but the question is how easily and naturally.
And yes, this sounds trollish but I honestly am interested in what applications promise new, interesting advances to computer interfaces. You can do do anything in EMACS, FORTH or MASM but the question is how easily and naturally.
After using emacs for some time, I find it very hard to use an environment that doesn't offer keyboard macros (quickly produced temporary programmable sequences of arbitrary commands, could do anything from altering the layout of each paragraph in a bunch of emails to subscribing to new mailing lists of a certain type or what have you). VI has something similar going on I hear.
Also, the unified UI of the emacs operating system[1] is hard to beat. Everything Is Text, so any portion of an interface can be copy-pasted and manipulated as text, sent through other commands, etc. (Again, using vi + screen and combinations of terminal commands can also achieve something like this.)
Lastly, the interface is not really 30 years old. I would never use the first edition of GNU EMACS -- it barely has any of the features I've become so dependent on. Would you use the first mouse-based Xerox UI? It's essentially the same as the mouse-based interfaces we have now...
[1] To me it feels like I'm running two operating systems at once: emacs, and Anything Else. But I hear some people just use it to edit text.
Also, the unified UI of the emacs operating system[1] is hard to beat. Everything Is Text, so any portion of an interface can be copy-pasted and manipulated as text, sent through other commands, etc. (Again, using vi + screen and combinations of terminal commands can also achieve something like this.)
Lastly, the interface is not really 30 years old. I would never use the first edition of GNU EMACS -- it barely has any of the features I've become so dependent on. Would you use the first mouse-based Xerox UI? It's essentially the same as the mouse-based interfaces we have now...
[1] To me it feels like I'm running two operating systems at once: emacs, and Anything Else. But I hear some people just use it to edit text.
The nice thing for me about emacs interfaces is that they are all very similar. I spend about 90% of my time in emacs. I use it for email (gnus), chat (jabber.el), writing code (locally, and remotely via TRAMP), interacting with version control repos, running shells, interacting with databases, planning and managing my work (org-mode), invoicing clients, etc.
In all of this the same basic keybindings work the same way. This reduces the context switching overhead as move from one task to another. Yeah it looks primitive, being text mode and all, but it's soooo productive.
In all of this the same basic keybindings work the same way. This reduces the context switching overhead as move from one task to another. Yeah it looks primitive, being text mode and all, but it's soooo productive.
Thanks to the wonders of "backtype", a direct commenter in the post asks for what do you use (in emacs, I assume) to invoice clients. If I don't ask here directly you would never know about the question!
Agreed with your "high productivity" statement. Being able to just write, kill and yank without even thinking what my fingers are doing feels so much better than context-switching to Cmd-C, Cmd-V, oh no this is in X11 Ctrl-C Ctrl-v....
Agreed with your "high productivity" statement. Being able to just write, kill and yank without even thinking what my fingers are doing feels so much better than context-switching to Cmd-C, Cmd-V, oh no this is in X11 Ctrl-C Ctrl-v....
Personally, I love reading how other people use computers. And just because an interface is old doesn't make it bad.
Because its there.
I really recommend "Mew"(http://www.mew.org/en/) as an option for email in Emacs. I was able to really customize it to my needs. Its exactly like Emacs in its extensibility.
Gnus is a little overrated in my opinion.
Gnus is a little overrated in my opinion.
Gnus carries a lot of usenet cruft, no question. But the last time I tried 'mew' or 'wanderlust' I got pretty angry:
http://homonculus.net/posts/nerdism/2009-10-08-attention_jap...
http://homonculus.net/posts/nerdism/2009-10-08-attention_jap...
Tomorrow evening I will give mew a try. Thanks for the suggestion! Any interesting configuration part I may need to be aware of?
I've been using gnus as my primary mail reader for seven years, but it feels like it's getting a bit long in the tooth. Pulling messages down from the IMAP server and putting it in the appropriate folders takes a good 20s, whereas for the webmail clients I occasionally resort to, retrieval of the header information for new mail is instantaneous.
The notmuch application sounds very useful. Searching my mail has been the other big pain point which often leads me to resort to another client.
The notmuch application sounds very useful. Searching my mail has been the other big pain point which often leads me to resort to another client.
Have you tried the development version of gnus (from git) in the last 2 or 3 months? Lars has returned and is making lots of improvements, especially in IMAP performance and search.
I have about 200 mail folders, and the dev version of gnus now starts up in about 5 seconds, whereas the version included in Emacs 23.2 took about 1 minute during IMAP session bring-up. Refreshing the group view is now quite fast for me, too.
I believe that nnir has been rewritten from scratch. Doing a 'G G' search against my dovecot server on a folder with several hundred messages takes only a couple of seconds. The new nnir also has support for various kinds of search filters, including a mode where you can type a raw IMAP search string, if the included filters aren't sophisticated enough for your queries. Unfortunately, it's not well-documented yet, so you have to search the mailing list or read the .el file to discover what it can do, at the moment.
Anyway, try the development version of gnus, and see if addresses any of your complaints.
I have about 200 mail folders, and the dev version of gnus now starts up in about 5 seconds, whereas the version included in Emacs 23.2 took about 1 minute during IMAP session bring-up. Refreshing the group view is now quite fast for me, too.
I believe that nnir has been rewritten from scratch. Doing a 'G G' search against my dovecot server on a folder with several hundred messages takes only a couple of seconds. The new nnir also has support for various kinds of search filters, including a mode where you can type a raw IMAP search string, if the included filters aren't sophisticated enough for your queries. Unfortunately, it's not well-documented yet, so you have to search the mailing list or read the .el file to discover what it can do, at the moment.
Anyway, try the development version of gnus, and see if addresses any of your complaints.
I'll also check the dev version: searching and moving mails is really slow. I hope this solves it, thanks for pointing it out!
No; I'll definitely try that tomorrow. Thank you.
I have exactly the same Gnus setup. fantastic! However, one of the most frustrating things currently is the wrong message counts when messages 'disappear' from the backend either because they moved or deleted. Also when you get prompted by Gnus on how many messages you want to fetch from the server is broken by the above.
I got a bit tired with Emacs' mail readers (VM and Gnus) and moved to mutt.
My next project that I will likely never finish (but perhaps start!) is a Spotlight importer for Maildirs, as my baroque mail reading setup involves syncing all my various mail accounts via offlineimap to my local machine, where I run an IMAP server. It'd sure be nice to get the benefit of Spotlight indexing without having to have Apple's wretched mail client running all the time.