Ask HN: Best way to implement natural language commands for todo's, reminder etc
11 comments
You're going up the wrong tree entirely. What you're trying to do is one of the oldest ideas in computing, and it does not work. They keep trying it, and it keeps failing. Not technically, but from a user point of view. Typing stuff out the way you want people to is not easy or convenient, and if there is even the hint of uncertainty, most people will prefer not to do it. Considering that you do not have a background in this, you will lots of unmatched items, and 2/10 fails will consitute too much uncertainty for your users.
I have two ideas for alternatives that may or may not work, but at least have not been tried as often:
1. Autocomplete. When the user is typing, autocomplete his message, such that there is always 100% certainty that the command will be executed properly. Also, simplify the syntax, so I write 'remind' : 'December ...' : 'Dentist'. Your syntax has to be consistent, so once learned, there is no relearning. I don't want to have to figure out what the system wants me to type in as a command.
2. Visual. Use blocks and lines to create the actions. Humans have a very good sense of geometry and relationships between physical objects. Leverage that to make your app. Models of physical things are one of the few things we can actually say is inherently intuitive for human beings.
I have two ideas for alternatives that may or may not work, but at least have not been tried as often:
1. Autocomplete. When the user is typing, autocomplete his message, such that there is always 100% certainty that the command will be executed properly. Also, simplify the syntax, so I write 'remind' : 'December ...' : 'Dentist'. Your syntax has to be consistent, so once learned, there is no relearning. I don't want to have to figure out what the system wants me to type in as a command.
2. Visual. Use blocks and lines to create the actions. Humans have a very good sense of geometry and relationships between physical objects. Leverage that to make your app. Models of physical things are one of the few things we can actually say is inherently intuitive for human beings.
Entirely agree - natural language processing has been a holy grail for decades, and nobody has ever gotten it quite right.
You're also right on the money with the notion that from a user perspective if there is a small chance of failure users will tend to distrust and ultimately discard the interface.
You're also right on the money with the notion that from a user perspective if there is a small chance of failure users will tend to distrust and ultimately discard the interface.
Despite all my obvious love for NLP and grammars, I would actually like to echo markessien's point here.
I agree that NLP might not be the best approach here, especially if you're having to learn all of it from scratch (I'm no expert but I learnt enough basics so that I could look into any aspect in more depth when it comes to it).
My first suggestion would be to take a step back and think about what the system is and what is the easiest, fastest and most intuitive way to input dates. Brainstorm this and you will come up with good ideas like those above.
I like the autocomplete idea and also there's no reason not to use some syntax like "Dentist @ 2pm", i.e. using @ to separate title from date. It's sufficient to indicate to the machine which is text and which is date info, but close enough to natural speech as well (also the @ is a very recognisable character now).
One thing to note is that Google Calendar accepts expressions such as "Dentist at 2pm"; perhaps you can try that and see how you like how that works?
I agree that NLP might not be the best approach here, especially if you're having to learn all of it from scratch (I'm no expert but I learnt enough basics so that I could look into any aspect in more depth when it comes to it).
My first suggestion would be to take a step back and think about what the system is and what is the easiest, fastest and most intuitive way to input dates. Brainstorm this and you will come up with good ideas like those above.
I like the autocomplete idea and also there's no reason not to use some syntax like "Dentist @ 2pm", i.e. using @ to separate title from date. It's sufficient to indicate to the machine which is text and which is date info, but close enough to natural speech as well (also the @ is a very recognisable character now).
One thing to note is that Google Calendar accepts expressions such as "Dentist at 2pm"; perhaps you can try that and see how you like how that works?
Auto complete would be a good option, however I want people to be able to send the commands in emails (with the option on the web page too).
Having started scanning about NLP, I can tell its a big area, and I don't think I want to invest lots of time into this. Perhaps a cutdown syntax is the way to go. For my small set of commands, all I have now are TODOs, Followups and Reminders. So a few ways of entering each of them and some thinking about dates should get me something usable.
That said, I do have concerns about the ambiguity of what can be entered, and what to do with failed commands etc.
Dates really are the pain point for entering things - date pickers are generally annoying, and a series of drop downs are a pain too and people write dates in so many different ways!
Having started scanning about NLP, I can tell its a big area, and I don't think I want to invest lots of time into this. Perhaps a cutdown syntax is the way to go. For my small set of commands, all I have now are TODOs, Followups and Reminders. So a few ways of entering each of them and some thinking about dates should get me something usable.
That said, I do have concerns about the ambiguity of what can be entered, and what to do with failed commands etc.
Dates really are the pain point for entering things - date pickers are generally annoying, and a series of drop downs are a pain too and people write dates in so many different ways!
Also remember that some ways are impossible to tell apart. 12/10/2008, for example, is 12th October for us in the UK, but 10th December for Americans. There's simply no way to guess at which it is without some context or the user saying which.
What that shows is that while NLP can make systems seem quite intelligent, it's really not a replacement for the user putting some thought into it too. Forms work nicely as they can force input into certain formats. I like NLP, so I always think systems could at least meet a human half way on this.
However, even if we were talking face to face, there's some social expectation you'll try to make some sense (i.e. speak in English). You wouldn't just speak Spanish to me if I hadn't indicated I could speak Spanish. AI-like systems are meant to be intelligent, not psychic -- a point often forgotten by some people.
A dialogue system would help uncertainties since it could ask for clarification (which, frankly, is all we humans do if something's not clear when talking to each other), but if you're talking over email, you really want to get it right the first time as it's not the fastest system.
Again, off the top of my head: I'd say you could make it clear that the subject of the email has to be one of "reminder", "followup" or "todo"... then do some aggressive normalisation on the subject lines that come in (grab a Porter Stemmer or even just take the first 3/4 letters maybe) then force to upper or lower and then check against a massive list of synonyms. Then you simply have to bounce with a help notice if someone really still managed to enter something stupid.
Examples: Subject: Reminder -> REMINDER -> REM -> matches rule for "REM/EVE/HOL" (Reminder, event, holiday) Subject: event -> EVENT -> EVE -> matches rule for "REM/EVE/HOL" Subject: follow-up -> FOLLOW-UP -> FOL -> matches rule for "FOL/REV" (follow-up, review)
That sort of idea. It accounts for RaNDom CapITAlisation, uncertain hyphen-usage and plurals. Make those match rules long enough and you might get error rate pretty low indeed.
I know you've probably already gone over this, but does it really need to be email? Is that the best interface? Is it worth taking a step back (going outside the box, as they say) and working out the most efficient method?
What that shows is that while NLP can make systems seem quite intelligent, it's really not a replacement for the user putting some thought into it too. Forms work nicely as they can force input into certain formats. I like NLP, so I always think systems could at least meet a human half way on this.
However, even if we were talking face to face, there's some social expectation you'll try to make some sense (i.e. speak in English). You wouldn't just speak Spanish to me if I hadn't indicated I could speak Spanish. AI-like systems are meant to be intelligent, not psychic -- a point often forgotten by some people.
A dialogue system would help uncertainties since it could ask for clarification (which, frankly, is all we humans do if something's not clear when talking to each other), but if you're talking over email, you really want to get it right the first time as it's not the fastest system.
Again, off the top of my head: I'd say you could make it clear that the subject of the email has to be one of "reminder", "followup" or "todo"... then do some aggressive normalisation on the subject lines that come in (grab a Porter Stemmer or even just take the first 3/4 letters maybe) then force to upper or lower and then check against a massive list of synonyms. Then you simply have to bounce with a help notice if someone really still managed to enter something stupid.
Examples: Subject: Reminder -> REMINDER -> REM -> matches rule for "REM/EVE/HOL" (Reminder, event, holiday) Subject: event -> EVENT -> EVE -> matches rule for "REM/EVE/HOL" Subject: follow-up -> FOLLOW-UP -> FOL -> matches rule for "FOL/REV" (follow-up, review)
That sort of idea. It accounts for RaNDom CapITAlisation, uncertain hyphen-usage and plurals. Make those match rules long enough and you might get error rate pretty low indeed.
I know you've probably already gone over this, but does it really need to be email? Is that the best interface? Is it worth taking a step back (going outside the box, as they say) and working out the most efficient method?
There is some good advice coming up in this thread.
A few people are suggesting email is not the correct medium, so I need to explain more what it is I am trying to achieve. My app is sort of like 'I Want Sandy' (I started working on this before I even heard about I Want Sandy, as I only came to know of it now that its about to be shut down).
Basically, I want to be able to take an email sent to me, and forward it to a drop box. I can either just send the email in as is (received emails have certain information extracted from them and displayed on a dash board), or I can send it in with a command on it, or several commands.
So I can fire in an email saying followup in 2 days/on Dec 12th etc, and in two days time, the app will send me an email along with the original email attached.
If I add a command as TODO, then it will add it to the TODO list attached to that drop box etc.
There will of course be a non-ambiguous web front end to all this to manage all the items, and add followups, todo's etc without an email, but my goal is to be able to add items from your inbox.
A few people are suggesting email is not the correct medium, so I need to explain more what it is I am trying to achieve. My app is sort of like 'I Want Sandy' (I started working on this before I even heard about I Want Sandy, as I only came to know of it now that its about to be shut down).
Basically, I want to be able to take an email sent to me, and forward it to a drop box. I can either just send the email in as is (received emails have certain information extracted from them and displayed on a dash board), or I can send it in with a command on it, or several commands.
So I can fire in an email saying followup in 2 days/on Dec 12th etc, and in two days time, the app will send me an email along with the original email attached.
If I add a command as TODO, then it will add it to the TODO list attached to that drop box etc.
There will of course be a non-ambiguous web front end to all this to manage all the items, and add followups, todo's etc without an email, but my goal is to be able to add items from your inbox.
Ah, that's quite a neat idea really. I do think you're going to have to stick to a fixed syntax here though. People are already familiar with the standards of "Re: and Fw:" on the Internet, so it's not that far-fetched for people to learn "Todo:", "Followup:" and "Remind:".
My overall suggestion would be to go with that (you could even tell users to press the "forward" button then replace "fw" with one of three commands). Then you can go with my suggestion of normalising caps, punctuation and even stemming to the first few letters to as to make it as forgiving as possible. e.g., FOL would even match a typo like Fololw-up ;-)
That way, it could expect all subject lines in the form:
I really don't think it's unreasonable to expect users to get the command bit right, so the problem really is reduced to getting the date bit parsed correctly for which I've seen two suggestions here already.
p.s. One possible future thing could be to make extensions for things like Thunderbird that actually gives a "Todo", "Remind" and "Follow-up" buttons alongside the "Reply" and "Forward" buttons ;-)
My overall suggestion would be to go with that (you could even tell users to press the "forward" button then replace "fw" with one of three commands). Then you can go with my suggestion of normalising caps, punctuation and even stemming to the first few letters to as to make it as forgiving as possible. e.g., FOL would even match a typo like Fololw-up ;-)
That way, it could expect all subject lines in the form:
Command: Text @ Date
I really don't think it's unreasonable to expect users to get the command bit right, so the problem really is reduced to getting the date bit parsed correctly for which I've seen two suggestions here already.
p.s. One possible future thing could be to make extensions for things like Thunderbird that actually gives a "Todo", "Remind" and "Follow-up" buttons alongside the "Reply" and "Forward" buttons ;-)
p.s. One possible future thing could be to make extensions for things like Thunderbird that actually gives a "Todo", "Remind" and "Follow-up" buttons alongside the "Reply" and "Forward" buttons ;-)
That would be very much in the future! I actually have a a prototype working at the moment, but its for my eyes only for a while yet until I can iron a few things out!
Good tip on the stemming - I will certainly give that a go.
That would be very much in the future! I actually have a a prototype working at the moment, but its for my eyes only for a while yet until I can iron a few things out!
Good tip on the stemming - I will certainly give that a go.
Note: I work in the NLP field.
I'll echo what others have been suggesting, that in this case a restricted grammar with a thesaurus for synonyms and making use of some extant date libraries might be your best option here.
I suggest this because your trigger input is purpose-driven. If you were going to be extracting this type of information from the middle of a email or document content, then I would be making some NLP(-ish) suggestions here. However, since you say that the commands to your application will be input directly by the user for that specific purpose, it will be much more user-friendly and easier (with more user confidence) if you have a pre-defined simple grammar with a range of available options and command synonyms.
That way, since the user is only typing for your application to understand, they know what they can/should type for the expected results. You can allow for a little variety, a little variation in date descriptions, but it should be constrained to a point.
The comments about auto-complete suggestions would work very well in reminding the user what types of commands are available. Have a list of commands (TODO, FOLLOWUP, NAGME, etc) and then when the user selects one of those commands, fill in the other parts of the syntax with placeholders (FOLLOWUP: [in X days|on DATE],...) perhaps.
I'd say that you could add a little intelligence to it by picking up on certain keys from the email to suggest commands, i.e. dates in the email could be options for the date part of the command, and other more complex (and less-accurate) extractions.
But a simpler solution should work better in this case than anything with a more NLP focus, in my opinion.
I'll echo what others have been suggesting, that in this case a restricted grammar with a thesaurus for synonyms and making use of some extant date libraries might be your best option here.
I suggest this because your trigger input is purpose-driven. If you were going to be extracting this type of information from the middle of a email or document content, then I would be making some NLP(-ish) suggestions here. However, since you say that the commands to your application will be input directly by the user for that specific purpose, it will be much more user-friendly and easier (with more user confidence) if you have a pre-defined simple grammar with a range of available options and command synonyms.
That way, since the user is only typing for your application to understand, they know what they can/should type for the expected results. You can allow for a little variety, a little variation in date descriptions, but it should be constrained to a point.
The comments about auto-complete suggestions would work very well in reminding the user what types of commands are available. Have a list of commands (TODO, FOLLOWUP, NAGME, etc) and then when the user selects one of those commands, fill in the other parts of the syntax with placeholders (FOLLOWUP: [in X days|on DATE],...) perhaps.
I'd say that you could add a little intelligence to it by picking up on certain keys from the email to suggest commands, i.e. dates in the email could be options for the date part of the command, and other more complex (and less-accurate) extractions.
But a simpler solution should work better in this case than anything with a more NLP focus, in my opinion.
How about just using a fixed syntax:
-- email start
* Pick Milk
* Buy Fags
* Drop Children at Mary @ 16:00
---
Then use a python date parse to parse whatever comes after @. You don't need to split reminders or followups, because an @ symbol would automatically imply a reminder, innit?
-- email start
* Pick Milk
* Buy Fags
* Drop Children at Mary @ 16:00
---
Then use a python date parse to parse whatever comes after @. You don't need to split reminders or followups, because an @ symbol would automatically imply a reminder, innit?
"Remind" might solve the date side of it, but you'll still need to parse it out. I've thought about a similar thing before and the approach I was looking to take was using something like: http://www.nltk.org/ to create grammar trees (read up on how grammars/parsers work at least on Wikipedia) for an acceptable subset of English.
You might notice things like a time is always after "at", a date "on" and a relative date "in". Ideas for rules:
Just some things to think about and if you don't know grammars, then read up until you understand the example I gave at least. I just wrote it off the top of my head too, so I'm sure someone might find a mistake with it :-)
p.s. If you're as weird as me, you might even consider Prolog for parsing ;-)
You might notice things like a time is always after "at", a date "on" and a relative date "in". Ideas for rules:
Event -> Title When
Title -> <anything>
When -> TimeSpecifier When
TimeSpecifier -> "at" Time
TimeSpecifier -> "in" RelativeTime
TimeSpecifier -> "on" Date
...
Just some things to think about and if you don't know grammars, then read up until you understand the example I gave at least. I just wrote it off the top of my head too, so I'm sure someone might find a mistake with it :-)
p.s. If you're as weird as me, you might even consider Prolog for parsing ;-)
One trick for training users on your grammar is to have the program always talk to the user with it. If the program imports my existing calendar and talks about stuff like "lunch with fendale in 3 days" all the time, I'll talk back to the program in similar language. Making users learn syntaxes, especially with any punctuation that you don't pronounce out loud, is lousy.
Then, don't be afraid of ambiguity. In many cases, you can Branch Both Ways. 12/10/2008 could refer to two dates, so put the item on both, showing clearly why it's there. "Lunch with Bill" could refer to a few people in my address book, so just link all of them. If your software manages to get some additional data, sort the choices so the likely one is on top.
Then, don't be afraid of ambiguity. In many cases, you can Branch Both Ways. 12/10/2008 could refer to two dates, so put the item on both, showing clearly why it's there. "Lunch with Bill" could refer to a few people in my address book, so just link all of them. If your software manages to get some additional data, sort the choices so the likely one is on top.
I like that tip about talking to the user in the language as the APP expects - hadn't though of that myself.
The cheap and cheerful way to cheat this problem is to use a restricted grammar that allows synonyms. And one of the many libraries that wrap strptime for date parsing.
In other words; schedule, remind, tell, add; might all represent the same operation.
If you go this route, do write a BNF grammar, do test with a reasonably large corpus of 'live' requests.
here is a link that should get you started. http://www.antlr.org/wiki/display/ANTLR3/Quick+Starter+on+Pa...
In other words; schedule, remind, tell, add; might all represent the same operation.
If you go this route, do write a BNF grammar, do test with a reasonably large corpus of 'live' requests.
here is a link that should get you started. http://www.antlr.org/wiki/display/ANTLR3/Quick+Starter+on+Pa...
The old unix reminder system, "remind", is what I use as the core of my calendar system. It's very good at parsing dates in different formats and is based on text files with lines such as "REM <date string> MSG <text relevant to event>".
The reason I like it is the fact systems such as your idea there could possibly be built on top of it. Say you parsed out any portion you saw as a date string and put that at the front, it might be smart enough to handle several date string formats.
The reason I like it is the fact systems such as your idea there could possibly be built on top of it. Say you parsed out any portion you saw as a date string and put that at the front, it might be smart enough to handle several date string formats.
DateJS (http://www.datejs.com/) might have some interesting ideas you can borrow.
It sounds like you want to rewrite Sandy (http://www.iwantsandy.com/), which is good because they're closing down.
They show several examples of commands that Sandy accepts. That might help give you a little direction designing your system.
They show several examples of commands that Sandy accepts. That might help give you a little direction designing your system.
Actually I started this before I heard about iwantsandy.com, which I only heard about last week when it was shutting down. At that point I simultaneously thought 'damit, there is a tool to do what I wanted and here is me wasting time writing one to scratch my own itch' and 'interesting, outrage at this shutting down and I am working on something similar'.
I have started taking a bit of a different approach to the problem though, in that I want to create a drop box per work area (in my day job, I tend to have many mini-projects to manage at once, each lasting only a month or two) to help me keep track of things without excel.
I guess Basecamp or Highrise may get me what I wanted too, but I wanted a small project that is achievable in a reasonable time working 10 hours a week on it, so I am giving it a go!
I have started taking a bit of a different approach to the problem though, in that I want to create a drop box per work area (in my day job, I tend to have many mini-projects to manage at once, each lasting only a month or two) to help me keep track of things without excel.
I guess Basecamp or Highrise may get me what I wanted too, but I wanted a small project that is achievable in a reasonable time working 10 hours a week on it, so I am giving it a go!
We already do that at ShareMeme - http://sharememe.com/welcome/quickmode. You have to a be registered user to use this, and we also offer up the simple forms as well.
I have seen a few places that do these things, but how is the question. Care to elaborate on how you go about it? What tools/libraries you use etc and how successful it has been?
Try searching for NLP - natural language processing
parsedatetime (http://code.google.com/p/parsedatetime/) is a fantastic Python library for parsing natural language date / time strings.
I am surprised that Bayesian Reasoning has not yet been suggested.
Followup in two days
Followup on November 23rd at 3pm
Remind me about my dentist appointment at 3.30pm on December 12th 2008
Clearly there can be lots of subtle variations on what exactly gets entered in these sort of scenarios which makes for a potentially tricky/interesting problem.
Listening to a podcast recently, Joel Spolsky mentioned that people with no knowledge of compilers would tend to tackle a parsing problem like this the 'hard way' (ie using regexes, searching strings etc) while they may be better using a lexer/compiler approach. I have no knowledge of compilers, so my thinking right now is to scan for keywords and then things that look like dates using regexes.
Has anyone implemented anything that does this sort of this command processing and have any tips or tools to share? My application is in Rails, but I am willing to experiment - I just don't want to have many hundreds of lines of code testing for all sorts of strange conditions making for a brittle maintenance nightmare!