Ask YC: How does Kayak or Mobissimo work?
6 comments
Many of the flight-related sites use http://www.itasoftware.com/ for fare and schedule querying.
Thanks for pointing me in this direction. I wonder how much licensing their software cost.
I will contact them.
I will contact them.
I'm pretty sure they've worked out deals with each of the target sites and have access to some private APIs. Probably this only works if you can convince them you're going to send serious business their way.
An alternative is something like beautiful soup.
http://www.crummy.com/software/BeautifulSoup/
http://www.crummy.com/software/BeautifulSoup/
Upvoted for Beautiful Soup. If you are into Python, its great for screen scraping websites and striping off markup, leaving only the necessary information which you can display on your own site. great for accessing sites that don't have a public API.
I'm pretty familiar with BeautifulSoup. My only concern is having to constantly monitor the mark up of hundreds of sites for changes.
Both sites search hundreds of other travel sites, hotel chains and airlines, show the end-user the aggregated results in a nice way and then direct them to the original sources passing an affiliate id code so they can profit.
I assume that the majority of the sites they pull information from don't provide any kind of APIs.
Does anyone know how they might be able to pull this off?
I was wondering if there is a web service out there that gives you access to query the majority of these sites from an central API or if they are just having to scrape hundreds of sites individually.