Scraping Hacker News on a Schedule with TaskPipes(blog.taskpipes.com)
blog.taskpipes.com
Scraping Hacker News on a Schedule with TaskPipes
http://blog.taskpipes.com/2015/09/22/scraping-hacker-news-on-a-schedule-with-taskpipes/
12 comments
Check out Diffbot @ http://www.diffbot.com/ -- you can setup repeat crawls and extract data into .csv/Excel file format or JSON with Crawlbot API, extract data automatically with automatic APIs, or Custom API Toolkit. You can get a free trial account at https://www.diffbot.com/plans/trial to try it out, and you can get paid plans at http://www.diffbot.com/pricing/.
This general category of tool seems very useful. Apparently there are many Yahoo Pipes followers. http://www.makeuseof.com/tag/12-best-yahoo-pipes-alternative...
Support for email and HTML tables seem to be some distinguishing features of this one.
Support for email and HTML tables seem to be some distinguishing features of this one.
I thought I remember reading that Yahoo was discontinuing the pipes tool?
Looks pretty but I can't find pricing information.
Chatted with one of the guys
We're starting to move TaskPipes out of beta at the moment, but our pricing will start at $50/month for the service
We're starting to move TaskPipes out of beta at the moment, but our pricing will start at $50/month for the service
I was looking for prices as well, or any other indicator for that matter that the tool will be available for a reasonable time at least.
+1
I love these tools but before I invest more of my time leveraging this in my workflow I'd want to know that it'll be around for another few months at least...
I love these tools but before I invest more of my time leveraging this in my workflow I'd want to know that it'll be around for another few months at least...
I tried NewsBlur as an alternative to Yahoo Pipes. Like it because it has an Android app as well.
I've seen something similar more than once on HN but my google-fu is failing. What other similar services are out there that'll let you smartly scrape a page into a spreadsheet.
In the past, I've tried Google spreadsheets with "ImportXML" option but got frustrated after a bit and resorted to python.
In the past, I've tried Google spreadsheets with "ImportXML" option but got frustrated after a bit and resorted to python.
What Open Source projects are similar? It seems like data transformation tasks like this are a universal problem. I'd just do map/reduce, but that's the advantage that comes with being a developer. Great tools for non-developers to do limited programming tasks are incredibly useful.