Ask HN: How to wget pages with javascript?

4 pointsby baconfacevor 15 Jahren8 comments

I need to wget a url, allow the javascript to execute, and capture the result in a file. I've tried wget, curl, httrack,plugins like lget, and links(2) and none of them execute JS.

I want something like $: firefox <url> -s <file>

Does anyone know how to accomplish this behavior without using JS?

4 comments

dsteinvor 15 Jahren

You can write a NodeJS script using the JSDom module to do this.

baconfacevor 15 Jahren

Thanks for the suggestion. Any idea how to accomplish this task without using JS?

I'm surprised chromium, konqueror and firefox don't offer this functionality for offline browsing.

dsteinvor 15 Jahren

Selenium (http://seleniumhq.org/) might work but it's not a simple command line script.

bartonfinkvor 15 Jahren

Selenium will do this, although it's not pretty. I spent about 4 months on a contract doing exactly that, and it sucked.

a3camerovor 15 Jahren

iMacro? Rhino?

profitbaronvor 15 Jahren

Personally, I'd use http://www.phantomjs.org/ and create a script which loads the page like a browser. Once you've done that you can get it to take screenshot or use JS to pull data off the page.

baconfacevor 15 Jahren

Thanks. I really want to get this done with the command line and it looks like Phantom JS will do the trick.

ddemchukvor 15 Jahren

perfect use case for celerity http://celerity.rubyforge.org/