I can switch to any platform, provided that would be best one. I will check this framework and try to create a demo application so that i can get better idea.
There are many companies that use web crawling. Are they using Pyton or something other ?
I have searched on web and have created one solution, this is semi automated means I have to put some URLs and some parameters to it to get desired result.