Ask HN: What you have automated with Python?
any script that you are using to automate your life? How's it been done? Appreciate you sharing codes.
47 comments
From reddit:
My brother in law has a "habit" of going to jail. All. The. Time. My wife wants to know when it happens, so I setup a script to poll the local jail inmate rosters, and send me an email when his name shows up. It works pretty well.
https://www.reddit.com/r/Python/comments/308ucq/how_do_you_u...
My brother in law has a "habit" of going to jail. All. The. Time. My wife wants to know when it happens, so I setup a script to poll the local jail inmate rosters, and send me an email when his name shows up. It works pretty well.
https://www.reddit.com/r/Python/comments/308ucq/how_do_you_u...
I have a script that logs into my bank's web interface, and extracts my current balance. I subtract out my upcoming bills/savings transactions to tell me how much money I can spend until my next paycheck. My next step is to have it email the amount spendable every morning.
I'm using python 3, selenium, and SQLAlchemy connected to a home MySQL server.
I'm using python 3, selenium, and SQLAlchemy connected to a home MySQL server.
Check out https://peterdemin.github.io/kibitzr-banks.html
It does almost the same
Same here, though I only store them in a plain text file.
My tip for others trying the same is to look into your bank's mobile site, if they have one. Mine was much simpler to parse, I only needed a single XPath expression to extract the number.
Also, if your bank's site is AJAX-y, look into the network tab of your browser, chances are you can find an URL that sends you a nice JSON back with the data you need.
My tip for others trying the same is to look into your bank's mobile site, if they have one. Mine was much simpler to parse, I only needed a single XPath expression to extract the number.
Also, if your bank's site is AJAX-y, look into the network tab of your browser, chances are you can find an URL that sends you a nice JSON back with the data you need.
This is a good idea and it's on my very long nice-to-have list. How are you driving the browser? Some sort of UI testing library? My bank has no API so it'd have to screen scrape.
EDIT: Sorry, I can see that it says it right there in your post: Selenium. Leaving up my stupid question as a marker of shame.
EDIT: Sorry, I can see that it says it right there in your post: Selenium. Leaving up my stupid question as a marker of shame.
That is a neat idea! I should deploy something similar. Thanks!
Could you share your codes?
The code's a bit rough, I have a bit of cleanup to go before I publish it.
Very cool, please let me know when you release it! I'd love to check it out.
[deleted]
[deleted]
I use Python to suck structured data out of our sales CRM's REST API and transform it into a nicely formatted narrative biweekly report in Excel for exec management. Pleasingly, it's reduced the aggregate time the sales team spends "writing about work" from about 80 hours a month to under 20. Subsequently I've extended that script to also automate the process around reminding relevant people about expiring legal agreements and the like.
Using Python 2.7, BeautifulSoup, XLSXWriter, Fuzzy Fuzzy. As a Python newbie, the hardest thing was working out how to even pick a version and an environment to start building on.
Using Python 2.7, BeautifulSoup, XLSXWriter, Fuzzy Fuzzy. As a Python newbie, the hardest thing was working out how to even pick a version and an environment to start building on.
A year ago I was lecturing mathematics and came up with a system to generate randomised coursework assignments (with corresponding markschemes, including graphs) for large classes of students (about 70 in my case). At the core of it was a python script that automated a slightly tricky latex compilation process.
You can read more about it and see the python code here: http://tug.org/TUGboat/tb37-1/tb115alsafi.pdf
You can read more about it and see the python code here: http://tug.org/TUGboat/tb37-1/tb115alsafi.pdf
I work for a botanical garden and spend a lot of my time working in the gift shop. We have a huge pile of t-shirts, dozens of designs each in several sizes. Most of the employees spend at least an hour going through the whole pile to figure out which shirts need to be restocked. I wrote a python script that takes an xls sales report generated by our POS system and outputs a list of all the shirts we've sold, by design, size and color. It takes me 5 minutes to go grab those shirts and put them out.
How do you handle shrinkage w o manual count
We do count the piles manually twice weekly
[deleted]
I've automated my build-test-commit workflow. My build script commits my changes at the beginning of the build. If the build or any of the tests fail, then the commit is rolled back, otherwise it is squashed together with other edits ready to be pushed. I automatically get a complete record of builds (in the reflog), and a fairly neat (if somewhat coarse-grained) history of changes with perfect traceability in my commit messages.
I would love to see what it looks like, even in a rough format. I have been meaning to do something similar, in particular for running tests + docs and pushing to a repo.
We, meaning 3 of us, have automated a greenhouse using Python code running on a raspberry pi -- this includes getting data from sensors within the greenhouse (temperature, humidity, soil moisture, soil temperature, carbon dioxide), from an accurite 5 in 1 weather station outside, and then running fans (intake and exhaust), water pumps and sprinklers, camera, etc... and pump data to web, so that we do not need to be present to see what is happening inside the greenhouse. No code is production ready, but we have had 2 good vegetables crops from the greenhouse already. The problem we working on is presented here : https://medium.com/@pravenj/a-bucket-a-day-a-hack-in-agricul...
Absolutely Inspiring. Are you guys hiring?
The company I work for ships a lot of packages each day. Shipments that are late, even ground shipments, are eligible for a refund if it is the carrier's fault. I have a python application that tracks shipments and let's me know which might be eligible for a refund. Saves us thousands each year.
Is there a SaaS that does this for companies? Maybe charging a percentage of the refund so there's no upfront cost to the customer?
If not, is anyone interested in building it with me?
I know there's a company that does something like this for parking tickets in NYC, and another one that does it for airline tickets.
If not, is anyone interested in building it with me?
I know there's a company that does something like this for parking tickets in NYC, and another one that does it for airline tickets.
There was one for a while that did this specifically. You had to give them access to your online accounts with UPS and FedEx. They would then get the refunds for you and keep/bill for 50% of the total refunded. A nice no-risk proposal.
Now, the carriers require that the shipper themselves (or whoever paid for the shipment to begin with) must claim the refund. So third party refund collections are a violation of terms of service.
There are still some very large logistics providers that claim to offer this, but I am not sure how they do this. They might actually route all shipments through them, making them the actual shipper, allowing them to collect refunds. But you have to commit to them for all logistics/shipping.
Unless you monitor it closely yourself, there is no way to get information about late shipments from the carriers. So they make it hard to get them. And not all late shipments are eligible (only if fault of carrier).
So there is value alone in being aware of what to seek refunds on. I've thought about rolling this out into a service. I'm still working on tweaking my setup.
Now, the carriers require that the shipper themselves (or whoever paid for the shipment to begin with) must claim the refund. So third party refund collections are a violation of terms of service.
There are still some very large logistics providers that claim to offer this, but I am not sure how they do this. They might actually route all shipments through them, making them the actual shipper, allowing them to collect refunds. But you have to commit to them for all logistics/shipping.
Unless you monitor it closely yourself, there is no way to get information about late shipments from the carriers. So they make it hard to get them. And not all late shipments are eligible (only if fault of carrier).
So there is value alone in being aware of what to seek refunds on. I've thought about rolling this out into a service. I'm still working on tweaking my setup.
I had heard of one before this... Not sure if the tou allow it anymore.
While I do not contribute code, I do contribute documentation and support what I can via issue triage & community support for Home Assistant (https://home-assistant.io/) which is a Python 3 open source Home Automation webapp with over 500 components.
You can check out my automations, configuration files & docs at GitHub https://github.com/brianjking/homeassistant-config.
You can check out my automations, configuration files & docs at GitHub https://github.com/brianjking/homeassistant-config.
I automated my search for a houseshare by scraping http://www.flatmates.com.au and searched each description for a series of keywords. Made a potentially awful task much more pleasent.
https://github.com/siquick/findahouseshare
Python + BeautifulSoup
https://github.com/siquick/findahouseshare
Python + BeautifulSoup
A simple GAE app (written in Python) to notify me of new posts in blogs I follow, https://github.com/adrian/feed2me
There are lots of managed services to do this of course but I figured it would be a useful learning exercise. And it was
There are lots of managed services to do this of course but I figured it would be a useful learning exercise. And it was
I use Python to control the GPIO pins on a Raspberry Pi to run my sprinklers. It just runs out of cron. I haven't polished it yet for publication, but here it is:
https://github.com/pjungwir/raspi-sprinklers
https://github.com/pjungwir/raspi-sprinklers
Pulling geometry data of chemical formulas off the cccbdb.nist.gov
Built for a friend who has to pull it manually all day at work.
https://github.com/marcelo-mason/cccbdb-geometry-parser
Built for a friend who has to pull it manually all day at work.
https://github.com/marcelo-mason/cccbdb-geometry-parser
In my country you can apply for social housing online - but you never know when new listings will be published. So I built a scraper that used a telephony api to give me a call when a new listing was published. On a Friday night I got my automated call, looked at the listing and immediately booked a place on the queue. I was second. But the first person must have said no to the flat, so I got it. Paid very low rent for the few years that I had it. They added a captcha soon after.
I've (technically) automated server monitoring.
I am a solo Sysadmin at a decently sized game server host and we have ~60 different dedicated servers. I wrote Platypus as a way of keeping track of it all. It's still definitely a work in progress (bugs are plentiful), but it now uses websockets so theres that. https://github.com/gmemstr/platypus
I am a solo Sysadmin at a decently sized game server host and we have ~60 different dedicated servers. I wrote Platypus as a way of keeping track of it all. It's still definitely a work in progress (bugs are plentiful), but it now uses websockets so theres that. https://github.com/gmemstr/platypus
Once upon a time I was writing an Android app. I wrote a simple Python script to monitor the APK file and copy it to my Dropbox when it changed so that I could easily install and test it on my phone. Uses the Win32 API, so not portable.
It's here: https://github.com/gusmd/vigil.py
It's here: https://github.com/gusmd/vigil.py
My last company used Atlassian Stash and switched from a single project and monorepo to lots of projects and repos. I wrote a python script that uses the Stash API (via the stashy library) to update or clone selected (or all, including new) repositories.
I used it fairly regularly to keep my local repos up to date, and I know one developer who used it to generate gource visualizations.
I used it fairly regularly to keep my local repos up to date, and I know one developer who used it to generate gource visualizations.
Me and my friends often share articles on our xmpp chatroom, some of them are behind paywalls and we all of use don't have subscriptions to the same websites.
So I have a parser that checks for url of paywall articles being posted. When it happens, the python script logs to the corresponding website and rehost the content of the article on my server. Finally it posts a new link — with no paywall this time — in the chatroom.
This required asking my friends for their login information on the websites where I don't have a subscription, but it made the whole process of sharing articles way easier.
So I have a parser that checks for url of paywall articles being posted. When it happens, the python script logs to the corresponding website and rehost the content of the article on my server. Finally it posts a new link — with no paywall this time — in the chatroom.
This required asking my friends for their login information on the websites where I don't have a subscription, but it made the whole process of sharing articles way easier.
Common practice, but still illegal. Recommend watching https://en.wikipedia.org/wiki/The_Internet%27s_Own_Boy if you haven't. Similar use case here.
Not in Python (yet), but I recently built a tiny Bash script to extract some useful information from Java Flight Recorder logs (by essentially converting them to XML and doing some XPath queries).
However, covering all corner cases in Bash is such a pain that I might just migrate it all to Python someday.
However, covering all corner cases in Bash is such a pain that I might just migrate it all to Python someday.
European Govt Bond market making including order management on inter dealer broker electronic markets, risk driven automated hedging with Eurex futures and ax driven quoting on dealer to client markets. Ten years ago. Can't share the code I'm afraid as it's proprietary.
I had a script that each week would process the schedule of a local movie theater (which only shows non-recent movies), fetch my individual score for each from Criticker[1] and send me an email with the entries scoring over 70%.
I managed to watch some classics on a Big Screen thanks to it!
I managed to watch some classics on a Big Screen thanks to it!
not a huge automation task, but i wrote a python script which plays songs from terminal with filters like title,genre and artist etc.
https://github.com/zuck007/xplay
Is Python a requirement? I mostly do my automation in bash. I use Python a lot for throwaway scripting for example if I want to go through a file and e.g. calculate/count/remove/add various points of it. It's a shortform of automation, I guess.
I was tired of going through arxiv's rss feeds so I wrote a script that sends me a mail if there is anything interesting.
http://myarxiv.org/
http://myarxiv.org/
EVERYTHING!!!
Buying and selling Bitcoins on btc-e.com. I thought about automating trollbox trolling too.