CLI tool for converting txn statements from Singaporean banks into a sane format(github.com)
github.com
CLI tool for converting txn statements from Singaporean banks into a sane format
https://github.com/ajensenwaud/dbs-tools
13 comments
Hello. I am an expat based in Singapore. I use GnuCash for managing my personal finances (it's a great tool). I was unable to find a way to easily import bank statements (CSV files with transactions) into GnuCash without all sorts of weird conversion errors. To streamline the process, I wrote a Perl-based tool for automating the conversion into something GnuCash can easily import. You might find this useful (or not!). Have fun!
I have about 15 Perl-based converters for various banks I use, from parsing Revolut's CSV exports to making sense of copying and pasting the statement from Citibank. Someone should really make a repo to collect these, although hopefully as more banks sign up to Salt Edge and friends it'll be less of an issue
Off topic - DBS is hilarious in Singapore, the "World's best digital bank" 4 years in a row (forget the years) - who did they pay-off for that title.
You can't even export bank transactions older than 6-months from your account in CSV. You have to download - individually mind you - pdf statements and then good luck. This is the worlds best digital bank?
You can't even export bank transactions older than 6-months from your account in CSV. You have to download - individually mind you - pdf statements and then good luck. This is the worlds best digital bank?
Yeah banks here are so far behind what real neo-banks like Monzo are doing.
I left Monzo in the UK and came to Singapore to join Bigpay. We're building a proper digital challenger.
1.1m customers in Malaysia and about 1/10th that in Singapore.
I left Monzo in the UK and came to Singapore to join Bigpay. We're building a proper digital challenger.
1.1m customers in Malaysia and about 1/10th that in Singapore.
My main pet peeve with Monzo is that it is App only which can be quite inconvenient. I wish they also offered a webpage. I hope you will consider that with Bigpay too.
DBS ain't too bad imho
DBS ain't too bad imho
Most countries have incredibly sub-par banking systems that work against the consumer. The UK is really on another level, though many countries are catching up.
I can't speak for Monzo but Revolut have presence in Singapore, and the product is incredibly feature rich.
I can't speak for Monzo but Revolut have presence in Singapore, and the product is incredibly feature rich.
> DBS is hilarious in Singapore, the "World's best digital bank" 4 years in a row (forget the years) - who did they pay-off for that title.
Euromoney [1], which, on first impression, seems to be one of those content farms that make most of its actual revenue off selling tickets to award ceremonies.
[1] https://www.euromoney.com/
Euromoney [1], which, on first impression, seems to be one of those content farms that make most of its actual revenue off selling tickets to award ceremonies.
[1] https://www.euromoney.com/
The six months limit is so hilarious for a so-called “digital” bank. They still use old mainframe at the back so they are obviously constrained a lot my batch processing still. It takes 2-3 days for most card payments to be cleared, which is weird given PayNow and other real time payments work in real time.
On the note of PDF statements, I found that Okular on Linux was able to copy the table format that KBC Ireland use in their PDFs. That at least let me paste into a spreadsheet, massage the columns slightly, and then save as CSV for import into KMyMoney.
[deleted]
Their deposits API ain't too bad. You can create bank accounts, get your transactions etc.
Thanks for the interest. It’s not really a super sexy project but it is useful for me. In addition, I really like writing software in Perl. I hope there will be a real resurgence of interest in classic Unix and Perl after people start copping it from the the towering complexity of other stacks …
Btw, the script currently cannot install Perl package dependencies. I will add code soon to make that happen (MakeMaker) so it becomes self-installing.
Feel free to add stuff for other banks!
Btw, the script currently cannot install Perl package dependencies. I will add code soon to make that happen (MakeMaker) so it becomes self-installing.
Feel free to add stuff for other banks!
I did a similar thing for Volksbank (a German bank) statements using awk:
https://github.com/airblade/volksbanker-awk/blob/main/vba
https://github.com/airblade/volksbanker-awk/blob/main/vba
Perl one-liners are fine. But I’m not interested in writing Perl beyond that due to the towering syntax complexity.
This is exactly the kind of projects I like. The stuff where people make something to solve a personal problem.
There's a few things that can be tidied up though:
1. Your README instructions references `bin/import_csv` but the file name in the repo is `import_bank_csv`
2. You have a bunch of `use` statements in your `import_bank_csv` file which are not being used. Weirdly including `switch` yet you have a `elsif` block.
3. `import_bank_csv` also has a stray `1;` at the end of it. This is only really needed for Perl modules.
4. In addition to 3, some of your Perl modules do have `1;` and some don't. There isn't any consistency there.
I do share your love for Perl even though it definitely has a lot of hidden traps (I'd even go as far as to say it's not a well designed language by modern standards). But the fact I still hear about Perl scripts I'd written for jobs 10+ years previously, and running on production systems without any additional maintenance, says a lot about the stability of the Perl ecosystem.
There's a few things that can be tidied up though:
1. Your README instructions references `bin/import_csv` but the file name in the repo is `import_bank_csv`
2. You have a bunch of `use` statements in your `import_bank_csv` file which are not being used. Weirdly including `switch` yet you have a `elsif` block.
3. `import_bank_csv` also has a stray `1;` at the end of it. This is only really needed for Perl modules.
4. In addition to 3, some of your Perl modules do have `1;` and some don't. There isn't any consistency there.
I do share your love for Perl even though it definitely has a lot of hidden traps (I'd even go as far as to say it's not a well designed language by modern standards). But the fact I still hear about Perl scripts I'd written for jobs 10+ years previously, and running on production systems without any additional maintenance, says a lot about the stability of the Perl ecosystem.
Thank you for the feedback!
1. Spot on, I have fixed the README instructions :) It was written in a rush. Apologies.
2. You are right. I have removed the unused use statements. I had the switch statement imported as I wanted to try out the 'given ...' construct but it was easier just to use if-elsif.
3. Good point, removed.
4. Yes, absolutely. Thank you!
Having spent almost 15 years away from Perl in various other languages, I actually tthink Perl is quite elegant. You can do some pretty neat code in a very succinct way, as long as you keep your code structured and avoid too many clever things. The Perl stack definitely allows you to hammer a nail into your own foot, but it doesn't mean you have to. :)
1. Spot on, I have fixed the README instructions :) It was written in a rush. Apologies.
2. You are right. I have removed the unused use statements. I had the switch statement imported as I wanted to try out the 'given ...' construct but it was easier just to use if-elsif.
3. Good point, removed.
4. Yes, absolutely. Thank you!
Having spent almost 15 years away from Perl in various other languages, I actually tthink Perl is quite elegant. You can do some pretty neat code in a very succinct way, as long as you keep your code structured and avoid too many clever things. The Perl stack definitely allows you to hammer a nail into your own foot, but it doesn't mean you have to. :)
BTW, I have now significantly improved the documentation and also added installation instructions. :)
Hi, I'm Singaporean and this is great! I use a similar tool for YNAB, though that requires a bit of manual work: https://github.com/aniav/ynab-csv
Unfortunately, there doesn't seem to be a way to export data from GrabWallet.
Unfortunately, there doesn't seem to be a way to export data from GrabWallet.
Brings back memories of why I liked Perl so much back in the 90's. Nice work solving this.
I'm curious about this bit:
It feels like you could safely shorten that down to:
I'm curious about this bit:
my $debitAmount = length(trim $fields->[4]) > 0 ? trim($fields->[4]) + 0.0 : 0.0;
It feels like you could safely shorten that down to:
$debitAmount=$fields->[4]+0;
Thanks for the input! It is indeed a bit annoying I have to do that. Initially my code had what you suggested, but it caused a lot of "type casting" (in lack of a better Perl term) errors:
Argument " " isn't numeric in addition (+) at lib/SGBankUtils/DBS.pm line 25, <$data> line 111.
Ah, it seems to be a warning, so it works, but is noisy. Something like what you're doing is needed then...I had thought space coercing to zero didn't do that.
That or a printf/sprintf: https://www.tutorialspoint.com/perl/perl_printf.htm
Had the same tool for years but never made it public since there was. no expectation of any interest.
At some point I was thinking of dealing with HSBC PDF statements, but they supply them in non-split form so you need to OCR it first and then process. Was too much hassle at the moment
At some point I was thinking of dealing with HSBC PDF statements, but they supply them in non-split form so you need to OCR it first and then process. Was too much hassle at the moment
HSBC Singapore (and I believe Australia) have a facility to export transactions for an account to CSV. YMMV in other countries but it appears standardised. The output format is ugly-as, of course.
I’m in Singapore as well. Eventually settled on CSV, ugly as it is as you’ve mentioned
I've written a bare bones script to get historical transactions out of the pdfs statements. It was a PINA.
Reminds me of https://plaintextaccounting.org/
Shameless plug for a tool our team have recently launched to convert bank statements into spreadsheet format. Free for 30 pages: https://oscars-on-it.demarq.com/
I had the exact same problem with the exact same bank, HSBC. It led me into creating a web app that extracts transactions from PDF files.
https://bankstatementconverter.com/
https://bankstatementconverter.com/
I have worked on software which does the opposite. Convers sane json into horrible "narrative" format
Please do one for Binance.US
I am not a US citizen so unfortunately cannot help you there ... is the problem the same on their international one?