Amazing! GitHub actions to compute a giant skim matrix is an incredible hack.
I pretty regularly work with social science researchers who have a need for something like this... will keep it in mind. For a bit we thought of setting something like this up within the Census Bureau, in fact. I have some stories about routing engines from my time there...
Worth noting that there already exists an ecosystem of these sorts of contracting firms (nava, skylight, truss I believe, forgetting even more) that, basically, pitch themselves as the antidote to beltway banditry
People from the federal civic tech nexus started them up over the past decade as they termed out of 18F, USDS, and PIF
Hi HN! I've spent a couple of months fiddling with OCR and wanted to share some of my findings.
The approach I share here (fine-tuning recent deep learning models) is the first one that's gotten me anything resembling high-quality OCR on these particular noisy historical documents. OCRing these has been something of a white whale for me for several years (except, a white whale that I have spent comparatively little time on).
At this point I think I am reasonably competent in OCR, but no expert... Curious for your thoughts.
Yeah, I think MS' is the best out there, but agree that the usability leaves something to be desired. 2 thoughts:
1. I believe the IR jargon for getting a JSON of this form is Key Information Extraction (KIE). MS has an out-of-the-box model for this. I just tried the screenshot and it did a pretty good (but not perfect) job. It didn't get every form field, but most. MS sort-of has a flow for fine-tuning, but it really leaves a lot to be desired IMO. Curious if this would be "good enough" to satisfy the use case.
2. In terms of just OCR (i.e. getting the text/numeric strings correct), MS is known to be the best on typed text at the moment [1]. Handwriting is a different beast... but it looks like MS is doing a very good job there (and SOTA on handwriting is very good). In particular, it got all the numbers in that screenshot correct.
Figures, too! Yeah you could write some logic essentially on top of a library like this, and tune based on optimizing for some notion of recall (grab more surrounding context) and precision (direct context around the word, e.g. only the paragraph or 5 surrounding table rows) for your specific application needs.
Using the models underlying a library like this, there's maybe room for fine-tuning as well if you have a set of documents with specific semantic boundaries that current approaches don't capture. (And you spend an hour drawing bounding boxes to make that happen).
Funnily enough, this is another great tactic for getting emails returned (looping in someone with more leverage than you or asking them to follow up for you)!
We should talk! I do work on automatically coding products for a shipping survey at the Census Bureau. One of the earliest production uses of ML here at Census :)
(I am no expert in the analytic underpinnings of the beta distribution or precisely how it is the conjugate prior to the binomial -- or, rigorously speaking, what conjugate prior means -- but the formula here lines up with his formula :P )
Thanks for the feedback! I wasn't particularly dedicating this to new users. This solves a problem for me, an experienced *nix user. I'm not even sure it /would/ solve a problem for new users (arguably ninite doesn't either) - you need to know what packages you care about in order to have an interest in downloading them.
This site gives me an interface to say, "oh yeah, I /do/ need VLC on this laptop, come to think of it", rather than having to lazy-evaluate that when I have a video downloaded and realize I don't currently have a means of playing it.
I spend a fair bit of time doing DevOps work, and this isn't meant to be some sort of configuration management or provisioning tool. This is for desktop / dev users, which is, I think, quite a different use case (but an increasingly relevant one). I have a dotfiles repo, which is to say I'm familiar with the "document things in scripts and repos" game - but it's not clear to me that something like a script-as-documentation is even a useful process to go through when any given computer I set up is going to be used differently. That laptop playing VLC is very different than the web dev box I spin up to mess around with a new project.
The motivation was basically a version of ninite[1] for Linux. So very desktop-focused. I actually got in touch with the ninite folks, who used to run a Linux installer [2]. They said there wasn't enough interest and package managers are "good enough." I respectfully disagree on that point - I think that having the ability to point-and-click for the core, most important packages you want to install is really useful when you're spinning up a new desktop from scratch.
I was running into this problem after basically doing ad-hoc setup for every new machine I turned on. Sure - I could write a shell script / list of packages and save it to my dotfiles repo, but where would the fun be in that? :)
(Plus, package lists on every machine are different)
A few good ones here - another really interesting, current initiative is Bolsa Família, a Brazilian assistance program that functions similarly to a basic income subsidy - the main difference is that it requires families to demonstrate up-to-date immunizations and good school attendance rates for children.
The issue here is the spec - no runtime errors at the source code level is, as I understand, a completely different specification than end-to-end implementation correctness.
In particular, the Muen Kernel report itself [1] explains:
By implementing the kernel in SPARK and proving the absence of runtime errors, we have
shown that the kernel is free from exceptions. While these proofs provide some evidence to
the correctness claim of the implementation, the application of these particular formal methods
do not provide any assurances beyond the error free execution of the kernel. Proving functional
properties such as the correspondence of the scheduler to a given formal specification is necessary
to further raise the confidence in systems based on the Muen kernel.
In other-words, we don't yet have formal confirmation that this thing actually does what we might expect it to - just that its execution is bug-free.