Wow! Thanks for the CUSPARSE.jl shoutout (I'm the maintainer of that package)!
I must take issue with "near complete" - CUSOLVER.jl, my other CUDA wrapper, is missing a lot of the RF (refactorization) functionality and doesn't really have a pretty high level API yet. The doc and testing situation is also pretty bad. Everyone else's packages in JuliaGPU (the overarching GPU org on GitHub) are in a much better state.
You are right, though, that writing CUDA bindings in Julia is very easy - so easy that I can do it! It's also thanks to packages like Clang.jl, which make it easy for us to automate the procedure of wrapping the low-level interfaces.
Finally I must add the caveat that although I should test CUSPARSE.jl and CUSOLVER.jl with MPI (and GPUDirect) I've been extremely busy recently and not able to do so. If anyone wants to help out in this regard I would be very appreciative!