It's actually base 10^7 :) The reason for that is that at the lowest level, the result of any operation on a single digit needs to fit in a JavaScript Number. Since you sometimes have to multiply two digits and 64-bit floats have about 15 digits of precision, the largest power of 10 that will never lose precision is 10^7 (because 10^7 * 10^7 = 10^14).
Of course, it would be more efficient to use a power of two, but using a power of 10 makes converting back and forth between strings (in decimal) quite a bit faster. In my original use case that was a large factor, so it's what I went with.
It's hard to say exactly how much more memory it takes, but yes, there is quite a bit of overhead, especially for small numbers. But that's the trade-off you have to make in order to support the large numbers. Obviously you wouldn't use BigIntegers for everything.
Of course, it would be more efficient to use a power of two, but using a power of 10 makes converting back and forth between strings (in decimal) quite a bit faster. In my original use case that was a large factor, so it's what I went with.