So far, we haven't had any particular need for native support for parquet files. This type of import can be done satisfactorily via RayPy, and there is also a plugin that allows you to access kdb+ instances.
I am pleased to announce that the RayforceDB columnar database, developed by Lynx Capital Partners, is now an open source project.
RayforceDB is an implementation of the array programming language Rayfall (in the same way that kdb+ is an implementation of k/q), which inherits the ideas embodied in k and q. However, RayforceDB uses Lisp-like syntax, which, as our experience has shown, significantly lowers the entry threshold for beginners and also makes the code much more readable and easier to maintain. However, the implementation of K syntax remains an option for enthusiasts of this type of notation.
RayforceDB is written in pure C with a minimum of external dependencies, the executable file size does not exceed 1 megabyte on all platforms (tested and actively used on Linux, macOS, and Windows), and the executable file is the only thing you need to deploy to get a working instance. Additionally, it is possible to compile to Webassembly and run in a browser. However, in this case, automatic vectorization is not available.
RayforceDB was developed by a company that provides infrastructure for the most liquid financial markets. As you might expect, the company has extremely high requirements for data processing speed. The effectiveness of the tool can be determined by visiting the following link: https://rayforcedb.com/content/benchmarks/bench.html
The connection with the Python ecosystem is facilitated by an external library, which is available here: https://raypy.rayforcedb.com
RayforceDB offers all the features that users of columnar databases would expect from modern software of this kind. Please find the necessary documentation and a link to the project's GitHub page at the following address: https://rayforcedb.com
Here's a new kdb+ like time-series database and programming language. This new implementation is being tried to avoid well known kdb+ drawbacks (including its famous price tag) and to add many new features now missed by many kdb+ programmers. And, well, it's going to become an open-source project soon.
every instance can export its monitoring information as 9p virtual filesystem, which easily can be mounted from outside
we debug server code in BEAM, Erlang on Xen is a deployment platform, if instance crashed we simply restart it.
intruder has very few chances to find breaking in beneficial - there's no shell inside which gives only minimal chances to snatch control, instance simply will crash. also, having of no OS leaves no holes to dig deeper
You're correct, it's exokernel-like approach.
We gain simplicity, much better resource consumption characteristrics, manageability at large scale and much better instance mobility. And, well, security.
BEAM ("official" Erlang VM) starts much longer. Erlang On Xen has it's own VM (written from scratch), it was designed for run without OS and to srtart very fast, particularly.
When you sat in container, you're limited with libraries of host OS, and there still will be some issues with migration and resource management (on large scales)
yes, very soon we will start to give attention to writing application over this platform. there is some work still needs to be done in management stack, but it's manageable amount of work.
I'm afraid, only few people are familiar with internals of Erlang VM, especially, if that VM is totally different from BEAM. Thus, there's no reason to opensource it, at least yet. Though, build.erlangonxen.org is open, contains the most recent stable version, and free to use.
Current implementation runs in Xen PV domain and uses Xen's hypercalls. There are no strong dependencies, though, Ling easily can be ported for other hypervisors. Xen is the priority, since it's the plaform for all major public clouds.
Scalable MapReduce, for example, super scalable web servers tolerant for any possible spikes of load, including abuse-resistant hostings, personal virtual appliances... there are many possible usecases, indeed