I use it on Linux and think it's great. My laptop has a screen with some crazy-high DPI and a monitor which doesn't. Changing the font sizes in settings to suit has never left me with a poorly rendered view.
Amen. Even with those 5k word monsters it's brutally hard. Andreas Fertig's cpp-insights is really helpful, when is able to complete the coroutine transform.
FWIW, I think a useful addition would be for compilers to output the intermediate source code, so you can reason more easily about behaviour and debug into readable code.
I don't find this surprising at all. Humans are tool-users, and valuing an object's utility and experiencing a feeling of something like loss when it's neglected or loses efficacy would seem to be an advantageous trait.
It's entirely possible, have done it at few times. For example, the `fby` verb[?] annoyed me one too many times, so I pulled it apart to see what was going on. In contrast to json.k it's quite short. I usually split each separable idea into a new line and introduce a bunch of new variables to track state that would otherwise be passed from right to left. Lengthy end-of-line comments are my chosen way of understanding q or k when I come back to anything later.
> I just don’t understand where they went wrong on curtailing free speech rights of their citizens, privacy rights, etc.
Security establishment's innate desire to read and listen to absolutely everything. Blair/Bush's war on terror. Id card proposals. Smart phone use sky rockets. Supposed E2E comms. Hate speech. Something must be done! Right wing policies on pretty much everything cause more protest. Tories criminalise (*some types of) protest. Labour government raises TCN to Apple.
Surely any peacetime military contract in the US (or, perhaps anywhere) is just a signal to get snouts in the trough and start hoovering up the cash? While wartime begets its own kind of profiteer, my sense is that it could only be done when businesses put the national interest first.
I've found that the challenge is to "think in vector operations" rather than of iterating over the same data. The tricky part is figuring out how to get an operator to do the right thing over an array of stuff on the left hand side and this list/bag/etc of arguments on the right
I'm not sure if anyone's yet suggested that you embed your code as a library for KDB, that it could load dynamically? There's some pointer walking fun involved which Rust may _hate_ but it's not that hard and after that you'd be left with a the numerical arrays you're interested in.
Hi, KDB is used for this kind of thing in probably all the Tier 1 banks, or has been at some point. I'm surprised that you seem to have been given so little help by the KDB guys as it really matters how you store your data. That's informed by the data itself and the access patterns you're likely to use. When you say you're saving them as complex-ish models it makes me think that it may not be optimal for KDB to process.
KDB is in some respects as dumb as a bag of rocks. There is no execution profiler nor explain plan, no query analysis at all. When running your query over tabular data it simply applies the where-clause constraints in-order, passing the boolean vector result from one to the next, which refines the rows still under active consideration. It's for this reason that newbies are always told to put the date-constraint first, or they'll try to load the entire history (typically VOD.L) into memory.
KDB really is very fast at processing vector data. Writing nested vectors or dictionaries to individual cells could easily be slowing you down; I've heard of one approach which writes nested dictionaries into vectors with the addition of a column to contain the dictionary keys. Then you get KDB to go faster over the 1-D data, nicely laid out on disk. You really do need to write it down in a way that is sympathetic to the way you will eventually process it.
You can create hashmap indices over column data but the typical way of writing down equity L1 data is to "partition by date" (write it into a date directory) and "apply the parted attribute" to the symbol column (group by symbol, sort by time ascending). Each of the remaining vectors (time, price, size, exchange, whatnot) are obviously sorted to match and finding the next or previous trade for a given symbol is O1 simplicity itself. I've never worked on options data and so can't opine on the problems it presents, but if you've been asked to write this down without any help, then it's pretty "rubbish" of the KDB guys in your firm. You have asked for help, right?
I'm really going on a bit but just a few more things:
- KDB will compress IPC data — if it wants to. The data needs to exceed some size-threshold and you must, I think, be sending it between hosts. It won't bother compressing it to localhost, at least, according to some wisdom received from one of the guys at Kx, many moons ago. The IPC format itself is more or less a tag-length-value format, and good enough. It evolved to support vectors bigger than INT32_MAX a while ago but most IPC-interop libraries don't tend to advertise support for the later version that lets you send silly-big amounts of data around, so my guess is you may not want to load data out of KDB a day at a time. Try to do the processing in KDB!
You said you're scared to do queries that return a lot of data, and that it often freezes. Are you sure the problem is at the KDB end? This may sound glib but you wouldn't be the first person to have been given a VM to do your dev-work on that isn't quite up to the job. You can find out the size of the payload you're trying to read by running the same query with the "-22!" system call. It'll tell you how many bytes it's trying to send. Surely there's help to be had from the KDB guys if you reach out?
- I'm confused by the use of the term "data lake": to me this includes unstructured data. I'm not sure I'd ever characterise a KDB HDB as such.
- If your firm has had KDB for ages there's a good chance it's big enough to be signed up to one of the research groups who maintain a set of test-suites they will run over a vendor's latest hardware offering, letting them claim the crown for the fastest Greeks or something. If your firm is a member you may be able to access the test-suites and look at how the data in the options tests is being written and read, and there are quite a few, I think.
- KDB can scale horizontally. It can employ a number (I forget whether it's bounded) of slave instances and farm-out work. I think I read that the latest version has a better work-stealing algo. It's often about the data, though: if the data for a particular symbol/date tuple is on that one server over there, then you're probably better off doing big historic-reads on that one server alone. I doubt very much you're compute-bound or you'd have told us that your KDB licence limited you to a single or N (rather than any number) of cores.
- Many years ago I was told never to run KDB on NFS. Except Solaris' NFS. I have no idea whether this is relevant ;)
What are the downsides of keeping the public fields in a struct which is in turn embedded in a private struct created by the library? Inferring the "outer offset" could get irritating but it depends on your tolerance of additional complexity (in every public function...).
Agree. With so many unknown brands on Amazon they had become a go-to, but I plan to spend my pounds elsewhere. A tiny contribution to try to effect change.