Java is better than C++ for high speed trading systems(news.efinancialcareers.com)
news.efinancialcareers.com
Java is better than C++ for high speed trading systems
https://news.efinancialcareers.com/uk-en/3004875/low-latency-java-trading-systems
482 comments
Yeah. Languages like C# and Java offer you so much facilities and QOL features compared to C and C++, but in practice you have to give up most of those facilities in order to have it perform almost on par with lower level languages; Even then I think it's still acceptable since you still have the tooling, the facilitated debugging, etc. C# for example, is keeping up with this trend of using higher level languages to code things closer to the metal and with more memory management awareness, and offering more and more features to make that kind of code more performant and with less heap allocations, since C# 7.0 or so: Span<T> for example is a revolution; But I use things like stackalloc, ref, fixed, Unsafe.CopyBlockUnaligned, &MemoryMarshal.GetReference etc, on a daily basis to get that extra performance where it matters;
That was exactly what made me pick C++ over C already in 1992, ability to code at all levels in a modern programming language.
Nowadays I rather use Java and .NET, which have been picking up on the features that should have been there at v1.0, like the ones you mentioned, but hey better later than never. :)
And on the few cases where it isn't enough, I can code a tiny library in C or C++, no need to throw everything away.
Nowadays I rather use Java and .NET, which have been picking up on the features that should have been there at v1.0, like the ones you mentioned, but hey better later than never. :)
And on the few cases where it isn't enough, I can code a tiny library in C or C++, no need to throw everything away.
So Java is nearly as low-latency as C if you're willing to write your Java as if it were C?
It can be done. It is just not very practical, you are better off writing C.
The only situation I can think of where Java would be right fit is where you have an application with a lot of logic and just a small part that needs to be highly optimized.
The only situation I can think of where Java would be right fit is where you have an application with a lot of logic and just a small part that needs to be highly optimized.
Most trading systems I work on only need a very small portion to be highly optimised. As long as most of the code is pretty efficient that is fine.
Real programmers can write FORTRAN in any language.
No. Java is not low-latency by any stretch. Java advocates would like to think it is, until a Bank, like BoA spends $8 million on Java in 2014, just to have it fail compliance testing. Entire project was scrapped.
US Navy thinks otherwise,
https://www.ptc.com/en/blogs/plm/ptc-perc-virtual-machine-te...
https://www.ptc.com/en/blogs/plm/ptc-perc-virtual-machine-te...
250ms is not real-time, nor high frequency.
Please, get informed on what real-time is before you comment (in general it is good idea to understand a concept before you place confident comments like yours).
https://en.wikipedia.org/wiki/Real-time_computing
Real-time has nothing to do with performance and everything to do with predictability and deadlines.
People are put off when they find out that real time versions of Java or Linux are actually slower than the regular ones. Until you think about it a little bit and if they were faster there would not be multiple versions.
Real-time versions of products are created because sometimes performance is achieved by means of stochastic processes or amortizing costs and you need to remove these optimizations to be able to support deadlines.
Real-time programming emphasizes ability to calculate how long an operation will take as more valuable than raw performance.
https://en.wikipedia.org/wiki/Real-time_computing
Real-time has nothing to do with performance and everything to do with predictability and deadlines.
People are put off when they find out that real time versions of Java or Linux are actually slower than the regular ones. Until you think about it a little bit and if they were faster there would not be multiple versions.
Real-time versions of products are created because sometimes performance is achieved by means of stochastic processes or amortizing costs and you need to remove these optimizations to be able to support deadlines.
Real-time programming emphasizes ability to calculate how long an operation will take as more valuable than raw performance.
250ms could easily be real time in the context of a given system.
Real time doesn't (necessarily) mean fast, it means the latency is bounded.
Real time doesn't (necessarily) mean fast, it means the latency is bounded.
It might not be high frequency, but it certainly is real-time, unless you are implying that the agencies that give this kind of government certifications are all a bunch of fools.
Doesn't real time just mean that you can guarantee to react within certain time limits? In principle, those time limits could be hours or days..
Yes, but those limits are part of certification processes like those done by companies like https://www.highintegritysystems.com.
Usually playing with numbers in these kind of deployments isn't part of the package.
Usually playing with numbers in these kind of deployments isn't part of the package.
The trading systems we work on have a wire-to-wire latency of 20 - 50 microseconds on the 99.9%ile. It really depends on what you are looking to achieve.
I think this may not due to programming language, but beyond that.
You're really just paying for the jvm overhead at that point. 20-30% seems in the right ballpark.
I'm so happy this is the top comment. I'm saving this as it's well written and highly qualified. Thanks for sharing.
> So I still prefer working low latency in C as it is more natural, native solution for managing problem where you absolutely need to control memory layouts, preallocated object pools, NUMA, compilation of decision trees to machine code, prefetcher, etc.
Out of curiosity, what was the rationale for writing the system in Java? Were the tradeoffs not obvious until too late, was the choice of language dictated by higher-ups, etc.?
Out of curiosity, what was the rationale for writing the system in Java? Were the tradeoffs not obvious until too late, was the choice of language dictated by higher-ups, etc.?
The rationale was there were no developers available with skills to work on a complex Common Lisp application.
Projects with limit time and resources can be completed faster with a high level language giving you more time to optimise it. A well optimised Java application can perform better than a poorly optimised C++ application.
Probably the usual suspect, it’s easier to find engineers that know Java than C/C++.
And it’s a foolish assessment. Finding Java programmers who can navigate all the hoops of low latency Java is more difficult than finding a C programmer who can do the same with C.
There is couple of reasons for this.
If you choose Java for your project it usually means you don't care a lot about latency and performance (you care some, but not all that much compared to, for example, linux kernel development). You care more about managing larger codebase, having easy access to whole lot of frameworks and libraries, good development tools and easy development / deployment process.
Developers who have experience with Java are inevitably people who have mostly had experience with those kinds of projects of which I would say most are corporate backend systems or Android applications. It really is difficult to find any other projects.
On the other C is inflicted pain of having much harder development environment for the size of the project. You only use low level language like C when you have a specific need. Maybe you need to develop Set Top Box software, or video codec, or some device controller, or high performance quant library, that kind of stuff.
So it isn't really fault of Java developers to not know the stuff. It is natural result of Java being suited to a different type of projects than C is.
If you choose Java for your project it usually means you don't care a lot about latency and performance (you care some, but not all that much compared to, for example, linux kernel development). You care more about managing larger codebase, having easy access to whole lot of frameworks and libraries, good development tools and easy development / deployment process.
Developers who have experience with Java are inevitably people who have mostly had experience with those kinds of projects of which I would say most are corporate backend systems or Android applications. It really is difficult to find any other projects.
On the other C is inflicted pain of having much harder development environment for the size of the project. You only use low level language like C when you have a specific need. Maybe you need to develop Set Top Box software, or video codec, or some device controller, or high performance quant library, that kind of stuff.
So it isn't really fault of Java developers to not know the stuff. It is natural result of Java being suited to a different type of projects than C is.
I'm curious, what role did Common Lisp play in the original version of your trading system?
Common Lisp was doing all the "high level" stuff. No CL code was on the actual critical path but it still was responsible for some critical things that would be difficult otherwise.
For example, I had piece of Common Lisp code optimize and compile decision trees to machine code. These were used, for example, to check whether the trade is allowed to go to market. This had to be very optimized as it was running in parallel to constructing the message and had to deliver verdict before the message was actually created to have a chance at actually stopping anything from reaching exchange. This was my solution to have order verification in essentially zero time (the cost was single branch instruction that would only be mispredicted if the verdict was to stop the order).
Another piece of Common Lisp generated machine code to parse XDP messages coming from exchange. The spec for these was in XML, Common Lisp macros created extremely optimized machine code that used absolute minimum instructions to parse out the information we were interested in without having to write all that code manually. I actually used concept from Practical Common Lisp book for this (http://www.gigamonkeys.com/book/practical-parsing-binary-fil...). Thanks, Peter Seibel.
The entire application was essentially a Common Lisp application that used FFI to call a library with optimized ANSI C code. Some Common Lisp code was running in parallel to constantly update things like decision trees that required regular recompiling based on market situation but most was really just to setup everything and give control to the C part.
The application did not talk to operating system (Linux) after it started (no syscalls at all) and used full kernel bypass to talk to NIC.
For example, I had piece of Common Lisp code optimize and compile decision trees to machine code. These were used, for example, to check whether the trade is allowed to go to market. This had to be very optimized as it was running in parallel to constructing the message and had to deliver verdict before the message was actually created to have a chance at actually stopping anything from reaching exchange. This was my solution to have order verification in essentially zero time (the cost was single branch instruction that would only be mispredicted if the verdict was to stop the order).
Another piece of Common Lisp generated machine code to parse XDP messages coming from exchange. The spec for these was in XML, Common Lisp macros created extremely optimized machine code that used absolute minimum instructions to parse out the information we were interested in without having to write all that code manually. I actually used concept from Practical Common Lisp book for this (http://www.gigamonkeys.com/book/practical-parsing-binary-fil...). Thanks, Peter Seibel.
The entire application was essentially a Common Lisp application that used FFI to call a library with optimized ANSI C code. Some Common Lisp code was running in parallel to constantly update things like decision trees that required regular recompiling based on market situation but most was really just to setup everything and give control to the C part.
The application did not talk to operating system (Linux) after it started (no syscalls at all) and used full kernel bypass to talk to NIC.
> did not talk to operating system (Linux) after it started (no syscalls at all) and used full kernel bypass to talk to NIC
That's really interesting... How does it work? How can a user space program talk to the NIC directly without using system calls? Shared memory?
That's really interesting... How does it work? How can a user space program talk to the NIC directly without using system calls? Shared memory?
[deleted]
Some Googling suggests https://blog.cloudflare.com/kernel-bypass/
A lot of NIC manufactures provide either some hooks into something like DPDK or simple LD_PRELOADable libraries that offload the network syscalls (socket, poll, recv, etc.) into a library that talks to the NIC directly using mmap'd IO I presume.
One interesting approach I heard about for doing garbage collection in finance is to get computers with a shitload of RAM and just turn off GC. At the end of the trading day you can reclaim memory by killing the process. Not sure how common that strategy is.
That is exactly what I did. It was much less elegant and performant, though, than my initial solution with Common Lisp and C which basically allocated buffers in memory and mapped them with huge pages so that memory access don't ever require switch to kernel for mapping.
We also had custom Intel CPUs that are auctioned individually. And cutthrough switches that stream the IP packet immediately when they receive the IP header, before they receive the payload. I have calculated that if the message from exchange was a beam of light it would be delayed by less than a meter.
We also had custom Intel CPUs that are auctioned individually. And cutthrough switches that stream the IP packet immediately when they receive the IP header, before they receive the payload. I have calculated that if the message from exchange was a beam of light it would be delayed by less than a meter.
> would be delayed by less than a meter
That's just latency to your NIC, right? 1 meter ~ 3 ns ~ 9 CPU cycles at 3 GHz, less than the integer pipeline depth last I checked.
That's just latency to your NIC, right? 1 meter ~ 3 ns ~ 9 CPU cycles at 3 GHz, less than the integer pipeline depth last I checked.
Latency through switch (ie. how much the switch added compared to if the signal traveled unimpeded in vacuum).
Decade ago.
Decade ago.
If that's your memory management strategy, then what do you actually gain by using Java?
May as well just use C and never free your memory. Lots of memory management issues go away if you never free memory.
May as well just use C and never free your memory. Lots of memory management issues go away if you never free memory.
Not all your services/code needs to have the lowest latencies so you can have a team which just needs to be able to translate the business requirements and a couple of people to optimise the critical code.
If you have multiple processes each with 24 GB of Eden space and you can produce 1 GB/hour of garbage in each service and still only GC once a day.
It works until it doesn't and you lose a ton of money
Sounds like it should be possible to wrap it all into some kind of library and set of guidelines. On a positive side you probably could have much more confidence in Java code, because lots of boring and hard to debug memory errors just not possible (but may be C professionals don't do those errors?).
The cost of environment not making it possible for you to make a mistake is always there, somewhere.
I think best hypothetical development environment would allow you to specify the tradeoff and choose whether you want strong typing, fully managed memory or ownership semantics.
I think best hypothetical development environment would allow you to specify the tradeoff and choose whether you want strong typing, fully managed memory or ownership semantics.
[deleted]
Was a specialized real-time JVM used for this? I know real time isn't necessarily low latency, but it sounds like both are desired properties in this sort of situation.
Realtime and low latency are two different properties. Realtime systems have timing concerns and can oftentimes be slower than non-realtime systems.
This is not strictly true.
Realtime means processing of at least some of the events and tasks in your application must be completed before a set deadline. For example, coin acceptor that detects what kind of coin you dropped in has x milliseconds to figure out if the coin is to be accepted or not. If it can't do it within the time you get unhappy customer.
Realtime systems don't have to be very performant, but they don't have something like 99th percentile latency shooting up. All events must be processed within deadline.
Low latency systems are soft realtime PLUS performance.
In a low latency trading system you have to respond to market within a deadline to have a chance to do something useful but if you don't normally it doesn't mean lives are lost, metal is bent, etc.
Realtime means processing of at least some of the events and tasks in your application must be completed before a set deadline. For example, coin acceptor that detects what kind of coin you dropped in has x milliseconds to figure out if the coin is to be accepted or not. If it can't do it within the time you get unhappy customer.
Realtime systems don't have to be very performant, but they don't have something like 99th percentile latency shooting up. All events must be processed within deadline.
Low latency systems are soft realtime PLUS performance.
In a low latency trading system you have to respond to market within a deadline to have a chance to do something useful but if you don't normally it doesn't mean lives are lost, metal is bent, etc.
Realtime requires low-latency. In trading, they require real-time, not low-latency.
Some people use Azul Zing but we typically use OpenJDK or Oracle JDK.
I’ve seen this sentiment before and worked on both a “low latency Java” team and low latency C++ teams.
I have some sympathy for the idea that the JVM is better since it means you won’t spend all your time chasing crash reports. The thing is, like another comment hinted at, is that this issue is generally more reflective of the environment you build in than the technology choice.
Here’s a good talk on the reasons for and limits of using C++ for low latency systems: https://m.youtube.com/watch?v=NH1Tta7purM
If I had to rank in terms of reliability of trading infrastructure I’ve used/worked on,
* Low Latency C / C++ infra at fully automated trading firm. Very much run by the programmers and quants and also fairly small. By far the fastest (near limits of what you could do) and also most reliable.
* Low Latency Java execution infrastructure. Pretty reliable, not that fast, had some issues with GC battling and manual memory management to avoid GC, etc. There was a pretty clear latency floor (still quite low) even when “doing everything right” that serious native infrastructure beat.
* C++ market making infra at a firm run by manual traders. It was by far the slowest and least reliable. Echos the experience of “spent hours debugging weird crashes”. The culture was very “A trader asked for this and needs it done yesterday. Also this refactoring business doesn’t sound like adding new features, drop it”.
What I saw is that if you hire people who have a good idea of what they’re doing and keep a culture of technical excellence, C++ is definitely a better choice if you care about latency. You really have to maintain a culture of high quality code, testing, and in general caring about the technology.
This is only really possible when all of the stakeholders are involved in the technology, or at least understand the benefits. Once you start down the path of “well this feature could be done a day quicker if...” this goes down the drain, and a few years later you find yourself getting run over on latency AND with an impossible to use trading system. It’s really the worst of both worlds.
I have some sympathy for the idea that the JVM is better since it means you won’t spend all your time chasing crash reports. The thing is, like another comment hinted at, is that this issue is generally more reflective of the environment you build in than the technology choice.
Here’s a good talk on the reasons for and limits of using C++ for low latency systems: https://m.youtube.com/watch?v=NH1Tta7purM
If I had to rank in terms of reliability of trading infrastructure I’ve used/worked on,
* Low Latency C / C++ infra at fully automated trading firm. Very much run by the programmers and quants and also fairly small. By far the fastest (near limits of what you could do) and also most reliable.
* Low Latency Java execution infrastructure. Pretty reliable, not that fast, had some issues with GC battling and manual memory management to avoid GC, etc. There was a pretty clear latency floor (still quite low) even when “doing everything right” that serious native infrastructure beat.
* C++ market making infra at a firm run by manual traders. It was by far the slowest and least reliable. Echos the experience of “spent hours debugging weird crashes”. The culture was very “A trader asked for this and needs it done yesterday. Also this refactoring business doesn’t sound like adding new features, drop it”.
What I saw is that if you hire people who have a good idea of what they’re doing and keep a culture of technical excellence, C++ is definitely a better choice if you care about latency. You really have to maintain a culture of high quality code, testing, and in general caring about the technology.
This is only really possible when all of the stakeholders are involved in the technology, or at least understand the benefits. Once you start down the path of “well this feature could be done a day quicker if...” this goes down the drain, and a few years later you find yourself getting run over on latency AND with an impossible to use trading system. It’s really the worst of both worlds.
The worst (messiest, laziest and most chaotic) engineer I know works at a brokerage, coding obscenely large and unreliable edifices (and in a number of languages, from C++ to Excel+VB)... He does what he is asked, and doesn't worry about telling them what they need. They never demand code that doesn't crash, they see that as a fact of life, and developers are an overhead. Basically someone else owns the risk, the brokers don't, so why should he?
People who don't like this style (or the code) leave.
The direction for quality has to come from the top and be supported by systematic mechanisms. In fact that would be a question to ask in interviews.
People who don't like this style (or the code) leave.
The direction for quality has to come from the top and be supported by systematic mechanisms. In fact that would be a question to ask in interviews.
^^This^^ is why you shouldn't call software developers "engineers" (disclaimer, am software developer).
"No-one ever explicitly specified that the bridge shouldn't fall down and kill everyone who was on it at the time", said no engineer, ever.
"No-one ever explicitly specified that the bridge shouldn't fall down and kill everyone who was on it at the time", said no engineer, ever.
I think it’s not that software engineers aren’t engineers; but more that they’re most equivalent to combat engineers — engineers operating under time-pressure and shipping-pressure, where the systems they build need to “work” (in the sense of a bridge getting people across a ravine) but may only need to work once; and where it may be perfectly fine/expected/required to need to “baby” the resulting engineered system along (i.e. letting a batch of people over the bridge, then closing it to traffic, examining the piles for faults, and shoring up the ones that are slipping; then reopening the bridge, in a cycle.)
It’s not that this type of engineering is “not engineering”; it’s that it’s engineering where the engineer themselves (and their ability to actively re-stabilize the system) is considered a load-bearing element in the design, such that the system will very likely fall apart once that element is removed.
Combat engineering is still engineering, in the sense that there are still tolerances to be achieved, and it’s still a disaster if the bridge falls over while it’s in active use for the mission it was built for. It’s just not considered a problem if it falls over later, once that mission is accomplished.
It’s not that this type of engineering is “not engineering”; it’s that it’s engineering where the engineer themselves (and their ability to actively re-stabilize the system) is considered a load-bearing element in the design, such that the system will very likely fall apart once that element is removed.
Combat engineering is still engineering, in the sense that there are still tolerances to be achieved, and it’s still a disaster if the bridge falls over while it’s in active use for the mission it was built for. It’s just not considered a problem if it falls over later, once that mission is accomplished.
you mean they're more like train engineers, or maybe to be charitable, like geordi laforge engineers, someone who runs an engine.
I think software developers have a range in terms of responsibility from plumbers/electricians to civil engineers. We could save the designation “software engineer” for someone who has to go through the requisite ethics and case study courses like all other engineers. Otherwise maybe programmer/developer. (I’m gatekeeping I guess)
I would think it's less about ethics and more about understanding consequences; being able to reason about what happens based on the choices made while implementing something.
Given the same conditions (i.e. people don't die) the same outcome would present itself. We have a history of bridges falling down, so much so that Wikipedia has a list on that: https://en.wikipedia.org/wiki/List_of_bridge_failures
A particular one that caught my eye: Cimarron River Rail Crossing Dover, Oklahoma Territory United States 18 September 1906 Wooden railroad trestle Washed out under pressure from debris during high water 4-100+ killed Entire span lost; rebuilt Bridge was to be temporary, but replacement was delayed for financial reasons.[8][9][10] Number of deaths is uncertain; estimates range from 4 to over 100.[11]
Emphasis mine. How is that different from software engineering?
A particular one that caught my eye: Cimarron River Rail Crossing Dover, Oklahoma Territory United States 18 September 1906 Wooden railroad trestle Washed out under pressure from debris during high water 4-100+ killed Entire span lost; rebuilt Bridge was to be temporary, but replacement was delayed for financial reasons.[8][9][10] Number of deaths is uncertain; estimates range from 4 to over 100.[11]
Emphasis mine. How is that different from software engineering?
The point isn't that bridges never fall down. The point is that engineers who build bridges know that it's their responsibility to do everything in their power to make sure it doesn't, even if no-one else around them seems to care about that, or specifically asked for it. That they should walk away from a project rather than sign off on one where they will not have the resources or support to do a competent job.
That said, all bridges have design specifications and limits. If someone drives a convoy of trucks carrying gold bullion over a bridge and exceeds its design max weight by a factor of 10, and it doesn't hold up, that's not the engineer's fault. If a bridge is designed for a geologically active area and is designed to withstand a quake 10x more powerful than the strongest that's ever been recorded in the area, or is predicted by seismologists, and the area gets hit by a 100x quake, that's not the engineer's fault.
And if a bridge designed to last say, 5 years, is neither re-certified for use, or just closed, when its intended lifespan is up then that's not the engineer's fault either.
But the engineer is responsible for finding out what the likely max load of a bridge will be, or how powerful the strongest quake will be, or how long its intended lifespan is - and for adding in safety margins on top of that. They can't just assume that any new bridge will only be needed for 5 years, just because that no-one told them that it will be needed for longer, and then claim "well, it was only temporary" afterwards.
That said, all bridges have design specifications and limits. If someone drives a convoy of trucks carrying gold bullion over a bridge and exceeds its design max weight by a factor of 10, and it doesn't hold up, that's not the engineer's fault. If a bridge is designed for a geologically active area and is designed to withstand a quake 10x more powerful than the strongest that's ever been recorded in the area, or is predicted by seismologists, and the area gets hit by a 100x quake, that's not the engineer's fault.
And if a bridge designed to last say, 5 years, is neither re-certified for use, or just closed, when its intended lifespan is up then that's not the engineer's fault either.
But the engineer is responsible for finding out what the likely max load of a bridge will be, or how powerful the strongest quake will be, or how long its intended lifespan is - and for adding in safety margins on top of that. They can't just assume that any new bridge will only be needed for 5 years, just because that no-one told them that it will be needed for longer, and then claim "well, it was only temporary" afterwards.
> The point isn't that bridges never fall down. The point is that engineers who build bridges know that it's their responsibility to do everything in their power to make sure it doesn't, even if no-one else around them seems to care about that, or specifically asked for it. That they should walk away from a project rather than sign off on one where they will not have the resources or support to do a competent job.
The law takes care of that though. The law and inspection. If it was up to the business, any engineers building bridges who would refuse to project it faster because of that would be fired.
The law takes care of that though. The law and inspection. If it was up to the business, any engineers building bridges who would refuse to project it faster because of that would be fired.
A thief thinks every man steals. An apathetic engineer thinks all engineers share his apathy.
The sarcastic/negative implication of your example is that software engineering today is where civil engineering was 100+ years ago.
Yes, but not because software engineers are worse human beings than civil engineers, which is all too often the undertone of these discussions.
Rather, because software engineering hasn't killed enough people to advance; or when it has, it wasn't obviously the culprit.
In which fields of software engineering do we find actual solid procedures and standards? Avionics. Medical hardware. Fields where the link between bug and death is as short as possible, and so the stakeholders have demanded solid engineering.
What are the consequences of GP's acquaintance writing poor code? Some trading firm becomes slightly less efficient at trading. Impossible to evaluate the net human loss from it (if it's a loss at all).
The civil engineering equivalent would be something like façade design or layout. If your building is ugly or confusing, it will annoy or waste the time of the people who live and work in it, but as long as it doesn't fall on their heads nobody is going to withdraw your certification.
Rather, because software engineering hasn't killed enough people to advance; or when it has, it wasn't obviously the culprit.
In which fields of software engineering do we find actual solid procedures and standards? Avionics. Medical hardware. Fields where the link between bug and death is as short as possible, and so the stakeholders have demanded solid engineering.
What are the consequences of GP's acquaintance writing poor code? Some trading firm becomes slightly less efficient at trading. Impossible to evaluate the net human loss from it (if it's a loss at all).
The civil engineering equivalent would be something like façade design or layout. If your building is ugly or confusing, it will annoy or waste the time of the people who live and work in it, but as long as it doesn't fall on their heads nobody is going to withdraw your certification.
> not because software engineers are worse human beings than civil engineers, which is all too often the undertone of these discussions
I think the undertone is usually that the software developer at fault had no idea what they were doing, and had no place working on critical systems, rather than that they had ill intent.
I think the undertone is usually that the software developer at fault had no idea what they were doing, and had no place working on critical systems, rather than that they had ill intent.
> software engineering hasn't killed enough people to advance
and where it had, you get pretty serious control and certification process put in place to avoid it (Boeing notwithstanding)
and where it had, you get pretty serious control and certification process put in place to avoid it (Boeing notwithstanding)
[deleted]
The 737 Max issues were not with the software.
There seems to be plenty of software issues found:
The MCAS software was modified to read from both angle of attack sensors and to be less aggressive in pushing the nose of the plane down. The software that controlled the indicator light that illuminated when the two angle of attack sensors disagreed was also fixed.
While reviewing the software systems, a number of other software issues were found.
The wiring bundle issue was also found during these reviews.
https://www.barrons.com/articles/these-6-issues-are-preventi...
https://simpleflying.com/boeing-737-max-software-update-3/
The MCAS software was modified to read from both angle of attack sensors and to be less aggressive in pushing the nose of the plane down. The software that controlled the indicator light that illuminated when the two angle of attack sensors disagreed was also fixed.
While reviewing the software systems, a number of other software issues were found.
The wiring bundle issue was also found during these reviews.
https://www.barrons.com/articles/these-6-issues-are-preventi...
https://simpleflying.com/boeing-737-max-software-update-3/
I phrased that poorly, I should have said The 737 Max issues that caused the crashes were not with the software implementation. They were at the level of requirements and high-level design, and were not specific to the discipline of software engineering. We're not talking about a missing break statement here.
> The MCAS software was modified to read from both angle of attack sensors and to be less aggressive in pushing the nose of the plane down.
That strikes me as a design issue in the domain of aeronautical engineering, rather than in software engineering. Software engineers aren't the ones with the domain expertise to determine the right aggression parameters.
> The software that controlled the indicator light that illuminated when the two angle of attack sensors disagreed was also fixed.
I thought the issue was that the sensor-disagree warning light was not included as standard, it was sold as an optional extra. [0]
> While reviewing the software systems, a number of other software issues were found.
Interesting, I wasn't aware of that. If I understand correctly these issues aren't thought to have a direct bearing on the crashes.
[0] https://www.nytimes.com/2019/05/05/business/boeing-737-max-w...
> The MCAS software was modified to read from both angle of attack sensors and to be less aggressive in pushing the nose of the plane down.
That strikes me as a design issue in the domain of aeronautical engineering, rather than in software engineering. Software engineers aren't the ones with the domain expertise to determine the right aggression parameters.
> The software that controlled the indicator light that illuminated when the two angle of attack sensors disagreed was also fixed.
I thought the issue was that the sensor-disagree warning light was not included as standard, it was sold as an optional extra. [0]
> While reviewing the software systems, a number of other software issues were found.
Interesting, I wasn't aware of that. If I understand correctly these issues aren't thought to have a direct bearing on the crashes.
[0] https://www.nytimes.com/2019/05/05/business/boeing-737-max-w...
Exactly. People seem to have skipped the V&V lectures. The code did exactly what it was asked to do. But the system/aero engineers made that decision. Even the lights issue was specified that way in the system diagram.
But the biggest defect was the decision that the pilots didn't have to be told about the risk (to avoid retraining and certification costs). If they had been told the pilots/airlines may well have protested about the lack of redundancy. All those higher ups in Boeing and the FAA should be in prison.
But the biggest defect was the decision that the pilots didn't have to be told about the risk (to avoid retraining and certification costs). If they had been told the pilots/airlines may well have protested about the lack of redundancy. All those higher ups in Boeing and the FAA should be in prison.
I don’t think that is too far off the mark. Given 8 more decades of progress we might even be able to produce reliable code. Webpages will be 100GB of JavaScript though
> Given 8 more decades of progress we might even be able to produce reliable code
As the risk of nitpicking, we already can produce reliable code, but as piaste's comment states, it's rare to make the necessary investment.
Avionics software and medical systems software is held to a very high standard, at great expense, but the average website or desktop application is not.
> Webpages will be 100GB of JavaScript though
I agree that the problem of bloat is likely to continue to worsen in the web, but that's just the web. Other sectors, like high-performance game engines, will continue to compete on efficiency.
As the risk of nitpicking, we already can produce reliable code, but as piaste's comment states, it's rare to make the necessary investment.
Avionics software and medical systems software is held to a very high standard, at great expense, but the average website or desktop application is not.
> Webpages will be 100GB of JavaScript though
I agree that the problem of bloat is likely to continue to worsen in the web, but that's just the web. Other sectors, like high-performance game engines, will continue to compete on efficiency.
I think reliability is, after a certain point, inverse proportional to the amount of code involved. So I'd disagree that we'll have even larger webpages then - the same way that in modern construction, heavy and thick stone & brick pillars have been replaced with comparatively light and strong steel-reinforced concrete. Maybe the equivalent would be just enough strongly typed code running on top of just enough, well tested & secure WASM?
Given software engineering is less than 200 years old while civil engineering is a few thousand years old, I don't see this as a negative at all.
I don't think that is the case - but that is certainly one interpretation of it. They were still engineers at the time though, and so are we.
made me chuckle.
"I need that bridge yesterday. If my army doesn't cross the river today, we are all dead. I don't care if the bridge collapse tomorrow, or if the rain wear it down, as long as I can use it today."
If it's your job to do what is requested from you, then you do it. It's not like having brittle unmaintainable code is morally wrong. It's not engineers responsibility to judge use case of the customer paying for the bridge.
If it's your job to do what is requested from you, then you do it. It's not like having brittle unmaintainable code is morally wrong. It's not engineers responsibility to judge use case of the customer paying for the bridge.
>It's not engineers responsibility to judge use case of the customer paying for the bridge.
Actually yes, yes it is! I am a civil engineer and there's a standard of ethics and personal responsibility among engineers that is very, very high. When we graduate, most engineers participate in a ring ceremony. They get a funny, angled ring on their right pinky. It was originally made from the metal from a bridge that fell down and killed people. It was meant to be a daily reminder, worn on the drafting hand, that every single drawing you signed carried the weight of public trust of their lives. Even pedestrian bridges get built with vehicle load standards, because you can't just assume that it will only be used by pedestrians.
Civil engineering is a licensure, and no matter what liability structure you use, when you stamp a drawing, you carry PERSONAL liability for anything that might happen if that design fails. Even small violations of ethics or operating outside your scope of knowledge are dealt with very aggressively by the licensing board.
In school, if we misplaced a decimal or got a calculation wrong or failed to adhere to a very specific standard, it was automatic failure because if you do that in real life, people die.
Actually yes, yes it is! I am a civil engineer and there's a standard of ethics and personal responsibility among engineers that is very, very high. When we graduate, most engineers participate in a ring ceremony. They get a funny, angled ring on their right pinky. It was originally made from the metal from a bridge that fell down and killed people. It was meant to be a daily reminder, worn on the drafting hand, that every single drawing you signed carried the weight of public trust of their lives. Even pedestrian bridges get built with vehicle load standards, because you can't just assume that it will only be used by pedestrians.
Civil engineering is a licensure, and no matter what liability structure you use, when you stamp a drawing, you carry PERSONAL liability for anything that might happen if that design fails. Even small violations of ethics or operating outside your scope of knowledge are dealt with very aggressively by the licensing board.
In school, if we misplaced a decimal or got a calculation wrong or failed to adhere to a very specific standard, it was automatic failure because if you do that in real life, people die.
It is difficult to compare software development with civil engineering as they operate on different levels.
Software development is better compared to building houses. No one expects single story house to support 4 extra levels or for internal walls to be resistant to hammer blows.
Imagine what civil engineering would produce if you were building a bridge from a to b and then 3/4 way through you were required to add two extra exits c and d that were miles apart from a and b and another 4 lanes and extra level for train traffic...all in the same time frame and for no extra cost.
Civil engineers would not be able to do it and no-one would even expect them to but this is regular occurrence for software developers and most of them actually manage to deliver.
Sure there are bugs but bridges need constant maintenance too, some quite substantial and that is expected.
For some reason, lots of non-software engineers expect software to be perfect when their own products are not, not even close.
This is a key common thread in all these discussions. At so many development shops, developers have no agency to push back against poor quality. They don't have a license on the line that they can stand behind and firmly say "NO we will not do it that way or I could be personally liable for its failure!" When they point out that they're shipping buggy crap, the response simply is: Ship the crap on time.
It's strange too, you hear in other threads about how high the demand is for software engineers, how high their salaries are, how much negotiating power they have with their companies, but when it comes to the actual product content, suddenly they have no power at all and it's just Yes boss, whatever you say, boss. How is this true?
It's strange too, you hear in other threads about how high the demand is for software engineers, how high their salaries are, how much negotiating power they have with their companies, but when it comes to the actual product content, suddenly they have no power at all and it's just Yes boss, whatever you say, boss. How is this true?
> I am a civil engineer and there's a standard of ethics and personal responsibility among engineers that is very, very high.
And that is because no one can just go to a 6-month bootcamp and call themselves a Civil engineer. And even if they did, no one would hire them or the employer would go to jail.
If we want Software Engineering to be as rigorous as other Engineering disciplines, we need to erect similar barriers to become a Software Engineer.
And that is because no one can just go to a 6-month bootcamp and call themselves a Civil engineer. And even if they did, no one would hire them or the employer would go to jail.
If we want Software Engineering to be as rigorous as other Engineering disciplines, we need to erect similar barriers to become a Software Engineer.
Honestly, I think it would be sensible to have a software engineering licensure, though it shouldn't be required for all developers or development. I'd say something like an engineering stamp would be pretty reasonable for anything requiring encryption, financial data or transactions, health data, etc. And that's not to say that developers couldn't work on it, but just that a licensed software engineer should be in responsible charge of their work to ensure that it has been performed according to best practices and standards.
Is this the same in Europe? I doubt it and at least in Germany an engineer (even a software engineer) is someone who has studied a technical subject at university. It has nothing to do with liability here.
The exact protected term varies by country but there is an equivalent license in Europe. The EU has a "Eur Ing." title issued by FEANI which is roughly equivalent. As I understand it, Germany is a bit unique even for the EU. They still have some things which require a certification of sorts, but I don't know exactly how that works.
That is a really weird analogy. Using extreme scenarios to argue your point is never going to put you on a strong footing.
Anyway for most systems deployed the customer usually wants to maintain a business using it. If the system constantly falls over, cant be readily changed etc etc that is going to cost the customers business compared to its competitors.
Add that a lot of developers work for the same company as the customer and that is just as much the developers responsibility.
99.999% of bridges built would be expected to hold up 10 years later and require minimal maintenance. I'd wager all bridges even temporary army ones would be expected to not unforeseeably fail. See the Morandi Bridge tragedy.
Anyway for most systems deployed the customer usually wants to maintain a business using it. If the system constantly falls over, cant be readily changed etc etc that is going to cost the customers business compared to its competitors.
Add that a lot of developers work for the same company as the customer and that is just as much the developers responsibility.
99.999% of bridges built would be expected to hold up 10 years later and require minimal maintenance. I'd wager all bridges even temporary army ones would be expected to not unforeseeably fail. See the Morandi Bridge tragedy.
Enginners where not requested to make the Voyagers last a hundred years, yet they're still up and running. Personally I think the correlation between quality of code and time to delivery are not linear. People can cram more work and quality in the same ammount of time if they want and have the right incentives to do so.
Dont argue with that. There is common trend to blame poor managerial decisions on lower level developers. just shows how crooked system is.
The state of software engineering these days resemble that of civil engineering of late 19th century: some masterpieces, some failures and lots of experiments.
Do we put the cockpit in front of the boiler or behind?
Do we make a walking steam engine? How about using gears instead of wheels?
Which gauge to pick for the rails?
We'll get there in software engineering at well. It will be very reliable and equally boring.
Do we put the cockpit in front of the boiler or behind?
Do we make a walking steam engine? How about using gears instead of wheels?
Which gauge to pick for the rails?
We'll get there in software engineering at well. It will be very reliable and equally boring.
There may be reasons for not calling software developers engineers but this is not one of them. The software "bridge" in this case has the following properties:
1. It can be rebuilt in a matter of hours by a single person.
2. The customers that ordered the bridge are also the people on the bridge and are totally fine with the bridge crashing on them from time to time.
1. It can be rebuilt in a matter of hours by a single person.
2. The customers that ordered the bridge are also the people on the bridge and are totally fine with the bridge crashing on them from time to time.
#2 was the most frustrating thing (to me) about writing software, and ultimately what got me to move out of coding over to other areas of the business. This universal acceptance of low quality and crashing. You always have the classic "Schedule, cost, quality, pick 2" tradeoff, and 99 out of 100 places you'll work will throw quality under the bus when push comes to shove.
I remember the exact turning point. We had a super buggy Windows application that had tons of crashes in it. Instead of root causing each crash and fixing them, I was asked to simply write a launcher app that sat in the background, waited for the application to crash, then re-launch it. That was the great solution. And it was totally acceptable to the customer. Arghhhhh! I remember thinking: I didn't spend four years in university to shit this kind of finished product out.
I remember the exact turning point. We had a super buggy Windows application that had tons of crashes in it. Instead of root causing each crash and fixing them, I was asked to simply write a launcher app that sat in the background, waited for the application to crash, then re-launch it. That was the great solution. And it was totally acceptable to the customer. Arghhhhh! I remember thinking: I didn't spend four years in university to shit this kind of finished product out.
I guess the key reason why software developers are not engineers is that there is no barrier to entry. If a necessary requirement to build a bridge is to hire a certified Civil Engineer who has undergone the necessary ethics and reliability training, then you have no choice but to pay up for that certification cost. You can't outsource that work to Eastern EU or Asia or hire another one willing to "hack" the bridge to deliver it on your tight timeline.
On the other hand, if someone takes software reliability seriously, either they are going to burn themselves out to meet the crazy deadlines or are going to "not deliver on time", in which case they will be replaced by someone who can "get shit done" cheaper and faster.
On the other hand, if someone takes software reliability seriously, either they are going to burn themselves out to meet the crazy deadlines or are going to "not deliver on time", in which case they will be replaced by someone who can "get shit done" cheaper and faster.
If you wanna design a bridge...you need an engineering background. If you wanna design a car...you need an engineering background.
If you want to design the software that designs the bridge or the car...just need a 8 week bootcamp.
If you want to design the software that designs the bridge or the car...just need a 8 week bootcamp.
You shouldn't compare bridge building to developing a trading apps (on causes death, the other merely financial loss)
Compare bridge building to developing software for pace maker.
Now compare number of failures in both cases.
You get what you pay for, if you pay for developer you'll get a developer.
Compare bridge building to developing software for pace maker.
Now compare number of failures in both cases.
You get what you pay for, if you pay for developer you'll get a developer.
You can convert between financial loss and loss of human life using implied cost of averting a fatality [1], which is about $10 million.
[1] https://en.wikipedia.org/wiki/Value_of_life
[1] https://en.wikipedia.org/wiki/Value_of_life
That is not a bidirectional transform. A financial loss in an investment fund just means that other traders made more profit.
> You get what you pay for, if you pay for developer you'll get a developer.
That's way too oversimplified. You can be as good and as thorough as you want, but if the root of the problem is that software is seen as a cost not as an integral part of the solution, you get bad results.
This often starts with unclear/vague requirements that change every other day as new information and understanding is gained.
But it doesn't stop there - unrealistic deadlines, lack of defined processes and quality control as well as disregard (and refusal to budget and schedule) for background tasks (documentation, refactoring, ...) are also contributing factors that can turn even the best and most diligent software developer into a messy code cowboy.
If gaming the system is rewarded more or actually doing a good job is even penalised, why do a good job?
That's way too oversimplified. You can be as good and as thorough as you want, but if the root of the problem is that software is seen as a cost not as an integral part of the solution, you get bad results.
This often starts with unclear/vague requirements that change every other day as new information and understanding is gained.
But it doesn't stop there - unrealistic deadlines, lack of defined processes and quality control as well as disregard (and refusal to budget and schedule) for background tasks (documentation, refactoring, ...) are also contributing factors that can turn even the best and most diligent software developer into a messy code cowboy.
If gaming the system is rewarded more or actually doing a good job is even penalised, why do a good job?
I remember reading about bad software in a radiotherapy machine inducing many deaths. And also about software in Toyota cars failing and also inducing deaths because the code was a huge mess. In the end, there is no such thing as a clear separation between "engineer" and "developers".
The Bridge analogy is often applied with flaws. Bridges are not free to build. Software essentially is.
Here is a 1992 truth bomb rewind about it:
https://www.developerdotstar.com/printable/mag/articles/reev...
Basically engineers care about the quality of the code because they know that is the design. As soon as caring about codebase quality leaves the building, so will the quality of the product.
Here is a 1992 truth bomb rewind about it:
https://www.developerdotstar.com/printable/mag/articles/reev...
Basically engineers care about the quality of the code because they know that is the design. As soon as caring about codebase quality leaves the building, so will the quality of the product.
A civil engineer will put their stamp on the design of a bridge certifying that it will not fail. If it does fail, they will be held responsible and may lose their license to practice.
Software engineers often hide behind a contract saying they aren't responsible for anything.
Software engineers often hide behind a contract saying they aren't responsible for anything.
Software engineers are held responsible to plenty of things (HIPAA) when there is a will to do so.
He is a qualified elec eng though, he's just found a non-engineering job that pays a lot more.
Software engineering is real and does require you to have a real understanding of risk (including through formal methods, model based design, systematic testing etc) in regard to the consequence. If you work in a more regulated industry (e.g. railways, aviation, automotive) these things are taken more seriously. This is why things like 'partial autonomy' driving needs more regulation -- no one is requiring Tesla to have a Chief Engineer sign off on anything meeting any standard.
Software engineering is real and does require you to have a real understanding of risk (including through formal methods, model based design, systematic testing etc) in regard to the consequence. If you work in a more regulated industry (e.g. railways, aviation, automotive) these things are taken more seriously. This is why things like 'partial autonomy' driving needs more regulation -- no one is requiring Tesla to have a Chief Engineer sign off on anything meeting any standard.
There are _plenty_ of places in engineering where defects / quality / etc are lesser concerns, just like there are places in software development where they're a major concern. There's also a _lot_ more call for quality up high (ie, regulations) for the various engineering disciplines.
I would expect that there are plenty of engineers that look at faults in the systems they're working on and say, "management doesn't care, not my problem".
I would expect that there are plenty of engineers that look at faults in the systems they're working on and say, "management doesn't care, not my problem".
On the opposite end you have programmed obsolescence for example that is definitely an engineering art.
It takes talent and effort to make things fail.
Sounds like defense software. Feature A is blatantly obvious but there is no "shall" requirement for it.
Except those who go through university level software engineering or computer science degrees
I would suggest that some in the field are developers while others are engineers. The former write some code that does what they tested it for, the latter designed code to account for every detail. The former thinks it does the job the latter knows. The former rushes through while the latter codes in ways that avoids redundant busy work to allow for the higher up front investment. The former slows over time as they slog through the "ball of mud" they call inevitable while the latter increasingly gains and shares insight on the deeper problems being solved and the opportunities afoot.
TL;DR: I think the extent to which we are engineers is a choice we individually make.
TL;DR: I think the extent to which we are engineers is a choice we individually make.
Nice flamebait.
I know exactly the type of guys you are talking about. They write tools for traders, a mix of Excel, VB, .Net, you name it. They are not obsessed with architectural beauty, patterns and frameworks. They just write what users ask them (and they do not tell traders what traders need - the response will be swift and insulting). The traders take these tools and generate profit. And they do not give a shit about test coverage or not sound architecture principles. If software generates profit, then it works.
From my experience, JVM can't do much when there's a deep stack.
I remember just refactoring all of foreach-loops/iterators into for-loops and got insane speedups.
Functions with inefficient iterations would get optimized if called directly but deep in the call stack and nothing happens.
These kinds of things are impossible with C++.
It's possible to write efficient Java but you have to not use some of the language features to do so.
I remember just refactoring all of foreach-loops/iterators into for-loops and got insane speedups.
Functions with inefficient iterations would get optimized if called directly but deep in the call stack and nothing happens.
These kinds of things are impossible with C++.
It's possible to write efficient Java but you have to not use some of the language features to do so.
>These kinds of things are impossible with C++
What? Anything you can do in C, you can do in C++ at the same performance level (pay only for what you use); just be aware of what you are doing and using.
What? Anything you can do in C, you can do in C++ at the same performance level (pay only for what you use); just be aware of what you are doing and using.
> I remember just refactoring all of foreach-loops/iterators into for-loops and got insane speedups.
Wow! Can you please provide a simple example when/where it happens?
> It's possible to write efficient Java but you have to not use some of the language features to do so.
Please tell us more.
Wow! Can you please provide a simple example when/where it happens?
> It's possible to write efficient Java but you have to not use some of the language features to do so.
Please tell us more.
Let's say you write an Iterable class so that you can use foreach on your data structure. It's quite possible iteration won't be completely inlined and the overhead of calling hasNext & next can become non-significant.
An even worse case that made massive speedups is just inlining everything. I believe modern IDEs can automatically inline all invocations of a function. You'd be surprised how much performance you can get with that.
As for writing efficient Java, it comes down to using primitive and value types.
Although, my experience is mostly with Java 8. I have no idea how streams or other new syntax works. After optimizing a really massive Java code base I've never used it since.
An even worse case that made massive speedups is just inlining everything. I believe modern IDEs can automatically inline all invocations of a function. You'd be surprised how much performance you can get with that.
As for writing efficient Java, it comes down to using primitive and value types.
Although, my experience is mostly with Java 8. I have no idea how streams or other new syntax works. After optimizing a really massive Java code base I've never used it since.
But you still have to make the next and hasNext calls in a for loop. What guarantees that they would be inline in a for loop?
I inline them manually.
How much was the performance improvement?
The biggest improvement I measured was about 15x. I'd say you can get 3x most of the time (go down to 33% of previous runtime).
That is insane speedup! Thanks for sharing. How do I find next and hasNext is the bottleneck? Should I run profiler to see if they are being called a large number of times?
Unfortunately, any profiler I tried does not show these issues. Because the code you are profiling will be optimized completely differently. Although it makes sense to do the optimizations/inlining inside the top-level functions that profiler reports to take a lot of time (even if the iteration happens in deeper levels).
You just have to try it and see. I remember it was quite easy in IntelliJ, I believe there's an option to inline all invocations of a function. I just went crazy with that option but did it systematically to really find where the bottleneck is and make a minimal change.
You just have to try it and see. I remember it was quite easy in IntelliJ, I believe there's an option to inline all invocations of a function. I just went crazy with that option but did it systematically to really find where the bottleneck is and make a minimal change.
[deleted]
I thought the actual approach taken these days is to disable GC for any JVM based trading systems and use arena based architectures or similar for application where you need super low latency, like a LOB mirror or execution engine, no?
Yeah, basically everything has to be that way.
At the super fast speeds you start running into things like:
* why won’t the devirtualizer trigger?
* these object headers are sure wasting a lot of cache
* there’s a lot of forced pointer indirection
And you just end up spending vast amounts of time and effort trying to shave off those few microseconds you’re wasting in the JVM. Once you add in all the effort trying to get around the GC, it’s bleak picture for competing with quality C++ without spending far more effort.
At the super fast speeds you start running into things like:
* why won’t the devirtualizer trigger?
* these object headers are sure wasting a lot of cache
* there’s a lot of forced pointer indirection
And you just end up spending vast amounts of time and effort trying to shave off those few microseconds you’re wasting in the JVM. Once you add in all the effort trying to get around the GC, it’s bleak picture for competing with quality C++ without spending far more effort.
Thanks for the insights - I must admit I only briefly ever even used Java so I am not an expert. Do you have any recommendations for documents or a PDF that covers the JVM memory model and scheduler in any depth that makes it appropriate, possibly with applications to hard-realtime systems?
Again, thanks for the response, this was insightful and I'd love to learn more.
Also, an aside - do you have any thoughts on the use of Rust in these same systems? It's a little bit more bleeding edge, but I'm curious to hear an expert's thoughts!
Again, thanks for the response, this was insightful and I'd love to learn more.
Also, an aside - do you have any thoughts on the use of Rust in these same systems? It's a little bit more bleeding edge, but I'm curious to hear an expert's thoughts!
What do you mean by ‘devirtualizer’ and why would you want that triggered? Sounds like something you wouldn’t want to trigger?
The devirtualizer (maybe this is the wrong JVM terminology, it’s basically what clang/GC call it) is part of the optimizer which sees that you have a virtual function call (like most in Java) where there is a unique caller, so you can replace the virtual call with a direct one and possibly inline it.
In the JVM I think this can only be done speculatively (you have to double check the type), but it still matters.
In the JVM I think this can only be done speculatively (you have to double check the type), but it still matters.
Ah right yes conflict of terminology.
In the JVM devirtualising means making a virtual object a full object again, so the opposite of what you want to be happening.
I don't think the JVM really names the optimisation you're talking about, but it does do it, through either a global assumption based on the class hierarchy, or an inline cache with a local guard.
In the JVM devirtualising means making a virtual object a full object again, so the opposite of what you want to be happening.
I don't think the JVM really names the optimisation you're talking about, but it does do it, through either a global assumption based on the class hierarchy, or an inline cache with a local guard.
What do you mean by “virtual object”, an object which has been broken up on the stack instead of a “real” object living as a single entity on the heap?
> What do you mean by “virtual object”, an object which has been broken up on the stack instead of a “real” object living as a single entity on the heap?
Yes... but let's not say 'broken up on the stack' - the object's fields become dataflow edges. The object doesn't exist reifed on the stack - fields may exist on the stack, or in registers, or not at all.
Yes... but let's not say 'broken up on the stack' - the object's fields become dataflow edges. The object doesn't exist reifed on the stack - fields may exist on the stack, or in registers, or not at all.
LLVM calls the process of breaking a struct/object into dataflow variables "Scalar Replacement of Aggregates".
https://llvm.org/doxygen/classllvm_1_1SROA.html#details
https://llvm.org/doxygen/classllvm_1_1SROA.html#details
Yes that's what the JVM calls it - and then the node the ties together the state is the virtual object.
Remember back in cs101 where you had class animal, with subclasses dog and cat. You can call speak() on animal and get either ruff or meow depending on what animal is. Animal is a virtual object. The system needs to do some work to figure out which speak to call every time, the work isn't too bad, but it turns out that cpus basically never have the right thing in the cache, so it ends up being very slow compared to most of what you are doing.
When we say devirtualize we mean that we know animal is always a cat so we don't have to look up which speak to use.
When we say devirtualize we mean that we know animal is always a cat so we don't have to look up which speak to use.
Please don't take others for complete morons.
And that makes literally no sense in-context, you're mapping those words directly to the concept of virtual methods but that's the opposite of the way chrisseaton uses them (and they're clearly familiar with the concept of virtual calls and it's not what they're using "virtual" for), hence asking them what they mean specifically by this terminology.
And that makes literally no sense in-context, you're mapping those words directly to the concept of virtual methods but that's the opposite of the way chrisseaton uses them (and they're clearly familiar with the concept of virtual calls and it's not what they're using "virtual" for), hence asking them what they mean specifically by this terminology.
[deleted]
the jvm names the optimization exactly like he said
IIRC it calls it monomorphisation.
[deleted]
See Aleksey Shipilёv on JVM method dispatch: https://shipilev.net/blog/2015/black-magic-method-dispatch/
[deleted]
Why not just write it in C? And pre-allocate all the data structures, make them global, put them into a queue, and constantly reuse them. No more need to instantiate objects.
This is why C++ is used over Java for very fast stuff, or “just” fast stuff if you don’t want to bother fighting the JVM
Why not just write it in assembly while you’re at it?
The answer to both questions is that many people prefer writing in higher-level languages with more safety guarantees.
The answer to both questions is that many people prefer writing in higher-level languages with more safety guarantees.
This is a bit of a false equivalency -- there aren't many perf benefits in rewriting a C program in asm. The cost you pay for that rewrite is also much greater.
Yes, of course $HIGH_LEVEL_LANG is preferable for many many use cases. In this context, we're discussing "high speed trading systems", for which native implementations are going to be the favorite.
Yes, of course $HIGH_LEVEL_LANG is preferable for many many use cases. In this context, we're discussing "high speed trading systems", for which native implementations are going to be the favorite.
> for which native implementations are going to be the favorite
...but the point of the article is they aren't always the favourite.
...but the point of the article is they aren't always the favourite.
They mention not even using most of the Java features (exceptions, built in GC) that make it safe. So I think it's a pretty fair question, since they are essentially using a very stripped down version of the language that removes most of the compelling reasons to use it, and seemingly fighting the language runtime along the way.
The reason these people still use Java is that everything else can be nice high-level Java code.
So they use regular Java for the build system, deployment, testing, logging, loading configuration, debugging, etc etc. There's a small core written in this strange way... but everything else is easier. And things like your profiler and debugger still work on the core as well.
So they use regular Java for the build system, deployment, testing, logging, loading configuration, debugging, etc etc. There's a small core written in this strange way... but everything else is easier. And things like your profiler and debugger still work on the core as well.
I write C++ for now almost 30 years, and always for the workloads where the cost of allocations was plainly visible, especially in the critical parts of the applications. So I have never stopped writing "non idiomatic" C++, at least from the point of view of typical language lawyers (and C++ attracted them a lot through the years). And I'm surely not the only one: there were different environments where it was recognized, through the times, first, that creating and destroying much objects was very bad, and later, with the growth of the C++ standard library, that even not everything in the C++ standard library should be treated the same, and that are better solutions than what's already available, and that the third party libraries are often bringing even more potential danger.
Depending on the goal, one has to be very careful when choosing what one uses. The good side is, C++ kept its C features: if I'm deciding how I'll do something, I don't have to follow the rules of the "language lawyers." I can do my work producing what is measurably efficient. And compiler can still help me avoiding some types of errors -- others can anyway be discovered only with testing (and additional tools). At the end, knowing good what one wants is the most important aspect of the whole endeavor.
Depending on the goal, one has to be very careful when choosing what one uses. The good side is, C++ kept its C features: if I'm deciding how I'll do something, I don't have to follow the rules of the "language lawyers." I can do my work producing what is measurably efficient. And compiler can still help me avoiding some types of errors -- others can anyway be discovered only with testing (and additional tools). At the end, knowing good what one wants is the most important aspect of the whole endeavor.
> I don't have to follow the rules of the "language lawyers."
I'm not exactly sure what specifically you are trying to imply here. When you proclaim to ignore language lawyers, it sounds like you are knowingly breaking the rules of the C++ standard. That takes a lot of faith in compilers doing what you meant to do, despite writing code that is incompatible with the standard those compilers implement...
I'm not exactly sure what specifically you are trying to imply here. When you proclaim to ignore language lawyers, it sounds like you are knowingly breaking the rules of the C++ standard. That takes a lot of faith in compilers doing what you meant to do, despite writing code that is incompatible with the standard those compilers implement...
Yep. I always say start with the "better C" part of C++ to get stronger type checking and then add in other features only as needed. All abstractions should be with minimal overhead with a strict pay only what you need policy.
Yep, the same thing I do - just write in C++ as if it was good old C, with very occasional use of templates, containers and exceptions.
To those who miss the point, nobody is here denying that C++ has something to bring, it's just that what it brings isn't what those who promote some fashion would claim that is to be universally used, and, honestly, there's no actual reason to believe such claims, for they being not more true now than at the times where "making complex OOP hierarchies" was the most popular advice -- I remember these times too. Or the times when managers wanted to believe that everybody will just use Rational Rose to draw nice diagrams and the actual programming won't be needed, at all. Every time has its hypes:
http://www.jot.fm/issues/issue_2003_01/column1/
https://wiki.c2.com/?UmlCaseVultures
One size doesn't fit all. Some solutions to some problems could be and are provably better than those typically promoted or "generally known" at some point of time.
If that all still doesn't mean anything to you, please read carefully and very, very slowly "The Summer of 1960", seen on HN some 9 years ago:
https://news.ycombinator.com/item?id=2856567
Edit: Answering the parallel post writing "When you proclaim to ignore language lawyers, it sounds like you are knowingly breaking the rules of the C++ standard."
No. The language lawyers, in my perception, religiously follow everything that enters the standard and proclaim that all that has to be used, because it's standardized. Including the standard libraries and some specific stuff there that isn't optimal for the problem I'm trying to solve. And especially that whatever is newer and more recently entered the standard is automatically better. It's understandable that they support their own existence by doing all that -- it's about becoming "more important" just by following/promoting some book or some obligatory rituals (it's an easy and time proven strategy through the centuries, and that's why I call it "religiously", of course -- and I am also not surprised that somebody who identifies themselves with being one of the "lawyers" wouldn't like this perspective -- you are free to suggest a better name). But it should also be also very obvious that it's not what's necessarily optimal for me to follow, as soon as I can decide what I'm doing. And yes, it's different in the environment where the "company policy" is sacred. There one has the company "policy lawyers", and typically every attempt of change can die if one isn't one of them.
http://www.jot.fm/issues/issue_2003_01/column1/
https://wiki.c2.com/?UmlCaseVultures
One size doesn't fit all. Some solutions to some problems could be and are provably better than those typically promoted or "generally known" at some point of time.
If that all still doesn't mean anything to you, please read carefully and very, very slowly "The Summer of 1960", seen on HN some 9 years ago:
https://news.ycombinator.com/item?id=2856567
Edit: Answering the parallel post writing "When you proclaim to ignore language lawyers, it sounds like you are knowingly breaking the rules of the C++ standard."
No. The language lawyers, in my perception, religiously follow everything that enters the standard and proclaim that all that has to be used, because it's standardized. Including the standard libraries and some specific stuff there that isn't optimal for the problem I'm trying to solve. And especially that whatever is newer and more recently entered the standard is automatically better. It's understandable that they support their own existence by doing all that -- it's about becoming "more important" just by following/promoting some book or some obligatory rituals (it's an easy and time proven strategy through the centuries, and that's why I call it "religiously", of course -- and I am also not surprised that somebody who identifies themselves with being one of the "lawyers" wouldn't like this perspective -- you are free to suggest a better name). But it should also be also very obvious that it's not what's necessarily optimal for me to follow, as soon as I can decide what I'm doing. And yes, it's different in the environment where the "company policy" is sacred. There one has the company "policy lawyers", and typically every attempt of change can die if one isn't one of them.
I guess this is for you, in case you haven't seen it already,
"Orthodox C++"
https://gist.github.com/bkaradzic/2e39896bc7d8c34e042b
"Orthodox C++"
https://gist.github.com/bkaradzic/2e39896bc7d8c34e042b
Haven't seen, but thanks. Lived and worked while following some of the ideas mentioned.
E.g. at the whole bottom of the page in some comment is a link to:
"Why should I have written ZeroMQ in C, not C++ (part II)"
https://250bpm.com/blog:8/
where the author writes "Let's compare how a C++ programmer would implement a list of objects..." and then "The real reason why any C++ programmer won't design the list in the C way is that the design breaks the encapsulation principle" etc.
I have indeed more than once used intrusive data structures in non-trivial C++ code, and the result was easy to read, and very efficient. There I really didn't care about some "thou shalt not" "breaking the encapsulation principle" because whoever thinks at that level of "verbot" 100% of times is just wrong.
The "encapsulation principle" is an OK principle for some levels of abstraction, but nobody says that one has to hold to it religiously (exactly my point before). I would of course always make an API which would hide what's behind it. But where I implement something ("the guts" of something), I of course have my freedom to use intrusive elements, if that solves the problem better. I have even created some "extremely intrusive" stuff (with a variable number of intrusive links in the structures). It worked perfectly. Insisting on doing anything as "ritual" all the time is just so, so wrong.
E.g. at the whole bottom of the page in some comment is a link to:
"Why should I have written ZeroMQ in C, not C++ (part II)"
https://250bpm.com/blog:8/
where the author writes "Let's compare how a C++ programmer would implement a list of objects..." and then "The real reason why any C++ programmer won't design the list in the C way is that the design breaks the encapsulation principle" etc.
I have indeed more than once used intrusive data structures in non-trivial C++ code, and the result was easy to read, and very efficient. There I really didn't care about some "thou shalt not" "breaking the encapsulation principle" because whoever thinks at that level of "verbot" 100% of times is just wrong.
The "encapsulation principle" is an OK principle for some levels of abstraction, but nobody says that one has to hold to it religiously (exactly my point before). I would of course always make an API which would hide what's behind it. But where I implement something ("the guts" of something), I of course have my freedom to use intrusive elements, if that solves the problem better. I have even created some "extremely intrusive" stuff (with a variable number of intrusive links in the structures). It worked perfectly. Insisting on doing anything as "ritual" all the time is just so, so wrong.
I've heard of people doing that 10-15 years ago. I'm sure it still works, but hopefully the GC situation has also improved.
A pause is still a pause though. One system I know of had a solution. Throw in a huge amount of RAM on the server and delay garbage collection until end of day.
But a pause that can happen when you want it is acceptable. A 3 second stall might be totally acceptable as long as it happens outside of a trade, and you can guarantee that.
If you can predict 3 seconds in advance if you’re going to need to trade or not, you’d make millions very quickly. In trading terms, that’s basically having a time machine.
You don't need to predict in advance, you just need the capability in the system. If a system takes X time from the beginning of a trade to being ready to process the next trade, with Y of that time being GC, it doesn't matter how long Y is if you can execute the entire trade before all of the GC happens.
I used to support a derivatives trading system at a company called Patsystems, so I’ve seen this. Our legacy system could process trade triggers from 4ms to 7ms, but tracing the behaviour of the new system it could do it in 3ms, great. Except every now and then a GC pause would halt order processing for around 50ms. Our customers hit the roof.
Trading systems trigger almost all their orders based on detecting market changes, which you cannot predict. If a symbol hits a price that triggers one of your trades you want that trade in the market ASAP every millisecond matters. Randomly add 50ms on to that and your trading system is out of business. The customers will go elsewhere.
Trading systems trigger almost all their orders based on detecting market changes, which you cannot predict. If a symbol hits a price that triggers one of your trades you want that trade in the market ASAP every millisecond matters. Randomly add 50ms on to that and your trading system is out of business. The customers will go elsewhere.
(Sorry, I missed this over the weekend) - I think the issue in this case is you're selling this as a guaranteed 3ms, going down from 4-7ms. If you have to pause for this 50ms GC once every X trades, then you force the GC to happen _after_ the X-1th trade has completed, but you need to provision an extra 15(?) machines to cover the pause. Otherwise, you're not delivering 3ms trades, you're delivering (3 + GCTime/X) ms. It doesn't matter if GC takes 1ms or 1s, you "simply" need to have the capacity to cover the extra GC time.
Why waste money hire best engineers when more ram does trick?
As I understand it (before my time), this was a practical solution found on original symbolics lisp machines; trigger the GC and go get lunch...
There are open source 'pauseless' (really: ultra low pause) GCs for the JVM now. Azul made good money selling such a thing to the finance industry for a long time.
Typical pauses may be far less than a millisecond in these GCs because they do almost everything in parallel.
Typical pauses may be far less than a millisecond in these GCs because they do almost everything in parallel.
What is an arena based architecture? A quick google didn't show any results. My assumption is that you mean pre-allocating all the memory you want to use at once, but I'm not sure and I'm curious.
Basically yeah. You allocate out of pools or arenas. In a language like C this usually means using a block of memory for objects that can all be freed together, so instead of calling malloc and free for each little object in your critical path (eg from request to response on some server, or a frame in a videogame), you somehow determine a maximum size, allocate that (and write enough bytes to have the OS actually give you the memory), and then allocations are just bumping a pointer into this block. At the end of the request, free the block (or return it to a pool to be used later).
Sometimes an arena may instead refer to a region of memory for allocations that are all the same size/type. Because everything is the same size, the data structures for tracking allocations may be simpler.
In a language like Java, you basically preallocate an array of objects of the same type with a bunch of fields set to zero, and then have some data structure to give you a fast interface a bit like malloc. Because you don’t do any extra allocation, the GC can be safely turned off.
Sometimes an arena may instead refer to a region of memory for allocations that are all the same size/type. Because everything is the same size, the data structures for tracking allocations may be simpler.
In a language like Java, you basically preallocate an array of objects of the same type with a bunch of fields set to zero, and then have some data structure to give you a fast interface a bit like malloc. Because you don’t do any extra allocation, the GC can be safely turned off.
Am I understanding correctly that those firms really really care about latency & have unlimited money & chose C++/JAVA with a standard off-the-shelve compiler + an OS and/or JVM??
If you'd really really care about latency and have the resources, wouldn't it be more effective to just go bare metal? Bare metal as in: no OS (talk to the hardware registers directly from code), possibly a custom compiler, use all the relevant instructions that your CPU offers you. Or at the very least mess with the compiler to optimize it more to the specific make/model of the CPU/hardware that it will run on?
What are the considerations in those companies for or against this?
If you'd really really care about latency and have the resources, wouldn't it be more effective to just go bare metal? Bare metal as in: no OS (talk to the hardware registers directly from code), possibly a custom compiler, use all the relevant instructions that your CPU offers you. Or at the very least mess with the compiler to optimize it more to the specific make/model of the CPU/hardware that it will run on?
What are the considerations in those companies for or against this?
I don't know about java, but when using c++ you can use the OS to start the program, set up the hardware mappings and then move it to an isolated core where you talk to the hardware directly without ever issuing a system call.
Compilers already offer flags to optimize for specific processor generations (and use the newly available instructions); you aren't going to be able to do better than that with a custom compiler.
Compilers already offer flags to optimize for specific processor generations (and use the newly available instructions); you aren't going to be able to do better than that with a custom compiler.
The jit compiler of typical jvms automatically uses the instruction sets (with limits afaik) that are available on the hardware that it is running on.
Oh, that's cool. However, you'd still be sharing at least 1 core (and all peripheral hardware and memory access in the system) with the OS. A core, other hardware and memory that you could otherwise access directly from your application without having to wait for the OS to finish its business?
I'm not sure whether it's worth it to go through this trouble (writing for bare metal vs just using an off-the-shelve OS) in a real life trading-app scenario. Hence my wonder :)
I'm not sure whether it's worth it to go through this trouble (writing for bare metal vs just using an off-the-shelve OS) in a real life trading-app scenario. Hence my wonder :)
Accessing other hardware isn't latency-sensitive, though; that's usually disk and the internal network, for logging, telemetry and control. Typically a trading application will keep a thread running on the shared core and send messages between that and its dedicated core thread using lock-free queues, or simple atomics. As you say, writing for bare metal is a lot of hassle and it's unnecessary for the parts of a trading application that don't need to be super fast; using a commodity OS and isolating resources gives the best of both worlds.
I've never seen that, how do you do it? Also, how do you deal with VM and scheduling?
Isolating a core: isolcpus and taskset, or possibly cgroups, plus nohz and there's a patch set that offers even stronger isolation guarantees (keeping kernel threads off the core). Hardware mappings are done as a ring buffer in memory mapped to the device; a kernel module helps set up that mapping but once it's in place reads and writes to those virtual addresses go directly (well - via MMU and memory bus) to the hardware. The general term for this is user-space networking (although obviously you can do it with other hardware as well). Some addresses may be mapped to device registers that trigger hardware actions on read/write.
VM - the key is to get everything mapped in at the start and avoid page faults subsequently. madvise can help with this and obviously you need to avoid memory leaks that could result in sbrk - but in any case allocating at all in the trading thread is generally unnecessary (after startup) and frowned on. This does mean that the (lock-free) queue back to the shared core needs to recycle memory and both sides need to service it regularly or you need a strategy to deal with exhaustion. Scheduling isn't an issue - you have a single thread per core and (with isolcpus) the kernel scheduler knows to avoid placing other threads on it.
VM - the key is to get everything mapped in at the start and avoid page faults subsequently. madvise can help with this and obviously you need to avoid memory leaks that could result in sbrk - but in any case allocating at all in the trading thread is generally unnecessary (after startup) and frowned on. This does mean that the (lock-free) queue back to the shared core needs to recycle memory and both sides need to service it regularly or you need a strategy to deal with exhaustion. Scheduling isn't an issue - you have a single thread per core and (with isolcpus) the kernel scheduler knows to avoid placing other threads on it.
Thank you for that writeup. Can you please list some resources (links/articles/books etc.) for us to read up on the above?
Brief intro, should bootstrap your knowledge:
https://codywu2010.wordpress.com/2015/09/27/isolcpus-numactl...
In addition to the peer comments, using JNI, these facilities can also be availed in Java. Chronicle queue does this.
Well...the article says:
"If you have an unlimited amount of time and resources, the best solution for speed will be coded in FPGA," says Lawrey, referring to the language that directly codes field-programmable gate arrays. "If you still have a lot of time and resources and you want to create multiple exchange adaptors, you'll choose C++. - But if you want to engage with 20+ exchanges, to go to market quickly, and implement continuous performance tuning, you'll choose Java."
"If you have an unlimited amount of time and resources, the best solution for speed will be coded in FPGA," says Lawrey, referring to the language that directly codes field-programmable gate arrays. "If you still have a lot of time and resources and you want to create multiple exchange adaptors, you'll choose C++. - But if you want to engage with 20+ exchanges, to go to market quickly, and implement continuous performance tuning, you'll choose Java."
I used to work at a market maker where they used FPGAs to beat HFTs. Of course it was a constant arms race, so I’m not sure what they would be using nowadays...
I would imagine they switched to ASICs which are more optimized than FPGAs (much smaller feature size, less general reconfigurable bloat, higher clocks, etc).
Only a few have, and even then only for a few fixed things. They have extreme versions of the inflexibility problems that fpgas have, and few exchanges have low enough jitter to justify ASICS over fpgas.
It might surprise people that relatively few hfts do latency arb (and the ones that do often have some other structural advantage), and at that the ones are more and more mixing in serious short-term alpha. In that context speed is simply becoming table stakes for playing the game, and not the one true race to win it all.
It might surprise people that relatively few hfts do latency arb (and the ones that do often have some other structural advantage), and at that the ones are more and more mixing in serious short-term alpha. In that context speed is simply becoming table stakes for playing the game, and not the one true race to win it all.
I don’t know what you think the practical benefit of running with no OS is?
More processing capacity for your application (no need to share with OS), no more interruptions by the OS that are unpredictable in duration, no more interruptions by the OS that are unpredictable in terms of timing and priority and above all, since this seems to be the main goal of the software we're talking about: lower overall latency. Standard C++/JAVA (as far as I know) talks to hardware through a software layer (the OS's system calls). This takes time. Time you can save by cutting out the middle man.
There are apparently ways to do this with an OS running in parallel as well though when you have multiple cores available apparently (see another comment to this thread).
There are apparently ways to do this with an OS running in parallel as well though when you have multiple cores available apparently (see another comment to this thread).
You can get all this by pinning to a core and DMA to hardware. Think of the OS like a library - if you don't call it and don't ask it to call you it stays out of the way. There's no need to throw away the OS.
This is exactly what is done. OS’s are really nice to have around and will stay out of the way. In the day and age of fpgas no reason to ruin the software side to maybe kind of sort of get a bit faster in your software?
- raw, unshared access to devices.
- no more interrupts.
> raw, unshared access to devices
Get to OS to set up DMA for you.
> no more interrupts
If the OS has no need to interrupt you, it will never interrupt you. The OS doesn't context switch from a usefully running application on its own core unless you ask it to.
Don't need virtualisation.
Get to OS to set up DMA for you.
> no more interrupts
If the OS has no need to interrupt you, it will never interrupt you. The OS doesn't context switch from a usefully running application on its own core unless you ask it to.
Don't need virtualisation.
There is only a limit amount of money to make. If your algorithm generates 100k in gross profit a year that doesn't even pay for the programmers to maintain it, unless they can do other things as well. (I'd guess common, there are a lot of algorithms) the sum total doesn't pay. Where it does pay you are better off spending money on fpga, or even asic.
Which one of these firms made the most money?
The third one was definitely the least, it was losing money (and specifically to trader-invisible things that never got worked on).
The whole of the trading strategy (and capital traded, return profiles, etc) between the first two was fairly different, so it’s hard to compare.
I would say though that the first firm was significantly outperforming its peers in a way the second firm did not, although both were very successful.
The technology wasn’t the whole story in either case, but the first firm had significantly more and better opportunities as a result of just having a better stack all around
The whole of the trading strategy (and capital traded, return profiles, etc) between the first two was fairly different, so it’s hard to compare.
I would say though that the first firm was significantly outperforming its peers in a way the second firm did not, although both were very successful.
The technology wasn’t the whole story in either case, but the first firm had significantly more and better opportunities as a result of just having a better stack all around
It's generally pretty rare for a market maker to lose money over a whole year. Colour me surprised.
There's costs that aren't the PnL of buy-low-sell-high. The coders, the data, the infra. All cost money, a lot of money. Add to that there's a competitive aspect in the models interacting with each other, there's no slam-dunk.
There's at least a couple of MMs struggling these days.
There's at least a couple of MMs struggling these days.
True but I feel that the market maker label gets applied pretty loosely nowadays, couldve been a firm that mixes in a lot of intraday position taking trades that blew up
[deleted]
Interestingly I have seen this pattern in all kinds of software businesses, but for most it tended to be less of a core business problem.
I'm still wondering if the businesses would work better if the focus on technical excellency could be instilled and especially which parts of the company would have to suffer for it and what the ultimate outcome would be.
I'm convinced that at least for some companies, it simply can't be the winning strategy.
I'm still wondering if the businesses would work better if the focus on technical excellency could be instilled and especially which parts of the company would have to suffer for it and what the ultimate outcome would be.
I'm convinced that at least for some companies, it simply can't be the winning strategy.
You missed one comparison: C infra (no C++).
(well strictly speaking, additionally you could also have hand-written assembly infra).
(well strictly speaking, additionally you could also have hand-written assembly infra).
You can write C style code in C++ and still get access to features that are still interesting in the problem domain. constexpr functions, typesafe enums, static assertions, and destructors come to mind. No need to even touch a template or inheritance if you don't want to.
The main benefits of C per se are ABI stability and availability of compilers.
The main benefits of C per se are ABI stability and availability of compilers.
[deleted]
Well there is a middle ground - Rust
From what perspective exactly is Rust a middle ground Java and C++?
Rust improves on Java in that it doesn't have a GC, so no pauses, and the code gen is similar to C++ in terms of performance.
Rust is worse than C++ in that it struggles to express some idiomatic high-performance code architectures, such as using DMA for I/O, because they violate axioms of Rust's memory safety model, or in other cases because Rust currently lacks the language features. To make it work in Rust requires writing more code and disabling safety features.
Rust fixes many language design and performance issues with Java, while offering some similar types of safety, but (like Java) it is missing some elements of expressiveness of C++ that are important for high-performance architectures.
Rust is worse than C++ in that it struggles to express some idiomatic high-performance code architectures, such as using DMA for I/O, because they violate axioms of Rust's memory safety model, or in other cases because Rust currently lacks the language features. To make it work in Rust requires writing more code and disabling safety features.
Rust fixes many language design and performance issues with Java, while offering some similar types of safety, but (like Java) it is missing some elements of expressiveness of C++ that are important for high-performance architectures.
Hm... unsafe {} + use pointers and you're back to c++-style? You've had unsound code in one language, now you have unsound code in other language, how come it's suddenly worse other than having to type unsafe explicitly?
I've done my fair share of C++, including low latency stuff, and in the grand scheme of things I'd say "expressiveness of C++" is completely overshadowed by its complexity and occasional ambiguity, lack of proper type system and proper generics, lack of proper module system, lack of dependency tracking, lack of a unified build systems, etc. I'm not exactly sure what you mean by expressiveness though.
I've done my fair share of C++, including low latency stuff, and in the grand scheme of things I'd say "expressiveness of C++" is completely overshadowed by its complexity and occasional ambiguity, lack of proper type system and proper generics, lack of proper module system, lack of dependency tracking, lack of a unified build systems, etc. I'm not exactly sure what you mean by expressiveness though.
The issue, beyond having no visible references at compile-time, is that DMA has no concept of objects and has its own rules for how and when it interacts with regions of memory. It is oblivious to ownership semantics. In things like databases, most of your address space is DMA-able memory and therefore "unsafe". In C++, you can completely hide these mechanics behind constructs that look like unique_ptr/shared_ptr but which are guaranteed to be DMA-safe. In this context, all references are treated as mutable because mutability is not a property of references per se. Conflicts can only be resolved at runtime.
Instead of using an ownership-based memory safety model, which is clearly broken for systems that use DMA, they can use schedule-based memory safety models, which are formally verifiable. These don't rely on immutable references for safety, and also eliminate most need for locking -- many of the concepts originate in research on automated lock-graph conflict resolution in databases. The evolution away from ownership-based to schedule-based models is evident in database kernels over the decades. It was originally motivated by improved concurrency; suitability for later DMA-based designs was a happy accident and C++ allows it to be expressed idiomatically.
As for expressiveness, beyond the ability construct alternative safety models, modern C++ metaprogramming facilities enable not only compile-time optimization but also verification of code correctness that would be impractical in languages like Rust.
Instead of using an ownership-based memory safety model, which is clearly broken for systems that use DMA, they can use schedule-based memory safety models, which are formally verifiable. These don't rely on immutable references for safety, and also eliminate most need for locking -- many of the concepts originate in research on automated lock-graph conflict resolution in databases. The evolution away from ownership-based to schedule-based models is evident in database kernels over the decades. It was originally motivated by improved concurrency; suitability for later DMA-based designs was a happy accident and C++ allows it to be expressed idiomatically.
As for expressiveness, beyond the ability construct alternative safety models, modern C++ metaprogramming facilities enable not only compile-time optimization but also verification of code correctness that would be impractical in languages like Rust.
Memory safety with no unpredictable gc latency
VHDL is faster no?
Isn’t everything you wrote from the wrong decade?
Isn’t everything you wrote from the wrong decade?
Fpgas have a huge amount of operational and practical difficulties. They’re great for fixed-function stuff like feed parsing or dumb triggers your software can direct in real-time, but absolutely terrible for something where you the ability to be flexible and adaptive.
Pure latency arb isn’t the only hft trade there is anyways.
Pure latency arb isn’t the only hft trade there is anyways.
VHDL is faster no?
The title of the article and the comments here are conflating several things. We had Peter Lawrey consult at one of my employers, and I've worked on both C++ and Java HFT systems.
We used Chronicle as a high performance messaging system. The secret sauce is just memory mapped files, single threaded processes, and using Java objects as flyweights that just write to the end of the file, or are read from the end of the file. No GC. When taking market data off the wire, or sending messages around a system, that is competitive with C++ and certainly has a fast time to market, and lower defect rate.
The other thing people are conflating is high frequency part: get the market data and... do a complex calculation as fast as possible, and place or pull an order. Placing or pulling orders can be further optimized too.
The fast complex calculation part is where Java isn't as competitive and you run into the GC unless you use object pooling, warming up the JVM and so on. We had a 50-50 system of Java for the bulk of the system, and C++ for the complex algorithms where we mark hot paths and check the assembly on godbolt (thank you Matt Godbolt).
We used Chronicle as a high performance messaging system. The secret sauce is just memory mapped files, single threaded processes, and using Java objects as flyweights that just write to the end of the file, or are read from the end of the file. No GC. When taking market data off the wire, or sending messages around a system, that is competitive with C++ and certainly has a fast time to market, and lower defect rate.
The other thing people are conflating is high frequency part: get the market data and... do a complex calculation as fast as possible, and place or pull an order. Placing or pulling orders can be further optimized too.
The fast complex calculation part is where Java isn't as competitive and you run into the GC unless you use object pooling, warming up the JVM and so on. We had a 50-50 system of Java for the bulk of the system, and C++ for the complex algorithms where we mark hot paths and check the assembly on godbolt (thank you Matt Godbolt).
I was an intern in a startup years ago where they had a product that was a mix of java and C++. the company went bankrupt on that decision. there was only one engineer who understood the java/C/C++ interface at the time and that interface was responsible for all the bugs and performance problems.
Well this was the reverse of a startup - an established company that had a working system, historically in Java, and then added a mixture of C++ processes, and Java-C++ interop libraries to improve performance.
Picking two significant languages as a start up is a bold choice.
Measurement and QA is everything in terms of performance.
Picking two significant languages as a start up is a bold choice.
Measurement and QA is everything in terms of performance.
I don't think that matters. It all stands or falls on whether you have people that are capable of bridging both worlds correctly, and those people are rare, and I see no advantage to using java. So why bother? Just hire good C++ developers and you don't need to go find the ultra rare masterminds who can bridge both worlds. The time to market argument is debatable as there are an enormous number of libraries for C and C++ available and a lot of language features have been added to recent standards to address this issue. I also think that one of the key problems with java is that it forces the OOP paradigm on the user.
I used to work on a high-speed messaging system built in Java, and garbage collection was a killer. No matter how much we tweaked the code and the GC parameters, and this eventually got escalated pretty high up in Sun itself, we'd get these occasional multi-minute pauses where the whole thing would just freeze up, buffers would fill and stuff would start dropping on the floor.
This was over 10 years ago, so things may have improved, but it would certainly give me pause about ever implementing anything time-critical in Java.
This was over 10 years ago, so things may have improved, but it would certainly give me pause about ever implementing anything time-critical in Java.
Currently developing a radio communications system using Java, with JNI calls for hardware integration, running on a quad-core 32-bit ARM processor. We have constraints of around 20ms for real-time audio packet processing. Recently some stop-the-world pauses introduced by the GC resulted in calls dropping. A few GC parameter tweaks brought our latency below 20ms and the application no longer drops calls.
This is using the GC that ships with JDK 11. Future improvements, such as Shenandoah and ZGC, may be able to further drop the interruptions to 5ms on average.
The radio repeater is designed for real-time, safety-critical operations dealing with voice communications. Java is up for the job; like others have mentioned, it's more about the technical team and whether management understands what it takes to write and maintain an excellent code base than it is about Java versus C++.
This is using the GC that ships with JDK 11. Future improvements, such as Shenandoah and ZGC, may be able to further drop the interruptions to 5ms on average.
The radio repeater is designed for real-time, safety-critical operations dealing with voice communications. Java is up for the job; like others have mentioned, it's more about the technical team and whether management understands what it takes to write and maintain an excellent code base than it is about Java versus C++.
>A few GC parameter tweaks brought our latency below 20ms and the application no longer drops calls.
Tick to trade latency for a modern software HFT stack needs to be under 10 _microseconds_, which is over three orders of magnitude more stringent.
Tick to trade latency for a modern software HFT stack needs to be under 10 _microseconds_, which is over three orders of magnitude more stringent.
10ms is target at which percentile for HFT? From what I could tell in my limited experience, GC impact is worse at the tail of latency distribution
10ms is milliseconds though not micro, that's 10us
> 10ms is target at which percentile for HFT?
They already just said they need 10 microseconds. 10 milliseconds is far far too slow.
They already just said they need 10 microseconds. 10 milliseconds is far far too slow.
1000x slow so its unacceptable .
It would be interesting to know what the trade-off for those tweaks was. Typically it's increased memory usage.
Another relevant aspect is what happens when one hits GC pauses again. Will there be any tweaks left? Will they conflict with the previous ones? Using such high-level levers to fix performance problems in a specific component is great when it works, but when it doesn't you're out of options.
Another relevant aspect is what happens when one hits GC pauses again. Will there be any tweaks left? Will they conflict with the previous ones? Using such high-level levers to fix performance problems in a specific component is great when it works, but when it doesn't you're out of options.
Another option is to use a JVM that offers hard real-time constraints to achieve guaranteed performance characteristics. Such JVMs are in use in the automotive and airline industries---JamaicaVM, for example, offers deterministic garbage collection.
You’re coding safety critical systems and depending on severely non-deterministic tweaks to GC to get it to run right?! Where is this for? (Just so I can avoid it)
No need to know where this person is deploying this, if you want to avoid people doing stupid shit with safety critical systems you should just avoid going outside. Actually, avoid leaving the bed, as everything software in this day and age is basically just duct tape and cable ties put together in a rush to hit deadlines.
Have direct knowledge or worked with many critical systems in a few different areas (cannot get more specific for several legal reasons). Can confirm. I'm given to understand that legacy systems (think 30ish years ago) were much better engineered, but there's not enough Proper Engineers(tm)(r) to design (and build) all the critical systems that are necessary.
The biggest issues are that an engineer with the knowhow is expensive, a team of them is prohibitively expensive, a lot of companies outsource this work to different countries and make them sign onerous and dubiously-enforceable contracts, pay little, demand too much, and have incredibly unrealistic deadlines for the type of work that needs to happen.
You know how it is, if it never fails, you're throwing money out. That's the mentality of a lot of management types I've dealt with.
The biggest issues are that an engineer with the knowhow is expensive, a team of them is prohibitively expensive, a lot of companies outsource this work to different countries and make them sign onerous and dubiously-enforceable contracts, pay little, demand too much, and have incredibly unrealistic deadlines for the type of work that needs to happen.
You know how it is, if it never fails, you're throwing money out. That's the mentality of a lot of management types I've dealt with.
> You’re coding safety critical systems and depending on severely non-deterministic tweaks to GC to get it to run right?!
That's certainly one way to look at over five years of engineering effort, extensive systems testing, countless hours of automated regression test development, independent product quality verification, rigorous code reviews, risk assessment processes, mandatory four 9s call quality requirements, integration analysis of JVMs with deterministic GC, and so on.
Judge not a galaxy from a single star.
That's certainly one way to look at over five years of engineering effort, extensive systems testing, countless hours of automated regression test development, independent product quality verification, rigorous code reviews, risk assessment processes, mandatory four 9s call quality requirements, integration analysis of JVMs with deterministic GC, and so on.
Judge not a galaxy from a single star.
> real-time, safety-critical operations
Doesn't Java itself say it shouldn't be used for anything like this?
Doesn't Java itself say it shouldn't be used for anything like this?
TRWTF is that Java Specification Request 1 (yes, THE very first one) targeted real-time: https://www.jcp.org/en/jsr/detail?id=1
Most software does. But, let's face it, it's not like C or C++ compilers should come with a manual saying "We strongly recommend using C for safety critical systems".
(MISRA-C and other odd dialects excepted)
(MISRA-C and other odd dialects excepted)
Nope, there are even two vendors specialized in real-time Java based solutions.
You should not have any full GC (pause) events, ever, in properly written production Java code.
Source: I worked on several extremely high performance Java systems at Sun and also worked optimizing various Java servers at subsequent jobs based on my Sun experience.
At one company (after Sun) I took on a backend service which was doing GC pauses every 3-5 minutes, the code was a dumpster fire mess. After cleaning things up we didn't hit a single full GC in two months of production use (they redeployed every two months so that's as long as the process lasted).
Source: I worked on several extremely high performance Java systems at Sun and also worked optimizing various Java servers at subsequent jobs based on my Sun experience.
At one company (after Sun) I took on a backend service which was doing GC pauses every 3-5 minutes, the code was a dumpster fire mess. After cleaning things up we didn't hit a single full GC in two months of production use (they redeployed every two months so that's as long as the process lasted).
Extremely high performance properly written Java code has been mentioned so many times in Java performance discussions that it has acquired an almost legendary status, right there with the sufficiently smart compiler.
I have observed the former, written by exceptional coworkers, but never the latter.
Maybe, but that just raises the question...If you're coding your system under the constraint that you should not be having any GC-event, what's the point of picking a GC language in the first place?
As others said, no full GC events. Young generation GC is cheap, no need to avoid that.
Productivity is the point. Even while paying attention to GC, it's much faster to write good code in Java than to worry about every malloc() in C. I love C, but if I need to churn out high performance code quickly, Java is the choice.
Productivity is the point. Even while paying attention to GC, it's much faster to write good code in Java than to worry about every malloc() in C. I love C, but if I need to churn out high performance code quickly, Java is the choice.
But here we're talking about C++: you don't write malloc, you use containers that encapsulate memory management - likely std::containers with pool allocators. And you put things on the stack. Code is as readable if not more than java - there isn't any "new," anywhere in sight
Modern C++ is probably about as good, with sufficient team discipline. I don't have enough recent experience with C++ to have a strong opinion.
IMO (possibly biased by how much I hated C++ in the 90s) it is easier to maintain clean code discipline in a large team with Java than it is with C++.
IMO (possibly biased by how much I hated C++ in the 90s) it is easier to maintain clean code discipline in a large team with Java than it is with C++.
C++ after 11 is a very different beast. It's more pleasant to write compared with pre-11. Memory model, move semantics, smart pointers, lots of stuff standardized instead of left to the libraries.
Definitely depends on your team of course.
Definitely depends on your team of course.
smart pointer template blah, instead of pointer blah, in a very large code base.. hmm not so pleasant.. the debug level function definitions, adding templated parameters, are literally unreadable. I will agree that the mental model is tamer.
GP was not talking about "any GC-event", but about "any full GC (pause) events". That's where the heap has gotten so messy that everything has to stop while the GC sorts it out. It's an absolute worst case scenario. Under normal conditions on a modern JVM, the GC runs in parallel with the application and does not cause any pauses.
They said no full GC. The GC runs constantly in parallel with your program and cleans up what it can, but never has to pause execution of the whole program to reclaim even more memory. Not claiming I know how to do this, but I believe this is what they claiming.
Well, the first place may have predated you and a rewrite is out of the question.
A few other reasons:
The rest of the platform for review and analysis doesn’t have such stringent performance requirements and it would be nice to reuse code.
There also may be some good libraries that are jvm only that your program relies on.
Packing up a fat jar can be a lot easier, less complex and more reliable than building a binary.
Your dev team has a lot Java expertise.
Getting Java to not allocate is not as hard as it seems if you set out to do it from the start. And even if not it’s not impossible because the tooling is so good. You just have to know the right tricks and it can be easier to incrementally learn those tricks for a dev team than learn C++ and it’s ecosystem.
A few other reasons:
The rest of the platform for review and analysis doesn’t have such stringent performance requirements and it would be nice to reuse code.
There also may be some good libraries that are jvm only that your program relies on.
Packing up a fat jar can be a lot easier, less complex and more reliable than building a binary.
Your dev team has a lot Java expertise.
Getting Java to not allocate is not as hard as it seems if you set out to do it from the start. And even if not it’s not impossible because the tooling is so good. You just have to know the right tricks and it can be easier to incrementally learn those tricks for a dev team than learn C++ and it’s ecosystem.
I agree, because I've seen that in my previous jobs...
Programmers used to say that the GC and its fuc*#=+ STW was a pure mess, but under the hood, they didn't realize (accept ?) in fact the code WAS the mess.
Don't (always) blame the tool !
Programmers used to say that the GC and its fuc*#=+ STW was a pure mess, but under the hood, they didn't realize (accept ?) in fact the code WAS the mess.
Don't (always) blame the tool !
Did you ever detailed on the Internet about those methods to attain no GC pause? It would be interesting to learn from your experience
It's not rocket science honestly. Mostly any Java performance guide will tell you how.
Allocate either locally on the stack or allocate for the life of the process so it never gets GC'd.
Of course, spend time profiling to make sure you got it. Figure out your redeployment cadence and that sets the ceiling for how much data can be an exception to these rules.
(e.g. if you redeploy nightly then it's fine to grow the memory consumption as long as you don't hit full GC in less than 24hrs.)
In a nutshell that's it.
Allocate either locally on the stack or allocate for the life of the process so it never gets GC'd.
Of course, spend time profiling to make sure you got it. Figure out your redeployment cadence and that sets the ceiling for how much data can be an exception to these rules.
(e.g. if you redeploy nightly then it's fine to grow the memory consumption as long as you don't hit full GC in less than 24hrs.)
In a nutshell that's it.
Another user commented here a while ago that they solved this problem by just throwing it on the largest instance by memory (1TB or so), disabled GC and restart it after market closes.
Largest instance sounds like aws. There’s your nr 1 performance bottleneck
well, you choose the node infrastructure by whatever network latency you need to have guaranteed - obviously public clouds have their limits there.
But it’s idiotic to discuss ns-optimizations, when your infrastructure and vms add many times more that.
Aws is slow. Not talking about the network.
Aws is slow. Not talking about the network.
ehh I didn't even mention AWS - all I did was using the word 'instance' which seems to have thrown you off.
What else can "instance" mean, other than an instance of a virtual server? Granted may not be on AWS, but still is virtual which is a latency killer.
IMO you can't automatically go from "we have a latency requirement" to "public cloud is out of the question". It depends on what latency requirement. If you can eat an intermittent delay of 20ms, AWS, to me still seems fine, you just need to plan ahead in terms of what resources you rent there. Between that, and the multiple seconds a GC pause could cost you, there is still a space where GC optimization on AWS would make sense. But if you're in HFT and target micro- or even nanosecond optimizations, you sure don't wanna run anything in that critical path on any public cloud.
Why did your system dynamically allocate memory in the first place? High speed messaging sounds like an area where it should be doable to figure out at startup what resources you need to meet your latency and throughput requirements and then allocate them once and basically never gc anything. Was there some fundamental difficulty with that, or was it more that by the time your realized that GC would kill you, all your code was already written in a way that would have made it prohibitively expensive to switch to an architecture that avoids creating temporary objects?
Pretty much the latter: it was initially architected for 1x volume, but ended up handling 100x what was originally anticipated. So the engines were rebuilt a few times while the plane was in flight. As a bonus, while in production the system had to run 24x7 and upgrades were a fairly fraught manual affair involving hours of testing on the customer side.
The other major complication was that this was doing a lot of complicated protocol conversion, so it wasn't enough for the core message processing & routing to be solid and leak-free, all the input parsers and output converters had to be too.
The other major complication was that this was doing a lot of complicated protocol conversion, so it wasn't enough for the core message processing & routing to be solid and leak-free, all the input parsers and output converters had to be too.
The best part about using a garbage collected language for high performance software is it's exactly the same as a non-garbage-collected language.
Sure, as long as your non-garbage-collected language from ten years ago had memory safety (and general lack of UB footguns), first class IDE support, excellent introspection and profiling facilities and a large potential hiring pool of elite programmers and wide acceptance in the industries you were trying to sell into.
Why is it you think most major programs are built using C++? Most OS's, most drivers, most browsers, most games, most major applications, most compilers, most servers etc etc.
C++ completely dominates games, browsers (and a few things you don't list, such as competitive programming). It is certainly not marginal in the other areas you list either, but were do you get the weird idea that most operating systems and drivers are written in C++ (let alone most compilers, servers or "major applications")? Unix has completely won the OS market (what percentage of OS installs these days are unix derivatives? 99%+?) and I can't think of a single one that is predominantly or at all written in C++. Apart from lacking a browser, I'd be surprised if you couldn't run a fully functional linux desktop after deleting every single C++ application and library on your system.
Most servers, definitely not.
Consider that large chunks of Google, Twitter, Amazon, Alibaba etc run on Java.
Consider that large chunks of Google, Twitter, Amazon, Alibaba etc run on Java.
Wait, are we still comparing C++ with Java here?
Yes? If you write zero or low allocation code in Java you pay a price for working against the language and you also lack several tools for writing fast code that C++ provides you with (as well as a bunch of useful abstractions), but your development experience will obviously be much better on a number of other dimensions.
It's changed quite a bit, with some of the biggest developments just in the last couple of years. It should still give you pause (depending on how time-critical) but now you have more choices in pause and more pause-knobs.
Cassandra has some of these issues as well, despite arena allocation, buffer reuse, and offheap manually allocated memory.
10 years has seen improvements to GC, but not eliminated the problem.
Cass 4.0 finally enables ZGC, so we'll see how that goes.
If you have stateless, you could do two JVMs and monitor the GC state, and reroute requests to a low-heap JVM and let the other clean up.
Garbage collection is hard.
I'm surprised that there aren't expert-level concepts like weak references and ways to sub-partition the heap, with say a new operator that can target a partition, so major GC doesn't have to do the whole heap. Although that starts getting into rust concepts and analysis.
10 years has seen improvements to GC, but not eliminated the problem.
Cass 4.0 finally enables ZGC, so we'll see how that goes.
If you have stateless, you could do two JVMs and monitor the GC state, and reroute requests to a low-heap JVM and let the other clean up.
Garbage collection is hard.
I'm surprised that there aren't expert-level concepts like weak references and ways to sub-partition the heap, with say a new operator that can target a partition, so major GC doesn't have to do the whole heap. Although that starts getting into rust concepts and analysis.
It definitely improved. The GC is the area where we see more work between the Java versions
While I might not agree with the conclusion of the OP, the performance of a programming language should be always paired with its "average" programmers. A lot of people tend to straw-man a language they dislike and iron-man (?) a language they like. For an average programmer, I think it might be possible that Java produces a more performant/maintainable code than C++.
This is very true. I've seen colleagues port Python to C++ and wonder why it's slower.
Of course Python is slow, but it will call into efficient C routines for a lot of things that aren't just available out of the box in C++. So what does the average programmer do? They implement their own, inefficient version of this in C++.
Now you've saved the overhead of copying your data from the Python to the C-world and back, but the main part of your computations is just not on the level of numpy.
So naturally C++ can be much faster than Python, because it doesn't have this overhead, but you have to do it right, which may be more effort.
Of course Python is slow, but it will call into efficient C routines for a lot of things that aren't just available out of the box in C++. So what does the average programmer do? They implement their own, inefficient version of this in C++.
Now you've saved the overhead of copying your data from the Python to the C-world and back, but the main part of your computations is just not on the level of numpy.
So naturally C++ can be much faster than Python, because it doesn't have this overhead, but you have to do it right, which may be more effort.
When I moved from C/C++ to Java on the 90s, I was amazed how casually things like hash tables would be used. A single function might create a couple hash tables, do some data juggling and sorting, and solve a problem in like 20 lines of readable, performant code. In my experience, C coders almost invariably employ less performant algorithms and data structures due to the incidental complexity involved in memory allocation, pointer indirection, etc.
absolutely. Few years ago i got back deep into C++ after 18 years of mostly Java, and i'm amazed, it is like travel back in time - we (a large C++ platform) are looping over vectors instead of using hashtables/maps, copying strings all the time (we have 4 major types of strings plus some minor ones), and don't get me started on multithreading and locking/synchronization - such an easy and natural thing in Java - even these days the people in C++ world is still seem to be hesitant to use locking and as result of that hesitance push themselves into a myriad of multithreading issues when they risk and/or are forced to use threads. It a sight to behold when a customer waits for that one core/thread to finish the operation on a 400 core 16TB RAM machine - if it were Java all the 400 cores would be burning, the performance would still feel kind of so-so, and the customer would be building up the nerve to upgrade to 800 core 32 TB :)
>pointer indirection
30+ years ago, a student at University, i loved how pointers allowed to code algorithms efficiently and expressively compare to the non-pointer languages of the time. In the industry today the situation is completely opposite as correctly noted by the parent.
>pointer indirection
30+ years ago, a student at University, i loved how pointers allowed to code algorithms efficiently and expressively compare to the non-pointer languages of the time. In the industry today the situation is completely opposite as correctly noted by the parent.
In many cases n is small enough that looping over a vector is faster than a hash lookup. The vector loop is very ry cache friendly, while the hash lookup typically involves a cache miss or two.
it would be for a vector where values are stored in the vector entries. Instead it is usually pointers to strings or objects. So you pull from memory by pointer (i.e. cache unfriendly) every string, compare, etc... And small n is usually small until customer actually generates couple orders of magnitude more of those objects than it was in test/dev :)
I feel like often the opposite can be true as well. Obviously hashtables are better for large collections but often collections are very small (sizes typically follow a power law; most strings will be smaller than the pointer to their first character; most collections will only have a few elements) and arrays can win here due to less indirection and better locality.
Compare this to many functional languages (or lisps) where the most convenient data structure to hand is the singly linked list. It was already bad for performance when those languages were invented but in modern times they are relatively much worse than before.
Sometimes I think that the performance of C programs come more from the fact that it such a massive pain to do anything in C that you can usually only do the simplest thing and this tends to be fast and reliable. The problem is that if you can’t do a simple thing it will either take a lot of refactoring or you’ll do a complicated thing and your program will randomly segfault.
This is also my theory as to why languages like C have better libraries than lisp: it is such a monumental pain to do anything in C that people go to the small extra effort to package up their achievement into a library either for others or in case they need to solve the trivial problem again themselves. Improvements can them come later as needed but get shared. Compare this to, for example, lisp where the attitude is usually that libraries aren’t flexible enough and generally not that hard to implement yourself (so long as you aren’t so worried about data structures), and I think this is the reason there don’t tend to be so many libraries, especially for trivial things.
Compare this to many functional languages (or lisps) where the most convenient data structure to hand is the singly linked list. It was already bad for performance when those languages were invented but in modern times they are relatively much worse than before.
Sometimes I think that the performance of C programs come more from the fact that it such a massive pain to do anything in C that you can usually only do the simplest thing and this tends to be fast and reliable. The problem is that if you can’t do a simple thing it will either take a lot of refactoring or you’ll do a complicated thing and your program will randomly segfault.
This is also my theory as to why languages like C have better libraries than lisp: it is such a monumental pain to do anything in C that people go to the small extra effort to package up their achievement into a library either for others or in case they need to solve the trivial problem again themselves. Improvements can them come later as needed but get shared. Compare this to, for example, lisp where the attitude is usually that libraries aren’t flexible enough and generally not that hard to implement yourself (so long as you aren’t so worried about data structures), and I think this is the reason there don’t tend to be so many libraries, especially for trivial things.
Yeah I agree with this (apart from the lisp commentary, which I have no insight into).
This is where I always hope a JVM will learn that, use escape analysis to keep things off the heap (they can do this), swap out data structures for more efficient ones for small amounts of data, or tune data structure initialization params (they can't do this).
For that reason I generally motivate not using python to people on grounds of correctness rather than performance as per se. I can't be bothered to explain how compiler optimizations work so I usually just resort to "it's magic" when it comes to performance.
Going from a statically typed language to using python to do number crunching genuinely makes me want to vomit. I don't understand how people convince themselves that it's productive to write so many tests and (say) check types at runtime etc.
I actually love Python's syntax but the language design seems like it was thrown together over a weekend.
Going from a statically typed language to using python to do number crunching genuinely makes me want to vomit. I don't understand how people convince themselves that it's productive to write so many tests and (say) check types at runtime etc.
I actually love Python's syntax but the language design seems like it was thrown together over a weekend.
Take a look at Nim if you haven't already -
It has a Pythonesque syntax, but strong static typing with extensive user control of semantics you seem to care about; e.g. floats and ints do not convert automatically unless you explicitly "import lenientops"[0] ; You can define 'operational transform' optimizations (such as: c <- c+a\b converts to multiply_accumulate(c,a,b) - which is a big performance difference for e.g matrices) that will be applied by the compiler so that your code shows what you mean (c=c+ab), and yet the compiler gets to compile the efficient version (using the relevant BLAS routine)
It's young, but has the best FFI for C,C++,Objective-C or JS you'll find anywhere on one hand, and already a good deal of native implementations, including e.g. a pure Nim BLAS that is comparable to within 5-10% with the best out there (including those with carefully hand optimized ASM kernels).
[0] https://nim-lang.org/docs/lenientops.html
It has a Pythonesque syntax, but strong static typing with extensive user control of semantics you seem to care about; e.g. floats and ints do not convert automatically unless you explicitly "import lenientops"[0] ; You can define 'operational transform' optimizations (such as: c <- c+a\b converts to multiply_accumulate(c,a,b) - which is a big performance difference for e.g matrices) that will be applied by the compiler so that your code shows what you mean (c=c+ab), and yet the compiler gets to compile the efficient version (using the relevant BLAS routine)
It's young, but has the best FFI for C,C++,Objective-C or JS you'll find anywhere on one hand, and already a good deal of native implementations, including e.g. a pure Nim BLAS that is comparable to within 5-10% with the best out there (including those with carefully hand optimized ASM kernels).
[0] https://nim-lang.org/docs/lenientops.html
I’d second that. I’ve been doing a bit of algorithm/NN stuff in Nim this year since it simplifies the “deployment” phase and it’s been surprisingly handy despite the limited libraries as there’s a C library for everything. It has a few rough edges but manageable.
It has the spirit of Python 2 but written by a compiler geek (in the good sense). If I were to write an HFT it’d be tempting to use. The new default GC is reference counting but without using atomic ref counts.
P.S. thanks for the lenientops tip
It has the spirit of Python 2 but written by a compiler geek (in the good sense). If I were to write an HFT it’d be tempting to use. The new default GC is reference counting but without using atomic ref counts.
P.S. thanks for the lenientops tip
I am involved with the foundation that runs the D programming language so I've found my home.
Other than syntax obviously.
FFI: Can nim use a C++ class and vtables? D does it all the time, nearly every language proclaims to have the best C++ interop but only D seems to be actually able to do it. Templates, classes, structs, and more all work.
We also have Mir, which I haven't benchmarked for a while but was faster than OpenBLAS and Eugene a few years ago and was recently shown to be faster than numpy (the c bits) on the forum.
Other than syntax obviously.
FFI: Can nim use a C++ class and vtables? D does it all the time, nearly every language proclaims to have the best C++ interop but only D seems to be actually able to do it. Templates, classes, structs, and more all work.
We also have Mir, which I haven't benchmarked for a while but was faster than OpenBLAS and Eugene a few years ago and was recently shown to be faster than numpy (the c bits) on the forum.
> FFI: Can nim use a C++ class and vtables? D does it all the time, nearly every language proclaims to have the best C++ interop but only D seems to be actually able to do it. Templates, classes, structs, and more all work.
Well, it depends on how you define "best". Beyond an ABI, FFI is obviously a function of the implementation rather than the language per-se.
The main Nim implementation can use C++ as a backend language, and when it does, it can use any C++ construct very easily by way of the .emit and .importcpp directives. For sure, classes, structs, exceptions all work, and IIRC templates do too (although you might need to instantiate a header yourself for each concrete type or something .... haven't done that myself). This implementation also means that it can use any C++17 or C++20 construct, including lambdas and friends. Does D's C++ interop support C++17? C++20? Can you guarantee it will support C++27? Nim's implementation already does, on every single platform you'll be able to use C++27 on (as long as C++27 can compile modern C++ code; there had been backward incompatible changes along the C++ history).
You can't just #include a C or C++ header and call it a day; You need to have a Nim compatible definition for any symbol (variable, macro, function, class, ...). There are tools that help you and make it almost as easy as #include, such as nimterop[0] and and nimline[1], and "c2nim" which is included with the Nim compiler is enough to generate the Nim definitions from the .h definitions (though it can't do crazy metaprogramming; if D can do that, then D essentially includes a C++ compiler. Which is a fine way to get perfect C++ compatibility - Nim does that)
But Nim can also do the same for JS when treating JS as a backend.
And it can basically do the same for Python, with nimpy[2] and nimporter, generating a single executable that works with your installed Python DLL (2.7, 3.5, 3.6, 3.7) - which is something not even Python itself can do. There was a similar Lua bridge, but I think that one is no longer maintained.
> We also have Mir, which I haven't benchmarked for a while but was faster than OpenBLAS and Eugene
There's quite a bit of scientific stack built natively with Nim. It is far from self-sufficient, but the ease with which you can use a C library makes up for it. I haven't used it, but Laser[3] is on par with or exceeds OpenBLAS speedwise, and generalizes to e.g. int32 and int64 matrix multiplication; Arraymancer[4] does not heve all of numpy's functionality but does have quite a few nice bits from scikit-learn, supports CUDA and OpenCL, and you can use numpy through nimpy if all else fails. Also notable is NimTorch[5]. laser and arraymancer are mostly developed by mratsim, who occasionally hangs out here on HN.
D is a fine language, I used it a little in the D1 days, and it was indeed a "better C++" but did not deliver enough value to be worth it for me, so I stopped. I know D2 is much better, but I've already found my better C++ (and better Python at the same time!) in Nim, so I haven't looked at it seriously.
[0] https://github.com/nimterop/nimterop
[1] https://github.com/sinkingsugar/nimline
[2] https://github.com/yglukhov/nimpy
[3] https://github.com/numforge/laser
[4] https://github.com/mratsim/Arraymancer
[5] https://github.com/sinkingsugar/nimtorch
Well, it depends on how you define "best". Beyond an ABI, FFI is obviously a function of the implementation rather than the language per-se.
The main Nim implementation can use C++ as a backend language, and when it does, it can use any C++ construct very easily by way of the .emit and .importcpp directives. For sure, classes, structs, exceptions all work, and IIRC templates do too (although you might need to instantiate a header yourself for each concrete type or something .... haven't done that myself). This implementation also means that it can use any C++17 or C++20 construct, including lambdas and friends. Does D's C++ interop support C++17? C++20? Can you guarantee it will support C++27? Nim's implementation already does, on every single platform you'll be able to use C++27 on (as long as C++27 can compile modern C++ code; there had been backward incompatible changes along the C++ history).
You can't just #include a C or C++ header and call it a day; You need to have a Nim compatible definition for any symbol (variable, macro, function, class, ...). There are tools that help you and make it almost as easy as #include, such as nimterop[0] and and nimline[1], and "c2nim" which is included with the Nim compiler is enough to generate the Nim definitions from the .h definitions (though it can't do crazy metaprogramming; if D can do that, then D essentially includes a C++ compiler. Which is a fine way to get perfect C++ compatibility - Nim does that)
But Nim can also do the same for JS when treating JS as a backend.
And it can basically do the same for Python, with nimpy[2] and nimporter, generating a single executable that works with your installed Python DLL (2.7, 3.5, 3.6, 3.7) - which is something not even Python itself can do. There was a similar Lua bridge, but I think that one is no longer maintained.
> We also have Mir, which I haven't benchmarked for a while but was faster than OpenBLAS and Eugene
There's quite a bit of scientific stack built natively with Nim. It is far from self-sufficient, but the ease with which you can use a C library makes up for it. I haven't used it, but Laser[3] is on par with or exceeds OpenBLAS speedwise, and generalizes to e.g. int32 and int64 matrix multiplication; Arraymancer[4] does not heve all of numpy's functionality but does have quite a few nice bits from scikit-learn, supports CUDA and OpenCL, and you can use numpy through nimpy if all else fails. Also notable is NimTorch[5]. laser and arraymancer are mostly developed by mratsim, who occasionally hangs out here on HN.
D is a fine language, I used it a little in the D1 days, and it was indeed a "better C++" but did not deliver enough value to be worth it for me, so I stopped. I know D2 is much better, but I've already found my better C++ (and better Python at the same time!) in Nim, so I haven't looked at it seriously.
[0] https://github.com/nimterop/nimterop
[1] https://github.com/sinkingsugar/nimline
[2] https://github.com/yglukhov/nimpy
[3] https://github.com/numforge/laser
[4] https://github.com/mratsim/Arraymancer
[5] https://github.com/sinkingsugar/nimtorch
>Going from a statically typed language to using python to do number crunching genuinely makes me want to vomit. I don't understand how people convince themselves that it's productive to write so many tests and (say) check types at runtime etc.
Totally agree, although I think strong typing is also important, not just static typing. C/C++ has a weak type system because of the automatic conversions it allows.
The problem with this discussion however is that the people that push for dynamic languages usually do it because it's "so fast" to do something in Python, and they won't test their software anyway.
Totally agree, although I think strong typing is also important, not just static typing. C/C++ has a weak type system because of the automatic conversions it allows.
The problem with this discussion however is that the people that push for dynamic languages usually do it because it's "so fast" to do something in Python, and they won't test their software anyway.
Python is great for small programs that barely need any tests because you can tell it is correct by reading the code or running it until you get the right answer.
when I have to maintain a large python program I agree, it is too easy to miss some error condition that doesn't work at all (anymore)
It made more sense before the time of IDEs and code completion. I used to greatly prefer python to c++ or java when I actually had to type full names and look up interfaces like a pleb.
Now I can't stand untyped python. Have to actually look at documentation to see what methods a class has, who has time for that?
Writing golang with tabnine is otherworldly. It's so regular, not just the syntax, but the variable naming conventions, and course structure even, that it feels like loosely nudging the autocomplete in the direction of the goal.
Now I can't stand untyped python. Have to actually look at documentation to see what methods a class has, who has time for that?
Writing golang with tabnine is otherworldly. It's so regular, not just the syntax, but the variable naming conventions, and course structure even, that it feels like loosely nudging the autocomplete in the direction of the goal.
An iron man is a triathlon, you are looking for steelman!
https://en.wiktionary.org/wiki/steelman
https://en.wiktionary.org/wiki/steelman
There is (was?) a Steelman triathlon too (actually, more than one of them, in different parts of the US).
When you write Java in a C like syntax : "Chronicle's system is built in what Lawrey describes as "C-like Java" and it's this that he encourages people to code low latency systems in."
I guess very limited number of constructors, and most of all the methods are static.
And they still struggle to find programmers that can code that way.
I guess very limited number of constructors, and most of all the methods are static.
And they still struggle to find programmers that can code that way.
Avoiding allocations is the key part: "...we avoid all the garbage". I found the same is true when trying to write high performance Java UI code for Android. If you have to do something complex each frame, make sure to pre allocate or pool your objects. If the hot parts of the code are written in a C-like style, then the JIT and other optimizations can give you C-like performance. You'll still need to write C code if you want vector operations or other processor-specific functionality, but writing in C-like Java can give you C performance while still letting you interact with Java libraries & APIs.
I've found that coding in a "C-like style" offers great performance too. Are there any languages that target the JVM and are designed for higher speed? If it enforced the "C-like style" at the language level, perhaps it would be easier to follow?
Of course at this point the usual answer is just use Rust, but is there a language that meets in the middle? Sometimes I just want the GC to do the work and I'm okay with that.
Of course at this point the usual answer is just use Rust, but is there a language that meets in the middle? Sometimes I just want the GC to do the work and I'm okay with that.
I find Java itself works quite well for this. Especially newer Java versions with functional interfaces. You keep your data in data centric classes (e.g. something akin to struct-of-arrays in C) and have some functional interfaces to access this data, the only place where you loop over it, maybe in batches if it's really large (test it though, up to 10s or 100s of millions of elements is still fast to loop through in memory). Then you got some algorithm centric classes that you can plug into these interfaces. And finally, after reducing the data you may have some classic objects, e.g. to represent results.
edit, to give you an example. Let's say you have double precision X/Y coordinates. Put that into a class with two double arrays for X and Y. In order to e.g. run an Euclidean distance computation against it, provide a (double, double) -> double interface against it. Then you have either an Euclidean distance class providing that function or just an inline lambda that you can plug into that interface, giving you a distance array. Let's now say you just want to have a list of 10 closes points - instead of sorting you're fastest to brute force it. Only after you reduced it down to the 10 points, you put them into Point objects because there's probably a consumer expecting it that way.
edit, to give you an example. Let's say you have double precision X/Y coordinates. Put that into a class with two double arrays for X and Y. In order to e.g. run an Euclidean distance computation against it, provide a (double, double) -> double interface against it. Then you have either an Euclidean distance class providing that function or just an inline lambda that you can plug into that interface, giving you a distance array. Let's now say you just want to have a list of 10 closes points - instead of sorting you're fastest to brute force it. Only after you reduced it down to the 10 points, you put them into Point objects because there's probably a consumer expecting it that way.
[deleted]
> Are there any languages that target the JVM and are designed for higher speed?
Yes: Java
Performant Java looks mostly like C, just with no manual malloc() calls.
Sure you could write Java in a J2EE way but.. that not a good choice.
Yes: Java
Performant Java looks mostly like C, just with no manual malloc() calls.
Sure you could write Java in a J2EE way but.. that not a good choice.
In Java, every object stored on heap (and they can't be stored on stack) has a JVM-induced space overhead. Also, the allocator is free to spread the objects across the heap in a chaotic manner, which will make for less than optimal utilization of cache lines.
That's... not how JVMs work.
All modern VMs (not just limited to Java here!) apply two key optimizations. The first is escape analysis, which checks if references to objects will escape the current function boundary. If not, the objects will be stored on the stack instead of the heap. The second is generational GC, where memory allocation looks like this:
All modern VMs (not just limited to Java here!) apply two key optimizations. The first is escape analysis, which checks if references to objects will escape the current function boundary. If not, the objects will be stored on the stack instead of the heap. The second is generational GC, where memory allocation looks like this:
void *new_ptr = heap_mem;
heap_mem += alloc_size;
if (heap_mem >= max_size)
outlined_function_to_get_larger_blocks_of_memory();
>"but writing in C-like Java can give you C performance while still letting you interact with Java libraries & APIs."
Assuming those libraries do not do allocations of their own negating all the efforts.
Assuming those libraries do not do allocations of their own negating all the efforts.
That's right. Java libraries even within JDK are notorious for unnecessary allocations and locks (which also do allocations for queues) putting GC pressure. C-like java means the need to roll out your own or using libraries like Chronicle.
I imagine that the other libraries would be used only on the non-critical parts of the application.
This is Go and sync.Pool, essentially. He may have better luck finding good Gophers.
> "but writing in C-like Java can give you C performance while still letting you interact with Java libraries & APIs."
It really can't. Quite the opposite. Writing C-like Java is already doomed from the start. Java is faster than C when it comes to OOP. Unless you product is a highly complex OOP nightmare, C will always beat Java to the curb.
GC is not free. The problem with GC is that you pay asynchronously, while allocations are essentially free. But this asynchronous cost is very very hard to measure and to control. Java does not offer the mechanism C/C++ offer for native resource management. It also doesn't do complex optimizations, most notably vectorization. Number crunching performance will always suck in Java. Even if they add all the optimizations in the world, the simple fact remains: Java as a language simply does not allow you to express code in a performant way. That leave the compiler at a double disadvantage. It needs to essentially "convert Java to C++ and guess the most performant interpretation" (we are decades away from this), and it needs to do that within milliseconds (because its a JIT).
It just makes no sense to talk about this. Use Java for the 99% of your product that is not a hotpath, use C for the remaining 1% where you need pure performance. Simple as that.
It really can't. Quite the opposite. Writing C-like Java is already doomed from the start. Java is faster than C when it comes to OOP. Unless you product is a highly complex OOP nightmare, C will always beat Java to the curb.
GC is not free. The problem with GC is that you pay asynchronously, while allocations are essentially free. But this asynchronous cost is very very hard to measure and to control. Java does not offer the mechanism C/C++ offer for native resource management. It also doesn't do complex optimizations, most notably vectorization. Number crunching performance will always suck in Java. Even if they add all the optimizations in the world, the simple fact remains: Java as a language simply does not allow you to express code in a performant way. That leave the compiler at a double disadvantage. It needs to essentially "convert Java to C++ and guess the most performant interpretation" (we are decades away from this), and it needs to do that within milliseconds (because its a JIT).
It just makes no sense to talk about this. Use Java for the 99% of your product that is not a hotpath, use C for the remaining 1% where you need pure performance. Simple as that.
Use the stack and not the heap.
Pre-allocate arrays, avoid collections.
Use LUTs.
Avoid string processing, operate on bits and bytes as much as possible.
Same applies to telecoms-development that has to be fast and predictable.
Pre-allocate arrays, avoid collections.
Use LUTs.
Avoid string processing, operate on bits and bytes as much as possible.
Same applies to telecoms-development that has to be fast and predictable.
>Same applies to telecoms-development that has to be fast and predictable.
Funny enough, I've always thought of it more of writing Java like Erlang than C.
Funny enough, I've always thought of it more of writing Java like Erlang than C.
> When you write Java in a C like syntax
And C is faster still when you write it with Fortran like (column order, and "restrict" mostly) semantics.
An old mentor said to me 30 years ago that "A good programmer writes FORTRAN no matter what language they are using" (he was referring to speed of execution in the context of "good"). It seemed super funny at the time, but there's a lot of truth in that.
And C is faster still when you write it with Fortran like (column order, and "restrict" mostly) semantics.
An old mentor said to me 30 years ago that "A good programmer writes FORTRAN no matter what language they are using" (he was referring to speed of execution in the context of "good"). It seemed super funny at the time, but there's a lot of truth in that.
[deleted]
Why does column order matter? If you translate to the equivalent C/C++ row-major order, it should be laid out in memory the same way.
What do you mean by "if you translate to C/C++ row major order?"
What I meant was: In FORTRAN, you idiomatically do Struct-Of-Array=Column-Major order by default, In C/C++ you do Array-of-Struct or Array-of-Pointer-to-Struct (both of which are Row-Major) by default. The former tends to be much more efficient than the latter in computational code.
It turns out that, often and especially in computationally intensive code, only a small number of fields of an object/record are used in every part of the computation pipelines, but almost all records are.
As a result, if you do your data in column major order, then the cache usage patterns reflect that - whereas if you are in row major order, a lot more field get loaded into cache when they are not needed (thus, much lower cache utilization and lower performance).
FORTRAN did not have structures until quite late (FORTRAN-95 has them for sure, but IIRC FORTRAN-77 and earlier didn't). Thus, the idiomatic way to do stuff is have an SoA=Coiumn-Major implementation, rather than the C/C++ AoS=Row-Major order.
Furthermore, FORTRAN didn't have any pointers. As a result, most code was written with very little pointer chasing / foreign key reference chasing -- which also contributes to efficient use of cache and memory bandwidth.
What I meant was: In FORTRAN, you idiomatically do Struct-Of-Array=Column-Major order by default, In C/C++ you do Array-of-Struct or Array-of-Pointer-to-Struct (both of which are Row-Major) by default. The former tends to be much more efficient than the latter in computational code.
It turns out that, often and especially in computationally intensive code, only a small number of fields of an object/record are used in every part of the computation pipelines, but almost all records are.
As a result, if you do your data in column major order, then the cache usage patterns reflect that - whereas if you are in row major order, a lot more field get loaded into cache when they are not needed (thus, much lower cache utilization and lower performance).
FORTRAN did not have structures until quite late (FORTRAN-95 has them for sure, but IIRC FORTRAN-77 and earlier didn't). Thus, the idiomatic way to do stuff is have an SoA=Coiumn-Major implementation, rather than the C/C++ AoS=Row-Major order.
Furthermore, FORTRAN didn't have any pointers. As a result, most code was written with very little pointer chasing / foreign key reference chasing -- which also contributes to efficient use of cache and memory bandwidth.
Sometimes I feel this approach is simpler than making everything into an Object. It usually results in less files to deal with.
This is also allegedly the programming style of C# used by StackOverflow to squizz performances.
Ex-Wall Street guy here. High speed trading has no societal value and relies on a large number of bad-faith bids and offers - these people are parasites.
Creating liquidity has some value - though not as much as bankers tell themselves.
But HFT does not add liquidity. No one needs subsecond liquidity, but even more important, HFT traders vanish like a fart in a windstorm the moment liquidity is actually challenged.
Market makers are the ones who provide consistent liquidity, because they have to keep inventory of the stocks.
Now look at what HFT does - they flood the wires with hugely out-of-the-money bids and asks, all of which are in bad faith, because they know they won't even get hit or lifted.
So everyone else pays for this, and they get to shake absolutely everyone down for a few pennies, a trillion times a year.
---
It would be trivial to stop them without changing the markets in the least. Even just charging a dollar for every "bad faith" bid or offer made that expires without being within, say, 10% of the actual price, would put a big dent in them. Punitive taxes on trades where the trader doesn't keep the position for more than a few seconds would also be very effective.
Creating liquidity has some value - though not as much as bankers tell themselves.
But HFT does not add liquidity. No one needs subsecond liquidity, but even more important, HFT traders vanish like a fart in a windstorm the moment liquidity is actually challenged.
Market makers are the ones who provide consistent liquidity, because they have to keep inventory of the stocks.
Now look at what HFT does - they flood the wires with hugely out-of-the-money bids and asks, all of which are in bad faith, because they know they won't even get hit or lifted.
So everyone else pays for this, and they get to shake absolutely everyone down for a few pennies, a trillion times a year.
---
It would be trivial to stop them without changing the markets in the least. Even just charging a dollar for every "bad faith" bid or offer made that expires without being within, say, 10% of the actual price, would put a big dent in them. Punitive taxes on trades where the trader doesn't keep the position for more than a few seconds would also be very effective.
I think your definition of HFT is very narrow. Many companies that use the term HFT are not involved in the type of activities described in The Big Short.
> High speed trading has no societal value
And investment banking does?
>Even just charging a dollar for every "bad faith" bid or offer made that expires without being within, say, 10% of the actual price, would put a big dent in them.
Tell me this, what is the actual harm of these orders?
And investment banking does?
>Even just charging a dollar for every "bad faith" bid or offer made that expires without being within, say, 10% of the actual price, would put a big dent in them.
Tell me this, what is the actual harm of these orders?
> And investment banking does?
Whataboutism.
Whataboutism.
most electronic market makers are doing HFT these days though.
"If you're not a really good C++ developer, you can shoot yourself in the foot and bring an entire project down."
I'm tired of hearing this and other myths like "C++ is for geniuses". When are we going to acknowledge that managers only care about shipping feature fast regardless of the quality of the codebase? I work professionally with "productive languages" and I could claim the same thing about javascript and others.
On this article it is very clear that companies won't invest in proper training, quoting: "Most Java developers are web developers and are used to modelling things they can see" and "We began by hiring people from trading floors, but have more recently hired Masters and PhD students"
This is a culture problem, not a language one. If anything, C++ will always perform much better than Java. Claiming the opposite because you assume your coders don't know what they're doing, it's just ridiculous.
I'm tired of hearing this and other myths like "C++ is for geniuses". When are we going to acknowledge that managers only care about shipping feature fast regardless of the quality of the codebase? I work professionally with "productive languages" and I could claim the same thing about javascript and others.
On this article it is very clear that companies won't invest in proper training, quoting: "Most Java developers are web developers and are used to modelling things they can see" and "We began by hiring people from trading floors, but have more recently hired Masters and PhD students"
This is a culture problem, not a language one. If anything, C++ will always perform much better than Java. Claiming the opposite because you assume your coders don't know what they're doing, it's just ridiculous.
As an HFT programmer I like this article because it means less competition for me.
I mean, you can write a very fast system in Java but by the time you accomplish that goal you’ll have spent as much or more time than if you had just used C++.
I mean, you can write a very fast system in Java but by the time you accomplish that goal you’ll have spent as much or more time than if you had just used C++.
Having done both I agree with you. I find it's harder to write non idiomatic code that pretends to be another language than to just spend the time learning that other language.
But I get why people might feel it's worth a try, cpp probably looks more daunting than any other language to a novice. There's a lot of concepts to be mastered before you can write decent cpp, where in a lot of languages you can get started and discover things later. With cpp the abstraction gets very detailed. You might not even care what the difference between stack and heap, value or reference passing is in other languages.
Eg you can write python pretty intuitively and get a hash table as part of the language. If you're writing cpp you need to know how types work, how templates work, and how the STL containers work (allocators, traits, constness etc) just to save a simple hash table.
But I get why people might feel it's worth a try, cpp probably looks more daunting than any other language to a novice. There's a lot of concepts to be mastered before you can write decent cpp, where in a lot of languages you can get started and discover things later. With cpp the abstraction gets very detailed. You might not even care what the difference between stack and heap, value or reference passing is in other languages.
Eg you can write python pretty intuitively and get a hash table as part of the language. If you're writing cpp you need to know how types work, how templates work, and how the STL containers work (allocators, traits, constness etc) just to save a simple hash table.
I recall listening to a recent interview with
Chris Lattner who said the more languages you learn the faster it is to pick up others because of the overlaps.
Working in other languages, the idea of references and values didn't kick in to me, that is until I learned basic C++. I think it's because C++ engineers (well, the ones I would watch anyway) were sticklers for performance and so they'd describe the subtle differences in implementation. Even though I don't code C++ anymore, using it for a short time really helped to develop a better understanding of how code actually works.
Working in other languages, the idea of references and values didn't kick in to me, that is until I learned basic C++. I think it's because C++ engineers (well, the ones I would watch anyway) were sticklers for performance and so they'd describe the subtle differences in implementation. Even though I don't code C++ anymore, using it for a short time really helped to develop a better understanding of how code actually works.
The question is how much of your remaining tick-to-trade is really in the software loop. Realistically at the bleeding edge more and more logic is going into FPGA (and maybe ASIC t some extent). Your software is primarily there to make sure that you've processed the previous tick and re-configured the FPGA rather than actually competing in tick to trade. Given that's the case the software doesn't need to be the fastest, it just needs to be fast enough.
Meanwhile if your logic isn't going into FPGA you may well just not be super competitive anyway and therefore the priority is time to market.
Meanwhile if your logic isn't going into FPGA you may well just not be super competitive anyway and therefore the priority is time to market.
+1, that's how things typically work these days. Note that "processing previous tick" may take a very-very long time (up to fractions of a millisecond) to execute, in which case you still want that part to be as fast as possible.
(Source: hft quant)
(Source: hft quant)
As an HFT developer myself I have to disagree. I worked for tier 1 banks that used both:java and cpp.
The trend is moving towards quicker time to market and simpler implementations and replacing cpp with java. The cpp code bases I had to deal with were old, hard to maintain and easy to break. More often than not cpp was a pretty bad lock in as well. For example a big evil bank very well known here struggled for 3 years to upgrade the compiler on one of those projects. Did not happen to this day, I'm told. Few years later they are still on RHEL6 and gcc 4.
Speed is an important factor in this business, but so is time-to-market.
The trend is moving towards quicker time to market and simpler implementations and replacing cpp with java. The cpp code bases I had to deal with were old, hard to maintain and easy to break. More often than not cpp was a pretty bad lock in as well. For example a big evil bank very well known here struggled for 3 years to upgrade the compiler on one of those projects. Did not happen to this day, I'm told. Few years later they are still on RHEL6 and gcc 4.
Speed is an important factor in this business, but so is time-to-market.
I am afraid tier 1 banks aren't tier 1 in HFT though. Best HFTs are at "small" houses and your tier 1 bank experience isn't reflective of what they do.
It's well known that Goldman Sachs runs on SecDb (a homemade language) and JP Morgan runs on Python.
Banks are not competing on latency. It's never been part of their business model.
Banks are not competing on latency. It's never been part of their business model.
How is replacing with Java better than replacing with C++? Legacy code is a problem, and bad programmers will write bad code in any language. If you need to start over does Java give you anything you can't get with modern C++?
From my personal experience I'd say java is much more forgiving than c/cpp and things like debugging or dependency management make it a tempting option in a time-constrained environment.
It depends on what the edge is. Presumably you're competing for a sub 3us edge but they're not all like that. For slower edges Java might be a better idea.
[deleted]
Do you think it’d be even faster if you used C?
The one thing C++ can do better than C is inline a pipeline of operations using templates/type strictness which is handy for compiling all the (socket -> feed decoding -> book -> model) permutations without some macro trickery.
But otherwise they’re probably the same.
But otherwise they’re probably the same.
Nope. Watch the CppCon video mentioned in one of the top level comments. It's all about not allocating. C or C++ is not the real difference.
Hearing them describe this "C-like Java", I'm surprised C# wasn't a better fit given that it has value-type "structs" that don't have to sit behind a reference on the heap. As far as I know, Java still doesn't have an equivalent feature.
Maybe the JVM's GC is just so much faster that it's worth it?
Maybe the JVM's GC is just so much faster that it's worth it?
A few things. Java had been heavily used in finance for longer than .NET, so there's experience, and lots of existing code. Java's GC is faster (even if it's not being used in the context of HFT). But probably the most important thing is that .NET on Linux is very new (banks are very conservative, small companies, not so much).
.NET on Windows for HFT is a non-starter. In comparison to Linux, you have practically zero control over the OS; you need to turn off unnecessary processes, pin processes to cores, exclude processes from cores and so on.
Having said all that, I do know of at least one hedge fund in London that is mostly C#-based (though I have no idea whether their 'deep' trading code is C# though).
.NET on Windows for HFT is a non-starter. In comparison to Linux, you have practically zero control over the OS; you need to turn off unnecessary processes, pin processes to cores, exclude processes from cores and so on.
Having said all that, I do know of at least one hedge fund in London that is mostly C#-based (though I have no idea whether their 'deep' trading code is C# though).
> But probably the most important thing is that .NET on Linux is very new (banks are very conservative, small companies, not so much).
They are. However I'd say, based on what I saw at Pivotal, that .NET Core is sweeping all before it. It's faster, less crufty and so on. But the real win is that (1) most of it is familiar to folks with previous .NET experience, but (2) your ops teams can get out of running two very different platforms in Windows and Linux.
They are. However I'd say, based on what I saw at Pivotal, that .NET Core is sweeping all before it. It's faster, less crufty and so on. But the real win is that (1) most of it is familiar to folks with previous .NET experience, but (2) your ops teams can get out of running two very different platforms in Windows and Linux.
And put your HFTS at the mercy of windows updates?
Joking aside, C# has only become a solid contender after the fairly recent performance improvements and better Linux support. I could be wrong but I doubt a lot of these systems are built on Windows.
Joking aside, C# has only become a solid contender after the fairly recent performance improvements and better Linux support. I could be wrong but I doubt a lot of these systems are built on Windows.
Am also wondering (I have no experience on that matter) how .NET performs in terms of GC as I remember at some point they introduced a threaded GC that dropped the stop-the-world approach and perform garbage collection in dedicated threads. Anyone got some insights?
If you're using "background GC" (new default), they are doing GC in multiple steps, by running part of GC work in another thread - there are still parts which require suspension of managed threads. Here's some more insight:
https://devblogs.microsoft.com/premier-developer/understandi...
https://devblogs.microsoft.com/premier-developer/understandi...
its on par with the jvm but still has to stop the world sometimes
.NET was slower than the JVM before .NET Core but nowadays it's faster.
> .NET was slower than the JVM before .NET Core but nowadays it's faster.
Nope .NET is still slower than molasses in comparison to most JVM implementations.
Nope .NET is still slower than molasses in comparison to most JVM implementations.
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
.net core is generally faster than java at this point.
.net core is generally faster than java at this point.
.net core is old now as if this last month. Now it’s just .Net 5, which is faster still.
Did I miss the announcement? But AFAIK .NET 5 isn't production ready yet.
.NET 5 was released Nov 10th: https://devblogs.microsoft.com/dotnet/announcing-net-5-0/
Ah yes I missed the announcement. Thanks!
They say VS 16.8 supports it (I have VS 16.8.2) but it doesn't offer .NET 5 projects only core and regular.
Java got popular for HFT before .NET core. c# was just generally slower at the time. It would likely be the preferred choice at this point. Though Rust is really perfect for this kind of thing.
I'm surprised I haven't heard anything about rust in this area. It seems like it could do well what hft people want without the problems of c++. Though I don't know rust, so I'm not sure how well you can force the correct optimizations.
We developed the high-speed routing engine GraphHopper in Java and have never regretted it (think Google Maps not network routers :)). Although I'm unsure if I would have used it for a trading systems as every GC hiccup can be problematic and for our use case GC hiccups need only to be under 10ms.
But still the topic ("trading system" or in our case "routing engine") is already very complicated due to the involved algorithms and C++ likely makes it a bigger challenge also regarding attracting good developers.
The memory waste in Java is the biggest challenge that you have to solve. Usually you solve this via primitive arrays i.e. do not use List<Point> but int[] x and y and before this gets complicated wrap this in an iterator or other thin Object (flyweight pattern?). And today you can use advanced GCs like ZGC or Shenandoah that work well also for >100GB of RAM and you can heavily reduce hiccups. And you need to be aware of some smaller pitfalls e.g. foreach loop or Java streaming API might be slower than the raw loop. But you should never follow rules or trust a "guru" and only use your performance tests to guide development :)
And in non-performance critical sections (70% of the code?) you just do not care and can program "idiomatic Java". Also do not underestimate the crazy work that a modern JIT can do for you.
And now after years of work we are for several aspects better or same memory-wise and speed-wise compared to other open source C++ routing engines. And the interesting thing is that IMO this is because of the tooling and habits (like rigorous testing) which reduces bugs and debugging sessions and slowly shifts your focus from language problems to the actual domain problems. But of course all very subjective.
But still the topic ("trading system" or in our case "routing engine") is already very complicated due to the involved algorithms and C++ likely makes it a bigger challenge also regarding attracting good developers.
The memory waste in Java is the biggest challenge that you have to solve. Usually you solve this via primitive arrays i.e. do not use List<Point> but int[] x and y and before this gets complicated wrap this in an iterator or other thin Object (flyweight pattern?). And today you can use advanced GCs like ZGC or Shenandoah that work well also for >100GB of RAM and you can heavily reduce hiccups. And you need to be aware of some smaller pitfalls e.g. foreach loop or Java streaming API might be slower than the raw loop. But you should never follow rules or trust a "guru" and only use your performance tests to guide development :)
And in non-performance critical sections (70% of the code?) you just do not care and can program "idiomatic Java". Also do not underestimate the crazy work that a modern JIT can do for you.
And now after years of work we are for several aspects better or same memory-wise and speed-wise compared to other open source C++ routing engines. And the interesting thing is that IMO this is because of the tooling and habits (like rigorous testing) which reduces bugs and debugging sessions and slowly shifts your focus from language problems to the actual domain problems. But of course all very subjective.
So they want super efficient but safe code ... sounds like Rust.
Does anyone know of Rust in HTF teams? I think Haskell and OCaml may also be a good fit (and I know for a fact those are used in HFT teams).
Not sure whether this counts but someone did post their experiences of it on reddit.
https://www.reddit.com/r/rust/comments/bhtuah/production_dep...
https://www.reddit.com/r/rust/comments/bhtuah/production_dep...
From job ads I saw a year ago, there's at least one company in London doing HFT Rust.
Also, if memory serves, one of Lawrey's ex-colleagues has written his own version of Chronicle in D that's being used somewhere too.
Also, if memory serves, one of Lawrey's ex-colleagues has written his own version of Chronicle in D that's being used somewhere too.
[deleted]
I have a number of patents revolving around this specific issue, and I can tell you certainly this guy has no idea what he is talking about.
First, let me preface this, the language does matter, but the larger complications revolve around the operating system. Most transactions have a hard real-time deadline of 20ms or less. JVM creates additional issues on top of what are already present from the OS. If you're on Linux, a real-time kernel is required to handle these transactions.
For Windows, there are asynchronous methods you achieve this functionality, it is possible, but very difficult to do.
After reviewing this guys LinkedIn profile, all he has ever done is Java. He is a Java advocate. I think Java is great language, but this is one place it doesn't belong.
First, let me preface this, the language does matter, but the larger complications revolve around the operating system. Most transactions have a hard real-time deadline of 20ms or less. JVM creates additional issues on top of what are already present from the OS. If you're on Linux, a real-time kernel is required to handle these transactions.
For Windows, there are asynchronous methods you achieve this functionality, it is possible, but very difficult to do.
After reviewing this guys LinkedIn profile, all he has ever done is Java. He is a Java advocate. I think Java is great language, but this is one place it doesn't belong.
This is not "why Java is better for fast things". It is not.
This is "why Java is better", and the answer is "it usually is better because your programmers often suck".
This is "why Java is better", and the answer is "it usually is better because your programmers often suck".
If programmers didn't suck, we would do everything in ASM. Since programmers are humans, we need abstractions like C or C++. Java, C#, Python, Go, Objecttive-C, and other garbage collected languages are there to make programming easier.
It's not even about "programmers suck". It's about productivity.
I can, and have, written high performance systems in assembly (long ago). I've written many high performance systems in C. Rarely, but sometimes, still do.
But doing it in assembly takes a long time. No way to justify time to market on that one today. In C, sometimes it's worth it.
But Java productivity is so much higher and (if written properly) leaves very little performance on the table. It's the sweet spot of performance vs. productivity still today and has been for about 15 years (~Java 1.4).
I can, and have, written high performance systems in assembly (long ago). I've written many high performance systems in C. Rarely, but sometimes, still do.
But doing it in assembly takes a long time. No way to justify time to market on that one today. In C, sometimes it's worth it.
But Java productivity is so much higher and (if written properly) leaves very little performance on the table. It's the sweet spot of performance vs. productivity still today and has been for about 15 years (~Java 1.4).
Be an amazing programmer then and go do everything in ASM, "wow" your stakeholders. But make sure you're eligible for unemployment benefits first.
These days the optimizer is so good few humans can do better over a large program. Even if they can, Intel will release a new cpu, or someone will buy amd and all your asm code is lost while those who write a compiled language just change a flag and get completely different binary.
Using assembler would be likely slower, when intrinsics are available.
I kind of cringe everytime i hear "you don't trust your programmers to get memory allocations right" but at the same time you trust them enough to not break havoc with the wallets of your customers and the frigging entire worldwide economy.
wreak havoc
You know, hedge funds can afford to hire, and in fact in many cases do hire, good programmers. So not every financial institution that uses java does so because its programmers suck.
Aren't HFT programmers highest paid?
If they do suck, then almost everybody in the whole industry sucks
If they do suck, then almost everybody in the whole industry sucks
The quote here is the main point. "But if you want to engage with 20+ exchanges, to go to market quickly, and implement continuous performance tuning, you'll choose Java."
The amount of changes you need to make due to customer preferences and regulations is continuous. If it's ok for your system to run in the 100 microsecond range then Java is a clear winner. If you are just focused on a single exchange, running in the same rack as the exchange then of course C or lower is what you will want.
The amount of changes you need to make due to customer preferences and regulations is continuous. If it's ok for your system to run in the 100 microsecond range then Java is a clear winner. If you are just focused on a single exchange, running in the same rack as the exchange then of course C or lower is what you will want.
I wonder if this will change in the future when rust may become a more mature language. It would offer the speed and the safety needed for this kind of task.
In what regard do you think it's not mature enough yet?
The thing about Rust is that so many features are considered unstable and require switching to the nightly toolchain. That's just not suitable for real-world production use - yes, stuff does get stabilized eventually, but it takes time to really nail down the best possible design. Still, it's way better than the huge mess that is C++.
> The thing about Rust is that so many features are considered unstable and require switching to the nightly toolchain.
The core features are stable for a long time. The days that you needed nightly for pretty much ever web framework are behind us.
The core features are stable for a long time. The days that you needed nightly for pretty much ever web framework are behind us.
It really depends on the edge that needs to be captured. If we're trading a low latency edge in SP500 E-Minis, then not only is C++ better but it might be the only option to successfully capture that edge.
On the other hand it we have some niche edge in a smaller market that's less speed competitive, Java might very well be the better choice given that we might be able to capture that edge with 8us tick to trade latency.
So the answer totally depends on what edge you're trying to capture. There are still a large number of edges where Java is good enough. GC is not really a problem, either using a commercial JVM or coding without much garbage and large memory.
On the other hand it we have some niche edge in a smaller market that's less speed competitive, Java might very well be the better choice given that we might be able to capture that edge with 8us tick to trade latency.
So the answer totally depends on what edge you're trying to capture. There are still a large number of edges where Java is good enough. GC is not really a problem, either using a commercial JVM or coding without much garbage and large memory.
So not only you have to write the same efficient code you could have written in C++, but on top of that you have to fight with Java features like GC to make sure they don't occur. Sounds like a winning strategy to me.
He doesn't define high speed or better. If you turn off gc and use pool based allocators, you can build a reasonably fast general purpose trading engine that can connect to lots of exchanges, use lots of apis etc. It will be fast enough for algo trading, but will never be fast enough for market making / hf / colo stuff.
You can never get around the boxing/unboxing, the larger sized objects, jvm inefficiencies, lack of direct access to assembly, etc.
But the cost of building a super low latency setup, and then doing anything useful with it is much higher than doing it with Java/c#
You can never get around the boxing/unboxing, the larger sized objects, jvm inefficiencies, lack of direct access to assembly, etc.
But the cost of building a super low latency setup, and then doing anything useful with it is much higher than doing it with Java/c#
"Chronicle's system is built in what Lawrey describes as "C-like Java" and it's this that he encourages people to code low latency systems in. "
Does anyone have any guidelines for this type of code?
Does anyone have any guidelines for this type of code?
Basically it sounds like they adhere to MISRA C principles (but for Java) which is used in industries where performance and safety is more critical. Think embedded systems in automotive, space, etc.
'The problem with C++ is its fallibility, says Lawrey: "If you're not a really good C++ developer, you can shoot yourself in the foot and bring an entire project down."'
'The only problem with low latency Java is that most experience(sic) Java programmers struggle with the new paradigm.'
So...badly written C++ isn't reliable, and well written, idiomatic Java isn't performant...so the solution proposed is, rather than make sure you write your code well in C++, to instead write your code well AND non-idiomatically in Java? Seems an...interesting take.
'The only problem with low latency Java is that most experience(sic) Java programmers struggle with the new paradigm.'
So...badly written C++ isn't reliable, and well written, idiomatic Java isn't performant...so the solution proposed is, rather than make sure you write your code well in C++, to instead write your code well AND non-idiomatically in Java? Seems an...interesting take.
Or use Rust as it fits this particular use case where you need a language that’s low-level and safe.
I mean, sure, but kind of left field there, no? I'm solely quoting the article and pointing out that the statements made, taken together, make the conclusion the guy came to seem very...strange.
Chronicle Software's OpenHFT stack has a few of their libraries: https://github.com/OpenHFT
Anyone has experience/review on this, looks promising.
Seems like bending over backwards to use Java. If you are using Java in a c-like fashion, what's so bad with C++ which is c-like by default and has some Java-like features?
The article mentions crash logs. I can sorta relate, my company has their core application built in C (or C++? idk), and they will often have one FTE slave away for weeks analyzing a crash report / core dump, because apparently the code doesn't give nearly enough information to debug a production crash.
whereas in Java you get an exception, a stack pointing neatly at where the issue was and an error that explains what's going on, and you can fix it and move on.
I'd argue that C-like Java is much easier to debug than C.
whereas in Java you get an exception, a stack pointing neatly at where the issue was and an error that explains what's going on, and you can fix it and move on.
I'd argue that C-like Java is much easier to debug than C.
Fair enough, however I don't get how this ties up to "high speed trading systems". It can be said to be a general developer productivity advantage, not an advantage from technical perspective for this specific domain (which is what I thought the article was about and why I clicked the link)
On second thoughts, maybe I underestimate developer productivity?
Some of their projects are open source with LGPL license.
https://github.com/OpenHFT
https://github.com/OpenHFT
The programmer is often more important than the PL.
Though most programmers will not modify the PL to fit their needs, hence the PL limitations become the programmers limitations.
It's easier for a programmer to learn a new PL (at the start of the project) than for a 500kLOC code base on GC-enabled JVM to migrate to Rust.
It's easier for a programmer to learn a new PL (at the start of the project) than for a 500kLOC code base on GC-enabled JVM to migrate to Rust.
[deleted]
in other words: we invested a ton of money into building java infrastructure and to make it work well for this application and now we need to find ultra rare developers who will maintain it! those who do java don't bother with performance or low latency and those who do C++ don't bother with java. Problem! We need to convince everyone that java is better!
Or you can use Apache 2.0 open source licensed libraries which are used in low latency trading systems and pay for the level of support your organisation needs and no more.
https://github.com/OpenHFT
https://github.com/OpenHFT
Well, now there's Rust. So Java isn't best technical (performance, safety, correctness) choice for most applications.
This seems to not really be about Java and more be that C++ is complex and error prone. If you reframe it as "C++ may not be the best choice for high speed trading systems" instead, then you can evaluate other languages, like Rust, or Erlang, or OCaml, or any of the newfangled C++ "replacements" like Zig.
I had a sign hanging from a column in one of my previous offices. It read "You must be this tall to use threads and shared mutable state -->". It was about 3 meters from the floor.
It's a bit harder to write some bugs in Java than it is in C or C++. I gather it'd be almost impossible to write them in Rust.
It's a bit harder to write some bugs in Java than it is in C or C++. I gather it'd be almost impossible to write them in Rust.
In my experience, most java developers even in this field (or "embedded" that has similar requirements) tend to resist when someone proposes to use "c-like java", too used to their fancy libraries and APIs.
> too used to their fancy libraries and APIs
This is where modern C++ (augmented with, e.g. the C++ Core Guidelines) and Rust are more convenient and even "better" than Java. Fancy libraries and tools, with zero additional overhead compared to the optimal low-level code.
This is where modern C++ (augmented with, e.g. the C++ Core Guidelines) and Rust are more convenient and even "better" than Java. Fancy libraries and tools, with zero additional overhead compared to the optimal low-level code.
Surely Rust will be better than both of these.
Not if you need to be able to hire a large number of developers for an investment bank. London has around 300K developers, many in fintech. There isn't that many Rust developers with say 10 -15 years fintech development experience.
This reads like what they actually mean is "I don't think we could rewrite this in C++".
That is a common sentiment. So much work went into this! We don't understand how we got here in the first place. It's so big! Look at all this legacy code and technical debt! There is no way we or anybody else could rewrite this in Java let alone C++ in any useful time frame! We sunk decades of work into this monstrosity!!
The text boils down to a "time to market" argument. I find that surprising because you'd think for high speed trading correctness is more important than time to market.
That is a common sentiment. So much work went into this! We don't understand how we got here in the first place. It's so big! Look at all this legacy code and technical debt! There is no way we or anybody else could rewrite this in Java let alone C++ in any useful time frame! We sunk decades of work into this monstrosity!!
The text boils down to a "time to market" argument. I find that surprising because you'd think for high speed trading correctness is more important than time to market.
Say you have a strategy which is only going to make money for a few weeks, every day you delay means you are losing money.
If you are in a Bank, correctness is always the most important and speed only helps you reduce cost/make money, but it's not the core of your business model.
Everyone using C++ for high-frequency trading hopes you will believe this article and use Java, yourself.
Each person who loses money on the market loses it to someone. They are ready to be that someone, for you.
Each person who loses money on the market loses it to someone. They are ready to be that someone, for you.
That assumes your commercial model is based on an arms races of just being the fastest. This is not a model investment banks have used for some time.
"If you have an unlimited amount of time and resources, the best solution for speed will be coded in FPGA"
With all that real estate on CPUs, I really would like an FPGA. If Oracle wanted to make money on Java, it could offer a commercial java that compiles bytecode to FPGA. Apple with Rosetta2 could have done an FPGA.
Emulation of lots of systems that will be abandoned by their commercial companies has always relied on the mhz and node shrink "free ride", and that's over. FPGAs could handle a lot of legitimate emulation.
With all that real estate on CPUs, I really would like an FPGA. If Oracle wanted to make money on Java, it could offer a commercial java that compiles bytecode to FPGA. Apple with Rosetta2 could have done an FPGA.
Emulation of lots of systems that will be abandoned by their commercial companies has always relied on the mhz and node shrink "free ride", and that's over. FPGAs could handle a lot of legitimate emulation.
Hi-speed trading is a zero-sum game, like poker. Every single gain is somebody's loss. Either you rip the other guy's heart out and eat it, or he rips your heart out and eats yours. There is no other choice.
Choosing Java over C++ is choosing to be that second guy.
Sure, Java coders are cheaper and easier to hire, and they produce lots more code. But if lots of code is a liability, lots of Java code is a bigger liability.
A trading house with a lot of Java code is a red warning.
Choosing Java over C++ is choosing to be that second guy.
Sure, Java coders are cheaper and easier to hire, and they produce lots more code. But if lots of code is a liability, lots of Java code is a bigger liability.
A trading house with a lot of Java code is a red warning.
A lot of code in any language is a red flag.
Has anyone used Golang for high speed trading? It seems like it hits all the sweet spots, yet it doesn't seem to be used that way. Does anyone know why?
I remember first seeing Go and taking a look at some of the interesting libraries it shipped with and thinking “this is a systems language” but it appears it mostly got relegated to web backend work.
Web backend work at scale is all about building and maintaining distributed systems. Just consider the company where Go was designed ... And indeed, it's still a language of choice for teaching systems design[1].
[1] MIT 6.824: Distributed Systems - https://www.youtube.com/watch?v=cQP8WApzIQQ&list=PLrw6a1wE39...
[1] MIT 6.824: Distributed Systems - https://www.youtube.com/watch?v=cQP8WApzIQQ&list=PLrw6a1wE39...
Discord has an interesting engineering blog post on moving from Go to Rust due to performance issues causes by the Go GC.
https://link.medium.com/ZS6jwQCkJbb
https://link.medium.com/ZS6jwQCkJbb
> Java is better than C++ for high speed trading systems
When someone says "C++" instead of C, you're really looking at developer issues, not technology issues. This is a one-off article trying to make some other point which seems like management hand-wringing to make sense of project failure.
> Java developers are better than C++ developers for high speed trading systems
I would read this, especially if it was an actual study.
When someone says "C++" instead of C, you're really looking at developer issues, not technology issues. This is a one-off article trying to make some other point which seems like management hand-wringing to make sense of project failure.
> Java developers are better than C++ developers for high speed trading systems
I would read this, especially if it was an actual study.
If you can have any developers you want and have unlimited time, C++ is a good choice.
However, if you are a bank, which not where every developer wants to work, you have a limited budget, limited time and you have a project which isn't technically exciting (but can make a lot of money for the bank) you might find the developers you have who understand the business are more productive in Java and the resulting trading system is faster than the one they might have written in C++.
When you have some teams developing in Java and some in C++ you get to see which teams are more productive for your organisation.
However, if you are a bank, which not where every developer wants to work, you have a limited budget, limited time and you have a project which isn't technically exciting (but can make a lot of money for the bank) you might find the developers you have who understand the business are more productive in Java and the resulting trading system is faster than the one they might have written in C++.
When you have some teams developing in Java and some in C++ you get to see which teams are more productive for your organisation.
[deleted]
Sorry, but this sounds more like they had really bad C++ developers and then complaining about the language.
Why not Fortran? Genuinely curious. It is more difficult to shoot yourself in the foot in Fortran than in C. The perils of garbage collector is missing from Fortran. The syntax is clean. Performance is as good as and sometimes even better than C and most definitely better than Java.
Probably because it’s not sufficiently C like to meet the threshold for adoption.
I switched to Rust for my high speed trading system with some C++ where needed. Not looking back.
If any company thinks they can just get a consultant that fixes these things they are just not going to make it.
Bosses and programmers need to be fired and senior developers put in place.
There are no free lunches.
Interesting to note the comment section there vs here.
Interesting to note the comment section there vs here.
... at the price a particular CEO is willing to pay for programmers. There, fixed it ! :)
Let's invest all our money in this company called NullPointerException!
"We only use the constructs that are fast and efficient, and we avoid all the garbage."
Is there any open source project that one can study to understand how to code like this?
Is there any open source project that one can study to understand how to code like this?
Could someone point to C-like Java code real-world example? It would be awesome to see how the things are managed at a GC platform with most things on the heap.
You can have a look at https://github.com/OpenHFT however it is designed to have most data off heap.
This article is misguiding. “Java is better to programmer weak in c++ and in general cs.” Should be the title. Stop spamming with your bulshit.
In the real world projects, with the developers you can get, and the time frames you are given, not the developer you might want and any amount of time, Java can be a better choice.
Always amazes me statements like: X is better than Y for Z.
What meter is how you use it and algorithms.
[deleted]
Rust would easily beat both in terms of speed and ease of programming/testing/ebugging.
from another discipline, but similar rigor -- a FOSS community with both C++ and Java servers, serving high-requirements responses.. a sort of friendly "coopetition" contest around 2012.. many hands and eyes, certainly top-level engineering on both sides.
The results were basically -- with the best teams, almost identical times; the obviously slowest entries were on the C++ side. Admitting bias now, I was surprised to see Java winning by a "hair's breadth" in certain cases, with certain entries.
Later, talking to a PhD in CS who wrote their thesis on graph-analysis of code conditions for JIT optimization, I found out that Java put a lot of excellent computer science into the problem, and that works very well. As alluded to in a different comment, the response times in both camps were appproaching the physical limits of the hardware, apparently. I had to recalibrate my expectations, since I thought that C++ would certainly win without fail.
The results were basically -- with the best teams, almost identical times; the obviously slowest entries were on the C++ side. Admitting bias now, I was surprised to see Java winning by a "hair's breadth" in certain cases, with certain entries.
Later, talking to a PhD in CS who wrote their thesis on graph-analysis of code conditions for JIT optimization, I found out that Java put a lot of excellent computer science into the problem, and that works very well. As alluded to in a different comment, the response times in both camps were appproaching the physical limits of the hardware, apparently. I had to recalibrate my expectations, since I thought that C++ would certainly win without fail.
How did they avoid GC penalty? Arena allocation and buffer reuse?
Java is pretty good, I've used it for over 20 years, but it wastes memory for basic use cases, and yet GC pause limits the utilization of huge memory cases.
Did anyone try Go? IMO Go tries to address the slowness of java startup and commands, but it punts GC/memory waste so the huge-memory systems are even worse.
Java is pretty good, I've used it for over 20 years, but it wastes memory for basic use cases, and yet GC pause limits the utilization of huge memory cases.
Did anyone try Go? IMO Go tries to address the slowness of java startup and commands, but it punts GC/memory waste so the huge-memory systems are even worse.
the results that counted were "warm" responses
no one tried go-lang, the elaborate specialty code needed for the actual work did not exist
the Java probably did use pre-allocation and flat heirarchy a lot, but I do not know the details. Some Java did win by a significant margin, in very odd corner cases, probably due to simple code-coverage miss on the C++ side
no one tried go-lang, the elaborate specialty code needed for the actual work did not exist
the Java probably did use pre-allocation and flat heirarchy a lot, but I do not know the details. Some Java did win by a significant margin, in very odd corner cases, probably due to simple code-coverage miss on the C++ side
C# even better! Discover your signals with Python and implement your order execution in C#.
I think Rust is better than C++ which is better than Java for it for any real time system.
I'm curious how Zig would fare as a high speed trading system language..
These discussions tend to zoom in on just one quality attribute, latency in this case. The point this article tries to make is that there are multiple quality attributes and that you trade them off against each other. There is no right or wrong here other than getting sucked into having these choices imposed on you by irrational arguments made by engineers about things they care about vs. things that are actually business relevant.
In the case of high speed trading systems, any second they are late to market, they are losing money. It might be the fastest thing ever once it's finished but while it's not running, its slower competitors have the market to themselves. Worse, any month you are late is time your competitors can use against you to learn from your mistakes and beat you at your own came. High speed trading is of course extremely competitive. So things change all the time. So, now we are talking speed of development and maintainability in addition to raw performance and throughput. The impact of most improvements you make are in any case typically temporary in the sense that they only provide you an advantage as long as your competitors don't catch up. So, getting bogged down into lengthy maintenance and bug fixing cycles while your competitors copy everything that you got right means you lose some or all of the market opportunity.
This article mentions what happens when C++ projects fail. Basically, you are late to market with something that doesn't work as advertised (slow, unstable). That's not an inherent risk of course but it's a risk none the less. Performance often comes at the price of complexity. Complexity has its own penalties as well in the form of e.g. poor maintainability, bugs, or stability. C++ is frankly notorious for this. You compensate with having skilled engineers acting in a disciplined way. And sometimes that actually works as advertised. The project mentioned in the article sounds like it had issues across the board. Probably, at least one of the root causes for that was that they were trying to take some short cuts to get it to market.
The JVM ecosystem has a lot going for when making such trade-offs. That's why it's so entrenched in the industry. And it's been around long enough that there are lots of solutions to all sorts of issues that you might need solutions for. Also, you always have the option to go native in Java; it's not an either or kind of thing.
For the same reason you see people opting for a not particularly fast language like python in a domain where you get to use fast GPUs to get things done quickly (i.e. machine learning). Python gets that job done, everything on the critical path is basically delegated to lower level native stuff running on a GPU (or custom hardware in some cases). I wouldn't be surprised to learn that some high frequency traders have lots of python code around in addition to their precious native code. It would be a pragmatic thing to do.
Understanding your domain and understanding the trade offs is what differentiates successful companies and engineers here. There are lots of valid reasons for preferring C++ for some things. Personally, I rarely encounter such requirements and I'd probably prefer Rust over C++ if I did. And since I don't, I actually have limited experience with Rust as well because other than the intellectual challenge I don't see how I'd be getting much real world advantage out of it. I'm pretty sure that holds true across large parts of our industry. It's why you see people actively not caring about performance by choosing to run on slow (virtual hardware) on slow cloud networks using slow run time environments that include slow interpreters and slow application frameworks because they can get stuff done quickly with it and can trivially scale by throwing dollars at the problem. Slow is fast enough for most.
In the case of high speed trading systems, any second they are late to market, they are losing money. It might be the fastest thing ever once it's finished but while it's not running, its slower competitors have the market to themselves. Worse, any month you are late is time your competitors can use against you to learn from your mistakes and beat you at your own came. High speed trading is of course extremely competitive. So things change all the time. So, now we are talking speed of development and maintainability in addition to raw performance and throughput. The impact of most improvements you make are in any case typically temporary in the sense that they only provide you an advantage as long as your competitors don't catch up. So, getting bogged down into lengthy maintenance and bug fixing cycles while your competitors copy everything that you got right means you lose some or all of the market opportunity.
This article mentions what happens when C++ projects fail. Basically, you are late to market with something that doesn't work as advertised (slow, unstable). That's not an inherent risk of course but it's a risk none the less. Performance often comes at the price of complexity. Complexity has its own penalties as well in the form of e.g. poor maintainability, bugs, or stability. C++ is frankly notorious for this. You compensate with having skilled engineers acting in a disciplined way. And sometimes that actually works as advertised. The project mentioned in the article sounds like it had issues across the board. Probably, at least one of the root causes for that was that they were trying to take some short cuts to get it to market.
The JVM ecosystem has a lot going for when making such trade-offs. That's why it's so entrenched in the industry. And it's been around long enough that there are lots of solutions to all sorts of issues that you might need solutions for. Also, you always have the option to go native in Java; it's not an either or kind of thing.
For the same reason you see people opting for a not particularly fast language like python in a domain where you get to use fast GPUs to get things done quickly (i.e. machine learning). Python gets that job done, everything on the critical path is basically delegated to lower level native stuff running on a GPU (or custom hardware in some cases). I wouldn't be surprised to learn that some high frequency traders have lots of python code around in addition to their precious native code. It would be a pragmatic thing to do.
Understanding your domain and understanding the trade offs is what differentiates successful companies and engineers here. There are lots of valid reasons for preferring C++ for some things. Personally, I rarely encounter such requirements and I'd probably prefer Rust over C++ if I did. And since I don't, I actually have limited experience with Rust as well because other than the intellectual challenge I don't see how I'd be getting much real world advantage out of it. I'm pretty sure that holds true across large parts of our industry. It's why you see people actively not caring about performance by choosing to run on slow (virtual hardware) on slow cloud networks using slow run time environments that include slow interpreters and slow application frameworks because they can get stuff done quickly with it and can trivially scale by throwing dollars at the problem. Slow is fast enough for most.
"Well, C++ isn't gonna make your socket faster"
High speed trading systems, what a stupid, useless, and artificial domain. You could shutdown all of this and nobody will care in all societies.
Why so many down votes? I agree with what he said. HFT is just a scam. Anybody remembers Flash sales? And there were consideration like digging tunnel from nyc to chicago due to small latency. HFT is just ran by powerful and rich people thats why the are still now. There are preety bad thing hft has done like Intensifying Volatility, Ripple Effects,Uncertainty, Stock Manipulation.
As a game HFT is interesting but in real life its just a plain scam. Yes you may have a job and you may work on HFT but that doesn't mean you have to give your heart and support it.
Disclaimer: I worked on HFT but I still don't like it and the money HFT generate is just insane.
As a game HFT is interesting but in real life its just a plain scam. Yes you may have a job and you may work on HFT but that doesn't mean you have to give your heart and support it.
Disclaimer: I worked on HFT but I still don't like it and the money HFT generate is just insane.
Couldn't agree more! What a waste of talent.
agreed
tldr - Java is a better language to write good and safe code in than C++.
C++ is a fantastic language but it somehow attracts people with big ego.
The smartest people I know would always try to simplify things, but many C++ developers (especially in financial industry) have this preference for complexity I can't explain. And C++ is the worst language to get clever with.
The smartest people I know would always try to simplify things, but many C++ developers (especially in financial industry) have this preference for complexity I can't explain. And C++ is the worst language to get clever with.
I switched to C++ after years of doing web development in python, javascript and typescript, go and other high level languages associated with web dev.
You're not wrong, the ego among C++ devs is astronomical. There's a huge amount of serious hate for python and this idea that being a C++ dev is superior.
Get this we use Nix and C++ and the nix part of the code base is just as complex as the C++ code base. The nix stuff is just infrastructure! It's insane.
You're not wrong, the ego among C++ devs is astronomical. There's a huge amount of serious hate for python and this idea that being a C++ dev is superior.
Get this we use Nix and C++ and the nix part of the code base is just as complex as the C++ code base. The nix stuff is just infrastructure! It's insane.
I'm primarily a Java programmer, but I had a go at learning Python a little while ago. I couldn't stand it - it just seemed so sloppy and poorly-defined. I don't consider myself as being superior to Python devs or having an astronomical ego, but Python-hate? Yeah.
I got myself my first job as a Python programmer 15 years ago by (among other things) mentioning this blog entry: "Python Is Not Java" [1]
[1] https://dirtsimple.org/2004/12/python-is-not-java.html
[1] https://dirtsimple.org/2004/12/python-is-not-java.html
It's strange, I think that python and C++ are great languages to complement one another. You can write your critical stuff in C++ and use python to glue it all together, and it'll be a pretty effective solution (especially with the sheer amount of options in the python library ecosystem).
This is so true. Put your critical stuff, memory management, etc in C++. Run your ML/computation/data analysis on the side on Python.
I worked at a large C++ shop (the kind that maintains a stdlib fork) and never saw them looking down at Python.
I think that's common for people that have a higher vision and comprehension of computers, to respect other choices and other tools. I prefer linux, but win10 and ios are great in the domain they target. C++ is great tools when you want to be low level, like managing memory, python is another great tool when you don't want to or don't need to or don't have the time to. Hey, guess what, it can be the same person that write python and c++ code !! And he even can like both !!! And be good at both !!!
That is because maybe it did not make sense to them to look up or down. To me it is like comparing a synthetizer to a violin and try to say which one is better.
I started with coding in C++, and tbh, I've only had a certain unique appreciation for Python. Both have their use cases, and comparing them is like comparing apples and oranges.
it's not hard to see why C++ would attract people who revel in complexity, there's like 7 ways to initialize a variable, and they're all slightly different in certain contexts
And to make things even more interesting, the rules change in subtle ways depending on the version of the language.
https://stackoverflow.com/a/620402/
https://stackoverflow.com/a/620402/
C++ lets you "get things done" when the other tooling you need is your brain. Unfortunately, that might mean template metaprogramming that makes a compile take >50 minutes, and where any syntax error typo generates a 4000-line error message -- but for some people, being to use C++ to get things done is an intellectual reward onto itself.
If you wanted to get the same performance from C, you'd sometimes need to write your own preprocessor/code-generator, and/or abuse the built in preprocessor, so you can specaialize. To get the same performance in Python you need to go down to C. To get the same performance in Java, you need to do stuff like what's mentioned in this thread (allocate 1TB of memory and disable GC, for example).
The only place where this is considered acceptable and commonplace, is the C++ community, I guess a chicken-and-egg problem.
In most places, you only need that performance in a small part of your pipeline - which in HFT teams will sometimes be done in FPGA or ASICs or NIC firmware. For those places, C++ is also often required, and network effects (and "impedance mismatch") often mean that it will get used for a whole lot more than where it's actually essential.
C++ is not the only such community for algo trading, by the way - the K programming language enables fast yet incredibly short implementations, and thus tends to attract competitions in "how short can you get the fastest code" (aka code golfing) to manifest all over implementations. And personally, I'd much rather have a oneliner K brainteaser that does alot than a huge 100-line C++ template monster that does the same thing 10% more quickly.
If you wanted to get the same performance from C, you'd sometimes need to write your own preprocessor/code-generator, and/or abuse the built in preprocessor, so you can specaialize. To get the same performance in Python you need to go down to C. To get the same performance in Java, you need to do stuff like what's mentioned in this thread (allocate 1TB of memory and disable GC, for example).
The only place where this is considered acceptable and commonplace, is the C++ community, I guess a chicken-and-egg problem.
In most places, you only need that performance in a small part of your pipeline - which in HFT teams will sometimes be done in FPGA or ASICs or NIC firmware. For those places, C++ is also often required, and network effects (and "impedance mismatch") often mean that it will get used for a whole lot more than where it's actually essential.
C++ is not the only such community for algo trading, by the way - the K programming language enables fast yet incredibly short implementations, and thus tends to attract competitions in "how short can you get the fastest code" (aka code golfing) to manifest all over implementations. And personally, I'd much rather have a oneliner K brainteaser that does alot than a huge 100-line C++ template monster that does the same thing 10% more quickly.
well, at least there's also constexpr now which reduces some amount of need for template metaprogramming
I don't want to start a flame war, but what are some of the programming languages that attract the first variety of people?
I'm not sure what you mean by "first variety". If you mean "the people who like to simplify thing" mentioned in the GP, then it depends on your definition of simplicity more than anything. I'd say from my experience if a person prefers C to C++ or Java, that's a good indication - but it could also mean they learned C in 1990 and didn't bother learning anything else since....
From my experience: (apologies if this looks like a flame; purely personal experience, with no offense intended).
- Python attracts all kinds of people.
- Java attracts people who prefer rigid structure, specified down to minute detail (which non Java people will often consider unneeded bureaucracy)
- C++ attracts people who like brain teaser challenges with lots of details and steps. The compiler is both their colleague and adversary in reaching the optimal speed goal (their "frenemy")
- C attracts people with more experience who just wants "the simple thing that works" without giving up the low-level control that they would have to give up using e.g. Python or Java.
- K/J/APL attract people who like brain teaser with short elegant solutions, even (and perhaps especially) if reaching and understanding that solution takes nontrivial effort -- which is basically also people who like math for math's sake.
- COBOL attracts people who like job security, good income, don't care much about job satisfaction or recent advances made in the last 30-40 years, and want everyone to get off their lawn :)
From my experience: (apologies if this looks like a flame; purely personal experience, with no offense intended).
- Python attracts all kinds of people.
- Java attracts people who prefer rigid structure, specified down to minute detail (which non Java people will often consider unneeded bureaucracy)
- C++ attracts people who like brain teaser challenges with lots of details and steps. The compiler is both their colleague and adversary in reaching the optimal speed goal (their "frenemy")
- C attracts people with more experience who just wants "the simple thing that works" without giving up the low-level control that they would have to give up using e.g. Python or Java.
- K/J/APL attract people who like brain teaser with short elegant solutions, even (and perhaps especially) if reaching and understanding that solution takes nontrivial effort -- which is basically also people who like math for math's sake.
- COBOL attracts people who like job security, good income, don't care much about job satisfaction or recent advances made in the last 30-40 years, and want everyone to get off their lawn :)
Go is the only example I know of (very explicit syntax, almost no "magic")
tldr - Java is safer and simpler than C++.
Sarcasm?
WWJS
What would josh.com say..?
What would josh.com say..?
Trust in Rust.
As it happens, I have developed one algorithmic, low latency trading system in Common Lisp / ANSI C, and then was asked to rewrite it in Java which I did.
It actually traded on Warsaw Stock Exchange and was certified by WSE and was connected directly to it (no intervening software).
Yes, it is possible to do really low latency in Java. My experience is my optimized Java code is about 20-30% slower than what very optimized C code which is absolutely awesome. We are talking real research project funded by one of larger brokerage houses and the application was expected to respond to market signals within single digit microseconds every single time.
The issue with Java isn't that you can't write low latency code. The issue is that you are left almost completely without tools as you can't use almost anything from Java standard library and left scratching your head how to solve even simplest problem like managing memory without something coming and suddenly creating a lot of latency where you don't want it to happen.
Can't use interfaces, can't use collections, can't use exceptions, etc.
You can forget about freely allocating objects, except for very short lived objects you must relegate yourself to managing preallocated pools of objects.
I don't know what is the current state of garbage collection but in our case garbage collection had to be turned off and nothing outside of Eden could be collected during program execution (Java 8).
Writing that kind of optimized code is all about control and Java is all about abstracting it so you don't have to worry. As you see, both goals are at odds.
So I still prefer working low latency in C as it is more natural, native solution for managing problem where you absolutely need to control memory layouts, preallocated object pools, NUMA, compilation of decision trees to machine code, prefetcher, etc.
I have about 10 years combined experience working with C and 15 years working with Java.