Rust vs. C++ Benchmark Game(benchmarksgame.alioth.debian.org)

51 pointsby tetevor 12 Jahren48 comments

benchmarksgame.alioth.debian.org

http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?test=all&lang=rust&lang2=gpp&data=u64q

50 comments

jyxentvor 12 Jahren

As pointed out in the reddit thread, this may be more a measure of the llvm backend used by rust to the gcc backend. A more accurate comparison would be against clang.

acqqvor 12 Jahren

It doesn't matter much what the backends are, it's important to be able to demonstrate the effective implementation of the problems posed by the tasks being benchmarked. Especially for the new language. Of course, if your language is designed as to have the limited chances of being universaly fast, it's another story. But Rust is designed to be able to reach at least the speeds of C and also allow for a more modern approach. That's the real worth of the contributions to the "benchmark game:" demonstrating the capabilites of the language, even if it's just on some of all of possible tasks.

See also Rust vs Haskell: http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t...

And Rust vs Go: http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t...

joelthelionvor 12 Jahren

It's not that easy to implement a language that's expressive with this kind of performance, even if you use llvm. Kudos to the Rust guys!

igouyvor 12 Jahren

The Mono C# programs also use llvm ;-)

A more accurate comparison? A different comparison would be against clang.

The benchmarks game might well be described as "a good starting point" http://algs4.cs.princeton.edu/faq/ but let's not confuse that with some kind of definitive final analysis.

(Edit: Say what you disagree with, don't just down vote.)

SwellJoevor 12 Jahren

I was under the impression that clang was still, generally, slightly slower than GCC for C++. Perhaps this has changed in recent years (as clang and llvm have been seeing tons of development effort and funding).

bsaulvor 12 Jahren

I've only started to investigate rust for server side programming ( websocket in particular) but saw many discussions about the limitations rust has regarding parallel programming. Yet, iirc servo did parallel rendering of html page, so that seems a contradiction. Anyone here knows about that ?

Specifically, i'm currently hesitating between golang and rust for coding a small Websocket chat server. I'm ready to use beta code because rust seems more promising to me in the long term than golang, but not if the language itself has severe fundamental limitations on server side programming.

rubiquityvor 12 Jahren

Parallel rendering of an HTML page means an improvement of one renderer to 2 or more renderers. Writing a web server means possibly handling thousands to millions of concurrent requests, which isn't really doable with Rusts 1:1 threading model.

But the reality of concurrency and hardware is that all we have at the end of the day are threads anyway. So in Rust you could have a thread running an event-loop that serves your web sockets. People are building these sorts of abstractions for Rust but they are a work in progress as far as I know. MIO[0] is one such example.

Rust is a programming language for systems programming and "server side programming" is just a subset of systems programming.

0 - https://github.com/carllerche/mio

bsaulvor 12 Jahren

Thanks for your enlightening response. Wouldn't building event loop based server require non-blocking i/o libraries ? Does rust have that ?

rubiquityvor 12 Jahren

To the best of my knowledge Rust's stdlib doesn't have non-blocking libraries yet. But since Rust is "close to the metal" you can easily use any of the non-blocking system calls in glibc. Mio (https://github.com/carllerche/mio#features) seems to be doing just that.

kxovor 12 Jahren

That's exactly what they require. mio is one such example, I am working on another implementation.

m0th87vor 12 Jahren

Depends on the kinds of parallelism you want. Rust has built-in support for parallelization, but tasks are mapped 1:1 to threads. Currently M:N threading is also supported, but that will be removed. Consequently, applications requiring lots of tasks (including highly-concurrent websocket servers) won't do well with the built-in mechanism.

Context: https://news.ycombinator.com/item?id=8459888

tormehvor 12 Jahren

Can someone explain me why that is? It just seems like it is the dumbest thing in the world to me. The OS may be good at scheduling, but why not have a built in library providing you with an alternative? Native threads are usually really slow to start and stop, because they have high memory use and even modern OSs can only handle on the order of 10000 of them.

When I saw this decision my first reaction was "Well, that was a promising language. Shame about that." I can't be the only one...

Jweb_Guruvor 12 Jahren

This is actually not the case on Linux. Starting up a thread is really fast. One of Rust's contributors found that it was just as fast at spawning threads as Rust was at spawning green threads when both were pinned to one core.

Additionally, for a variety of reasons Rust's tasks were not smaller than OS threads, because they need to be bigger for a lot of things (calling into C, for example) and segmented stacks were found to be much too slow. Go gets around these problems by allocating a new stack when the current one runs out of space, but doing that properly requires being able to trace all the pointers into the stack to figure out where to transfer them--in other words, it requires a garbage collector :) So that solution wasn't really viable for Rust.

Finally, having to support both concurrency models required Rust to make significant sacrifices in its libraries, in both complexity and performance. Go was able to make the decision to focus entirely on green threading, but Rust did not take that approach, so it was stuck in a "worst of both worlds" situation.

It is possible that there is a better solution that avoids all these problems, but it will have to be found outside the main Rust tree.

steveklabnikvor 12 Jahren

A systems programming language without access to a systems feature isn't much of a systems language.

Also, it doesn't _preclude_ green threads. As others have mentioned, Rust is low-level enough that all of this is a library issue, not a language issue. You can write whatever concurrency primitives you need, and they'll be no less privileged as the ones the standard library gives you.

tormehvor 12 Jahren

Well, yes, but anything that's merely a convenience and not in the standard library won't see much use, as I've understood programmer nature.

steveklabnikvor 12 Jahren

Rust's standard library will be much less privileged than in other languages, due to Cargo existing from the start. It's going to be much more slim than most, we've already pulled lots of things out into their own packages. Also, given that only part of the standard library will be stable at 1.0.

higherpurposevor 12 Jahren

Sounds like Go is better for web servers because of better concurrency support for millions of users, while Rust is better suited for desktop apps that take advantage of all the PC's cores?

carllerchevor 12 Jahren

Actually, Rust is much closer to the system than Go is making it possible to implement lighter weight web services using Rust than Go. Rust features like not having a GC allows for much more predictable performance.

Go's concurrency features are nice if it's your kind of thing, but will get in your way when hitting that "last mile" of performance. For example, if you were to implement something like haproxy, you would probably want to use Rust over Go for performance.

fread2281vor 12 Jahren

If you want true lightweight threads, you need haskell or erlang.

ricticvor 12 Jahren

Green threads (M:N) do seem to be available as a library though: http://doc.rust-lang.org/green/

m0th87vor 12 Jahren

That too is being removed: https://github.com/rust-lang/rfcs/pull/230

adrusivor 12 Jahren

Its being removed from the standard library. As I understand it will still be maintained as a separate library.

joelthelionvor 12 Jahren

It's not being removed, it's being moved from the core of the language to an external library.

cmrx64vor 12 Jahren

Their new resting place: https://github.com/alexcrichton/green-rs

bhauervor 12 Jahren

Although not directly related to your use-case, I wanted to mention that we received a pull request [1] implementing a test of the Iron framework. Expect to see this in Round 10 of our framework benchmarks project.

[1] https://github.com/TechEmpower/FrameworkBenchmarks/pull/1101

[deleted]vor 12 Jahren

pptr1vor 12 Jahren

Rust really interest me. However I want them to get to 1.0 before I can evaluate whether or not to use Rust in my app. Currently I am using Go and it stable for my needs. Rust could make me switch once they get to 1.0 and people start writing more projects in Rust.

I want to add that stability in deciding to use a language in an application is very important. How good the standard library is and how much traction the language is gaining in projects and particularly open source projects is also important.

tetevor 12 Jahren

I agree. However sometimes it's good to do tiny side projects in a language. This allows you to learn, see if it's really nice, help the language and its ecosystem to develop and become stable.

For use outside of production it seems okay. There are two Sinatra/Flask/Express style web frameworks (Nickel and Iron), a browser engine (Servo), a game engine (Piston) actively developed. So if you have interest in any of those or interest in implementing something like that you might be able to make your project shine. Also really nice on your portfolio if you ever want to get a job where Rust is already in use.

Or one just has fun with it. :D

pptr1vor 12 Jahren

Agreed. I would do a small project just for fun.

sqrt17vor 12 Jahren

Rust seems to do worse on the string-y tasks (fannkuch, k-nucleotides, regex-dna). A lot if you look at single-thread performance. Does it handle strings differently from C++?

(From what I understood, Rust gives you similar control over memory allocation as [a sane subset of] C++, so intuitively it should come out relatively close)

burntsushivor 12 Jahren

With respect to regex-dna, there is still substantial room for improvement in the Rust library. It's currently using a full NFA simulation (albeit specialized to native Rust code with the `regex!` macro), while the competing C++ library (RE2) has been tricked out by Russ Cox with a parallel DFA implementation. :-)

tetevor 12 Jahren

Found on the Rust subreddit.

https://pay.reddit.com/r/rust/comments/2jm4vq/rust_0120_vs_c...

capnrefsmmatvor 12 Jahren

Incidentally -- reddit supports https by default now, so you don't need to use pay.reddit.com anymore.

general_failurevor 12 Jahren

Very impressive. What we need now is some sort of a good mainstream all that uses it to spread its popularity..

How is rust when it comes to deployment. C++ is a real pain to deploy for large projects. Compilation setup is also a massive headache in c++.

steveklabnikvor 12 Jahren

With Cargo, getting a nice repeatable build is much, much easier. There's still work to be done on this front, but I'm _incredibly_ excited by this RFC: https://github.com/rust-lang/rfcs/pull/403

steveklabnikvor 12 Jahren

This issue on the Rust repo is very relevant: https://github.com/rust-lang/rust/issues/18085

shmerlvor 12 Jahren

The server is down for me.

tux3vor 12 Jahren

Same, but google had the time to cache it.

https://webcache.googleusercontent.com/search?q=cache%3Ahttp...

[deleted]vor 12 Jahren

igouyvor 12 Jahren

And up, presumably once this discussion thread moved off the front page ;-)

farresitovor 12 Jahren

While I'm not a huge fan of benchmarks, this is quite impressive, given the age of Rust. C++ does better memory-wise in pretty much all benchmarks.

dbauppvor 12 Jahren

The main problem is the large (multi-megabytes) runtime that Rust currently is forced to load by default. This is disappearing.

igouyvor 12 Jahren

Don't read too much into those memory use measurements - they are just enough to suggest programmers might have made a space/time trade-off, nothing more.

zokiervor 12 Jahren

Considering that memory management is one of the headline features of Rust, memory use is certainly important factor to analyze.

igouyvor 12 Jahren

That may well be, but the benchmarks game measurements are not a good basis for that analysis.

pjmlpvor 12 Jahren

Actually LLVM vs GCC backend benchmark.

igouyvor 12 Jahren

Is there a Rust tool chain based on GCC?

adrusivor 12 Jahren

Not with full up to date language support, but there's a mature c++ toolchain on top of llvm

rando289vor 12 Jahren

I wish there was a D comparison. It seems to have similar goals.

igouyvor 12 Jahren

Make your wish reality - take the program source code and the measurement scripts and publish your own measurements.

http://benchmarksgame.alioth.debian.org/play.html#languagex

If you have a linux install the benchmarks game scripts should work fine, and you'd probably get help from the D community --

http://forum.dlang.org/thread/[email protected]?p...