Where Rust really shines(manishearth.github.io)
manishearth.github.io
Where Rust really shines
http://manishearth.github.io/blog/2015/05/03/where-rust-really-shines/
12 comments
I don't really think that's a problem with Rust so much as inexperience with languages that have no garbage collection. If anything, the problems you had would probably be exacerbated by languages like C and C++, which in many ways are deceptively simple, yet there are tricks to be aware of in every turn. Doing memory management is hard because of the nature of the problem.
I care a lot about figuring out how to teach people like you to use Rust. I've primarily worked in Ruby for the past several years, but I did have that degree, and C was my second language. So it's a bit harder for me, as I had my 'aha' moment with a lot of this stuff quite a while back.
I'm hoping as time goes on to include more stuff in the docs that's about generic systems stuff, rather than 100% Rust specific, exactly for this reason. The stack & heap chapter of the book (which is upcoming) is part of that. But, I gotta finish off the Rust-specific stuff first...
I agree with wycats: Rust is going to bring systems programming to a lot of people who haven't done systems programming before, just like Node brought a whole new group of people to server-side code. It's our job to give you the support you need to be successful at it.
I'm hoping as time goes on to include more stuff in the docs that's about generic systems stuff, rather than 100% Rust specific, exactly for this reason. The stack & heap chapter of the book (which is upcoming) is part of that. But, I gotta finish off the Rust-specific stuff first...
I agree with wycats: Rust is going to bring systems programming to a lot of people who haven't done systems programming before, just like Node brought a whole new group of people to server-side code. It's our job to give you the support you need to be successful at it.
I'm considering writing a GC for Rust next. It won't be as easy to use as the other GCs, but you will be able to put the ownership of stuff in the hands of the GC if you wish.
Yeah, a lot of people are excited about Rust for webdev, but there is a steep learning curve and it's quite different from the languages most are used to. For webdev I personally think Go will make it, it feels a lot like Python/Ruby when programming, even though it's compiled and probably faster.
Yeah, a lot of people are excited about Rust for webdev, but there is a steep learning curve and it's quite different from the languages most are used to. For webdev I personally think Go will make it, it feels a lot like Python/Ruby when programming, even though it's compiled and probably faster.
So I'm not sure I fully understand why you would target Rust as a web development language in the first place except pure curiosity of working in the language, but it appears there's more to the story. You're excited about Rust's possibilities -- in web development or just in general? And you gripe it's "demanding for web devs who work with a garbage collector." This makes me pause and think, why are we talking about web devs in the first place? From what I understood, Rust is a systems language and by systems language, it means stuff a lot lower level than `route '/me' => controller.toMe!`
Thing is about web dev is that (1) it's a get-things-done playing field and (2) primarily just data delivery. The idea that you would want a language that doesn't have garbage collection is just silly. Why? What complex operations are you doing in each request/response cycle that demands this sort of computational horsepower? And if you are doing said complex operations, maybe you should rethink how you're delivering that data and creating it? GitHub is huge and runs on Ruby. Surely, you can't be wanting to use Rust to improve your performance are you?
> Mutating state across threads is hard
Yep, so maybe you should target a language / framework that abstracts this away from you? I wouldn't say anything usually but you say it's "surprisingly difficult to figure out." Come on really.. programming languages are not created just for web devs...
I don't know. If you're trying to be productive AND change your toolset, it sounds like you just want a typed language. I wouldn't put bet too much of your time-and-focus-chips on Rust becoming a language for web developers. Do you see many people using C++ to write their servers? Nope. And don't take my message the wrong way. It really just bewilders me because I see a comment like "Yea I want to help people like you!" and I'm like Huh??
Thing is about web dev is that (1) it's a get-things-done playing field and (2) primarily just data delivery. The idea that you would want a language that doesn't have garbage collection is just silly. Why? What complex operations are you doing in each request/response cycle that demands this sort of computational horsepower? And if you are doing said complex operations, maybe you should rethink how you're delivering that data and creating it? GitHub is huge and runs on Ruby. Surely, you can't be wanting to use Rust to improve your performance are you?
> Mutating state across threads is hard
Yep, so maybe you should target a language / framework that abstracts this away from you? I wouldn't say anything usually but you say it's "surprisingly difficult to figure out." Come on really.. programming languages are not created just for web devs...
I don't know. If you're trying to be productive AND change your toolset, it sounds like you just want a typed language. I wouldn't put bet too much of your time-and-focus-chips on Rust becoming a language for web developers. Do you see many people using C++ to write their servers? Nope. And don't take my message the wrong way. It really just bewilders me because I see a comment like "Yea I want to help people like you!" and I'm like Huh??
FWIW Rust is intended to be more than a systems language, but is designed as a systems language first (but other use cases are all taken into consideration)
I agree, the #rust IRC channel is very helpful, with a lot of people nice enough to help you. It should be considered a feature of the language : having a helpful community.
While I like the overall Rust community, I'm a bit skeptical to this, because all the language communities I've seen claim to be friendly and helpful.
There may be some difference in degree, though.
There may be some difference in degree, though.
As a novice I find that when the compiler starts telling me do to things it will go one of two ways: either what the compiler tells me to add will quickly propagate until it's happy OR it will propagate until lifetime arguments have spread through the entire codebase, turning everything into token salad. The hard part is having a mental model of the data so you can reason about lifetimes. The compiler may help you to get it formally correct, but what would be really cool would be if it would help you find the "inversions" or "isolations" that you will need in order to not end up with lifetime soup.
Yum, salad. In my experience this happens when you overspecify a lifetime or something. After I got this code working, I messed with the lifetimes a bit -- I added one extra `'a` tying the lifetimes of something else to existing bound references. That propagated like wildfire and finally became unusable (proving that those two references could not/would not have the same lifetime).
In more complex situations, it's generally worth thinking which lifetime should be bound together for the first few functions and structs. After that point just trust the compiler.
It's not too hard to get a mental model of lifetimes, really, and once you have it you can look at lifetime parameters and figure out what they mean and how they work with each other. You're anyway implicitly thinking of them as scopes. I usually just ignore lifetime parameters -- they're something you learn to gloss over whilst reading Rust code, and I only really read them (and try to understand them) when I have lifetime errors. Sometimes the lifetime error is due to something unfixable, in which case the compiler will often lead me through a wild goose chase saladifying the entire codebase, or the lifetime error is fixable, in which case the compiler's suggestions usually work. Sometimes they don't, but in those situations you can still fix it by trying to understand the why of it (like how I did in this post, though in the case of this post the entire "understand the why of it" was an afterthought) because things worked.
In more complex situations, it's generally worth thinking which lifetime should be bound together for the first few functions and structs. After that point just trust the compiler.
It's not too hard to get a mental model of lifetimes, really, and once you have it you can look at lifetime parameters and figure out what they mean and how they work with each other. You're anyway implicitly thinking of them as scopes. I usually just ignore lifetime parameters -- they're something you learn to gloss over whilst reading Rust code, and I only really read them (and try to understand them) when I have lifetime errors. Sometimes the lifetime error is due to something unfixable, in which case the compiler will often lead me through a wild goose chase saladifying the entire codebase, or the lifetime error is fixable, in which case the compiler's suggestions usually work. Sometimes they don't, but in those situations you can still fix it by trying to understand the why of it (like how I did in this post, though in the case of this post the entire "understand the why of it" was an afterthought) because things worked.
I bet there are a few common "patterns" to rust ownership, and equally many pitfalls and rookie mistakes when one fails to pick one of these patterns. It becomes extra hard if you approach Rust from a Java/Python/C# perspective where the slightest bit of allocation/RAII and mostly even thinking of the stack feels completely alien. You have to learn that and Rust.
I'd love to see a kind of rust ownership tutorial in which you are asked to address a simple CS problem where these patterns occur in Rust. For example, many seem to find it hard to write a factory type. The next problem could be a doubly linked list and so on.
I'd love to see a kind of rust ownership tutorial in which you are asked to address a simple CS problem where these patterns occur in Rust. For example, many seem to find it hard to write a factory type. The next problem could be a doubly linked list and so on.
The article says you have to clone the vector, as if it's not completely normal to have one object hold a reference to another, "unsafely," in C++, without the world coming to an end. Having one object contain a reference or pointer to another in C++ is completely normal and something you certainly would "dream" of doing -- it happens all the time, you can find it everywhere in the STL, so I don't know what the blog author is getting at. Looking at the diff, the type SubstructureFields already has the lifetime parameter on it, and making FieldInfo, which is only used as a field in the SubstructureFields type (which owns its FieldInfo objects), be subject to the same lifetime constraint, is something that would be perfectly ordinary C++ and it registers near the bottom of the use-after-free risk scale.
> Looking at the diff, the type SubstructureFields already has the lifetime parameter on it, and making FieldInfo, which is only used as a field in the SubstructureFields type (which owns its FieldInfo objects), be subject to the same lifetime constraint, is something that would be perfectly ordinary C++ and it registers near the bottom of the use-after-free risk scale.
Which is why it works. But isn't it nice when the compiler is aware of the "use-after-free risk scale" for your data structures?
Which is why it works. But isn't it nice when the compiler is aware of the "use-after-free risk scale" for your data structures?
I prefer technical accuracy in technical blog posts. If you want somebody to preach at about the merits of compiler-checked safety, it's not me.
It was accurate. See my comment below -- just because `SubstructureFields` had a lifetime parameter already doesn't mean it was borrowing data of a similar lifetime. That data being borrowed could probably live longer than the attributes, but fortunately the attributes live long enough that I can equate the lifetimes. I was able to use the same lifetime parameter by luck -- there was a good chance it wouldn't have compiled and I would have had to introduce another lifetime. (Though since this was a fix the compiler suggested I didn't need to worry too much about that happening -- generally those just work)
Just having a lifetime doesn't mean that it's safe to put any random borrowed data in. In C++ we could have a single pointer to a very long-lived struct ("SubstructureFields"), and wish to introduce another struct ("FieldInfo") which contains a pointer to something that is shorter lived. Note that in large codebases knowing which is "longer lived" is not easy, so from the programmer's perspective there are just two pointers. Assuming that "Okay, we don't have any segfaults now due to the first pointer, introducing the second FieldInfo pointer should be fine then" would be fallacious -- we might be accessing data during a period of time when the first, original pointer is alive, but the second is invalidated. Use after free.
Just having a lifetime doesn't mean that it's safe to put any random borrowed data in. In C++ we could have a single pointer to a very long-lived struct ("SubstructureFields"), and wish to introduce another struct ("FieldInfo") which contains a pointer to something that is shorter lived. Note that in large codebases knowing which is "longer lived" is not easy, so from the programmer's perspective there are just two pointers. Assuming that "Okay, we don't have any segfaults now due to the first pointer, introducing the second FieldInfo pointer should be fine then" would be fallacious -- we might be accessing data during a period of time when the first, original pointer is alive, but the second is invalidated. Use after free.
I'm trying to look more closely at the situation and will compose an answer, hopefully soon, but after going to Staples to get a $50 DVI cable so I can plug this workstation into a monitor (I just moved) and running make -j33 to build the rust compiler and play around with it, it's been thirty minutes and this is what I see: https://i.imgur.com/2JL66Fg.png so bear with me.
Update: You don't have to wade through a lot of code at all. All the code is taking AST parameters that are the source of these Attribute vectors by const reference and returning stuff like a P<Expr> and P<Item> and the like, AST stuff that by its nature doesn't have references with tricky lifetime dependencies to other far-flung AST stuff. So in C++ you can see right from the type signatures (and some basic institutional knowledge) that you're OK.
You can see that the lifetimes are not long or dangerous. SubstructureFields is used in Substructure. A simple grep for that type shows a bunch of functions that take a Substructure by const reference and return a P<Expr>. There is nothing holding wiggly little references to Substructure objects or the like.
Update: You don't have to wade through a lot of code at all. All the code is taking AST parameters that are the source of these Attribute vectors by const reference and returning stuff like a P<Expr> and P<Item> and the like, AST stuff that by its nature doesn't have references with tricky lifetime dependencies to other far-flung AST stuff. So in C++ you can see right from the type signatures (and some basic institutional knowledge) that you're OK.
You can see that the lifetimes are not long or dangerous. SubstructureFields is used in Substructure. A simple grep for that type shows a bunch of functions that take a Substructure by const reference and return a P<Expr>. There is nothing holding wiggly little references to Substructure objects or the like.
You can easily write a function in C++ that takes an object by const reference and creates a dangling reference without returning it. Such functions regularly result in use-after-free (sometimes causing security vulnerabilities) in the wild. Running grep on the signatures of functions is not a sound analysis.
You can pretty well see by grepping whether it's going to create a dangling reference or not. For example, in C++ codebases I'm most familiar with, anything that takes a const foo& isn't going to retain a reference, and for unfamiliar stuff that takes a const foo* or foo*, you can look at the outputs, which, if it's a P<Expr>, all you need is the institutional knowledge that Exprs don't have dangling references.
You don't need to debate with me the merits of replacing visual analysis with sound analysis. (If I have a negative opinion of Rust on the matter, it's that it's not good enough at that.) My beef here is with the way the blog article overstates the case, saying you just wouldn't do this sort of thing or this specific thing. You so would make temporary references deep into an AST while expanding derived implementations.
You don't need to debate with me the merits of replacing visual analysis with sound analysis. (If I have a negative opinion of Rust on the matter, it's that it's not good enough at that.) My beef here is with the way the blog article overstates the case, saying you just wouldn't do this sort of thing or this specific thing. You so would make temporary references deep into an AST while expanding derived implementations.
Substructure objects contain other objects contain other objects and so on. Are you going to grep for all of these?
Yes, in this situation owned Expr pointers are returned, and fortunately I know that Expr contained no borrowed data (with a lack of lifetime annotation that is easy, but in C++ you might have to check)
But there could be other pointers being passed around internally (eg replacing the old pointer from a field of an argument with a new one, one which actually is short lived or is being iterated over. Just because a pointer isn't returned doesn't mean we're safe. With const we get some safety until interior mutability and casts come into the picture -- as I said before, consting a complex codebase is not easy unless you keep switching back and forth.
Yes, in this situation owned Expr pointers are returned, and fortunately I know that Expr contained no borrowed data (with a lack of lifetime annotation that is easy, but in C++ you might have to check)
But there could be other pointers being passed around internally (eg replacing the old pointer from a field of an argument with a new one, one which actually is short lived or is being iterated over. Just because a pointer isn't returned doesn't mean we're safe. With const we get some safety until interior mutability and casts come into the picture -- as I said before, consting a complex codebase is not easy unless you keep switching back and forth.
Oh, the general situation is something that might be common in C++, but in this specific situation -- the vector was created and held (and most probably being mutated) somewhere far away, and I have no clue what is being done with it -- and the relevant portion of the codebase is huge -- holding a reference is something I wouldn't dream of doing.
SubstructureFields could have had a lifetime parameter of something that was supposed to live longer, we can't be easily sure.
SubstructureFields could have had a lifetime parameter of something that was supposed to live longer, we can't be easily sure.
You can wrap the vector parameter or pointer/size parameter in a type, chase the compilation errors up to the places where the original vector exists, and make sure the usages there be well-protected.
That original vector could be used in a bajillion places all over the codebase (hint: It is. It's central to the AST representation in this case and is used everywhere). I'd be wading through a lot of code which doesn't even come close to the expansion phase.
This trick doesn't let me track the flow of a program to find out which portions of code will be run while my pointer is active. Tracking the flow only happens in Rust.
This trick doesn't let me track the flow of a program to find out which portions of code will be run while my pointer is active. Tracking the flow only happens in Rust.
So it turns out you wouldn't be wading through a lot of code. Especially when it's in an AST, which, even if it's the sort of compiler that mutably updates its AST, probably isn't going to have surprise updates to source-position-tied attributes at macro-expansion time. Especially not when you have some basic const correctness to convince you of this in C++ (the equivalent of what we're seeing in this Rust code), but even in C and without const safety, and with a compiler that practices mutating an AST (with metadata) in-place, this would be a comfortable thing to do. (In C there'd be a clearly named function that adds "internal" attributes and you'd see its existence and where it's getting called, while in C++ if you didn't have type safety you'd make the field private, give the functions modifying it funny names, and grep for them.)
> Especially not when you have some basic const correctness to convince you of this in C++ (the equivalent of what we're seeing in this Rust code),
Const correctness has nothing to do with it. It does not enforce reference safety in C++, and the Rust compiler does not use "const correctness" to enforce proper memory management.
Const correctness has nothing to do with it. It does not enforce reference safety in C++, and the Rust compiler does not use "const correctness" to enforce proper memory management.
A const parameter in C++ tells you that nothing is going to be modifying the object in the body of the function you're writing, in which you're making a temporary reference to said object.
Unless you're trying to say that it's not a formal proof of correctness, but that's not an argument that engages with the claim the blog article made about whether this is something you would or wouldn't, or couldn't do, in C++.
Unless you're trying to say that it's not a formal proof of correctness, but that's not an argument that engages with the claim the blog article made about whether this is something you would or wouldn't, or couldn't do, in C++.
[deleted]
This is really useful; I have been recently working on a huge c++ project and I had the exact same problem you mentioned with large vectors. The way I handled this problem was to wrap vector in a class which provides locking mechanism for expanding operations so while the vector is growing nobody can access it.
I am excited to try Rust soon.
[deleted]
For an outsider, Rust is still barely usable. I tried building a simple demo using the beta release today, and regretted right away. Lots of "unstable feature" errors that I can't figure out how to get rid of. Many useful features are unstable. I don't think I'm going to touch it again for a few months.
Sorry you had a bad experience. Just curious, have you tried asking for help on /r/rust or irc? I find Rust community is pretty friendly and helpful.
What was unstable that you wanted to use?
(Not trying to deny your experience: reporting this helps the Rust team determine what to prioritize)
Collections. I tried to set the flag to allow them. This used to work in the dev release I downloaded 2 months ago, but there is a new error that says unstable features are not allowed in the beta release and I cannot override an outer flag –– or something like that. At that point, I just gave up.
Most of the collections API is stable, & there may be a fairly simple alternative for the particular unstable feature you wanted to use. On the other hand, you could also download the nightly branch of Rust to be able to use all of the unstable features.
I think the nature of the 1.0 release and the train model is not very easy to communicate: the Rust team hopes to commit strongly to semver, so "stable Rust" is only that which they are comfortable committing to providing backwards compatibility for indefinitely. It's really pretty fine to be on the nightly track if you don't mind occasionally having to change a method call or something when an unstable API you use change.
Not trying to dismiss what you've said - I just think that the strength with which you expressed it ("Rust is still barely usable") was not accurate & that if you ask in the IRC channel or users.rust-lang.org, there is likely a no-or-little-hassle workaround for whatever feature you wanted.
I think the nature of the 1.0 release and the train model is not very easy to communicate: the Rust team hopes to commit strongly to semver, so "stable Rust" is only that which they are comfortable committing to providing backwards compatibility for indefinitely. It's really pretty fine to be on the nightly track if you don't mind occasionally having to change a method call or something when an unstable API you use change.
Not trying to dismiss what you've said - I just think that the strength with which you expressed it ("Rust is still barely usable") was not accurate & that if you ask in the IRC channel or users.rust-lang.org, there is likely a no-or-little-hassle workaround for whatever feature you wanted.
I also qualified it by saying "for an outsider." If you're just trying to get up and running, small things can be a big deal. The feature I was trying to use used to work and then it stopped working all of a sudden when I upgraded today, which was frustrating. But I haven't given up on Rust, and that's why I was saying I'd try again in a few months. I'm going to try the IRC next time around.
Yeah transitioning from 0.12 to 1.0 when you're not super committed to the project is probably frustrating because of the stability shift. The easiest thing to do if you don't care about targeting stable Rust promptly with what you're doing is to install nightly instead of beta.
This is really exciting, and clearly more powerful than what you get in C/C++. That being said:
>In a language like C++ there’s only once choice in this situation; that is to clone the vector.
What about a shared_ptr to the vector?
>In a language like C++ there’s only once choice in this situation; that is to clone the vector.
What about a shared_ptr to the vector?
A shared_ptr does not prevent iterator invalidation caused by reallocation, because the pointed-to vector is not locked, or in any other form stopped from being modified.
In Rust, when you have an immutable reference to a memory location, you know for certain that it won't be modified by distant code as long as you hold the reference. This guarantee also allows more compiler optimizations, because v[5] has the same value after a function call as before.
In Rust, when you have an immutable reference to a memory location, you know for certain that it won't be modified by distant code as long as you hold the reference. This guarantee also allows more compiler optimizations, because v[5] has the same value after a function call as before.
Thanks for the explanation, I missed that!
Doesn't help, because you don't necessarily know when the other shared pointer could push something to vector, potentially reallocating the internal array and thereby invalidating any references you had to the contents of the vector.
Ah okay - I didn't realize rust was guaranteeing that no one else would _modify_ the vector while an instance of that struct was around. That's pretty cool.
Rust follows multiple-readers or single-writer even in single-threaded situations. You can hold multiple immutable references to an object, or a single mutable one.
Initially this gets annoying, but as time passes one realizes how awesome this is. It prevents iterator invalidation (and vector-move-invalidation), for one (it's also necessary to get memory safety for our ADT enums since you can otherwise change the variant and invalidate a reference to its contents). In general you realize that in a large codebase, it's almost as bad as a multithreaded situation -- you don't know what objects are being modified by the methods being called and the code is too large to easily figure it out.
Initially this gets annoying, but as time passes one realizes how awesome this is. It prevents iterator invalidation (and vector-move-invalidation), for one (it's also necessary to get memory safety for our ADT enums since you can otherwise change the variant and invalidate a reference to its contents). In general you realize that in a large codebase, it's almost as bad as a multithreaded situation -- you don't know what objects are being modified by the methods being called and the code is too large to easily figure it out.
It's very cool. Rust guarantees that at any point in time there will only be one part of the code that can modify the contents of some data, and will fail to compile if it detects a situation where this is not the case.
Remember that this was an API, exposing the `&[Attributes]` to external consumers. I may not want to hold pointers to the insides, but a consumer of the API might.
Also, iterator invalidation.
Also, runtime overhead of shared_ptr.
(I added a footnote to the post to explain the issues with shared pointers)
Also, iterator invalidation.
Also, runtime overhead of shared_ptr.
(I added a footnote to the post to explain the issues with shared pointers)
Or scoped_ptr to transfer ownership.
There's certainly more than one choice, though.
There's certainly more than one choice, though.
Not really, and I say this as a fan of C++.
These things protect against one part of the problem - deletion of the vector, but not the other part - mutation of the vector, causing a new internal allocation, and leaving references to any elements of the previous vector invalid.
It's possible to make sure this isn't happening in C++ if a) your code base is small enough and b) you are careful, but the point of the article is that with Rust, you can make the changes and be confident that the compiler will fail if you do anything dangerous.
With C++, you can make the changes and through careful checking be reasonably confident everything works (perhaps only to find out later that you were wrong). It's very different from having the compiler verify that you are not doing something unsafe.
These things protect against one part of the problem - deletion of the vector, but not the other part - mutation of the vector, causing a new internal allocation, and leaving references to any elements of the previous vector invalid.
It's possible to make sure this isn't happening in C++ if a) your code base is small enough and b) you are careful, but the point of the article is that with Rust, you can make the changes and be confident that the compiler will fail if you do anything dangerous.
With C++, you can make the changes and through careful checking be reasonably confident everything works (perhaps only to find out later that you were wrong). It's very different from having the compiler verify that you are not doing something unsafe.
I couldn't have moved the vector, it was being used elsewhere.
Maybe I'm missing something, but why is it crucial that you hold on to a reference to the contents of the vector, and not the vector itself? Because, to me, the obvious "C++ way" to do it would be something like
struct FieldInfo {
//
const std::vector<ast::Attribute>& attrs;
}Well, if the object that owns the vector is destroyed then that reference will go dangling. The compiler can't (soundly) check this.
That operates under the assumption that no FieldInfo will be accessible after the original vector is deallocated, and similarly requires auditing to ensure this property holds (while making no guarantees about the future).
Actually having a constant reference as a member works as expected in C++ because the compiler will force you to initialize it a construction.
It is true it will not check at compilation that you access a const reference of a destroyed object, but to be honest with RAII this is not really a problem as the lifetime of your objects should be pretty clear, by construction.
In debug mode your program will quickly assert if you access a destroyed STL container.
To be honest I think in this case:
- either you want to snapshot the value and therefore you should clone the container (almost no performance cost for small containers)
- you want to access the current value but then it means you know about the lifetime.
It is true it will not check at compilation that you access a const reference of a destroyed object, but to be honest with RAII this is not really a problem as the lifetime of your objects should be pretty clear, by construction.
In debug mode your program will quickly assert if you access a destroyed STL container.
To be honest I think in this case:
- either you want to snapshot the value and therefore you should clone the container (almost no performance cost for small containers)
- you want to access the current value but then it means you know about the lifetime.
"the lifetime of your objects should be pretty clear, by construction"
Given the potentially-unbounded consequences of triggering UB in C++, "pretty clear" seems like not enough.
Given the potentially-unbounded consequences of triggering UB in C++, "pretty clear" seems like not enough.
> And this is despite the fact that I still don’t know where that particular vector is modified or destroyed — I didn’t explore that far because I didn’t need to! (or want to :P)
I find that a bit worrying - while it might well be an example of what's technically possible in some cases of using Rust to prevent segfault-type crashes, it doesn't mean there aren't now logic errors in the code which would do something just as bad. I guess it depends on how well you know the code and the context in which it's being used.
I find that a bit worrying - while it might well be an example of what's technically possible in some cases of using Rust to prevent segfault-type crashes, it doesn't mean there aren't now logic errors in the code which would do something just as bad. I guess it depends on how well you know the code and the context in which it's being used.
> it doesn't mean there aren't now logic errors in the code which would do something just as bad
Yes, it does. He wants to have immutable access to some data, the Rust borrow checker's rules provide certainty that the data is not deleted or mutated while his code is accessing it.
What could happen that is "just as bad"? Data being referenced cannot be changed.
Yes, it does. He wants to have immutable access to some data, the Rust borrow checker's rules provide certainty that the data is not deleted or mutated while his code is accessing it.
What could happen that is "just as bad"? Data being referenced cannot be changed.
I think an important thing to make sense of this is that Rust doesn't guarantee that what Manishearth did will work! Just that if it doesn't work, the code will not compile. It could have been the case that this vector was mutated or deleted at some point during the lifetime of his new struct; in that case, his code would not have compiled.
In this case it does.
I was more talking about the overall not caring theme. I'd hope that's only because they know the code so can make those sort of assumptions.
Are you saying Rust can find and prevent at compile time logic errors as well?
I was more talking about the overall not caring theme. I'd hope that's only because they know the code so can make those sort of assumptions.
Are you saying Rust can find and prevent at compile time logic errors as well?
You "don't care," I'm sure, about all kinds of things when you're programming - depending on what level you're at. Unless you're drawing up CPU schematics, there is something that you have abstracted away as no longer your concern.
Rust allows you to not care about more things than C/C++ despite compiling to a comparable binary. In particular, Rust makes refactoring and extending a code base much more care-free than many other languages (including high level languages) because of the strong type and memory safety guarantees of its semantics.
EDIT (re: your edit) - No one said anything about preventing arbitrary logic errors, just that what Manishearth did will not introduce logic errors because of Rust's guarantees. This could not be the data that Manishearth wants, but the compiler guarantees that Manishearth cannot change the data or delete the data and that the data is not changed or deleted while he is referencing it. That is what is so great.
Rust allows you to not care about more things than C/C++ despite compiling to a comparable binary. In particular, Rust makes refactoring and extending a code base much more care-free than many other languages (including high level languages) because of the strong type and memory safety guarantees of its semantics.
EDIT (re: your edit) - No one said anything about preventing arbitrary logic errors, just that what Manishearth did will not introduce logic errors because of Rust's guarantees. This could not be the data that Manishearth wants, but the compiler guarantees that Manishearth cannot change the data or delete the data and that the data is not changed or deleted while he is referencing it. That is what is so great.
What edit? My second comment?
That was my original point regarding logic errors and not caring - yes, it may well be that Rust gives you much more confidence that you don't do invalid/wrong things. But that excludes logic errors (it may well reduce the possibility of logic errors, but it doesn't prevent them), and there's still room in large code bases for things to break even with very good static type checking.
That was my original point regarding logic errors and not caring - yes, it may well be that Rust gives you much more confidence that you don't do invalid/wrong things. But that excludes logic errors (it may well reduce the possibility of logic errors, but it doesn't prevent them), and there's still room in large code bases for things to break even with very good static type checking.
> I was more talking about the overall not caring theme.
> Are you saying Rust can find and prevent at compile
> time logic errors as well?
Would you find it equally worrying if stack-based languages claimed they improved over x86 because you needn't worry about clobbering values in registers? I guess it depends on how well you know the code and its context. Might still be logic bugs.
And man, functional languages. It's pretty much 100% logic bugs there. I worry most about those sort of programs.
And man, functional languages. It's pretty much 100% logic bugs there. I worry most about those sort of programs.
If you only have 100% logic bugs in your code, I'd say the code is in damn good shape. That means all your bugs are "meaningful".
Different programming platforms have different "incidental complexity" causing bugs not related to actual program logic, but merely how it is put together, so to speak. In particular, Rust seems objectively better than C++ in this regard.
Different programming platforms have different "incidental complexity" causing bugs not related to actual program logic, but merely how it is put together, so to speak. In particular, Rust seems objectively better than C++ in this regard.
What sort of logic errors would be possible? The slice cannot be mutated, and the compiler has verified for me that nobody else is mutating it.
I was just holding on to a reference for an API to consume.
Of course, in more complicated situations you need to worry about logic errors. But in this case I was pretty sure I didn't have anything to worry about.
Still you never have to worry about memory unsafety, which is a huge cognitive load off my mind. Similar to how you don't have to worry about some types of memory unsafety in languages like Java. Or how most programmers don't worry about cache coherence since the processor handles it. Rust just raises the bar. If you know what it can and can't prevent, you can be carefree about many things.
I was just holding on to a reference for an API to consume.
Of course, in more complicated situations you need to worry about logic errors. But in this case I was pretty sure I didn't have anything to worry about.
Still you never have to worry about memory unsafety, which is a huge cognitive load off my mind. Similar to how you don't have to worry about some types of memory unsafety in languages like Java. Or how most programmers don't worry about cache coherence since the processor handles it. Rust just raises the bar. If you know what it can and can't prevent, you can be carefree about many things.
A C++ function returning pointer to vector, or a public field of that type, would be considered bad API for the reasons mentioned in this article. The solution is to not do that. I'm not sure that Rust's ability to allow shared ownership of a vector makes this API pattern any better.
But it's a bad pattern in other languages because they don't guarantee safe access. Rust does, thus making it a good pattern.
It's like saying that driving your car off a cliff is always a bad idea even when you've acquired a car that can fly.
It's like saying that driving your car off a cliff is always a bad idea even when you've acquired a car that can fly.
It's a bad pattern in any language because it exposes too much details of storage layout and violates encapsulation. You probably don't want to expose internal implementation details in your API unless such exposure is critical to performance, and if it is critical to performance it seems unlikely that you'd be writing it in anything other than C++ anyway.
This seems orthogonal to internal implementation hiding, especially if the "API" is not a public library API, say, but an internal one.
Then again, contracts about who can modify what data are also part of any public API; historically, there have been plenty of (quite decent) APIs that have returned pointers to internal structures that have limited lifetimes, in languages like C that don't enforce lifetimes at all.
Then again, contracts about who can modify what data are also part of any public API; historically, there have been plenty of (quite decent) APIs that have returned pointers to internal structures that have limited lifetimes, in languages like C that don't enforce lifetimes at all.
What's wrong with that API pattern if the compiler guarantees that it will be used correctly? It's certainly simpler and allows for better code reuse.
I would assume it would be bad API design in Rust too.
For all use-cases that are possible this kind of API it would be safe to use a const reference to a vector it C++ too. Of course with the difference that in Rust the correct use can be statically verified.
But I think for a really flexible API that allows parallel modification you should go for a copy or some kind of immutable-vector oder copy-on-writer-vector type in both languages.
For all use-cases that are possible this kind of API it would be safe to use a const reference to a vector it C++ too. Of course with the difference that in Rust the correct use can be statically verified.
But I think for a really flexible API that allows parallel modification you should go for a copy or some kind of immutable-vector oder copy-on-writer-vector type in both languages.
Why would it be bad API design in Rust? Rust ensures that it's safe.
(Besides, all these were internal functions that wouldn't be used as an API)
You don't always want parallel modification.
(Besides, all these were internal functions that wouldn't be used as an API)
You don't always want parallel modification.
Yes - it guarantees that it's safe at the moment.
But it limits how the API (especially on provider site) can be used in the future. If there comes up a need that that the data might need to be mutated at one point in future you might need to change it to an other interface in both of the languages and then change probably lot's of consumers according to it.
If it's only an internal API that might be ok - for a public API it would be to inflexible for my taste.
But it limits how the API (especially on provider site) can be used in the future. If there comes up a need that that the data might need to be mutated at one point in future you might need to change it to an other interface in both of the languages and then change probably lot's of consumers according to it.
If it's only an internal API that might be ok - for a public API it would be to inflexible for my taste.
Generally external APIs which return pointers will be only to get views into the internal data. And you can return an `&mut` pointer if you want others to be able to mutate it, if possible (as in, if the borrow checker accepts it). If not possible, then you don't want to do that anyway, unless you redesign the internals.
Generally to add mutation you might add another method in the situation you describe, but I can't think of such a situation coming up in a well-designed Rust API. (You're treating the API as if it would be designed the C++ way -- it wouldn't)
FWIW most libraries do use a COW type in many places to get this flexibility. But in most situations you can just return mutable or immutable slices.
Generally to add mutation you might add another method in the situation you describe, but I can't think of such a situation coming up in a well-designed Rust API. (You're treating the API as if it would be designed the C++ way -- it wouldn't)
FWIW most libraries do use a COW type in many places to get this flexibility. But in most situations you can just return mutable or immutable slices.
My comment was not directly about having the ability to mutate or not.
It's about that this API design causes close coupling between the API and the internal representation of the API. If you change your internal software structure and the data is stored in another way or you choose the make your library concurrent internally where different parts could mutate the vector while the user might request the same data in parallel then returning the slice would no longer be possible. And so you would have to change your external API and all consumers of the API only because you wanted to change internal things.
But as I said: Depends on the use-case.
It's about that this API design causes close coupling between the API and the internal representation of the API. If you change your internal software structure and the data is stored in another way or you choose the make your library concurrent internally where different parts could mutate the vector while the user might request the same data in parallel then returning the slice would no longer be possible. And so you would have to change your external API and all consumers of the API only because you wanted to change internal things.
But as I said: Depends on the use-case.
> If you change your internal software structure and the data is stored in another way or you choose the make your library concurrent internally where different parts could mutate the vector while the user might request the same data in parallel then returning the slice would no longer be possible
In that case you wouldn't want to return a slice, because it would be dangerous. The compiler would prevent you from invalidating an API use case. I don't see a problem here.
In that case you wouldn't want to return a slice, because it would be dangerous. The compiler would prevent you from invalidating an API use case. I don't see a problem here.
People don't choose a language. They choose a platform (or stack or environment). If you want to evaluate a language look at the language's platform. Currently Rust is not part of any platform but may become the language of the 'Mozilla platform' in the future.
There's no overhead in caling a C api nor in other code calling Rust code as if it were a C api. There's FFI setup and some caveats both ways but a lot of the larger Rust projects involve (e.g Servo, Piston) interfacing with pre-existing C libraries.
People choose standards, not platforms. Some use just platform-specific standards, others use cross-platform standards. Rust doesn't need a platform, it needs support for cross-platform standards.
This platform advocacy comes from what business needs (ability to extort money by gatekeeping), not what's technically sensible nor useful for the people.
This platform advocacy comes from what business needs (ability to extort money by gatekeeping), not what's technically sensible nor useful for the people.
I don't think Mozilla plans to create a Rust-based platform, but the community is already making libraries that are useful at various levels of a stack.
Garbage collected languages handle this fine too.
They handle it fine by tracking references at runtime. The advantage here is compile time verification that the reference will stay valid for the life of the struct - zero runtime overhead.
Iterator invalidation is still a problem.
Plus, having a GC for this would be quite heavy. The codebase was a compiler -- performance is necessary since a compiler's going to be slow anyway.
Plus, having a GC for this would be quite heavy. The codebase was a compiler -- performance is necessary since a compiler's going to be slow anyway.
But only for memory, and not for other kinds of resources.
I just spent an entire week working on a chatbot in Rust, and I found the steep learning curve to be very challenging. While Rust seems to be a big win for teams that are accustomed to C++/C development, it does seem to be very demanding for web devs who always work with a garbage collector.
In particular, I was trying to use Iron to write an http server that could receive web hooks. Iron, quite sensibly, handles each incoming request to a server in a separate thread. However, as a beginner it's very challenging to figure out how to persist data between requests without writing to disk or persisting to a db. Mutating state across threads is hard. Currently, there's also no version of channels that are single producer, multiple consumer. I'm sure as the ecosystem develops, more examples will make this easier, but for now it's surprisingly difficult to figure out.
Fortunatey, the #rust IRC channel supported me all the way. It's a very friendly and active community, but without their support I would have probably given up.
I look forward to seeing more posts at this level of detail help those of us struggling to learn Rust understand how the experts use Rust.