Three new utility functions in C++23(mariusbancila.ro)
mariusbancila.ro
Three new utility functions in C++23
https://mariusbancila.ro/blog/2022/11/08/three-new-utility-functions-in-cpp23/
17 comments
1. It is a compiler defined function (`__builtin_unreachable()`), but the issue is that MSVC doesn't have it, so you need a different implementation per compiler [0]. Plus, if a new compiler shows up (besides MSVC/GCC/LLVM), you'd need to investigate what the correct way to express `__builtin_unreachable` is.
From a compiler perspective, using a function makes the most sense, since that fits into the existing control-flow analysis that the compiler will do. Pragmas are processed by the pre-processor, so they aren't appropriate for expressing control flow hints.
2. `std::to_underlying(t)` is a wrapper around `static_cast<std::underlying_type_t<std::remove_cv_t<std::remove_reference_t<decltype(t)>>>>(t)`. So a lot fewer characters. It's also useful since `std::underlying_type_t<T>` behaves weirdly if `T` is not an enum type.
I think you are maybe missing the context that C++ allows the representation of an enum to be defined, e.g. `enum class X : unsigned char {};` vs `enum class Y : unsigned long long {};`. So you can't always cast to `int`. Technically, this isn't the case in C either: the type defaults to `int`, but the compiler will pick a larger type if necessary, e.g. `enum Z { a = ((long long)INT_MAX) + 1 };`
3. `htonl` are not standardized, so they were not part of the C++ standard library. Also, on Windows, I believe you'd need to include `winsock.h` to get access to them, which has its own idiosyncratic issues. You are also missing the context of C++ defining operator overloading, so you can call `std::byteswap(0ull)` and get an `unsigned long long` and you can call `std::byteswap(std::uint16_t{0})` and get a 16 bit unsigned integer.
[0]: https://stackoverflow.com/questions/60802864/emulating-gccs-...
From a compiler perspective, using a function makes the most sense, since that fits into the existing control-flow analysis that the compiler will do. Pragmas are processed by the pre-processor, so they aren't appropriate for expressing control flow hints.
2. `std::to_underlying(t)` is a wrapper around `static_cast<std::underlying_type_t<std::remove_cv_t<std::remove_reference_t<decltype(t)>>>>(t)`. So a lot fewer characters. It's also useful since `std::underlying_type_t<T>` behaves weirdly if `T` is not an enum type.
I think you are maybe missing the context that C++ allows the representation of an enum to be defined, e.g. `enum class X : unsigned char {};` vs `enum class Y : unsigned long long {};`. So you can't always cast to `int`. Technically, this isn't the case in C either: the type defaults to `int`, but the compiler will pick a larger type if necessary, e.g. `enum Z { a = ((long long)INT_MAX) + 1 };`
3. `htonl` are not standardized, so they were not part of the C++ standard library. Also, on Windows, I believe you'd need to include `winsock.h` to get access to them, which has its own idiosyncratic issues. You are also missing the context of C++ defining operator overloading, so you can call `std::byteswap(0ull)` and get an `unsigned long long` and you can call `std::byteswap(std::uint16_t{0})` and get a 16 bit unsigned integer.
[0]: https://stackoverflow.com/questions/60802864/emulating-gccs-...
> 3. `htonl` are not standardized, so they were not part of the C++ standard library.
I was questioning the motivation of this to facilitate network-byte-order issues as stated in the blog post. The proposal[1] (that the post linked to) also confirmed that the motivation was to expose more machine code intrinsics rather than deal with network byte order like I expected.
Concerning 1.) yeah, I guess I'd prefer a #pragma aesthetically, but didn't think about the fact that it wouldn't be exposed to the compiler.
Thanks for the well thought out reply!
[1] : https://isocpp.org/files/papers/P1272R4.html#motivation
I was questioning the motivation of this to facilitate network-byte-order issues as stated in the blog post. The proposal[1] (that the post linked to) also confirmed that the motivation was to expose more machine code intrinsics rather than deal with network byte order like I expected.
Concerning 1.) yeah, I guess I'd prefer a #pragma aesthetically, but didn't think about the fact that it wouldn't be exposed to the compiler.
Thanks for the well thought out reply!
[1] : https://isocpp.org/files/papers/P1272R4.html#motivation
one easily overlooked use of hton ntoh is portable binary persistent data. you want to read and write binary files in different cpu architecture hosts? mount a pluggable "disk" from different devices? use hton and ntoh to write/read binary data...
> Technically, this isn't the case in C either: the type defaults to `int`, but the compiler will pick a larger type if necessary, e.g. `enum Z { a = ((long long)INT_MAX) + 1 };`
if you read this and alarm bells didn't ring in your head I really invite you to immediately go check any enum you may have defined in your code because this is absolutely false with MSVC in C++ (https://gcc.godbolt.org/z/6bqW9rE81) (and C is often compiled as C++ on windows)
if you read this and alarm bells didn't ring in your head I really invite you to immediately go check any enum you may have defined in your code because this is absolutely false with MSVC in C++ (https://gcc.godbolt.org/z/6bqW9rE81) (and C is often compiled as C++ on windows)
In practice I don't think that the latitude of making enum size depend on the enumeration has ever been used as it would be too easy to break the ABI. IIRC compilers have allowed forward declaring enums as an extension before c++11 which obviously doesn't work if the size depends on the definition.
I don't think that's true. Neither C nor C++ would allow you to invoke a function with an incomplete type, regardless of whether it is a `struct` or an `enum`, so there's no ABI issues with forward declaring them, but you wouldn't be able to define or invoke a function that takes an incomplete type argument by value.
struct A;
enum B; // as you pointed out, not allowed by the C++ standard
// fine to declare, define, and invoke
void fooA(struct A*);
void fooB(enum B*);
// ok to forward declare, but you can't call them
void barA(struct A);
void barB(enum B);Well I misremembered it seems. You can use forward declared enum class or enum : <type> in C++ as per standard:
Enums in GCC do always default to int or unsigned int, unless -fshot-enums is used which is ABI breaking.
Probably I was misremembering this combination of the int ABI and the forward declaring extension.
enum class A; // defaults to int
enum B :int;
enum C;
fooA(A);
FooB(B);
FooC(C);
fooA((A)0);
fooB((B)0);
fooC((C)0); // error
Enums in GCC do always default to int or unsigned int, unless -fshot-enums is used which is ABI breaking.
Probably I was misremembering this combination of the int ABI and the forward declaring extension.
>Pragmas are processed by the pre-processor
This is wrong a lot of pragmas, I would even say most pragmas are not handled by preprocessor. Some examples: pragma pack, warning control, pragma GCC unroll, per function optimization setting changes, all the pragmas which 1:1 map to c++11 style attributes. None of that is handled by preprocessor, pragma once seems like rare one which is. Yes all of them are compiler specific, but handling compiler specific behavior is one of the main purpose of pragmas.
This is wrong a lot of pragmas, I would even say most pragmas are not handled by preprocessor. Some examples: pragma pack, warning control, pragma GCC unroll, per function optimization setting changes, all the pragmas which 1:1 map to c++11 style attributes. None of that is handled by preprocessor, pragma once seems like rare one which is. Yes all of them are compiler specific, but handling compiler specific behavior is one of the main purpose of pragmas.
I meant that they are processed before any syntactical analysis, so they're not particularly useful for this kind of thing. For example, these are all wrong usages of our hypothetical `#pragma unreachable`:
Also, I think you are conflating pragmas with `__attribute__`, which is how you set per function optimization settings. If you do that with pragmas, then it isn't limited to a single function.
void foo();
#pragma unreachable
class bar
{
#pragma unreachable
};
namespace foobar {
#pragma unreachable
}
Also, I think you are conflating pragmas with `__attribute__`, which is how you set per function optimization settings. If you do that with pragmas, then it isn't limited to a single function.
This is the intent and purpose of _Pragma; it provides a way to use existing #pragmas that are tokenized and handled a bit later, so they can e.g. be included in macro expansions.
I think the only benefit of `_Pragma` is that it enables defining a macro that expands into a `#pragma` definition. I don't think there's any other use case beyond that.
If pragma works for omp for it would work for unreachable. But it is just ugly and there really wouldn't be any reason to use it here.
> From a compiler perspective, using a function makes the most sense, since that fits into the existing control-flow analysis that the compiler will do. Pragmas are processed by the pre-processor, so they aren't appropriate for expressing control flow hints.
This seems par for the course for all C++ stuff: it's designed from a compiler perspective, not from a programmer perspective.
The correct way, IMHO, is to design features that supports the user's workflow, not to design the same feature in a way to make the compiler's job easier.
> `htonl` are not standardized, so they were not part of the C++ standard library.
They're POSIX standardised. The decision should have been to adopt something that exists in an existing and widespread standard rather than the worrisome not-invented-here syndrome that I see here.
This seems par for the course for all C++ stuff: it's designed from a compiler perspective, not from a programmer perspective.
The correct way, IMHO, is to design features that supports the user's workflow, not to design the same feature in a way to make the compiler's job easier.
> `htonl` are not standardized, so they were not part of the C++ standard library.
They're POSIX standardised. The decision should have been to adopt something that exists in an existing and widespread standard rather than the worrisome not-invented-here syndrome that I see here.
> `htonl` are not standardized
They are: https://pubs.opengroup.org/onlinepubs/9699919799/functions/h...
They are: https://pubs.opengroup.org/onlinepubs/9699919799/functions/h...
Yeah, but that's platform-specific.
It's not part of the C or the C++ standard library.
> Pragmas are processed by the pre-processor, so they aren't appropriate for expressing control flow hints.
I don't thing this is remotely true. C++ pragmas were designed with the express purpose of providing additional information to compilers.
I don't thing this is remotely true. C++ pragmas were designed with the express purpose of providing additional information to compilers.
Attributes are better suited for that. #pragma has always just been a grandfathered in hack.
> Attributes are better suited for that. #pragma has always just been a grandfathered in hack.
I'm not so sure attributes are better. They have political traction, but that does not mean better. All major compilers use pragmas effectively to implement custom compiler flags. See for instance how Visual C++ uses pragmas extensively to toggle specific compiler warnings, not to mention the infamous #pragma once
I'm not so sure attributes are better. They have political traction, but that does not mean better. All major compilers use pragmas effectively to implement custom compiler flags. See for instance how Visual C++ uses pragmas extensively to toggle specific compiler warnings, not to mention the infamous #pragma once
[deleted]
> You are also missing the context of C++ defining operator overloading, so you can call `std::byteswap(0ull)` and get an `unsigned long long` and you can call `std::byteswap(std::uint16_t{0})` and get a 16 bit unsigned integer.
I can believe this is useful in explicitly-typed form, i.e. using std::byteswap<T> with T specified. But letting T be inferred seems quite dangerous: C++ loves changing integer types around all by itself (via type promotion, for example), and byteswap<int> and byteswap<long> are (on UNIXy systems) simply not the same operation. For that matter, byteswap should really only be used on uintN_t.
byteswap(a+b) is just asking for trouble.
I can believe this is useful in explicitly-typed form, i.e. using std::byteswap<T> with T specified. But letting T be inferred seems quite dangerous: C++ loves changing integer types around all by itself (via type promotion, for example), and byteswap<int> and byteswap<long> are (on UNIXy systems) simply not the same operation. For that matter, byteswap should really only be used on uintN_t.
byteswap(a+b) is just asking for trouble.
> Plus, if a new compiler shows up (besides MSVC/GCC/LLVM)
They are still far from being the only game in town.
They are still far from being the only game in town.
That was unclear on my part, but I meant a new compiler in terms of "new to the project", not "new to the C++ community". The linked SO answer only covers those three compilers, which illustrates the issue I was talking about (you need to add a new `#elif` branch)
Longtime C, C++, and embedded programmer here.
1. Why "should" it be something different? It is semantically part of code flow; making it a pragma breaks that model. This replaces the nonstandard __builtin_unreachable().
2. This doesn't exist to save typing. static_cast<auto> doesn't make sense to me (that reads like a no-op). static_cast<underlying> introduces a new reserved word which is a big no-no.
3. This is not the same as htonl, which only does anything on little-endian machines. (htonl is also a POSIX function, not a C++ function.)
1. Why "should" it be something different? It is semantically part of code flow; making it a pragma breaks that model. This replaces the nonstandard __builtin_unreachable().
2. This doesn't exist to save typing. static_cast<auto> doesn't make sense to me (that reads like a no-op). static_cast<underlying> introduces a new reserved word which is a big no-no.
3. This is not the same as htonl, which only does anything on little-endian machines. (htonl is also a POSIX function, not a C++ function.)
1. Isn't std inadequate in "embedded" programming?
In principle the C++ standard does define a subset of its standard library which is available in Freestanding environments (ie without an operating system).
In practice what is actually available and whether it works satisfactorily varies considerably more than for the hosted environments. Almost everything is up for grabs and so you probably can't rely on the standard much. Your compiler vendor probably couldn't care less what the standard says anyway.
In practice what is actually available and whether it works satisfactorily varies considerably more than for the hosted environments. Almost everything is up for grabs and so you probably can't rely on the standard much. Your compiler vendor probably couldn't care less what the standard says anyway.
Depends on what's you're targeting, but if you're coding for say, AVRs, you're not using C++, but rather C, in which case `unreachable` is your friend. [1]
[1] https://en.cppreference.com/w/c/program/unreachable
[1] https://en.cppreference.com/w/c/program/unreachable
In some cases not even C, just a tiny subset full of chip vendor extensions.
A lot of it, sure. But there are a lot of header only types which do not dynamically allocate memory on their own. For instance <algorithm>, <atomic>, <bitset>, <limits>, <iterator>, <type_traits>, etc.
> ...`std::unreachable`. But shouldn't this be a compiler directive?
Probably "committee pragmatism", a stdlib change might be easier to get approved than a language change, and compilers already have builtins for this, they're just not compatible (but the differences can be wrapped in a macro, and since C++ doesn't like to expose macros, it's probably still a macro, but hidden inside a stdlib template).
FWIW it looks like C23 will also just get a macro:
https://thephd.dev/ever-closer-c23-improvements#unreachable-...
Probably "committee pragmatism", a stdlib change might be easier to get approved than a language change, and compilers already have builtins for this, they're just not compatible (but the differences can be wrapped in a macro, and since C++ doesn't like to expose macros, it's probably still a macro, but hidden inside a stdlib template).
FWIW it looks like C23 will also just get a macro:
https://thephd.dev/ever-closer-c23-improvements#unreachable-...
I checked LLVM's libc++ source, and it's an inline void function defined in a header, which contains only a call to __builtin_unreachable() when available or std::abort() otherwise.
Also from a standardization point of view there isn't really much of a difference between a builtin and a plain library function. Library functions can be scoped in namespaces and that's a plus.
std::unreachable() feels like it's what std::assert(false) should be, except that std::assert(false) is bizarrely defined (as a no-op) when NDEBUG is defined. This is one of those strange cases where the spec defines behavior that I'd strongly expect to be undefined.
What's bizarre about asserts being disabled if you explicitly ask for it?
I'm not asking for the assert to be disabled. I'm asking for debugging features to be disabled. (NDEBUG == "no debug.") The debugging feature of an assert statement is that it performs work to check if the assertion is true and provides diagnostics if not. Disabling the debugging feature of an assert would, in my mind, make it purely an assertion -- an assumption that the compiler can count on (since this is true by design and demonstrated during debugging). Which is to say, it becomes undefined (contrary to compiler assumptions) if the condition is false.
Another way of thinking of this: With NDEBUG undefined, the sequence `assert(condition); if(!condition) action();` is perfectly reasonable and equivalent to the naked assert. But, because assert(false) is defined, defining NDEBUG changes the semantics of the code, with the two examples having identical behavior when NDEBUG is unset, and different behavior when NDEBUG is set. This is contrary to my understanding of the intent of NDEBUG, to suppress debugging information without changing semantics.
A third view: With NDEBUG undefined (which is normal for test and development), /I have no way of testing what happens past an assert(false)/. If I then define NDEBUG for a release build, I'm by definition exposing my user to untested code. The compiler might (with a defined assert(false)) define the behavior of this new code, /but I cannot, since I can't test it./ It just makes more sense to acknowledge that anything past an event horizon is undefinable, and call it undefined.
Another way of thinking of this: With NDEBUG undefined, the sequence `assert(condition); if(!condition) action();` is perfectly reasonable and equivalent to the naked assert. But, because assert(false) is defined, defining NDEBUG changes the semantics of the code, with the two examples having identical behavior when NDEBUG is unset, and different behavior when NDEBUG is set. This is contrary to my understanding of the intent of NDEBUG, to suppress debugging information without changing semantics.
A third view: With NDEBUG undefined (which is normal for test and development), /I have no way of testing what happens past an assert(false)/. If I then define NDEBUG for a release build, I'm by definition exposing my user to untested code. The compiler might (with a defined assert(false)) define the behavior of this new code, /but I cannot, since I can't test it./ It just makes more sense to acknowledge that anything past an event horizon is undefinable, and call it undefined.
The sole intent of NDEBUG in the Standard is to suppress assert(). There's a reasonable argument to be made here that it's poorly named (although I suspect that ANSI simply standardized existing practice here, as they did with much of the standard).
Surely you mean ISO and not ANSI? Were such ugly hacks already a part of C89?
In fact, it turns out that it was already in K&R C ten years before C89:
https://github.com/dspinellis/unix-history-repo/blob/Researc...
https://github.com/dspinellis/unix-history-repo/blob/Researc...
> Were such ugly hacks already a part of C89?
Yes.
Yes.
The D Programming language actually uses assert(0) in that way. Except its not undefined behaviour if the code is reached in release builds. The compiler will put in a special assembly instruction which chrashes the programm.
1.) Functions are fine for this stuff. The compiler can be trusted for its ability to "inline" the language-level `__builtin_unreachable()` or equivalent at the relevant optimization levels.
2.) static_cast<int> is shorter, if you know that the underlying type is int. But even if you know, you might not want to spell out int, to be more robust to code changes, which might involve a change of the underlying type of the corresponding enum.
In generic context you might not even know the underlying type.
3.) I absolutely agree. I think it was a mistake to include. The included `std::byteswap` can be expressed in terms of `std::ranges::reverse(std::as_writable_bytes(obj))`. It could be quality of implementation detail to get a bswap instruction from the latter.
I would be happy with equivalents of the `htonl` functions in the standard library, but I have strong opinions of the appropriate function signatures of it for C++.
2.) static_cast<int> is shorter, if you know that the underlying type is int. But even if you know, you might not want to spell out int, to be more robust to code changes, which might involve a change of the underlying type of the corresponding enum.
In generic context you might not even know the underlying type.
3.) I absolutely agree. I think it was a mistake to include. The included `std::byteswap` can be expressed in terms of `std::ranges::reverse(std::as_writable_bytes(obj))`. It could be quality of implementation detail to get a bswap instruction from the latter.
I would be happy with equivalents of the `htonl` functions in the standard library, but I have strong opinions of the appropriate function signatures of it for C++.
Re 3), all the <bits> instructions can be expressed in high level code, and compilers have been able to convert high level code to the actual machine instructions for a while.
The issue is exactly which code pattern is detected by which compiler varies a lot, so to avoid having to rely on that there was a strong push to add all these explicit intrinsics. Also std::bytesewap is more readable than the longer reverse+as_writeable_bytes.
The issue is exactly which code pattern is detected by which compiler varies a lot, so to avoid having to rely on that there was a strong push to add all these explicit intrinsics. Also std::bytesewap is more readable than the longer reverse+as_writeable_bytes.
There is one strong difference between byteswap and other operations within <bit>.
byteswap semantically works on the object representation of the integer, while all other operations semantically work on the integer value, expressed in powers of two.
The C++ language has no strong requirements on the object representation of the underlying integer. The C++20 guarantee of two's complement also only just expressed in terms of integer value, not bit representation.
Therefor `byteswap` somewhat sticks out. It's possibly useful for `std::endian::little == std::endian::native` or `std::endian::big == std::endian::native`, and for `std::has_unique_object_representations_v<T> == true` (aka. no padding bits) for the corresponding integral type T. CHAR_BITS also change the semantics.
So, in this sense it is quite low level, and you have to check all of these to make use of it, or you have to implicitly rely on implementation defined values of these for the targets you care about.
I'm not saying that byteswap is not useful though. But it has the wrong interface. It reverses sequence of bytes, it does not work on integers.
edit:
Oh, in addition of having no padding bits, you also want no trap representations. I don't know if you can check for that.
edit2:
Apparently, you don't necessarily need to check for padding bits, byteswap does it for you, and fails to compile if there are padding bits:
https://eel.is/c++draft/bit#byteswap-2
I don't see trap representations being handled at all. This might be a defect.
byteswap semantically works on the object representation of the integer, while all other operations semantically work on the integer value, expressed in powers of two.
The C++ language has no strong requirements on the object representation of the underlying integer. The C++20 guarantee of two's complement also only just expressed in terms of integer value, not bit representation.
Therefor `byteswap` somewhat sticks out. It's possibly useful for `std::endian::little == std::endian::native` or `std::endian::big == std::endian::native`, and for `std::has_unique_object_representations_v<T> == true` (aka. no padding bits) for the corresponding integral type T. CHAR_BITS also change the semantics.
So, in this sense it is quite low level, and you have to check all of these to make use of it, or you have to implicitly rely on implementation defined values of these for the targets you care about.
I'm not saying that byteswap is not useful though. But it has the wrong interface. It reverses sequence of bytes, it does not work on integers.
edit:
Oh, in addition of having no padding bits, you also want no trap representations. I don't know if you can check for that.
edit2:
Apparently, you don't necessarily need to check for padding bits, byteswap does it for you, and fails to compile if there are padding bits:
https://eel.is/c++draft/bit#byteswap-2
I don't see trap representations being handled at all. This might be a defect.
Thanks, I didn't catch this difference between byteswap and the other <bits> functions.
> 2.) static_cast<int> is shorter, if you know that the underlying type is int.
> But even if you know, you might not want to spell out int, to be more robust to
> code changes, which might involve a change of the underlying type of the
> corresponding enum.
>
> In generic context you might not even know the underlying type.
Thanks for pointing this out! I'm aware that the underlying type of an enum isn't always `int`. This is what I meant by saying the `static_cast` feels more expressive to me, i.e. reading the code, at least I'll now on sight what type is being used here. I'd probably have to 'let go' more to effectively use CPP, but this category of mechanism to cope with a generic/type system so complicated that I can't figure out the type of anything anymore. In the same vein, 'making code more robust to changes' feels like kicking the can down the road (to me). At some point you'll need to deal with the actual type ...
> In generic context you might not even know the underlying type.
Thanks for pointing this out! I'm aware that the underlying type of an enum isn't always `int`. This is what I meant by saying the `static_cast` feels more expressive to me, i.e. reading the code, at least I'll now on sight what type is being used here. I'd probably have to 'let go' more to effectively use CPP, but this category of mechanism to cope with a generic/type system so complicated that I can't figure out the type of anything anymore. In the same vein, 'making code more robust to changes' feels like kicking the can down the road (to me). At some point you'll need to deal with the actual type ...
> But shouldn't this be a compiler directive?
Lisp perspective: anything that can be a function almost certainly should be a function (and not a special operator or macro).
std::unreachable() does not have any arguments; therefore it doesn't need any special argument evaluation semantics that would require a compiler built-in.
The way C and C++ work, compiler directives are keywords and not identifiers. Keywords are not namespaced. Introducing a keyword called "unreachable" is problematic; far more so than a new element in the std namespace.
My only problem with std::unreachable is that I would never use it over std::abort.
Nobody needs a function whose only job is to invoke undefined behavior (from which it is then assumed that it is not reached).
It's a good cold day, so I can almost hear the Rust people laughing in the distance.
Lisp perspective: anything that can be a function almost certainly should be a function (and not a special operator or macro).
std::unreachable() does not have any arguments; therefore it doesn't need any special argument evaluation semantics that would require a compiler built-in.
The way C and C++ work, compiler directives are keywords and not identifiers. Keywords are not namespaced. Introducing a keyword called "unreachable" is problematic; far more so than a new element in the std namespace.
My only problem with std::unreachable is that I would never use it over std::abort.
Nobody needs a function whose only job is to invoke undefined behavior (from which it is then assumed that it is not reached).
It's a good cold day, so I can almost hear the Rust people laughing in the distance.
To be clear, Rust has this exact same functionality, with an intrinsic as well as a regular function that wraps said intrinsic.
The only difference is that it’s marked unsafe.
https://doc.rust-lang.org/std/hint/fn.unreachable_unchecked....
There is a safe version that is just a wrapper around panic! with a standardized message.
The only difference is that it’s marked unsafe.
https://doc.rust-lang.org/std/hint/fn.unreachable_unchecked....
There is a safe version that is just a wrapper around panic! with a standardized message.
[deleted]
Is there syntax available in C++ to write some kind of instruction to the compiler which is not some kind of call? Even __builtin_trap is a call isn't it? What else could you attach a directive to?
c++11 style attribute. There are already similar attributes [[unlikely]] and [[likely]] which can be attached to any statement. Having [[unreachable]] wouldn't be that surprising. C++23 also introduced [[assume(expr)]] which is basically conditional unreachable. I don't see a reason why standard committee couldn't have defined that [[assume(false)]] can be used for that purpose instead of new builtin. Either of the two would have been more consistent in my opinion, only reason I see for choosing a builtin function is because GCC and Clang already had __builtin_unreachable() builtin functions, so making it a function might in theory reduce the work for them.
[[assume(expr)]] has a lot of controversy about what you can do with it. I haven't read the latest papers (I really should), but there is a good argument to be made that the expression means flag an error if this happens, but otherwise keep running. There are embedded systems where they have to keep running no matter what. They want to use [[assume]] as a hint to program provers about what shouldn't happen, but they still want the compiler to generate the error handling code for that condition. Unreachable by contrast means don't generate code and nobody will argue otherwise.
Again, I'm not up on the latest papers, but that was the situation last I checked.
Again, I'm not up on the latest papers, but that was the situation last I checked.
There are also conditional expressions to consider. With the function approach, you can effectively mark any subexpression as unreachable, so e.g. this is possible:
auto x = (y == 1) ? foo :
(y == 2) ? bar :
...
std::unreadchable();You could can 'inject' it via writing a function never_fails, that tests the condition, and branches to unreachable if it fails, then returns the same condition. That should let the compiler understand that the condition is always true.
Sure, you can adapt a different solution to this case - but why, when a magic function can be used everywhere a pragma or similar could, and also covers this case naturally? A pragma or an attribute would make more sense if you needed to do that in the middle of class declarations, say. But here, we're only concerned about executable code.
Nor is it unprecedented to have a standard library function triggering UB when called - it's just that this one has a precondition that's always false, so it's always UB.
Nor is it unprecedented to have a standard library function triggering UB when called - it's just that this one has a precondition that's always false, so it's always UB.
[deleted]
Apparently you can add attributes to case labels, so yeah that does work.
Directives are attached to functions, like abort() has the [[noreturn]] attribute, so compilers don't have to emit any code that would run after it returns, or save anything on the stack.
std::unreachable() goes a step further, and tells the compiler that it doesn't even have to emit the call instructions or any instructions leading up to it.
std::unreachable() goes a step further, and tells the compiler that it doesn't even have to emit the call instructions or any instructions leading up to it.
> std::unreachable() goes a step further, and tells the compiler that it doesn't
> even have to emit the call instructions or any instructions leading up to it.
But I don't see a reason why this is not be possible with an attribute? (I realize this is a purely aesthetic call from me .. a function that doesn't actually get called seems out of place). My guess would be that using a function allows niche compilers that don't support attributes to support `unreachable` and is easy to implement for compilers already using __builtin
But I don't see a reason why this is not be possible with an attribute? (I realize this is a purely aesthetic call from me .. a function that doesn't actually get called seems out of place). My guess would be that using a function allows niche compilers that don't support attributes to support `unreachable` and is easy to implement for compilers already using __builtin
[deleted]
Yeah obviously it's a compiler intrinsic.
That's not what I or the person I was replying to was saying, and wasn't the relevant point.
We were saying how else could you give the compiler an intrinsic, without a call of some kind? Like what other syntactic mechanism is there you can treat as an intrinsic in the compiler.
That's not what I or the person I was replying to was saying, and wasn't the relevant point.
We were saying how else could you give the compiler an intrinsic, without a call of some kind? Like what other syntactic mechanism is there you can treat as an intrinsic in the compiler.
Sorry, I misread it as wanting a different mechanism. Which I don't think is needed. Some compilers used #pragma for it, but that was ugly and ill-specified, and made it hard to work around missing features by defining your own version.
I definitely agree with 1. I find the inclusion of core language features in std:: to be rather disconcerting. I've always thought of std:: as a set of standard useful library functions, separate from actual language features.
That's what it is. It's a useful utility function over non standard compiler built-ins.
The alternative would be a new keyword, but then the go-to complaint would just be about c++ keyword bloat and source compat regressions
The alternative would be a new keyword, but then the go-to complaint would just be about c++ keyword bloat and source compat regressions
Hmm. How often do people actaully want to std::byteswap as opposed to "convert this value from native byte order to big-endian" or "convert this value from little-endian to native byte order"?
i.e., the functions documented in https://man7.org/linux/man-pages/man3/endian.3.html (why oh why are they not also documented in the GNU C Library Manual...)
i.e., the functions documented in https://man7.org/linux/man-pages/man3/endian.3.html (why oh why are they not also documented in the GNU C Library Manual...)
Yeah, for portable code you also need a function to tell you if you're on an architecture where you need to do a byteswap for the data you have. e.g. you know you have data in little-endian format - do you need to swap it to work with it natively? That depends.
Maybe having something like convert_be() and convert_le(), one of which is a no-op and the other does the byteswap (depending on your arch) would be better. It removes the duplication of e.g. htobe() and betoh() which are exactly the same function, while allowing the caller to not worry about which architecture their code is compiled on.
Maybe having something like convert_be() and convert_le(), one of which is a no-op and the other does the byteswap (depending on your arch) would be better. It removes the duplication of e.g. htobe() and betoh() which are exactly the same function, while allowing the caller to not worry about which architecture their code is compiled on.
I view it more as a platform-independent building block to be used by library authors. For instance, here's our suite of byte order macros that this will not simply replace:
https://bloomberg.github.io/bde-resources/doxygen/bde_api_pr...
https://bloomberg.github.io/bde-resources/doxygen/bde_api_pr...
Yes, there is also std::endian::{big, small, native} that can be used to check if you need to byteswap or not. So a generic hton* is just:
edit: in practice I think std::endian and std::byteswap is a compromise between those that wanted a simple {to,from}_network_endian and those that wanted strongly typed wrappers to prevent mixing object with distinct endianess (a-la boost::endian). As the commitee couldn't reach consensus, this is the compromise and you can build your own thing with these portable bits.
auto to_network_endian(auto std::integral value) {
if constexpr (std::endian::native != std::endian::big) {
return std::byteswap(value);
} else {
return value;
}
}
edit: in practice I think std::endian and std::byteswap is a compromise between those that wanted a simple {to,from}_network_endian and those that wanted strongly typed wrappers to prevent mixing object with distinct endianess (a-la boost::endian). As the commitee couldn't reach consensus, this is the compromise and you can build your own thing with these portable bits.
Huh. You know I had never considered why we have htobe32 and be32toh which do exactly the same thing. Maybe because of oddball architectures which are neither big nor little endian?
For instance... /usr/include/x86_64-linux-gnu/bits/endian.h has:
#define __LITTLE_ENDIAN 1234 #define __BIG_ENDIAN 4321 #define __PDP_ENDIAN 3412
Apparently this is called 'middle-endian'... yikes!
On the other hand I sort of like having one function that's clearly used for importing values to the host's byte order, and another for exporting values from the host's byte order. But maybe that's just because I'm used to having them...
For instance... /usr/include/x86_64-linux-gnu/bits/endian.h has:
#define __LITTLE_ENDIAN 1234 #define __BIG_ENDIAN 4321 #define __PDP_ENDIAN 3412
Apparently this is called 'middle-endian'... yikes!
On the other hand I sort of like having one function that's clearly used for importing values to the host's byte order, and another for exporting values from the host's byte order. But maybe that's just because I'm used to having them...
Those are in <bit> as of c++20
I've always thought that it should have been possible in C and C++ to declare endian-ness as the property of a member variable's type, and that's it: the compiler would then automatically choose where to swap the byte-order to/from he native byte order.
The benefits are obvious: Code is declarative, and there's no risk of a bug where you missed to call std::byteswap() or did it twice.
Also, the compiler could automatically extract and insert values from/to little-endian and big-endian bitfields (which can be a handful...), and it could optimise to reduce the number of byteswaps in the code.
The benefits are obvious: Code is declarative, and there's no risk of a bug where you missed to call std::byteswap() or did it twice.
Also, the compiler could automatically extract and insert values from/to little-endian and big-endian bitfields (which can be a handful...), and it could optimise to reduce the number of byteswaps in the code.
Sounds like a pretty easy class you could write in C++. Template it for anything that's integral if you want to get fancy so you can do BigEndian<int32_t> or BigEndian<int64_t> and provide operators that convert to/from the "native" type transparently.
Is that generically useful enough to be part of the C++ standard? Seems like the type of thing that more belongs in a helper utility (aka, surely something like Boost already has this)
Is that generically useful enough to be part of the C++ standard? Seems like the type of thing that more belongs in a helper utility (aka, surely something like Boost already has this)
Of course that has crossed my mind, but I am not sure that there aren't opportunities for optimisation left if the compiler would retain higher-level information.
I love how over the past decade my own C++ utility library has been continuously shrinking because with each update there are more and more utility functions (like the to_underlying this article mentions) and even complete libraries (like <filesystem>) which replace self-written or 3rd party code.
I always worry as much as anyone else on each new release for the additional complexity ("the committee is out of control!!!!1!!eleven!"), but on each compiler upgrade when I actually get to use the new versions of the standard I'm always pleasantly surprised about all the little low-key quality of life improvements.
While that's true, when you get to a point where you have to write a small library - so that you need to cater to all of the language and standard additions - that's when you start experiencing pain. How do I expose the right iterators and sentinels? What do I have to specialize? Do I need to define concepts? Do I need to use concepts from elsewhere? I am often at a loss...
While this is true to some extent, trying to be overly generic and solving for all cases is the root of the analysis-parslysis. I end up solving just the part that's needed for the problem in hand but accepting meaningful compromises and trade offs when faced with this issue.
Premature generalisation is the root of all evil.
According to Stepanov generalisation is something you are supposed to discover as you develop a program, not something you do from first principles.
According to Stepanov generalisation is something you are supposed to discover as you develop a program, not something you do from first principles.
That may be relevant when you're writing a program. When you're writing a _library_, the "discovery" has already happened. But you can't just use the minimum level of generalization useful for your own program(s), you need to consider the conceivable needs of fellow programmers.
I understand why it's there, but I do find it fun that when many people are trying to reduce undefined behaviour in their code, std::unreachable is literally defined as "this is undefined behaviour, use that to optimise".
I suspect 99.9% of uses of std::unreachable would be better replaced by abort. (There will be those times when the code is correct and the optimisation gains are worth it -- but they will be rare).
I suspect 99.9% of uses of std::unreachable would be better replaced by abort. (There will be those times when the code is correct and the optimisation gains are worth it -- but they will be rare).
+100
I felt physically ill when I read:
It’s intended to be used when you know you have an execution path in your code that cannot be reached but the compiler cannot figure that out.
It felt like saying to the compiler, "please, find a way to make my program break even more easily". Exactly not what I need.
I felt physically ill when I read:
It’s intended to be used when you know you have an execution path in your code that cannot be reached but the compiler cannot figure that out.
It felt like saying to the compiler, "please, find a way to make my program break even more easily". Exactly not what I need.
You might not need it, but gcc/clang have __builtin_unreachable and msvc has __assume and both are used extensively for optimizations. std::unreachable is just standardizing existing diverging practice.
If a code path that's supposed to be unreachable is reached then the program is already broken. Unless it has a bug, a compiler will not make a program more broken. At worst (or best, depending on how you look at it) it will only make any bugs it already has more obvious.
Yes, but what was unreachable may get reachable as I do changes to the program. I see the utility when doing highly optimized library code.
But for my purposes I much prefer to stick something which flags me (exception or logging or whatever) of "this should never happen" instead of crashing. (Undefined.)
But for my purposes I much prefer to stick something which flags me (exception or logging or whatever) of "this should never happen" instead of crashing. (Undefined.)
In a real codebase, this would be wrapped by a macro to abort in a debug build but gain the optimizations in a release build.
Can't help but feel offended that my codebase isn't real. Thanks for the laugh though :-D
I'd rather have solid logging and/or abort handling than some extra yak hairs shaved off the speed on my release builds. I'm weird that way.
I'd rather have solid logging and/or abort handling than some extra yak hairs shaved off the speed on my release builds. I'm weird that way.
> At worst (or best, depending on how you look at it) it will only make any bugs it already has more obvious.
An abort would make them obvious, surely unreachable makes them less obvious?
An abort would make them obvious, surely unreachable makes them less obvious?
The point is that you're telling the compiler "I don't care what happens if control reaches here. Assume it never will and use that information to better optimize the rest". It's not an alternative to abort() or throwing because the compiler still needs to generate code for them.
Yes, that’s what I mean, the compiler can completely remove a branch which is unreachable, along with any checks. If it was forced to keep the checks and report an error instead, any bug would be more obvious. If the compiler removes checks for bugs, then the application could silently continue, obscuring the bug.
So this is a “hide bugs but maybe improve performance” function.
So this is a “hide bugs but maybe improve performance” function.
No, it doesn't hide bugs. It has no defined effect on the obviousness of bugs. The reason is that modern compilers are pretty much theorem solvers, and they're able to propagate truth values in order to deduce facts about programs. For example, a compiler could deduce that since a branch is never reached, a particular pointer is never null, and could therefore skip a null check that it deduced was redundant that would have prevented a null dereference. If it turns out that the branch is reachable and the pointer is null, the pointer would be dereferenced and the program would crash immediately (typically). But UB is UB; once you hit it all behaviors are permissible, from immediate crash, to silent data corruption, to nasal demons.
It's undefined behavior. The compiler is free to do whatever, including making your program more broken.
I find it unhelpful to frame undefined behavior as an “escape hatch” that lets the compiler do anything it wants. Compiler writers aren’t gleefully hunting for issues and using its presence as an excuse to make your life miserable. Instead, the weirdness arises because the compiler makes certain assumptions and transforms the code in ways that depend on them. For example:
1. Null pointers are never dereferenced.
2. Ptr *p is dereferenced.
3. p therefore cannot be null.
4. Ergo, we can omit checking whether p is null.
If the initial premise isn’t actually true (i.e., you slip up and dereference a null pointer), the chain of logic breaks down and boom! Applying many such rules can certainly lead to weird emergent behavior—and maybe you should act as if anything can happen—but it’s not a total free-for-all.
1. Null pointers are never dereferenced.
2. Ptr *p is dereferenced.
3. p therefore cannot be null.
4. Ergo, we can omit checking whether p is null.
If the initial premise isn’t actually true (i.e., you slip up and dereference a null pointer), the chain of logic breaks down and boom! Applying many such rules can certainly lead to weird emergent behavior—and maybe you should act as if anything can happen—but it’s not a total free-for-all.
If you have undefined behavior, your program is already broken. No such thing as "more broken"; there's already no theoretical limit to what might happen if it gets triggered.
Undefined behavior is considered worse than crashing, which is typically the alternative when reaching "unreachable" codepaths.
Compare these two blocks similar to the article
N.b. fixed last code example- thanks afiori.
Compare these two blocks similar to the article
switch (ch) {
case 'a': do_a(); return;
case 'd': do_d(); return;
// ch is guaranteed to be 'a' or 'd' by previous code.
default: assert(0);
}
switch (ch) {
case 'a': do_a(); return;
case 'd': do_d(); return;
default: std::unreachable();
}
if (ch == 'a') { do_a(); }
else { do_d(); }
return;
N.b. fixed last code example- thanks afiori.
I take your point, but you can get the behavior of your first example, while still marking the default branch with std::unreachable(), by asserting the preconditions before the switch. This seems to me to be a pretty general equivalence.
So what does std::unreachable() do here? In this particular case, and with NDEBUG defined and any level of optimization selected, I suspect that, at a minimum, the switch would be replaced as you have shown in all versions - it would take a more complex example to show how std::unreachable() makes a difference. The point is, now we have a choice - and it is one that is being offered without creating any backwards-compatibility issues.
Furthermore, the original function, without assertions, is not guaranteed to crash, with or without std::unreachable(). You need some explicit checks to get a desirable response in the case where a mistake has been made, and that option is just as available whether or not you use std::unreachable().
Therefore, while I agree you have shown that not all broken variants of a given program are equivalent, this does not show that std::unreachable() is harmful.
So what does std::unreachable() do here? In this particular case, and with NDEBUG defined and any level of optimization selected, I suspect that, at a minimum, the switch would be replaced as you have shown in all versions - it would take a more complex example to show how std::unreachable() makes a difference. The point is, now we have a choice - and it is one that is being offered without creating any backwards-compatibility issues.
Furthermore, the original function, without assertions, is not guaranteed to crash, with or without std::unreachable(). You need some explicit checks to get a desirable response in the case where a mistake has been made, and that option is just as available whether or not you use std::unreachable().
Therefore, while I agree you have shown that not all broken variants of a given program are equivalent, this does not show that std::unreachable() is harmful.
I suspect you meant to omit the `if (c == 'd')` part in the last example
I get your point, but assert is usually removed in release builds. Abort would probably be better if you want to crash the program.
One thing people are missing is that you may need to use this to satisfy conditions that the compiler might encounter to emit a warning (or error if equivalent of -Werror is enabled). If you have strict warnings on, but no way to tell the compiler that a location should not be reachable, you wind up in situations where you are doing something like `assert(!"not reached!")` and it won't be enabled in all build types. This is similar to how `__attribute__((noreturn))` is used in a "fatal" function that isn't always enabled but needs to convey its intention to static analyzers so that they stop evaluating branches past that call.
For that abort() is just fine. Being UB, unreachable is more about optimizations .
The literal example in the Clang documentation is not about optimization:
> For example, without the __builtin_unreachable in the example below, the compiler assumes that the inline asm can fall through and prints a “function declared ‘noreturn’ should not return” warning.
> For example, without the __builtin_unreachable in the example below, the compiler assumes that the inline asm can fall through and prints a “function declared ‘noreturn’ should not return” warning.
void myabort(void) __attribute__((noreturn));
void myabort(void) {
asm("int3");
__builtin_unreachable();
}> the compiler assumes that the inline asm can fall through and prints a "function declared 'noreturn' should not return" warning.
It actually can (Hint: what happens if the operating system `iret`s from its int 3 handler?), although it's probably not a issue in practice. Regardless, you don't need __builtin_unreachable to write:
However, if you're implementing built-in/standard functions like abort, you presumably know what compiler you're using and don't need a std interface in the first place. There's zero legitimate reason to use a undefined-behaviour-based `unreachable` in application code.
Claiming std::unreachable is useful for implementing abort is like proposing a std::manual_copy function because your compiler optimized a implementation of memcpy to a call to itself - at some point you do in fact have to resort to implementaion-specific details to define the abstractions that abstract away said details, and "in literally the same function as the (also-nonstandard, IIRC) inline assembly that hopefully doesn't return" seems at if not noticeably past that point.
It actually can (Hint: what happens if the operating system `iret`s from its int 3 handler?), although it's probably not a issue in practice. Regardless, you don't need __builtin_unreachable to write:
void myabort(void) __attribute__((noreturn));
void myabort(void) {
asm("int3");
myabort(); // might need `return myabort();` to force TCO,
// but gcc doesn't like that and it should work anyway
}
# Assuming tail-call optimization etcetera, this produces:
myabort:
int3
jmp myabort
However, if you're implementing built-in/standard functions like abort, you presumably know what compiler you're using and don't need a std interface in the first place. There's zero legitimate reason to use a undefined-behaviour-based `unreachable` in application code.
Claiming std::unreachable is useful for implementing abort is like proposing a std::manual_copy function because your compiler optimized a implementation of memcpy to a call to itself - at some point you do in fact have to resort to implementaion-specific details to define the abstractions that abstract away said details, and "in literally the same function as the (also-nonstandard, IIRC) inline assembly that hopefully doesn't return" seems at if not noticeably past that point.
Right elsethread I said that it is both about optimizations and static analysis. That's a static analysis case.
It seems like optimization cases where this makes sense are generally of the sort where `noreturn` can't be used because it is conditional. That makes sense to me (although, could this have covered most use cases by making `noreturn` support being passed the name of a function argument?).
One example was interesting, though. I could see someone believing these might produce the same optimized assembly (-O2):
I'd love to see these things illustrated with more real-world examples.
One example was interesting, though. I could see someone believing these might produce the same optimized assembly (-O2):
void a(int& x, int& y) {
if (&x == &y) __builtin_unreachable();
x ^= y; y ^= x; x ^= y;
}
void b(int& __restrict x, int& __restrict y) {
x ^= y; y ^= x; x ^= y;
}
a(int&, int&): # @a(int&, int&)
mov eax, dword ptr [rdi]
xor eax, dword ptr [rsi]
mov dword ptr [rdi], eax
xor eax, dword ptr [rsi]
mov dword ptr [rsi], eax
xor dword ptr [rdi], eax
ret
b(int&, int&): # @b(int&, int&)
mov eax, dword ptr [rsi]
mov ecx, dword ptr [rdi]
mov dword ptr [rsi], ecx
mov dword ptr [rdi], eax
ret
I'd love to see these things illustrated with more real-world examples.
Attempting to emulate restricted with __builtin_unreachable has been one of the first thing I tried when I learned about unreachable. I periodically try again, but generally I have been underwhelmed with trying to give gcc value range informations with it.
Technically these would still produce different results if y started two bytes after x -- I can't remember if that is itself legal however!
No that wouldn't be legal. Distinct objects can't partially overlap, unless one is a subject of the other.
>> and it won't be enabled in all build types.
The typical usecase would to wrap this in a #define, so that it aborts on a debug build but you get faster code in a production build.
The typical usecase would to wrap this in a #define, so that it aborts on a debug build but you get faster code in a production build.
[deleted]
It's been useful to me in one very specific place so far: to remove a range-check in front of the jump-table lookup in a big switch-case statement. For 'very hot' code paths like in the instruction decoder of an emulator or interpreting byte-code VM this might actually translate to a measurable difference.
In debug mode it makes a lot of sense to replace the __builtin_unreachable with an abort() though.
In debug mode it makes a lot of sense to replace the __builtin_unreachable with an abort() though.
https://gcc.gnu.org/onlinedocs/gcc/extensions-to-the-c-langu...
The rational is better explained there :)
Most programmers will write an error message and halt execution flow (return EXIT_FAILURE, abort(), exit() or even an exception). The hint about assembler give cares about special (maybe optimized or machine specific) code or embedded stuff. And the second example refers explicitly to functions which do exit() and never return actually. The committee tidies up stuff. Something which compilers provide individually is becoming standard.
The rational is better explained there :)
Most programmers will write an error message and halt execution flow (return EXIT_FAILURE, abort(), exit() or even an exception). The hint about assembler give cares about special (maybe optimized or machine specific) code or embedded stuff. And the second example refers explicitly to functions which do exit() and never return actually. The committee tidies up stuff. Something which compilers provide individually is becoming standard.
> this is undefined behaviour, use that to optimise
Optimization is a very large part of the reason for having undefined behavior at all. Viewed from that perspective, I don’t see how the existence of std::unreachable is at all odd. Also, it’s not like anyone is required to use it.
Optimization is a very large part of the reason for having undefined behavior at all. Viewed from that perspective, I don’t see how the existence of std::unreachable is at all odd. Also, it’s not like anyone is required to use it.
You can compromise by using abort in debug builds and std::unreachable in release builds.
I love it when switching debug mode on changes the control flow graph of my program. Keeps life interesting.
well, that's already the case.
The point of this is that you can control that behavior at the compiler level using different flags instead of changing the code itself (via preprocessor defines).
Then just use abort!
>Network protocols specify big endian for the order of transmission
Only in the parts specified by the protocol (headers etc). I encourage everyone sending data over network in a novel way to just use little-endian.
Only in the parts specified by the protocol (headers etc). I encourage everyone sending data over network in a novel way to just use little-endian.
That's not a good advice. Only if the sender and receiver are guaranteed to be running on little endian architecture you can make such a claim. A better advice is to always consider the endian-ness when designing protocols and have a strategy to handle it.
How is that "better advice"? Big endian architectures are pretty much dead (x86, ARM and RiscV are all little endian; some ARM chips are bi-endian, but not Apple's) and there's no discernible compelling advantage that would allow a comeback. You absolutely want to specify the byte order in new protocols as little endian.
> and there's no discernible compelling advantage
Actually, for wire encodings, there is, although I've (I-think-)literally never seen any proponent of big endian bring it up (versus the bullshit "it's human-readable" nonsense[0][1]): big endian encodings of unsigned numbers have lexicographic order that matches their numeric order.
The most obvious concrete example of why this is useful is a keys-sorted encoding of a hash table: if you encode keys in size-type-value format, you can check sortedness by lexicographic order of type-value strings (which means you can add new types without old software needing to know how to compare them), and you'll get integer keys in inspection-friendly numeric order rather than semi-random order. (Encoding negative numbers with a type id of T_UINT-1 lets you extend this to them as well.)
At a more abstract level, where (zero-padded) little-endian numbers have the same value at different granularities, this means that big-endian numbers have invariant lexicographic order at different granularities: two strings viewed as bits, bytes, or uint32s are consistently in the same order.
You can kind of use reverse-lexicographic order for some of this, but there are obvious problems with sending data in value-type order rather than type-value, so forward-lexicographic tends to be strongly enforced.
0: "You mean for arabic numerals, except not actual arabic numerals, because Arabic is written right-to-left, so the numbers are little-endian there, but ended up big endian because they stayed least-signifiant-digit-right rather than least-signifiant-digit-first when imported into Latin."
1: "Also, so (supposedly) is decimal and sign-magnitude, but we've (agonizingly slowly) learned that those aren't good ideas."
Actually, for wire encodings, there is, although I've (I-think-)literally never seen any proponent of big endian bring it up (versus the bullshit "it's human-readable" nonsense[0][1]): big endian encodings of unsigned numbers have lexicographic order that matches their numeric order.
The most obvious concrete example of why this is useful is a keys-sorted encoding of a hash table: if you encode keys in size-type-value format, you can check sortedness by lexicographic order of type-value strings (which means you can add new types without old software needing to know how to compare them), and you'll get integer keys in inspection-friendly numeric order rather than semi-random order. (Encoding negative numbers with a type id of T_UINT-1 lets you extend this to them as well.)
At a more abstract level, where (zero-padded) little-endian numbers have the same value at different granularities, this means that big-endian numbers have invariant lexicographic order at different granularities: two strings viewed as bits, bytes, or uint32s are consistently in the same order.
You can kind of use reverse-lexicographic order for some of this, but there are obvious problems with sending data in value-type order rather than type-value, so forward-lexicographic tends to be strongly enforced.
0: "You mean for arabic numerals, except not actual arabic numerals, because Arabic is written right-to-left, so the numbers are little-endian there, but ended up big endian because they stayed least-signifiant-digit-right rather than least-signifiant-digit-first when imported into Latin."
1: "Also, so (supposedly) is decimal and sign-magnitude, but we've (agonizingly slowly) learned that those aren't good ideas."
There are still big-endian-only ARM chips out there, and some of them are basically the only processors in their class (TMS570 in particular); while I wish they were bi-endian, big endian platforms are still alive and well.
There's still way, way more little-endian platforms, and it's not likely to change in the future. So it makes sense to use the representation that's most efficient for them when designing protocols.
The efficiency difference is negligible; the cost of an in-register byte swap is basically zero compared to the cost of getting the word from memory into the register to begin with. The bigger deal is making sure that people remember that we /are/ still in a bi-endian world, and writing protocols and code with that in mind.
The performance cost in isolation is not high, but the benefits of being able or reinterpret cast the buffer you got form the network and access it directly without any intermediate preprocessing is high. Most programming languages do not allow accessing fields in an endian agnostic way, you need accessors and a lot of custom code, so the code complexity benefits of ignoring the issue are great.
There are practically no big endian architectures anymore. Little endian is a sensible default. The weird architectures should bare the burden of complexity.
There are still big-endian-only ARM chips out there, and some of them are basically the only processors in their class (TMS570 in particular); while I wish they were bi-endian, big endian platforms are still alive and well.
That’s just a waste of cycles on encoding and decoding, as most/all senders and receivers are LE nowadays.
ARM big endian, so you're wrong, and it takes literally nanoseconds to byte swap.
Can you provide any numbers showing that more than 0.1% of networked ARM devices are big-endian? Pretty much all modern consumer facing ARM devices (including anything made by Apple as well as Android phones, as well as Nintendo) are little endian either exclusively or per standard configuration.
ARM supports both big & little endian, but basically everyone only uses little endian on ARM.
Not true. Most are setup to run little endian but support both.
>Byte swapping is important when transferring data between system that use different order for the sequence of bytes stores in memory.
That seems like a glaring footgun to me, to the point where I think I must be missing something.
What I want when dealing with endianess are "from_little_endian/to_little_endian", "from_big_endian/to_big_endian" function pairs that expand to either nop or a byte swap depending on the host architecture.
Exposing the byte swapping directly without this layer on top is asking for trouble because every user will have to make sure that they correctly detect the local endianess before attempting a swap. That's the potentially tricky part, not swapping the bytes.
That seems like a glaring footgun to me, to the point where I think I must be missing something.
What I want when dealing with endianess are "from_little_endian/to_little_endian", "from_big_endian/to_big_endian" function pairs that expand to either nop or a byte swap depending on the host architecture.
Exposing the byte swapping directly without this layer on top is asking for trouble because every user will have to make sure that they correctly detect the local endianess before attempting a swap. That's the potentially tricky part, not swapping the bytes.
The std::endian enum makes it very easy to find the native endianness.
That's irrelevant, because there's nothing useful you can do with the native endianness (other than implement (load/store)_(big/little)_N for various combinations of options).
you can use it to implement native_to_be or native_to_le, it seems pretty useful to me. In fact, what can't it do?
Kind of off topic, but is there a good "catch up" guide for people who stopped paying attention after C++11? Like a quick summary of just the useful, practical things you'd actually want to use in production, rather than the parts that are only interesting to computer science academics? Most of the "what's new in C++X" guides seem to just dump everything on you at once. I feel like I should get back into C++ but I don't think I really need to care about folding expressions or std::bit_cast.
I found Stroustrop's book A Tour of C++ very useful for catching up with C++14/17. It's relatively short and it's easy to skim through parts you already know well.
It was just updated for C++20 (with some coverage of C++23).
There's always a bunch of CppCon talks for this too. Just pop on over to their YouTube channel.
It was just updated for C++20 (with some coverage of C++23).
There's always a bunch of CppCon talks for this too. Just pop on over to their YouTube channel.
[deleted]
Why is it called "byteswap" and not "bytereverse"? "Swapping" can be done in any number of ways, but only one way to reverse.
I don't know, but 'swap' seems to have become the standard term for the operation, used in places like the C bswap family of functions and the x86 'bswap' instruction. My (totally unsupported and unresearched) guess is that it became popular as a term when 16-bit architectures were common -- "swap the bytes in a 16 bit value" is unambiguous.
The Arm architecture does call this operation "reverse bytes", though, so it's not universal to call it "swap".
The Arm architecture does call this operation "reverse bytes", though, so it's not universal to call it "swap".
That makes sense.
There's an x64 instruction called BSWAP.
Reversing bytes sounds like reversing their bit orders to me.
That would be "bitreverse" I guess, plus their description of byteswap[0] is exactly that:
"Reverses the bytes in the given integer value n."
I would expect the description for "byteswap" to be something like "swaps the given bytes of an integer value n." and be called like byteswap(x, a, b), where a and b are the indicies of the bytes to be swapped.
[0]: https://en.cppreference.com/w/cpp/numeric/byteswap
"Reverses the bytes in the given integer value n."
I would expect the description for "byteswap" to be something like "swaps the given bytes of an integer value n." and be called like byteswap(x, a, b), where a and b are the indicies of the bytes to be swapped.
[0]: https://en.cppreference.com/w/cpp/numeric/byteswap
Could be called "byte-wise reverse".
[deleted]
Because the reversal is done by swapping the first byte with the last, etc.
But on the same note, sorting is also done by swapping elements.
Not if you look at the result.
What do you mean? Sorting is done by swapping elements ( swap(a[i], a[j]) )
Well, first, whether swap() is used or not depends on the algorithm; second, the result of the sorting, as opposed to reversal, does not necessary look like the elements were swapped.
Oh, based on your other comment, I finally understood what you meant initially.
You were saying that the final array doesn't have many indices i,j such that a[i] = sorted_a[j] and a[j] = sorted_a[i].
My initial comment was not referring to the final order of the array, but the operations made to reach that order (one or multiple swaps). Another example could have been saying that we could have named the "sort" method "compare" because it uses comparisons in its algorithm (which was a parallel to your initial comment that it's called "swap" because the reverse operation uses swaps to achieve this).
You were saying that the final array doesn't have many indices i,j such that a[i] = sorted_a[j] and a[j] = sorted_a[i].
My initial comment was not referring to the final order of the array, but the operations made to reach that order (one or multiple swaps). Another example could have been saying that we could have named the "sort" method "compare" because it uses comparisons in its algorithm (which was a parallel to your initial comment that it's called "swap" because the reverse operation uses swaps to achieve this).
> whether swap() is used or not depends on the algorithm
I didn't mean the std::swap() function itself, but the swapping of two elements a[i] and a[j]. Is there any sorting algorithm that doesn't rely on swapping two elements (or two parts of the array)? I guess, only if the sort is not being done in-place and the result is stored in a different variable/memory.
> does not necessary look like the elements were swapped
The only way elements don't look like they are swapped is elements are changed, added or removed, which doesn't happen during sort.
I didn't mean the std::swap() function itself, but the swapping of two elements a[i] and a[j]. Is there any sorting algorithm that doesn't rely on swapping two elements (or two parts of the array)? I guess, only if the sort is not being done in-place and the result is stored in a different variable/memory.
> does not necessary look like the elements were swapped
The only way elements don't look like they are swapped is elements are changed, added or removed, which doesn't happen during sort.
There are in-place algorithms that use shifting rather than swapping (the so called in-place merge sort, for instance).
If you look at the result and at the original, generally you will not find many pairs in which the elements exchanged their positions. It takes special initial arrangements for this to be true for all elements.
If you look at the result and at the original, generally you will not find many pairs in which the elements exchanged their positions. It takes special initial arrangements for this to be true for all elements.
No thanks, I'm going to keep calling abort (ANSI C, 1989) to indicate "control stops here":
Code after an abort() call is unreachable. (Or after any function attributed noreturn).
GCC and Clang know this, and do things accordingly.
For instance, I've seen GCC emit code which assumes that ptr is not null after ASSERT(ptr != NULL), because the custom ASSERT macro called an __attribute__((noreturn)) function in the null case.
abort() has defined behavior; it terminates the program abnormally, as if by raising the SIGABRT signal.
Your own function attributed __noreturn__ can have whatever behavior you want it to have. The one in the ASSERT macro I alluded to above calculates and prints a backtrace and other useful information.
Code after an abort() call is unreachable. (Or after any function attributed noreturn).
GCC and Clang know this, and do things accordingly.
For instance, I've seen GCC emit code which assumes that ptr is not null after ASSERT(ptr != NULL), because the custom ASSERT macro called an __attribute__((noreturn)) function in the null case.
abort() has defined behavior; it terminates the program abnormally, as if by raising the SIGABRT signal.
Your own function attributed __noreturn__ can have whatever behavior you want it to have. The one in the ASSERT macro I alluded to above calculates and prints a backtrace and other useful information.
Silly question from someone who hasn’t written C++ in 20 years and only very vaguely remembers it - if a code path is unreachable, why have the path at all?
Is a default on a switch required? Is lack thereof a compiler warning or something? It’s been a very long time; I only ever seem to recall that kind of “all paths must be handled” from functional languages.
Is a default on a switch required? Is lack thereof a compiler warning or something? It’s been a very long time; I only ever seem to recall that kind of “all paths must be handled” from functional languages.
There are a number of reasons.
Having an unreachable block communicates to other humans that you didn't forget the else case, it shouldn't exist. Code is about communicating to the next maintainer of the code.
Unreachable communicates to static analyzers, which can throw an error if it detects a code path that would reach this code, even though otherwise that code path is fine. Also static analysis will stop analysis at this point, and since static analysis often is running into the halting problem having a forced halt means some other heuristic elsewhere will get more time to run and so it can find bugs in a different code path that it wouldn't have analyzed before.
> Is a default on a switch required?
Many style guide do not allow a default case on a switch. If you don't have a default case and you add a new item static analysis will flag an error (compiler warning), thus ensuring you look at that section of code that you may not have known about. So unreachable is a way to mark a lot of not possible cases as ones you have thought about, without either skipping them or adding a default.
> I only ever seem to recall that kind of “all paths must be handled” from functional languages.
C++ doesn't require all paths be handled, but realistically as a programmer you want to handle all paths. Marking a path as not reachable is a useful way to handle impossible code paths.
Having an unreachable block communicates to other humans that you didn't forget the else case, it shouldn't exist. Code is about communicating to the next maintainer of the code.
Unreachable communicates to static analyzers, which can throw an error if it detects a code path that would reach this code, even though otherwise that code path is fine. Also static analysis will stop analysis at this point, and since static analysis often is running into the halting problem having a forced halt means some other heuristic elsewhere will get more time to run and so it can find bugs in a different code path that it wouldn't have analyzed before.
> Is a default on a switch required?
Many style guide do not allow a default case on a switch. If you don't have a default case and you add a new item static analysis will flag an error (compiler warning), thus ensuring you look at that section of code that you may not have known about. So unreachable is a way to mark a lot of not possible cases as ones you have thought about, without either skipping them or adding a default.
> I only ever seem to recall that kind of “all paths must be handled” from functional languages.
C++ doesn't require all paths be handled, but realistically as a programmer you want to handle all paths. Marking a path as not reachable is a useful way to handle impossible code paths.
It comes up from time to time, for example sometimes you'll have a big enum where most of the values can be handled in a straightforward way, and things are weird and complex for the other values. So you write something like
int process(my_enum x, state_t state) {
if (x == my_enum::COMPLICATED) return do_stuff(state);
switch (x) {
case my_enum::SIMPLE0: return 0;
case my_enum::SIMPLE1: return 1;
case my_enum::SIMPLE2: return 2;
}
}
It can also improve codegen and is a building block of building an assume like macro to convey preconditions to the compiler and optimize potentially on that. The other is at the end of a function that can be guaranteed to never reach that point so that the compiler can know. Its a low level tool and shouldn’t be used without caution
It can help both optimizations and static analysis. For example in the switch case in principle the compiler can avoid doing bound checks on the switch jump table. For static analysis it can help flags paths that can actually happen as erroneous.
Not C++ specific, but the example given - a default case that is not expected to ever be hit - is a "cover all bases" thing, to avoid undefined behaviour. Without the default case, what should happen? In the example given, it would just do nothing, but it would do so silently, which could lead to hours of developer time wasted trying to figure out why it doesn't do anything.
But I can imagine other use cases that are like "this is not supposed to happen" that can cause major issues like buffer overflows. Better to be more defensive and write code that basically says you are aware of code that shouldn't be reachable.
But I can imagine other use cases that are like "this is not supposed to happen" that can cause major issues like buffer overflows. Better to be more defensive and write code that basically says you are aware of code that shouldn't be reachable.
I get that covering all bases is good, but am I reading right that this just tells the compiler not to generate that path? Seems like the opposite.
Wouldn't it be safer just to trigger a panic or something?
Wouldn't it be safer just to trigger a panic or something?
> Wouldn't it be safer just to trigger a panic or something?
Tradeoffs. The panic can result in a lot of code generation which in turn makes the reachable paths slower. Sure we are talking nanoseconds, but this is C++, if performance isn't important you shouldn't be using C++.
Tradeoffs. The panic can result in a lot of code generation which in turn makes the reachable paths slower. Sure we are talking nanoseconds, but this is C++, if performance isn't important you shouldn't be using C++.
‘default’ is there by default anyway. This new feature allows to explicitly state that there isn’t one.
I think the examples are not the best...
Why is std::unreachable a function, and why does it require including a header?
It sounds like the functionality of unreachable is to inform the compiler of something, which is what a core language keyword, like "if" or "for", does. Of course then there could be name collisions with the new keyword, so being in std:: might be a solution for that. But that is inconsistent, sometimes they solve this with prepending/appending __ or _t instead, or using obscure enough names like "constexpr" or "nullptr" that probably don't clash
It sounds like the functionality of unreachable is to inform the compiler of something, which is what a core language keyword, like "if" or "for", does. Of course then there could be name collisions with the new keyword, so being in std:: might be a solution for that. But that is inconsistent, sometimes they solve this with prepending/appending __ or _t instead, or using obscure enough names like "constexpr" or "nullptr" that probably don't clash
Why shouldn't it be a function? The global namespace is a scarce resource.
constexpr can't be a function because it is not used a such. Nullptr could have been std::nullptr, but it is used often enough that it made sense to put it in the global namespace (but note that it had to be nullptr instead of null to avoid collisions, so only two chars saved). The type is still std::nullptr_t.
The C++ standard doesn't really use __ as a prefix, it is a namespace reserved for the implementor. The _t suffix in the global namespace is from POSIX originally then adopted by C; C++ still puts _t names under std.
std::unreachable is a very obscure functionality, polluting the global namespace for it wouldn't have been a good idea.
constexpr can't be a function because it is not used a such. Nullptr could have been std::nullptr, but it is used often enough that it made sense to put it in the global namespace (but note that it had to be nullptr instead of null to avoid collisions, so only two chars saved). The type is still std::nullptr_t.
The C++ standard doesn't really use __ as a prefix, it is a namespace reserved for the implementor. The _t suffix in the global namespace is from POSIX originally then adopted by C; C++ still puts _t names under std.
std::unreachable is a very obscure functionality, polluting the global namespace for it wouldn't have been a good idea.
They missed a trick with `std::to_underlying`. It should work on all types, not just enums.
By not doing that you still have to do `std::is_enum_v<T>` somewhere.
For types that aren't enums it should do nothing (maybe `std::identity`)
By not doing that you still have to do `std::is_enum_v<T>` somewhere.
For types that aren't enums it should do nothing (maybe `std::identity`)
Seems easy enough to add your own overloaded function that does that. But then again, std::to_underlying() doesn't do much to begin with; it just casts to std::underlying_type<T>::type.
In D an unreachable branch can be indicated with:
assert(0);
The problem with the C++23 std::unreachable is that it invokes undefined behaviour. Calling abort (or panic, or whatever D's assert boils down to when the condition fails), would be a prefectly reasonable way to define unreachable. (That is, for example, basically how I define it in my own code:)
#define unreachable die("unreachable code reached")As the article states, they're doing different things. It's not a runtime check. The point of making it undefined behaviour is that it enables the programmer to provide information to the compiler to enable better optimisation. The compiler is permitted to assume the programmer is correct about the path being unreachable, and may infer accordingly.
Why on earth are these in C++23 and not C++11? There are a ton of things that should’ve been standardized over a decade ago but only show up in C++20 or later. `std::span<T>` is a huge one. The mind boggles.
Trying to get everything is why C++11 wasn't c++07. The committee spend several years trying to polish things instead of doing a new release. (the committee thought they weren't allowed to release anything before 06/07, otherwise some of what was in C++11 could have been in C++01 - or maybe c++03 but with more in it). Eventually you need to say stop right here, what is done is what we will release, what isn't done will have to wait.
Backwards compatibility and a slow moving industry; adding more to a standardization process will only postpone the release.
More and more programming languages are moving to a more lightweight or scheduled release schema though, e.g. Java that had been stuck in limbo for nearly a decade due to design-by-committee and backwards compatibility concerns by major players.
More and more programming languages are moving to a more lightweight or scheduled release schema though, e.g. Java that had been stuck in limbo for nearly a decade due to design-by-committee and backwards compatibility concerns by major players.
To get something into C++ someone has to actually write a paper and propose it to the standards committee who has to vote and agree on it.
You can't just have some dude write a span.hpp and go "yep, that's going into my compiler v2" - not how standards work.
You can't just have some dude write a span.hpp and go "yep, that's going into my compiler v2" - not how standards work.
I would’ve much rather have gotten span into C++11 than variadic templates, for instance. Definitely seems to me like a paper about span would’ve been a lot shorter and easier to write than one about variadic templates.
That’s a pretty surprising opinion. Variadic templates are extremely widely used and there’s no simple replacement (e.g. how would you implement something like emplace_back without them?) Span is trivial to replace with a third party library.
Very much. In many cases a single variadic replaces tens of thousands of lines of code, often machine generated, greatly simplifying code bases and speeding up compilation. It also immediately obsoleted the whole of boost.mpl. Together with lambda they are probably my favourite feature of c++11.
Span is nice especially as a vocabulary type, but it wasn't hard to implement something functionally similar and in practice most of codebases already did.
Span is nice especially as a vocabulary type, but it wasn't hard to implement something functionally similar and in practice most of codebases already did.
Heck I'm still using gsl::span, though that's mainly for compatibility with a C++17 compiler. It's great that std::span was standardized, maybe it was overdue, but it's just a drop-in replacement for stuff that already existed and worked fine.
It took 10 years to get c++11 out as it was...
[deleted]
I think -- I hope? -- we're seeing the death spiral of this language.
C++'s Biggest Problem: unbelievable, extraordinary, just mind-blowing levels of complexity.
Solution: add more stuff!
C++'s Biggest Problem: unbelievable, extraordinary, just mind-blowing levels of complexity.
Solution: add more stuff!
I don’t necessarily disagree with the underlying point, but this it’s a weird change to comment on.
The complexity explosion in C++ is due to extra language features etc. not 3 extra, probably well documented, quality of life functions in the standard library.
std::unreachable is the most atypical “function” in this post, but from a code readability perspective, it is pretty clear what it does and it doesn’t really add any new concepts to the language (UB has always been a footgun).
Also, if you want to preserve backward compatibility, all you CAN do is add more abstractions and hope they are simpler to understand and cover the majority of the use cases that the old ones did.
The complexity explosion in C++ is due to extra language features etc. not 3 extra, probably well documented, quality of life functions in the standard library.
std::unreachable is the most atypical “function” in this post, but from a code readability perspective, it is pretty clear what it does and it doesn’t really add any new concepts to the language (UB has always been a footgun).
Also, if you want to preserve backward compatibility, all you CAN do is add more abstractions and hope they are simpler to understand and cover the majority of the use cases that the old ones did.
While C++ is indeed a complex beast, mastering the whole of Java, C#, F#, Scala, Python, OCaml, Haskell in their latest versions, standard libraries and changes between major releases isn't any easier.
> I think -- I hope? -- we're seeing the death spiral of this language.
In a lot of metrics C++ has never been more popular. And given the direction of heterogeneous programming, C++ is here to stay.
> Solution: add more stuff!
In a lot of ways, yes, that is the solution.
In a lot of metrics C++ has never been more popular. And given the direction of heterogeneous programming, C++ is here to stay.
> Solution: add more stuff!
In a lot of ways, yes, that is the solution.
> A typical use case for this function are switch statements on a variable that can take only a limited set of values from its domain. For instance, an integer that can only be between 0 – 9. Here is a simple example with a switch that checks a char value and executes operations. Only a limited number of commands are supported but the argument is checked before invoking the function so it shouldn’t be possible to receive other values than already handled in the switch.
This is a bad practice. Parse, don't validate - https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...
This is a bad practice. Parse, don't validate - https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...
Okay, what about when you are writing the parser?
1.) Coming from embedded programming, I can see the utility of `std::unreachable`. But shouldn't this be a compiler directive? Or a standardized #pragma? Can someone more knowledgeable in C++ say whether using functions as markers is a common mechanism in std:: ?
2.)Maybe the example is bad here, as it doesn't even save typing. (18 chars for `std::to_underlying` vs. 16 for `static_cast<int>`. The 'old' variant seems more expressive to boot. If anything, how about `static_cast<auto>`? or <underlying> ...
3.) The usefulness of `std::byteswap` to convert data to network byte order seems trivial vs. the venerable old `htonl` family of functions. `std::byteswap` seems more like intrinsics meant to expose possibly present target machine instructions to the user. Like `std::unreachable` this is probably of most use to embedded / low-level programming. This may be a deficiency in the article...
If it's not obvious, I'm not a big C++ fan and read the article with C-tinted glasses :)