Race Conditions == Random Number Generation(github.com)
github.com
Race Conditions == Random Number Generation
https://github.com/dasmithii/RCRand
54 comments
A lot of the time, the randomness used must be secret (for nonces etc.) but your solution is an interesting way to implement a CRS (http://en.wikipedia.org/wiki/Common_reference_string_model) which is very useful in some cryptographic primitives. A CRS is a common string, typically random, that anyone can access, but cannot be controlled by any one party. Another typical thought-implementation of this is using stock market prices -- predict those accurately and you'd do better things with your time than break some crypto protocols.
Using OS thread functionality to do some useful work is always amusing.
This reminds me of sleep sort. For those unfamiliar, sleepsort is where you have an unsorted array, then have 1 thread per element sleep for the amount of time specified by that element, then append it to a list.
This reminds me of sleep sort. For those unfamiliar, sleepsort is where you have an unsorted array, then have 1 thread per element sleep for the amount of time specified by that element, then append it to a list.
Interesting.
Though sleep sort is obnoxious (& hilarious), I'm sure it could be of use in educational situations. It's both elegant and intuitive. Thought provoking, maybe?
Though sleep sort is obnoxious (& hilarious), I'm sure it could be of use in educational situations. It's both elegant and intuitive. Thought provoking, maybe?
In reality sleep sort is not actually doing the sorting, it's just asking the OS to do it using a traditional sorting algorithm, because the OS will have to sort all the sleep timers to determine the order in which to wake up the threads.
And OSs either use a simple list for the timer queue (which means n^2 overall complexity), or something based on a heap (like a binary tree), which ultimately becomes O(n lg n).
> because the OS will have to sort all the sleep timers
Not necessarily (though yes, in any efficient real life it would). It could wait for a tick of the clock and on each scan an unordered list of things to see if any are due to happen.
Of course if the list gets big enough that the list takes a long time to scan then it will break sleep sort because if enough time passes for two or more events to have happened between two checks, those events will happen in an undefined order.
Not necessarily (though yes, in any efficient real life it would). It could wait for a tick of the clock and on each scan an unordered list of things to see if any are due to happen.
Of course if the list gets big enough that the list takes a long time to scan then it will break sleep sort because if enough time passes for two or more events to have happened between two checks, those events will happen in an undefined order.
That's pretty much a selection sort.
On a compile-to-hardware architecture it would do the sorting. Say Fleet (assuming fleet supports 'sleep' or something like it).
I don't think that fixes it. The closest you could come is to have n physical timers to sort n elements, but then your "sorting algorithm" can't sort more than n elements. As soon as you had more elements than timers you would still need some actual sorting algorithm to determine priority.
It's kind of like claiming you have an O(n) sorting algorithm because you can run an O(n) selection algorithm to find the kth smallest element for each of 0..n-1 and then run n of them in parallel on n cores to get an O(n) sorting algorithm. If you actually have n cores then the wall clock time for executing that algorithm will be O(n), but the number of operations won't be and it all falls apart when you run out of hardware parallelism.
It reminds of me of bucket sort. There is an intellectually curious implementation to sort a list of unique fixed-width (e.g. 32-bit) integers that goes like this: You allocate an m bit (e.g. 2^32 bit / 512MByte) array and zero it (O(m)), then you inspect every element and set that bit of the array (O(n)), then you traverse the array and add an element to the end of the sorted list for every set bit (O(m)). So the overall algorithm is actually O(1) in the number of elements, because you're doing m+n+m work where m > n, which means you're doing at most m+m+m. The problem is that in practice for most real lists the constant factor is so much larger than n that using an O(n log n) sorting algorithm will be significantly faster. And it's clearly Do Not Use if you're sorting e.g. 128-bit ints. But for n sufficiently close to m it can actually be useful. It would probably be tough to beat for sorting a 3 billion element array of 32-bit ints. Or a 200 element array of 8-bit ints.
It's kind of like claiming you have an O(n) sorting algorithm because you can run an O(n) selection algorithm to find the kth smallest element for each of 0..n-1 and then run n of them in parallel on n cores to get an O(n) sorting algorithm. If you actually have n cores then the wall clock time for executing that algorithm will be O(n), but the number of operations won't be and it all falls apart when you run out of hardware parallelism.
It reminds of me of bucket sort. There is an intellectually curious implementation to sort a list of unique fixed-width (e.g. 32-bit) integers that goes like this: You allocate an m bit (e.g. 2^32 bit / 512MByte) array and zero it (O(m)), then you inspect every element and set that bit of the array (O(n)), then you traverse the array and add an element to the end of the sorted list for every set bit (O(m)). So the overall algorithm is actually O(1) in the number of elements, because you're doing m+n+m work where m > n, which means you're doing at most m+m+m. The problem is that in practice for most real lists the constant factor is so much larger than n that using an O(n log n) sorting algorithm will be significantly faster. And it's clearly Do Not Use if you're sorting e.g. 128-bit ints. But for n sufficiently close to m it can actually be useful. It would probably be tough to beat for sorting a 3 billion element array of 32-bit ints. Or a 200 element array of 8-bit ints.
[deleted]
I think that's only true in a single threaded system
No, the OS still needs to maintain a sorted list, and that list likely has either a single thread that maintains it or some locking mechanism. Whenever the OS is looking for something to run it will inspect the head of that list to see if the current time >= the time at which that thread should be woken up. And if it can't find anything there then it will check the 'runnable' queue (processes that are CPU bound) and if there is nothing there then it will default to some idle loop.
> I'm sure it could be of use in educational situations.
It is a good example of how algorithm design can be done naively and algorithm analysis can be done badly, so definitely has at least some educational value.
Excluding implementation specific edge cases that would break the underlying assumptions regarding thread wakeup times, it is a O(N) time complexity sort in all (best, worst, average) cases. A miracle! Get thee to the patent office! Eat my shorts Shell!
Of course the constant time multiplier involved is so massive that it would not do better than an n(log n) or even an n*n sort in any practical situation, and this is before considering space requirements and thread processing overhead.
It is a good example of how algorithm design can be done naively and algorithm analysis can be done badly, so definitely has at least some educational value.
Excluding implementation specific edge cases that would break the underlying assumptions regarding thread wakeup times, it is a O(N) time complexity sort in all (best, worst, average) cases. A miracle! Get thee to the patent office! Eat my shorts Shell!
Of course the constant time multiplier involved is so massive that it would not do better than an n(log n) or even an n*n sort in any practical situation, and this is before considering space requirements and thread processing overhead.
"it is a O(N) time complexity sort in all cases"
The OS needs to pick which thread to run from a pool of N threads - absolute best case an O(log N) operation, either when the thread is queued or when it's being selected. This happens N times, so the runtime seems like it would actually scale as O(N log N).
On the other hand, if you've got a scheduler that can pick the thread with the smallest sleep time from a list in O(1) time, you could just use whatever algorithm that's using to make an O(N) insertion sort...
The OS needs to pick which thread to run from a pool of N threads - absolute best case an O(log N) operation, either when the thread is queued or when it's being selected. This happens N times, so the runtime seems like it would actually scale as O(N log N).
On the other hand, if you've got a scheduler that can pick the thread with the smallest sleep time from a list in O(1) time, you could just use whatever algorithm that's using to make an O(N) insertion sort...
But if the time multiplier used is high enough the time spend choosing a thread at each period is small.
So while that overhead could increase processing time, wall-clock time need to be affected.
Again this is why I think it makes a good example for illustrating critical thinking in the area of algorithm design and analysis. It is an obviously silly example which could actually work, within certain practical boundaries, and the many obvious problems with it are hidden in more complex processes/structures.
So while that overhead could increase processing time, wall-clock time need to be affected.
Again this is why I think it makes a good example for illustrating critical thinking in the area of algorithm design and analysis. It is an obviously silly example which could actually work, within certain practical boundaries, and the many obvious problems with it are hidden in more complex processes/structures.
Unless your thread scheduler can fairly choose from linearly many threads in constant time with 0 cost context switching and your threads take 0 time to call sleep(), at large input sizes, you eventually have to make the base sleep time longer, so it's not really linear time complexity.
Mainly for the purposes of making my earlier statement correct, I consider that to be an implementation specific problem which we do not need to consider in theoretical discussions!
Hilarious, yes, elegant, no. There's no guarantee that the list will actually come out sorted (unless you dig through the implementation of sleep() down to the kernel and confirm that the order is always deterministic). It's funny precisely because it has nothing at all going for it, and yet almost kind of works.
This is a remarkably poor idea.
Thread scheduling may be difficult to predict precisely. This is a radically different property than being random.
A good source of randomness is becomes no more predictable given previous outputs. Being able to predict the next output precisely would obviously break a random number generator. But being able to weight the odds of the next output also break a random number generator; it's less obvious, but most certainly enough to beat the house in any gamble.
Which thread wins race conditions is certainly not random. CPU cache lines will create biases, the implementation of your processor's pipelining of memory fences is likely predictable, and on and on.
Use a CSPRNG.
A counterargument might begin with "But I don't need cryptographically valid randomness!", to which I would respond "then why are you thinking about this at all?" Cryptographically valid randomness is not hard from either a programmer braintime perspective (there are libraries, unless you take up deep issues with modern CSPRNGs) or a CPU-time perspective (CSPRNGs can spit bytes faster than your disk can accept them). If you don't need cryptographic randomness, then there are other even faster sources of psudeorandomness; use those, not this mechanism for creating CPU waste heat.
Thread scheduling may be difficult to predict precisely. This is a radically different property than being random.
A good source of randomness is becomes no more predictable given previous outputs. Being able to predict the next output precisely would obviously break a random number generator. But being able to weight the odds of the next output also break a random number generator; it's less obvious, but most certainly enough to beat the house in any gamble.
Which thread wins race conditions is certainly not random. CPU cache lines will create biases, the implementation of your processor's pipelining of memory fences is likely predictable, and on and on.
Use a CSPRNG.
A counterargument might begin with "But I don't need cryptographically valid randomness!", to which I would respond "then why are you thinking about this at all?" Cryptographically valid randomness is not hard from either a programmer braintime perspective (there are libraries, unless you take up deep issues with modern CSPRNGs) or a CPU-time perspective (CSPRNGs can spit bytes faster than your disk can accept them). If you don't need cryptographic randomness, then there are other even faster sources of psudeorandomness; use those, not this mechanism for creating CPU waste heat.
I don't think the author was presenting this as an alternative to a CSPRNG. (Or rather, I hope they weren't.) It seems like more of a fun hack than anything else. I wish they explicitly warned people not to actually use this, but it's still an interesting side project.
Right. As an amusement, this is great, and I can hardly improve upon the visual aid of monkeys frenzying over a lego structure.
And indeed even as an educational opportunity to explore how very much not-random CPU scheduling over unguarded memory is, this is also probably a gem.
But I should hate to see anyone mistake this concept for useful.
And indeed even as an educational opportunity to explore how very much not-random CPU scheduling over unguarded memory is, this is also probably a gem.
But I should hate to see anyone mistake this concept for useful.
I've added a note in the readme.
Sorry to have been misleading.
Sorry to have been misleading.
[deleted]
I once found a bad implementation of this idea in a common commercial crypto library. I calculated from observation of a bunch of runs that they were achieving about 45 bits of entropy in practice, and taking six seconds of wall time and a shitload of power in the process.
To the OP: for additional amusement and possibly some educational epiphanies, try compiling and running this on your smartphone (assuming it's a multi core ARM).
The cache coherency rules are different in ARM and Intel CPUs, you should see quite different result from unsynchronized access to same memory locations. On ARM, you need to be more careful with explicit memory barriers, especially when you attempt to write lock-free code, or you're in kernel space.
I expect that on ARM, you're going to see a lot less random because an entire cache line written to by one core will replace another cache line written to by another core, thereby showing only the randomness contribution of one core.
The cache coherency rules are different in ARM and Intel CPUs, you should see quite different result from unsynchronized access to same memory locations. On ARM, you need to be more careful with explicit memory barriers, especially when you attempt to write lock-free code, or you're in kernel space.
I expect that on ARM, you're going to see a lot less random because an entire cache line written to by one core will replace another cache line written to by another core, thereby showing only the randomness contribution of one core.
I made a silly one that uses the GPU to generate numbers.
http://www.anfractuosity.com/projects/rndgpu/
http://www.anfractuosity.com/projects/rndgpu/
Color me impressed. I do wonder if there's any way to reliably influence the pattern of the mutations. On the surface, with the differences in thread-scheduling patterns, I'd say it seems like a meaningful improvement on the randomness of the system PRNG.
If you have a sufficiently well-developed model of thread-scheduling patterns, then the randomness reduces in quality to that of the system's PRNG. But thread-scheduling is a serious pain, so this seems promising.
If you have a sufficiently well-developed model of thread-scheduling patterns, then the randomness reduces in quality to that of the system's PRNG. But thread-scheduling is a serious pain, so this seems promising.
I'm pretty sure you can reduce the entropy of the output by saturating the processor or memory. If the various accesses are being spread out more because other processes are executing then you're going to have substantially less entropy. Sounds like bait for a DoS attack... hit the server really hard and all of a sudden the RNG is producing bad random numbers.
I doubt it's more efficient than something like Mersenne Twister. Generating entropy this way is pretty damn expensive in terms of processor cycles.
I doubt it's more efficient than something like Mersenne Twister. Generating entropy this way is pretty damn expensive in terms of processor cycles.
run it through dieharder please, just for fun.
How does this compare to dakarand? http://dankaminsky.com/2012/08/15/dakarand/
I'd find it way more interesting if it started from a non-random buffer. Like this I can't tell how much of the randomness comes from the initial seed...
Those who are pointing out that the CPU & threads operate in a deterministic manner should consider that the universe may operate under the same principles. Maybe some day a computer can watch live video of someone rolling a die & predict what it will settle on before it is settled. There may be no concept of "random", only more or less predictable to humans or algorithms they create.
[author here]
It's difficult to have meaningful discussion (w/ regard to randomness) when involving more philosophical principles. For those who believe wholeheartedly in a deterministic universe, the concept is utter nonsense.
That's why it's important that pragmatic folk focus on practical and statistical randomness, rather than universal randomness.
By practical, I mean that the information required to predict generated numbers is simply not available. It may exist; it may not; but regardless, it's inaccessible to those in need of it.
In this case, I'm not certain if sufficient thread-scheduling information is available to derive accurate predictions, or if separate processes could be of effect without altering the speed at which RNG state mutation occurs simultaneously. Being interdependent on other thread & process CPU usage, my approach might be another case of "the act of observation changing the thing observed".
But then again, I'm know little of both cryptography and CPU scheduling. This comment, like this project, is largely speculative.
It's difficult to have meaningful discussion (w/ regard to randomness) when involving more philosophical principles. For those who believe wholeheartedly in a deterministic universe, the concept is utter nonsense.
That's why it's important that pragmatic folk focus on practical and statistical randomness, rather than universal randomness.
By practical, I mean that the information required to predict generated numbers is simply not available. It may exist; it may not; but regardless, it's inaccessible to those in need of it.
In this case, I'm not certain if sufficient thread-scheduling information is available to derive accurate predictions, or if separate processes could be of effect without altering the speed at which RNG state mutation occurs simultaneously. Being interdependent on other thread & process CPU usage, my approach might be another case of "the act of observation changing the thing observed".
But then again, I'm know little of both cryptography and CPU scheduling. This comment, like this project, is largely speculative.
The issue I've always had with this & dakarand are that they might not be easily predictable, but it is probably possible to influence their output into something you can more easily predict.
There's a book "The Newtonian Casino" which describes doing something similar with roulette wheels .. in the 1980s.
Doesn't "sensitive dependance on initial conditions" place severe limitations on what we can predict?
I think the Linux /dev/random already do this to some extent.
generator.c:
I'm not a cryptographer, but this smells extremely insecure to me. If the random seed is all zeroes this generator will generate all zeroes for perpetuity, no?
Even if you get a decent initial seed, I still strongly suspect this RNG strategy will be very statistically leaky, will tend to equilibria, and will generally be easily breakable in practice. An RGB visualization in the context of RNG means next to nothing.
Please children, don't do your own crypto.
// For each byte in the buffer, use its value to index into another
// byte and XOR the two.
I'm not a cryptographer, but this smells extremely insecure to me. If the random seed is all zeroes this generator will generate all zeroes for perpetuity, no?
Even if you get a decent initial seed, I still strongly suspect this RNG strategy will be very statistically leaky, will tend to equilibria, and will generally be easily breakable in practice. An RGB visualization in the context of RNG means next to nothing.
Please children, don't do your own crypto.
It also uses 100% CPU until the threads are stopped. The author did not make any claims that it was cryptographically secure. It's just a toy that he shared - an interesting way to perhaps generate something useful from something that is normally undesirable.
I suppose the 100% CPU is (very) unnecessary.
Maybe I'll include a variable sleeping time determined by current buffer values. That way, thread synchronization would be further disorganized, and CPU power wouldn't be wasted quite as much.
In that case, it's important that mutations take place between every byte outputted tough.
Maybe I'll include a variable sleeping time determined by current buffer values. That way, thread synchronization would be further disorganized, and CPU power wouldn't be wasted quite as much.
In that case, it's important that mutations take place between every byte outputted tough.
The visualisation means next to nothing for proving it's good. But the visualisation seems to show patterns, which says plenty.
If I look at it too long, I see "patterns" in the output of OS X's /dev/random too: http://codeanarchy.com/files/yarrow.png
Random doesn't always look random. To make matters worse, we're biased towards seeing patterns in noise: http://en.wikipedia.org/wiki/Clustering_illusion
Random doesn't always look random. To make matters worse, we're biased towards seeing patterns in noise: http://en.wikipedia.org/wiki/Clustering_illusion
...and my repost from https://news.ycombinator.com/item?id=4821942.
A bookmarklet (paste in address bar) to show truly random and "evened out" dot distributions side by side. (Don't laugh at my Javascript):
javascript:"<html><body><canvas id=\"tutorial\" width=\"200\" height=\"200\">foo</canvas><-><canvas id=\"tutorial2\" width=\"200\" height=\"200\">foo</canvas><script>var canvas = document.getElementById('tutorial');var ctx = canvas.getContext('2d');ctx.fillStyle = \"rgb(000,0,0)\";for (var i=0;i<400;i++) {ctx.fillRect (Math.random() * 200,Math.random() * 200, 2, 2); };</script><script>var canvas = document.getElementById('tutorial2');var ctx = canvas.getContext('2d');ctx.fillStyle = \"rgb(000,0,0)\";for (var i=0;i<20;i++) for(var j=0;j<20;j++) for(k=0;k<1;k++) {ctx.fillRect (i * 10 + Math.random() * 10, j * 10 + Math.random() * 10, 2, 2); };</script></body></html>"
A bookmarklet (paste in address bar) to show truly random and "evened out" dot distributions side by side. (Don't laugh at my Javascript):
javascript:"<html><body><canvas id=\"tutorial\" width=\"200\" height=\"200\">foo</canvas><-><canvas id=\"tutorial2\" width=\"200\" height=\"200\">foo</canvas><script>var canvas = document.getElementById('tutorial');var ctx = canvas.getContext('2d');ctx.fillStyle = \"rgb(000,0,0)\";for (var i=0;i<400;i++) {ctx.fillRect (Math.random() * 200,Math.random() * 200, 2, 2); };</script><script>var canvas = document.getElementById('tutorial2');var ctx = canvas.getContext('2d');ctx.fillStyle = \"rgb(000,0,0)\";for (var i=0;i<20;i++) for(var j=0;j<20;j++) for(k=0;k<1;k++) {ctx.fillRect (i * 10 + Math.random() * 10, j * 10 + Math.random() * 10, 2, 2); };</script></body></html>"
"Patterns"? Of course there are patterns. If there were no patterns, that would be suspicious. Do you claim to be able to look at a picture of a random number generator data and determine the quality based on that? You do realize that even perfectly random data may have patterns?
It serves mostly as a litmus test against blatant predictability. As said above, sequential lines or otherwise obvious shapes would indicate poor randomness.
I wouldn't evaluate a generator solely on the basis of a bytemap. But these visualizations are of value.
I wouldn't evaluate a generator solely on the basis of a bytemap. But these visualizations are of value.
It seems to be an improvement from the notoriously poor rand() of PHP.
(not confident in the credibility of this article) http://boallen.com/assets/images/randbitmap_computer.png
(not confident in the credibility of this article) http://boallen.com/assets/images/randbitmap_computer.png
It's not a PHP function, it's a C library function that PHP is using.
On Linux rand() doesn't suffer from the problem.
On Linux rand() doesn't suffer from the problem.
"Patterns" doesn't mean much. Repeated diagonal lines would be obviously broken.
[deleted]
I'm not sure undefined behaviors are so much unpredictable...
strace -Tiv -ttt nice -n 19 curl -Lv --raw https://google.com/news 2>&1 | shasum -a 512 | dd bs=1 count=2 2>/dev/null
This assumes Google News serves frequently changing content. The noncanonical Google News URL generates extra entropy from an HTTP 302 Moved Temporarily to an HTTP 301 Moved Permanently to an HTTP 200 OK with the final news page. strace and nice and curl add entropy from OS syscall timings and HTTP headers.
(OS X doesn't have strace, so you can just skip it.)