staticfloat

8 か月前議論

I love that this article includes a test program at the bottom to allow you to verify its claims.

9 か月前議論

This is a very old answer about the M1, but yes what you’re saying is possible: https://stackoverflow.com/a/67590869/230778

staticfloat

11 か月前議論

While you can cause the compiler to run longer to squeeze the binary size down, the compiler has a baseline number of compiler passes that it runs over the IR of the program being compiled. These compiler passes generally take time proportional to the input IR length, so a larger program takes longer to compile. Most compiler passes aren't throwing away huge amounts of instructions (dead code elimination being a notable exception, but the analysis to figure out which pieces of dead code can be eliminated still is operating on the input IR). So it's not a perfect proxy, but in general, if the output of your compiler is 2MB of code, it probably took longer to process all the input and spit out that 2MB than if the output of your compiler was 200KB.

staticfloat

2 年前議論

Just a small piece of feedback, it looks like there's a small typo on the last paragraph of the page:

> Interviews can be scheduled (and rescheduled) interviews around the clock, 7 days a week.

I believe the second `interviews` should be omitted.

staticfloat

2 年前議論

Other posters have pointed out that this is incorrect, but I wanted to give a bit of intuition as to how signals can be received when they are below the noise floor.

First, as a definition, below the noise floor means that the power of my signal at any given time is smaller than the power of the ambient noise in my channel, and usually this implies that you're only interested in a particular segment of frequency spectrum (e.g. within the 10MHz band centered at 1.8GHz). If we were doing a simple frequency-shift keying or amplitude-modulated signal, once the noise power exceeds the signal power, there is basically no hope of recovering anything useful, as those are both demodulation schemes that rely upon obtaining instantaneous estimates of the frequency or amplitude of the signal of interest.

However, spread-spectrum methods make a time/frequency tradeoff, where the signal of interest is "spread" across multiple points in time and frequency. A very simple example of this is to say "if I want to transmit a 1, instead of transmitting one cycle of a sinusoid at 18.GHz, I will transmit 10 cycles". Then, at the decoder stage, you average across 10 cycles of your carrier in order to detect whether a signal was sent or not. By doing this averaging across time, you get a 10x gain versus the noise which is expected to cancel itself out as often as not.

True spread-spectrum techniques are more advanced than this, they actually use wave shapes that are more complicated than just a sinusoid to make it easier to detect when they start and stop (whereas with a sinusoid there's a fair amount of ambiguity if you shift one period to the left or right) but the fundamental idea of averaging across time is the same.

Through this mechanism we are able to rescue out signals from far below the noise floor, although it reduces your maximum transmission rate. When dealing with digital radio systems we can even rescue out signals from below our quantization floor, although not too much lower, as eventually you lose the ability to average out a signal that is fluctuating by significantly less than a single bit.

Whenever I talk about making tradeoffs in transmission speed to aid in reception, I am reminded of the ELF systems in submarines [0]. While they did not use spread-spectrum techniques, (they just jumped between two frequencies, 76Hz and 80Hz) they still correlated across time to boost up their effective SNR. [0] https://en.wikipedia.org/wiki/Communication_with_submarines#...

staticfloat

2 年前議論

I ran into an issue like this in my first ever job! I accidentally filled up a cluster with junk files and the sysadmin started sending me emails saying I needed to fix it ASAP but rm wouldn’t work. He taught me that file truncation usually works when deletion doesn’t, so you can usually do “cat /dev/null > foo” when “rm foo” doesn’t work.

staticfloat

3 年前議論

Yes, that’s what it means for me. I’ve never heard someone use the word arbitrary to mean anything other than “a random choice”, or even “a poorly thought-out choice”.

My professors in grad school explicitly discouraged use of that word anywhere in technical writing, as they felt it would immediately give the reader the impression that the actions taken in the research were not thought through. Example: “This new technique enables arbitrary manipulations of data” should instead be replaced by something like “this technique enables a wide range of manipulations of data”.

staticfloat

3 年前議論

I really like the integration of lower-level memory control in a superset of Python. Trying to maintain compatibility with such a large and varied ecosystem is a daunting task, and has the opportunity to be very valuable for many people who don't want (or are unable) to move away from the Python ecosystem. Kudos to you all for tackling such a difficult problem! Good luck, I look forward to seeing how you guys help to increase efficiency across the board!

staticfloat

3 年前議論

It is a little disappointing that they're setting the bar against vanilla Python in their comparisons. While I'm sure they have put massive engineering effort into their ML compiler, the demos they showed of matmul are not that impressive in an absolute sense; with the analogous Julia code, making use of [LoopVectorization.jl](https://github.com/JuliaSIMD/LoopVectorization.jl) to automatically choose good defaults for vectorization, etc...

    julia> using LoopVectorization, BenchmarkTools, Test
           function AmulB!(C,A,B)
               @turbo for n = indices((C,B),2), m = indices((C,A),1)
                   Cmn = zero(eltype(C))
                   for k = indices((A,B),(2,1))
                       Cmn += A[m,k]*B[k,n]
                   end
                   C[m,n]=Cmn
               end
           end
           M = K = N = 144; A = rand(Float32, M,K); B = rand(Float32, K,N); C0 = A*B; C1 = similar(C0);
           AmulB!(C1,A,B)
           @test C1 ≈ C0
           2e-9*M*K\*N/@belapsed(AmulB!($C1,$A,$B))
    96.12825754527164

I'm able to achieve 96GFLOPs on a single core (Apple M1) or 103 GFLOPs on a single core (AMD EPYC 7502). And that's not even as good as what you can achieve using e.g. TVM to do the scheduling exploration that Mojo purports to do.

Perhaps they have more extensive examples coming that showcase the capabilities further. I understand it's difficult to show all strengths of the entire system in a short demonstration video. :)

EDIT: As expected, there are significantly better benchmarks shown at https://www.modular.com/blog/the-worlds-fastest-unified-matr... so perhaps this whole discussion truly is just a matter of the demo not showcasing the true power of the system. Hopefully achieving those high performance numbers for sgemm is doable without too much ugly code.

staticfloat

4 年前議論

> The Julia correctness issues are things like basic math functions, called in simple normal ways, returning wrong numbers.

If you have an example of this, I'd be interested in tracking it down. I didn't see this in Yuri's blog post, nor am I aware of any egregious examples of this right now.

staticfloat

4 年前議論

Small correction; it’s DisplayLink, not DisplayPort. You have to use DisplayLink because they do some part in software as a part of their driver (which you must download and install) whereas the native Apple stuff has some internal hardware limitation.

I run three 1080p monitors off of my M1Pro, and I don’t notice any heavy CPU load from whatever software load the driver may be adding.

staticfloat

4 年前議論

We actually use this in our CI system to limit write access outside of the build environment’s build folder.

You can see some Julia code that generates the sandbox config rules here: https://github.com/JuliaCI/sandboxed-buildkite-agent/blob/ma...

staticfloat

4 年前議論

This linear scaling per-core doesn't match my experience with using the AMX co-processor. In my experience, on the M1 and the M1 Pro, there are a limited number of AMX co-processors that is independent of the number of cores within the M1 chip. I wrote an SO answer exploring some of the performance implications of this [0], and since I wrote that another more knowledgable poster has added more information [1]. One of the key takeaways is that appears to be one AMX coprocessor per "complex", leading us to hypothesize that the M1 Pro contains 2 AMX co-processors.

This is supported by taking the code found in the gist [2] linked from my SO answer and running it on my M1 Pro. Compiling it, we get `dgesv_accelerate` which uses Accelerate to solve a medium-size linear algebra problem, that typically takes ~8s to finish on my M1 Pro. While running, `htop` reports that the process is pegging two cores (analogous to the result in my original SO answer on the M1 pegging one core; this supports the idea that the M1 Pro contains two AMX co-processors). If we run two `dgesv_accelerate` processes in parallel, we see that they take ~15 seconds to finish. So there is some small speedup, but it's very small. And if we run four processes in parallel, we see that they take ~32 seconds to finish.

All in all, the kind of linear scaling shown in the article doesn't map well to the limited number of AMX co-processors available in Apple hardware, as we would expect the M1 Max to contain maybe 8 co-processors at most. This means we should see parallelism step up in 8 steps, rather than 20 steps as was shown in the graph.

Everything I just said is true assuming that a single processor, running well-optimized code can completely saturate an AMX co-processor. That is consistent with the tests that I've run, and I'm assuming that the CFD solver he's running is well-written and making good use of the hardware (it does seem to be doing so from the shape of his graphs!). If this were not the case, one could argue that increasing the number of threads could allow multiple threads to more effectively share the underlying AMX coprocessor and we could get the kind of scaling seen in the article. However, in my experiments, I have found that Accelerate very nicely saturates the AMX resources and there is none left over for future sharing (as shown in the dgesv examples).

Finally, as a last note on performance, we have found that using OpenBLAS to run numerical workloads directly on the Performance cores (and not using the AMX instructions at all) is competitive on larger linear algebra workloads. So it's not too crazy to assume that these results are independent of the AMX's abilities!

[0] https://stackoverflow.com/a/67590869/230778 [1] https://stackoverflow.com/a/69459361/230778 [2] https://gist.github.com/staticfloat/2ca67593a92f77b1568c03ea...

staticfloat

5 年前議論

My guess is that it’s due to git looking at its own base name to figure out which command it’s supposed to run as, kind of like busybox does.

staticfloat

5 年前議論

To add to Keno's sibling comment, Julia, as a JIT compiler, essentially creates large chunks of standalone, "static" code, and runs those as much as it can, breaking out into the "dynamic" runtime when it has reached the limits of type inference or for some other reason needs to return to the runtime to perform dynamic dispatch etc... in these instances, we break out of the standalone code and start using the runtime to do things like determine where to jump next (or whether to compile another chunk of static code and jump to that). Note that these chunks of static code can be both smaller or larger than a function, it all depends on what Julia can compile in one go without needing to break out into the dynamic environment.

staticfloat

5 年前議論

In the Julia world, we make redistributable binaries for all sorts of things; you can find lots of packages here [0], and for LLVM in particular (which Julia uses to do its codegen) you can find _just_ libLLVM.so (plus a few supporting files) here [1]. If you want a more fully-featured, batteries-included build of LLVM, check out this package [2].

When using these JLL packages from Julia, it will automatically download and load in dependencies, but if you're using it from some other system, you'll probably need to manually check out the `Project.toml` file and see what other JLL packages are listed as dependencies. As an example, `LLVM_full_jll` requires `Zlib_jll` [3], since we build with support for compressed ELF sections. As you may have guessed, you can get `Zlib_jll` from [4], and it thankfully does not have any transitive dependencies.

In the Julia world, we're typically concerned with dynamic linking, (we `dlopen()` and `dlsym()` our way into all our binary dependencies) so this may not meet all your needs, but I figured I'd give it a shout out as it is one of the easier ways to get some binaries; just `curl -L $url | tar -zxv` and you're done. Some larger packages like GTK need to have environment variables set to get them to work from strange locations like the user's home directory. We set those in Julia code when the package is loaded [5], so if you try to use a dependency like one of those, you're on your own to set whatever environment variables/configuration options are needed in order to make something work at an unusual location on disk. Luckily, LLVM (at least the way we use it, via `libLLVM.so`) doesn't require any such shenanigans.

[0] https://github.com/JuliaBinaryWrappers/ [1] https://github.com/JuliaBinaryWrappers/libLLVM_jll.jl/releas... [2] https://github.com/JuliaBinaryWrappers/LLVM_full_jll.jl/rele... [3] https://github.com/JuliaBinaryWrappers/LLVM_full_jll.jl/blob... [4] https://github.com/JuliaBinaryWrappers/Zlib_jll.jl/releases [5] https://github.com/JuliaGraphics/Gtk.jl/blob/0ff744723c32c3f...

staticfloat

5 年前議論

Aha! A chance to plug one of my favorite CGP Grey videos that explores this very question: https://www.youtube.com/watch?v=JEYh5WACqEk

staticfloat

5 年前議論

Imagine instead of getting a grid of pixels once every 30th of a second, you instead get one pixel’s value, alone with its location, along with the time stamp at which the pixel’s change was noticed. Event cameras can have very fine time stamp resolution (orders of magnitude better than 1/30th of a second) and so a bright moving pixel can be tracked very accurately.

staticfloat

5 年前議論

Note that the restriction on external monitors can be worked around by using a DisplayLink compatible dongle/dock, since it uses a custom driver (I assume it does something in software that would otherwise be limited in hardware).

I use the Dell D6000 and I run three (1080p) external monitors in addition to the built in monitor.

staticfloat

5 年前議論

In case there's anyone out there with an M1 MBP that desperately wants multiple monitors, if you use a DisplayLink compatible external dock, something about the alternate driver that such a dock uses allows for multiple monitors. I personally use the Dell D6000 to run three external displays in addition to the built-in display off of my M1 MBP and it works quite well. You can see a list of DisplayLink docking stations here [0].

[0] https://www.displaylink.com/products/universal-docking-stati...

コメント