itamarst · HackerTrans

itamarst

5,611 karmajoined 10 tahun yang lalu

I write about speeding up Python software development, and code, at https://pythonspeed.com

I write about fundamental engineering skills and programmer career advice at https://codewithoutrules.com

I also write a weekly email about all the mistakes I've made both coding and in my career over the past 20 years, so that you can learn and avoid them: https://softwareclown.com

Submissions

[untitled]

1 pointsby itamarst4 bulan yang lalu0 comments

[untitled]

1 pointsby itamarst5 bulan yang lalu0 comments

[untitled]

1 pointsby itamarst6 bulan yang lalu0 comments

[untitled]

1 pointsby itamarst7 bulan yang lalu0 comments

Testing the compiler optimizations your code relies on

pythonspeed.com

1 pointsby itamarst11 bulan yang lalu0 comments

500× faster: Four different ways to speed up your code

pythonspeed.com

3 pointsby itamarsttahun lalu0 comments

Loading Pydantic models from JSON without running out of memory

pythonspeed.com

134 pointsby itamarsttahun lalu45 comments

The surprising way to save memory with BytesIO

pythonspeed.com

4 pointsby itamarsttahun lalu0 comments

Faster pip installs: caching, bytecode compilation, and uv

pythonspeed.com

2 pointsby itamarst2 tahun yang lalu0 comments

Catching memory leaks with your test suite

pythonspeed.com

3 pointsby itamarst2 tahun yang lalu0 comments

Reducing CO₂ emissions with faster software

pythonspeed.com

3 pointsby itamarst2 tahun yang lalu0 comments

Using portable SIMD in stable Rust

pythonspeed.com

1 pointsby itamarst2 tahun yang lalu0 comments

Beyond multi-core parallelism: faster Mandelbrot with SIMD

pythonspeed.com

4 pointsby itamarst2 tahun yang lalu0 comments

Let's build and optimize a Rust extension for Python

pythonspeed.com

4 pointsby itamarst2 tahun yang lalu0 comments

It's time to stop using Python 3.8

pythonspeed.com

5 pointsby itamarst2 tahun yang lalu5 comments

Running 15× faster with a situation-specific algorithm

pythonspeed.com

4 pointsby itamarst2 tahun yang lalu0 comments

Jevons Paradox doesn't always apply to software

pythonspeed.com

2 pointsby itamarst2 tahun yang lalu0 comments

Not just Nvidia: GPU programming that runs everywhere

pythonspeed.com

1 pointsby itamarst2 tahun yang lalu0 comments

Profiling your Numba code

pythonspeed.com

67 pointsby itamarst2 tahun yang lalu23 comments

Beware of misleading GPU vs. CPU benchmarks

pythonspeed.com

3 pointsby itamarst3 tahun yang lalu1 comments

comments

7 hari yang laludiscuss

When this was posted to lobsters someone shared this relevant link: https://curiouscoding.nl/posts/static-search-tree/

7 hari yang laludiscuss

The bucket boundaries are often chosen from a random sample of the data, if the input data is very large. Sorting is O(nlogn), but using binary search per value to assign a bucket is O(n), plus the cost of creating the buckets on a sample. So once you hit a large enough number of values this scales better.

Binary search does give random access but in this case there's only up to 255 buckets typically, so it's random access on cached memory.

12 hari yang laludiscuss

Better guesses reduce the number of guesses, so there will be less branch misprediction, but there will still be mispredictions for each remaining branch. So I would guess branchless interpolation search would still help.

In practice because the real code in scikit-learn is used in parallel, memory bandwidth starts being a problem in real usage. Plus, in the overall algorithm (this is just a small part) the time spent on binary search is now low enough that there are other, more significant bottlenecks elsewhere. So in practice the branchless optimization had enough impact on the original motivating code base that there didn't seem much point spending more time on it.

tahun laludiscuss

That's great! Would also be cool (separately from Pydantic use case) to add jiter backend to ijson.

tahun laludiscuss

https://remarkjs.com/

tahun laludiscuss

I talk about this more explicitly in the PyCon talk (https://pythonspeed.com/pycon2025/slides/ - video soon) though that's not specifically about Pydantic, but basically:

1. Inefficient parser implementation. It's just... very easy to allocate way too much memory if you don't think about large-scale documents, and very difficult to measure. Common problem with many (but not all) JSON parsers.

2. CPython in-memory representation is large compared to compiled languages. So e.g. 4-digit integer is 5-6 bytes in JSON, 8 in Rust if you do i64, 25ish in CPython. An empty dictionary is 64 bytes.

tahun laludiscuss

Once you switch to ijson it will not save any memory, no, because ijson essentially uses zero memory for the parsing. You're just left with the in-memory representation.

tahun laludiscuss

msgspec is much more memory efficient out of the box, yes. Also quite fast.

tahun laludiscuss

You can't add extra attributes that weren't part of the original dataclass definition:

  >>> from dataclasses import dataclass
  >>> @dataclass
  ... class C: pass
  ... 
  >>> C().x = 1
  >>> @dataclass(slots=True)
  ... class D: pass
  ... 
  >>> D().x = 1
  Traceback (most recent call last):
    File "<python-input-4>", line 1, in <module>
      D().x = 1
      ^^^^^
  AttributeError: 'D' object has no attribute 'x' and no __dict__ for setting new attributes

Most of the time this is not a thing you actually need to do.

tahun laludiscuss

For my PyCon 2025 talk I did this. Video isn't up yet, but slides are here: https://pythonspeed.com/pycon2025/slides/

The linked-from-original-article ijson article was the inspiration for the talk: https://pythonspeed.com/articles/json-memory-streaming/

tahun laludiscuss

One key question in these sort of things is how free() works: it is given a pointer, and it has to decide whether this was sampled or not, with _minimum_ effort.

Poireau does this, IIRC, by putting the pointers it sampled in a different memory address.

Sciagraph (https://sciagraph.com), a profiler I created, uses allocation size. If an allocation is chosen for sampling the profiler makes sure its size is at least 16KiB. Then free() will assume that any allocation 16KiB or larger is sampled. This may not be true, it might be false positive, but it means you don't have to do anything beyond malloc_usable_size() if you have free() on lots and lots of small allocations. A previous iteration used alignment as a heuristic, so that's another option.

2 tahun yang laludiscuss

It's somewhat domain specific. Pure Python libraries have easier time supporting new releases than libraries that rely on C APIs, and even slower are those that deal with less stable implementation details like bytecode (e.g. Numba). But definitely getting better and faster every release.

2 tahun yang laludiscuss

It's much faster! There's been significant performance improvements since 3.8.

2 tahun yang laludiscuss

There's overhead in transferring data from CPU to GPU and back. I'm not sure how this works with internal GPUs, though, insofar as RAM is shared.

In general, though, as I understand it (not a GPU programmer) you want to pass data to the GPU, have it do a lot of operations, and only then pass it back. Doing one tiny operation isn't worth it.

2 tahun yang laludiscuss

You can use Numba to speed up some Pandas calculations: https://pandas.pydata.org/docs/user_guide/enhancingperf.html...

3 tahun yang laludiscuss

NumPy default is that you iterate over the earlier dimensions first.

The slow code is likely at least partially slow due to branch misprediction (this is specific to my CPU, not true on CPUs with AVX-512), see https://pythonspeed.com/articles/speeding-up-numba/ where I use `perf stat` to get branch misprediction numbers on similar code.

With SIMD disabled there's also a clear difference in IPC, I believe.

The bigger picture though is that the goal of this article is not to demonstrate speeding up code, it's to ask about level of parallelism given unchanging code. Obviously all things being equal you'll do better if you can make your code faster, but code does get deployed, and when it's deployed you need to choose parallelism levels, regardless of how good the code is.

3 tahun yang laludiscuss

Neat! Unfortunately at the moment it still ignores cgroups, it's just a wrapper around sched_getaffinity().

https://github.com/python/cpython/blob/6a69b80d1b1f3987fcec3...

3 tahun yang laludiscuss

That's not it. I updated the article with an experiment of processing 5 items at a time. The fast function doing 5 images at a time is slower than the slow function doing 1 image at a time (24*5 > 90).

If your theory was correct, we would expect the optimal number of threads for the fast function processing 5 images at a time to be similar to that of the slow function processing 1 image at a time.

In fact, the optimal threads in this case (5 images at a time) was 20 for slow function, 10 for fast function, so essentially the same as the original setup.

3 tahun yang laludiscuss

Visiting NYC a few weeks ago the bike-based restaurant delivery people were interesting to see, not a thing where I live.

Relevant to "active time", they seemed to spend a lot of time during the day just waiting around...

3 tahun yang laludiscuss

There's presumably a reason they've spent the past 20 years adding additional instructions to CPUs, yeah :) And a large part of the Python ecosystem just ignores all of them. (NumPy has a bunch of SIMD with function-level dispatch, and they add more over time.)