itamarst · HackerTrans

itamarst

5,611 カルマ登録 10 年前

I write about speeding up Python software development, and code, at https://pythonspeed.com

I write about fundamental engineering skills and programmer career advice at https://codewithoutrules.com

I also write a weekly email about all the mistakes I've made both coding and in my career over the past 20 years, so that you can learn and avoid them: https://softwareclown.com

投稿

[untitled]

1 ポイント投稿者 itamarst4 か月前0 コメント

[untitled]

1 ポイント投稿者 itamarst5 か月前0 コメント

[untitled]

1 ポイント投稿者 itamarst6 か月前0 コメント

[untitled]

1 ポイント投稿者 itamarst7 か月前0 コメント

Testing the compiler optimizations your code relies on

pythonspeed.com

1 ポイント投稿者 itamarst11 か月前0 コメント

500× faster: Four different ways to speed up your code

pythonspeed.com

3 ポイント投稿者 itamarst昨年0 コメント

Loading Pydantic models from JSON without running out of memory

pythonspeed.com

134 ポイント投稿者 itamarst昨年45 コメント

The surprising way to save memory with BytesIO

pythonspeed.com

4 ポイント投稿者 itamarst昨年0 コメント

Faster pip installs: caching, bytecode compilation, and uv

pythonspeed.com

2 ポイント投稿者 itamarst2 年前0 コメント

Catching memory leaks with your test suite

pythonspeed.com

3 ポイント投稿者 itamarst2 年前0 コメント

Reducing CO₂ emissions with faster software

pythonspeed.com

3 ポイント投稿者 itamarst2 年前0 コメント

Using portable SIMD in stable Rust

pythonspeed.com

1 ポイント投稿者 itamarst2 年前0 コメント

Beyond multi-core parallelism: faster Mandelbrot with SIMD

pythonspeed.com

4 ポイント投稿者 itamarst2 年前0 コメント

Let's build and optimize a Rust extension for Python

pythonspeed.com

4 ポイント投稿者 itamarst2 年前0 コメント

It's time to stop using Python 3.8

pythonspeed.com

5 ポイント投稿者 itamarst2 年前5 コメント

Running 15× faster with a situation-specific algorithm

pythonspeed.com

4 ポイント投稿者 itamarst2 年前0 コメント

Jevons Paradox doesn't always apply to software

pythonspeed.com

2 ポイント投稿者 itamarst2 年前0 コメント

Not just Nvidia: GPU programming that runs everywhere

pythonspeed.com

1 ポイント投稿者 itamarst2 年前0 コメント

Profiling your Numba code

pythonspeed.com

67 ポイント投稿者 itamarst2 年前23 コメント

Beware of misleading GPU vs. CPU benchmarks

pythonspeed.com

3 ポイント投稿者 itamarst3 年前1 コメント

コメント

7 日前議論

When this was posted to lobsters someone shared this relevant link: https://curiouscoding.nl/posts/static-search-tree/

7 日前議論

The bucket boundaries are often chosen from a random sample of the data, if the input data is very large. Sorting is O(nlogn), but using binary search per value to assign a bucket is O(n), plus the cost of creating the buckets on a sample. So once you hit a large enough number of values this scales better.

Binary search does give random access but in this case there's only up to 255 buckets typically, so it's random access on cached memory.

12 日前議論

Better guesses reduce the number of guesses, so there will be less branch misprediction, but there will still be mispredictions for each remaining branch. So I would guess branchless interpolation search would still help.

In practice because the real code in scikit-learn is used in parallel, memory bandwidth starts being a problem in real usage. Plus, in the overall algorithm (this is just a small part) the time spent on binary search is now low enough that there are other, more significant bottlenecks elsewhere. So in practice the branchless optimization had enough impact on the original motivating code base that there didn't seem much point spending more time on it.

昨年議論

That's great! Would also be cool (separately from Pydantic use case) to add jiter backend to ijson.

昨年議論

https://remarkjs.com/

昨年議論

I talk about this more explicitly in the PyCon talk (https://pythonspeed.com/pycon2025/slides/ - video soon) though that's not specifically about Pydantic, but basically:

1. Inefficient parser implementation. It's just... very easy to allocate way too much memory if you don't think about large-scale documents, and very difficult to measure. Common problem with many (but not all) JSON parsers.

2. CPython in-memory representation is large compared to compiled languages. So e.g. 4-digit integer is 5-6 bytes in JSON, 8 in Rust if you do i64, 25ish in CPython. An empty dictionary is 64 bytes.

昨年議論

Once you switch to ijson it will not save any memory, no, because ijson essentially uses zero memory for the parsing. You're just left with the in-memory representation.

昨年議論

msgspec is much more memory efficient out of the box, yes. Also quite fast.

昨年議論

You can't add extra attributes that weren't part of the original dataclass definition:

  >>> from dataclasses import dataclass
  >>> @dataclass
  ... class C: pass
  ... 
  >>> C().x = 1
  >>> @dataclass(slots=True)
  ... class D: pass
  ... 
  >>> D().x = 1
  Traceback (most recent call last):
    File "<python-input-4>", line 1, in <module>
      D().x = 1
      ^^^^^
  AttributeError: 'D' object has no attribute 'x' and no __dict__ for setting new attributes

Most of the time this is not a thing you actually need to do.

昨年議論

For my PyCon 2025 talk I did this. Video isn't up yet, but slides are here: https://pythonspeed.com/pycon2025/slides/

The linked-from-original-article ijson article was the inspiration for the talk: https://pythonspeed.com/articles/json-memory-streaming/

昨年議論

One key question in these sort of things is how free() works: it is given a pointer, and it has to decide whether this was sampled or not, with _minimum_ effort.

Poireau does this, IIRC, by putting the pointers it sampled in a different memory address.

Sciagraph (https://sciagraph.com), a profiler I created, uses allocation size. If an allocation is chosen for sampling the profiler makes sure its size is at least 16KiB. Then free() will assume that any allocation 16KiB or larger is sampled. This may not be true, it might be false positive, but it means you don't have to do anything beyond malloc_usable_size() if you have free() on lots and lots of small allocations. A previous iteration used alignment as a heuristic, so that's another option.

2 年前議論

It's somewhat domain specific. Pure Python libraries have easier time supporting new releases than libraries that rely on C APIs, and even slower are those that deal with less stable implementation details like bytecode (e.g. Numba). But definitely getting better and faster every release.

2 年前議論

It's much faster! There's been significant performance improvements since 3.8.

2 年前議論

There's overhead in transferring data from CPU to GPU and back. I'm not sure how this works with internal GPUs, though, insofar as RAM is shared.

In general, though, as I understand it (not a GPU programmer) you want to pass data to the GPU, have it do a lot of operations, and only then pass it back. Doing one tiny operation isn't worth it.

2 年前議論

You can use Numba to speed up some Pandas calculations: https://pandas.pydata.org/docs/user_guide/enhancingperf.html...

3 年前議論

NumPy default is that you iterate over the earlier dimensions first.

The slow code is likely at least partially slow due to branch misprediction (this is specific to my CPU, not true on CPUs with AVX-512), see https://pythonspeed.com/articles/speeding-up-numba/ where I use `perf stat` to get branch misprediction numbers on similar code.

With SIMD disabled there's also a clear difference in IPC, I believe.

The bigger picture though is that the goal of this article is not to demonstrate speeding up code, it's to ask about level of parallelism given unchanging code. Obviously all things being equal you'll do better if you can make your code faster, but code does get deployed, and when it's deployed you need to choose parallelism levels, regardless of how good the code is.

3 年前議論

Neat! Unfortunately at the moment it still ignores cgroups, it's just a wrapper around sched_getaffinity().

https://github.com/python/cpython/blob/6a69b80d1b1f3987fcec3...

3 年前議論

That's not it. I updated the article with an experiment of processing 5 items at a time. The fast function doing 5 images at a time is slower than the slow function doing 1 image at a time (24*5 > 90).

If your theory was correct, we would expect the optimal number of threads for the fast function processing 5 images at a time to be similar to that of the slow function processing 1 image at a time.

In fact, the optimal threads in this case (5 images at a time) was 20 for slow function, 10 for fast function, so essentially the same as the original setup.

3 年前議論

Visiting NYC a few weeks ago the bike-based restaurant delivery people were interesting to see, not a thing where I live.

Relevant to "active time", they seemed to spend a lot of time during the day just waiting around...

3 年前議論

There's presumably a reason they've spent the past 20 years adding additional instructions to CPUs, yeah :) And a large part of the Python ecosystem just ignores all of them. (NumPy has a bunch of SIMD with function-level dispatch, and they add more over time.)