Using vectorization to speed up UTF-8 character counting(daemonology.net)

17 pointsby cperciva18 lat temu3 comments

Using vectorization to speed up UTF-8 character counting

http://www.daemonology.net/blog/2008-06-05-faster-utf8-strlen.html

2 comments

I thought the title meant it was using Altivec or SSE, but it's merely operating on a chunk of 4 bytes at a time dealing with misaligned data up front. Still a good article, despite my initial disappointment.

A similar article which originally taught me these tenants is:

http://rentzsch.com/papers/straightenUpAndFlyRight

cperciva18 lat temu

it's merely operating on a chunk of 4 bytes at a time

If you have a modern CPU, that code operates on 8 bytes at a time. :-)

ajross18 lat temu

I don't think I believe those benchmarks. Many of those numbers are significantly slower (!) than main memory bandwidth, and for code like strlen() which can be naively implemented in about three instructions per byte.

Something is wrong with the testing, I think.