Boosting Vector Search Speed by 40% with AVX512 on Emerald Rapids(weaviate.io)
weaviate.io
Boosting Vector Search Speed by 40% with AVX512 on Emerald Rapids
https://weaviate.io/blog/intel
2 comments
nice, inference is still primarily a compute workload.
Yes, personally, I'm actually very bullish on CPU inference for both models and the vector DB.
We explain how we attained a ~40% QPS speed up at 90% Recall in Weaviate running on Intel’s new Xeon Processor, Emerald Rapids.
We do a technical deep dive into different implementations for vector distance calculations and optimizations enabled by Intel’s new 5th Gen Xeon Processor - Emerald Rapids, parallelization techniques using SIMD with AVX-256 and AVX-512 instruction sets, loop unrolling and compiler optimizations by transpiling C to Go assembly.
We do a technical deep dive into different implementations for vector distance calculations and optimizations enabled by Intel’s new 5th Gen Xeon Processor - Emerald Rapids, parallelization techniques using SIMD with AVX-256 and AVX-512 instruction sets, loop unrolling and compiler optimizations by transpiling C to Go assembly.