For userspace probes DTrace uses INT3 AFAIK. Taking a signal each time a probe gets hit can be high overhead for certain usages though (like low overhead profiling as opposed to intrusive debugging).
This works since x86 guarantees atomicity for aligned reads and writes w.rt. instruction fetch. LOCK prefix can be used for unaligned reads/writes to ensure atomicity of data reads and writes. But Intel SDM says (in 8.1.2 Volume 3A) "Locked instructions should not be used to ensure that data written can be fetched as instructions", which suggests instruction fetch is not atomic even with LOCK'd instructions, specially when the accesses are unaligned. Intel SDM (8.1.3 Volume 3A) suggests a cross modification protocol which requires global synchronization to ensure correct operation in such scenarios. Anyhow recently we unwittingly hit this limitation in our work and this led to some exploratory work for finding a way to relax the global synchronization requirement. Our work can be found at http://conf.researchr.org/event/pldi-2016/pldi-2016-papers-l... if anyone is interested.