Using Gentoo, (no GUI just bash command line environment) and ALSA back-end, I got a latency of 64 samples, which seemed indistinguishable from dedicated sampler hardware when triggering audio files in Csound via a USB MIDI keyboard. This was using a 1.2 ghz Pentium M Toshiba laptop with basic on-board Intel soundcard. An i5 540m @ 2.67 ghz goes down to 16 samples (at least with no complaints from ALSA or Csound). An external USB soundcard on the i5 seems to go no lower than 32 samples.
The main thing is making sure ALSA connects directly to your soundcard without going through dmix. This of course limits you to using one audio app at a time. If you need to have multiple applications making/receiving at low latencies , you will need to deal with the tedious per-session configuration of JACK.