Interesting...I used a Youtube clone software to build a niche video sharing service. Of course, I never hope to reach Youtube mass...and I don't intent to.
If you would have hit the frontpage regurarly then they will come regurarly...otherwise the number will fall. It's like a commercial: you need to push it more than once to have an effect.