Agree - partially. I often made the experience in my life, that if I undergo a period of bad mood (especially depression) that lasts for weeks or months, I don't realize how down I am. But when that period is finally over, I suddenly become aware of how deep down I actually was. The comparison to yesterday might be a clever trick to deal with that inability of perception.
The pitfall that might araise from that rating system now is that you might tend to rate positive changes higher than negative changes or vice versa. If that is the case, it would introduce a drift again.