Chances are you have more than 50 bookmarks. Mainstream users have a hard time finding what they saved on their bookmarks. Except for being synced on the cloud. We are still confined to the same bookmark experience we had 20 years ago.
Don't think the problem is with the amount of indexed pages. Just the way we need to access what we already know and use. Chrome provides us with the top 8 most frequent sites. This keeps us going again and again to the same sites. Harder to make a habit around a new site.