In reply to Lee.

It's definitely all about the user. Even if you gain plenty of traffic what does it matter if that traffic doesn't find something they want on your site.

It's not so much that Google punishes poorly constructed sites than they might rank well constructed sites better or at least give them some kind of boost for being well constructed.