This doesn’t prove anything! Google has quality control people checking to see if algorithms are working well? So? Surely the point is to figure out how to get algorithms that will pick up on problems, so they can be improved, not to try to solve quality rating by having people do it. That would be insane and go against every way that Google works.

Heck, I do the same thing at LibraryThing. I’ve had LT employees rate six or seven different recommendation results. From that, I can see which of my algorithms performs the best. That doesn’t mean I’m asking employees to rate recommendations for 3 million titles!