PFF does their own player comparison, correct? They watch and grade each player on each play, and then the cumulative grade of the game is what they give for that player. Furthermore the outcome of the play may or may not dictate the grade they give that player.
For example if a RB rushes for 15 yards on a given play. The RB could end up with a positive, neutral or even minus grade depending on how much work he actually had to do on the play. Was there a hole he could have rushed for 50 yards and a touchdown, did the offensive line do all of the significant effort which allowed him to gain those 15 yards, was the hole none existent and the RB is the sole reason he ended up with 15 yards. Each of these would result in a different grade for the RB. While this sounds extremely simply, its not, its no where close to this simple. What if you watch the play and think the RB made the best decision, what if i think he made the worst decision. You give him a plus grade, i give him a minus grade for the exact same play. See where we start to run into issues?
To further complicate this is the fact that they cant have the same guy grading each and every game, which means you ahve multiple people grading multiple games. I always think a certain type of play is worthy of a plus grade, you always think a certain type of play is worth a negative grade. It just so happens team X runs a lot of the plays you like, you naturally give a higher grade to those players.
Even more complication is the fact that on many plays you dont get to see all 22 players. So on a running play a the line of scrimmage that goes for 5 yards to the left, and thats where the cameras zoom to. Tell me what grade you give the WR on the far right side of the field? Did he block his guy effectively?
What about on the passing play where the QB gets sacked because the RT completely whiffed on his block. What grade do you give the WR who is 20 yards down field that ran a great route, but you didnt get to see it
The point is that all of this is subjective scoring, it is by definition biased.
edit: i am NOT arguing against the usage of PFF, what I am cautioning everyone against is using them as any sort of definitive player grading. They make the waters EXTREMELY blurry when it comes to player comparison, because its not based upon statistical output of the player there is no way to double check the work they do.
