This may be part of the explanation:
The thing is: The Lakers should welcome this strategy from their opponents. James is perfectly fine (if not actually good) as an on-ball defender. Just let one of his stans tell you....
It’s while he’s off the ball on the weak side that his energy and focus tends to wane. Watch him when he isn’t guarding the ball and you’ll get to see the ways he can actually hurt the defense. All too often he’ll just wander over to help somewhere that doesn’t need it, and then hardly attempt to recover to the shooter he left open.
LeBron James says he doesn’t care if people want to criticize his defensive effort with the Lakers this year.
www.silverscreenandroll.com
So if that's the case, perhaps some of the stats don't properly assign fault to James for poor off-ball defense, and focus too much on his on-ball defense. If that's true, then he's really a "worse" defender than the stats may show.
Also, LBJ is still so immensely talented physically that he can be decent on defense even while putting force below-average effort. So perhaps some of those pundits are seeing that below average effort. And then perhaps the last component is that no matter how badly you get beaten, it still only counts as 2 points. So if James has a bunch of horrendous plays where he doesn't do anything and gets beaten badly, they don't count any more against his defensive rating than if he contested and just got beaten. So that also may be part of the "eye test" factor on defense.