Hey, so LeBron James isn't the primary reason our defense sucks, I guess.
I have another theory. All of our players suck at defense. The only guy on the roster who made a career of playing elite defense is too old to do it now (Marion). The only other guy who has played elite defense at points in his career doesn't feel like doing it all the time (James). And then the only guy who played above average defense for large portions of his career is hurt for the season, but even before that was too old and worn down to play at that level anymore (Andy).
After that, it's a team of minus defenders. There are guys who have the tools to be good at defense, like Waiters, but they aren't and they never have been. There are guys who play hard on defense, but are too athletically limited to play it well for any length of time, like Delly. And then there are just the flat turnstiles who are straight up embarrassments on that end, like Mike Miller.