I've been outside the car and felt it rattle, also you can rattle it with your hand.
it's also doing it on both sides, one side worse than the other.
I've had the entire door apart, down to the two skins, and it's got to be the handle. If the speakers were blown, would it sound like shit?
It sounds clear as day