The upper control arm spherical joint locates the axle side to side but the trailing arms are really important in controlling diff rotation and yaw. When the trailing arm bushings are either soft or worn, the axle will yaw or rotate around the spherical joint. I never would have considered this if I didn't take that video of my diff while driving the car. When I drove into my back lane at an angle to the curb, the diff wobbled around like a drunken hula dancer. That made sense to me when I thought about how my car handled when pushed hard around a corner. The rear end seemed very loose and the car would "rear steer" where the rear end would step out but the tires weren't sliding, it was the diff rotating around the spherical joint. With your wheel spacers, there won't be much room for that kind of diff movement.
