When you say there is play in the distributor shaft, exactly how are you pushing? Are you pushing straight sideways against the rotor, or are you turning the rotor, pushing against the side of the distributor, or what?
I'm no distributor expert, but there is some play in there for the weights to adjust the timing. Did you have the arm that opens the points on the top of the cam when you set the gap? If not, then turning the rotor would probably change the gap.