The Ball and lever of the joystick is supposed to rotate. Every joystick does this with the exception of flight joysticks (used or flying games)
And yes if you lift up on the stick the thing lifts up a tad bit.
There is a spring that centers the joystick, returns it back to neutral, there is this gap that the spring takes up, this gap allows the stick to move around.
So by lifting the handle you are closing the gap that their.
As for the Ball top, if you pop open your Atrox controller and look at the underside of the joystick, right at the bottom of the level, it almost looks like a screw head.
You would use a screw driver in that slot to loosen or tighten the ball top so you can replace your ball top of your joystick if you want to (like another color, design or style of joystick top).
Part why the whole level rotates is that in the arcade people can’t steal the ball top that easily, and that everything in that lever assembly is round.
That round needs gives you more feedom of movement than if everything was square, the trade off is the level and top will rotate.