The orientation of that thread guide should not be an issue. Anyway, it is held in place by a tiny set screw coming from the top. See picture (yellow circle).
Top thread look correctly threaded to me. So what is going on with the bottom thread. Or what is the overall problem? is it not sewing or not picking up the bottom thread?