Several factor cause offset when setting: (all of which I ran into and have to correct once they were pointed out to me)
holes too big in leather - 2 size punches needed for the two posts on the male or female snap - will cause issues even with press
post too long fro leather thickness - yes often you will need to snipe the post down a little - good side snipers then open the post up again with a nail setter or modelling tool etc. - will cause issues even with press
When using hand set you need to angle the setter a little off plum and rotate it around the post with continuous light tapes (thin of the setter following a conical path - check out the how to video on Tandy as this is well demonstrated.
Hope this helps