Back in the day, servants put on mens battle gear, i.e. belt and sword, armor, etc. Buckle was placed on the right, so servants could use their dominant(right hand) to manipulate the buckle. Similar story for buttons.
When fashionable buttons first came about, only the rich had garments with them and men would dress themselves, so that's why the buttons are on the right. However, women had servants dress them, so buttons are on the left so its easier for the servants.