Are you an archer?
I know champion archers and they pretty much ALL wear armguards. And, no, the buttons would actually be very hard to hook. Believe me, I've tried. The guards must be bigger than you think and they wrap around the arm. The buttons are beyond the sides of the arm. These were made from a tried and true design.
I've been shooting quite a while and, even though I rarely hit my arm, I still wear a guard.