I've been satisfied with the results of using gum tragacanth and hand burnishing with a Sharpie marker. I apply the gum trag to the edge, and rub it with the side of the marker until smooth. i keep going all the way around the holster doing this. By the time I get back to where I started, the gum is dry enough that I can rub a little harder with the marker to bring the edge to a slick, shiny finish. It takes a little time, but I'm a hobbyist. I'd seek a quicker method were I trying to make a living at it.
I'm guessing your wood dowel isn't hard or slick enough. You also could be turning it too fast. I've heard of people using wheels made from micarta or some sort of hard plastic in drill motors. I've not tried it myself.