Yeah if your provider doesn't offer the service, that can't work at all. If they did it wouldn't matter if you knew the number, because you can block the "last caller". The system knows the number even if it isn't displayed to you.
May still be worth calling them for options.
Good luck,
Kevin
ps. You could always forward your phone to police station or something...(just kidding)