What do you need holes for?
Regardless, here's how I would drill into conduit. I think a vice would be a big help. I'd hit it the pipe with a center punch so the drill bit won't slip around. Maybe a little bit of filing before the punch can ensure the punch doesn't slip. I'd drill a small hole as a pilot so a larger bit has a guide. After drilling my final hole, I'd de-burr it with a bit that is much larger than the hole size. I may even use a small round file to deburr the inside edge as well.