The A2Z holders use a set screw to lock the bit in place. Does you bit have a flat on one side for this purpose? There are a couple of problems I could think about with the A2Z style holder.
1) The holder or bit shanks is not perfectly clean, a bit of dirt could cause the bit to 'tip' as you tighten the set screw.
2) The bit shank is not properly sized causing it to be pushed off center by the set screw.
3) Using a really long bit in combination with the holder would increase vibration and chatter for the reasons Dean mentioned.
_________________
Happy Machining!
Jeff Birt - Soigeneris.com
Proud Dealer of Taig, Precisebits, Gecko 540,
SmoothStepper and A2ZCNC products.