Adjustable non-releasing tap holders are used in reversing spindle automatic screw machines and CNC lathes. The tap is mounted in the tool holder directly or in an appropriately sized bushing; the holder assembly is adjustable to accurately align the tap with the work-piece. The non-rotating tool holder is free to move axially, permitting the tap to lead itself after being started in the workpiece. Automatic spindle reversal withdraws the tap from the hole and a spring draws the tool holder back into the head when the tap clears the workpiece.