There are many methods available to align ligands in 3D in the absence of protein information. However, they fall into three main classes: atom matching/pharmacophore-based methods (GALAHAD, Open3DALIGN, Align-it), shape/electrostatics-based methods (Cresset, ROCS, Phase Shape, ShaEP) and QM potential-derived methods (CosmoSim3D). Each of these methods has different strengths and weaknesses. We have tested representatives of each class of alignment method on the recently-published AZ data set (Giangreco et al., J. Chem. Inf. Model. 2013, 53(4), 852-866) and the PharmBench data set (Cross et al., J. Chem. Inf. Model. 2012, 52(10), 2599-2608) in order to get a better understanding of what types of ligands are best aligned by each class. We demonstrate significant problems with alignment success criteria published in the literature, and suggest a robust alternative leveraging the large amounts of data available in these new validation suites.