http://www.abanet.org/media/youraba/200509/article06.html A FAQs from the ABA Standing Committee which issues the rankings.
Q: What does the committee’s rating mean? What is it based upon?
A: Well, we actually have two ratings in this case. We have a rating that was issued by our committee when Judge Roberts was nominated to be an associate justice, which was unanimous “well qualified.” And then we did a supplemental evaluation by the 2005-2006 committee because its membership turned over at the end of the Annual Meeting, and that was limited to the question of his administrative and leadership abilities to be Chief Justice. We had a new nomination. And that came in unanimous “well qualified” too — even with seven new people out of 15 on the committee.
What does it mean? It means that John Roberts has been found by the American Bar Association’s Standing Committee on Federal Judiciary to be of the highest standing for his integrity, his professional competence, his judicial temperament, and all the component parts that that would suggest that he is not just qualified to serve on the U.S. Supreme Court, but to serve as Chief Justice of the United States. He demonstrated all of the elements one would hope to see of collegiality, of intelligence, of fairness, of willingness to work with others, ability to build consensus, that one could ask for in trying to determine the qualifications of anybody who’s been nominated for that high office.