The career advice website Vault released its "best firm to work for" and "best firm for diversity" rankings. These are released each year and, as I explained in a prior post, they rely on the honor system. A firm's score is based on the opinions of the firm's own lawyers, and only the firm's lawyers. The idea is that if asked about their firm's diversity or quality of life -- lawyers would be conscientious, thoughtful and ethical enough to answer honestly.
The problem with this approach is that O'Melveny would pressure its attorneys to give positive reviews -- so that it could receive a high ranking. Pressure works. Pressure was reportedly how Trump University achieved a 98% student approval rating, and I suspect it's how authoritarian regimes achieve curious 100% approval ratings. (For comparison, O'Melveny's approval rating was 99.5%. Remember, this is a firm that reportedly hired investigators to identify associates who criticized the firm online.) Beyond employer pressure, attorneys might themselves choose to lie, especially if they're part of an unethical and rankings-obsessed culture. Vault knows how unreliable this approach is. That's why for its most widely-read ranking, the "Vault Law 100" ranking -- it does not allow attorneys to rank their own firm.
So when reading Vault's diversity and "best firm to work for" rankings, it's important to know if the respondents were being honest. In prior posts, I offered a way to do that. First, you can compare the "best firm for diversity" ranking with actual diversity data. Second, you can compare the "best firm for compensation" sub-ranking with actual compensation data. Third, you can compare the "best firm for hours" sub-ranking with actual hours data. If a firm ranks itself high on diversity, compensation, and hours -- when the objective data states otherwise -- you can conclude that the firm's attorneys were lying when they filled out the Vault survey.
This year, O'Melveny again graded themselves at the top of the rankings, and issued a press release congratulating themselves. But this may have been for naught, because the rankings have lost some relevance. Last year, a reporter told me that she doesn't write about Vault's diversity ranking, because it's self-graded. A few weeks after this year's results came out, I did a Twitter search for the words "Vault" and "law" to see all tweets about them. I did this so that I could take a minute and reply to the tweets, to let them know about the flawed methodology. But only a handful of people mentioned the rankings in that two-week period. Almost nobody cared. Gamesmanship may have ruined what could have been a useful tool.
So when reading Vault's diversity and "best firm to work for" rankings, it's important to know if the respondents were being honest. In prior posts, I offered a way to do that. First, you can compare the "best firm for diversity" ranking with actual diversity data. Second, you can compare the "best firm for compensation" sub-ranking with actual compensation data. Third, you can compare the "best firm for hours" sub-ranking with actual hours data. If a firm ranks itself high on diversity, compensation, and hours -- when the objective data states otherwise -- you can conclude that the firm's attorneys were lying when they filled out the Vault survey.
This year, O'Melveny again graded themselves at the top of the rankings, and issued a press release congratulating themselves. But this may have been for naught, because the rankings have lost some relevance. Last year, a reporter told me that she doesn't write about Vault's diversity ranking, because it's self-graded. A few weeks after this year's results came out, I did a Twitter search for the words "Vault" and "law" to see all tweets about them. I did this so that I could take a minute and reply to the tweets, to let them know about the flawed methodology. But only a handful of people mentioned the rankings in that two-week period. Almost nobody cared. Gamesmanship may have ruined what could have been a useful tool.
