New rant just dropped. "The Structure and Interpretation of Computer Science Academic Metrics": https://www.williamjbowman.com/blog/2025/04/02/the-structure-and-interpretation-of-computer-science-academic-metrics/
@wilbowma I used to think basically this, and I definitely think there could be a better ranking of CS departments, but I've become convinced that the plausible alternatives are not better.
@samth But why do we need to rank CS departments
@wilbowma We don't "need" to but we do need to be able to summarize relative prestige of CS departments for a variety of purposes and our culture means we are going to rank them.
@samth I accept that "some people are going to", and this forces us (if we wish to survive) to play silly games. I don't understand why we need to summarize relative prestige of CS departments, though.
@wilbowma @samth idk. I never think about the rankings?? I think the only compelling reason to rank relative prestige is to make sure potential undergrad students have all the facts, but quality of research (which is what these rankings try to approximate) is kind of unrelated to the quality of teaching. It matters for PhD applicants making choices too, but anyone who is about to do a PhD had better already be able to figure this out on their own without looking up a Prestigimeter.
Is there another reason we need to do these comparisons?
@jonmsterling @wilbowma I think providing information to PhD students is the best reason, and I don't think the information is obvious to many PhD applicants.
@jonmsterling @wilbowma But aside from that, lots of people care about prestige of departments but cannot be expected to work it out from first principles, such as our deans. (One can say that they shouldn't care about this, but that seems hopeless even if I agreed.)
@jonmsterling @wilbowma No, because I don't think (a) deans are evil or that (b) caring about the prestige of departments is evil.
@samth @wilbowma Maybe 'evil' is too strong, but yeah lol, I do think that caring too much about the 'prestige' of your department is pretty dumb. I also don't see how this could be a controversial point.
And I can tell you from experience that an environment where people are focused on the prestige of their department to the detriment of .... doing the business of scholarship is not a good thing.
@jonmsterling @wilbowma I mean, this is exactly why aligning the two is an improvement, and why csrankings, for all its serious flaws, is better than "harvard is probably really good at CS".
@samth @wilbowma I don't see that to be the case at all...
The thing that makes "Gee, I bet Harvard / Cambridge / Whatever is really good because it is famous!" shower-thoughts fundamentally different from "This extremely detailed website says that CMU is the best CS department ever" is precisely that the latter comes with some "evidence" and so it carries more weight automatically among unfamiliar people, whereas the evidence-free "Harvard Cool!" thing carries comparatively less weight among unfamiliar people.
So if you consider the relative consequences of "Harvard is best b/c it is the most famous name!" being wrong and "CMU is best because of all this evidence!" being wrong, obviously the consequences of the latter are far more grave because people would have taken it more seriously.
@jonmsterling @wilbowma But they aren't just "shower thoughts"! US News puts out a whole ranking system that is basically that and also rarely updated. Prior to the creation of csrankings that was the most important ranking of CS departments.
Additionally, while it's possible that the psychological explanation you gave is true, the opposite is also possible and as far as I know there's no evidence for it.
@samth @wilbowma I mean, CS Rankings is bad for the same reason that US News rankings was bad... lmao
(EDIT: the commonality is they both dressed up shower thoughts with pseudo-evidence that looks really compelling to people who don't know anything. So both are more damaging than whatever random crap people assume without evidence, because people know the difference between things that are not backed up by anything, and things that are backed up by an authoritative source.)
@jonmsterling @wilbowma csrankings broadly corresponds to my own assessment of deparments in PL; does it not for you?
@samth @jonmsterling @wilbowma Its intended purpose (graduate applications) is not to rank whole departments, but departments in a particular subfield. It's rubbish at that, because "top two conferences" is too narrow a metric.
I do know that my own department has a strong group in computational biology, particularly in organ modelling. I don't know how strong, relatively speaking, but it is about one tenth of a strong department. You wouldn't know from csrankings. Not that it is misranked: it doesn't appear at all. Csrankings only considers two conferences, both in molecular biology; my colleagues publish in journals, and not in mol bio.