Google Scholar is Manipulatable (arxiv.org) 16
Abstract of a paper [PDF] the on pre-print server Arxiv: Citations are widely considered in scientists' evaluation. As such, scientists may be incentivized to inflate their citation counts. While previous literature has examined self-citations and citation cartels, it remains unclear whether scientists can purchase citations. Here, we compile a dataset of about 1.6 million profiles on Google Scholar to examine instances of citation fraud on the platform. We survey faculty at highly-ranked universities, and confirm that Google Scholar is widely used when evaluating scientists. Intrigued by a citationboosting service that we unravelled during our investigation, we contacted the service while undercover as a fictional author, and managed to purchase 50 citations. These findings provide conclusive evidence that citations can be bought in bulk, and highlight the need to look beyond citation counts.
Goodhart's Law (Score:2)
This is an example of Goodhart's Law [wikipedia.org]:
"When a measure becomes a target, it ceases to be a good measure"
There is an article in this week's Economist about how the peer review system and promotions based on publishing and accumulating citations are failing. Companies that might benefit from research mostly see research from universities as worthless, yet also do much less of their own research because of perverse incentives. This leads to less innovation and lower productivity growth.
Universities are failing [economist.com]
Re: (Score:2)
"Of course, there is a place for speculative research unconstrained by economic concerns, but perhaps we have gone too far in that direction". But that's the very opposite of what's happening. It's getting harder and harder to get funding for research unless you can demonstrate the likelihood of short-term commercialization.
Re: (Score:2)
It's getting harder and harder to get funding for research unless you can demonstrate the likelihood of short-term commercialization.
Do you have a citation for that? That's the opposite of what the Economist says.
Or, more accurately, the Economist says that the research isn't actually useful, but not necessarily that it wasn't expected to be useful.
Re: (Score:2)
No, I don't have a citation, I have 40 years of experience. If you want a citation you'll need to demonstrate the short-term commercial benefit of performing such a study.
Re: (Score:3)
Companies that might benefit from research mostly see research from universities as worthless, yet also do much less of their own research because of perverse incentives.
For-profit companies do their own research because they want proprietary innovations. If companies just relied on public research, then they become manufacturers of commodities, which is not a great position for those companies.
Most published work is either incremental or not all that relevant. That's true for both academia and corporate research. However, a small percentage is useful, and an even smaller percentage is seminal. Missing out on the useful and especial seminal work is a bad thing.
Most good
Other ways to get citations... (Score:2)
Another is to take one research project but publish several papers on it, each reporting a different feature or aspect, rather than
Re: (Score:2)
Who knows of any more?
Publishing essentially the same thing in multiple places.
Getting your name added to your students' papers even though you did little to no actual work on those papers.
Working on research with a team and then break your findings into multiple parts, then all cite each other.
Contributing something to the CERN, then getting your name added to thousands of papers that you know nothing about.
Letting a pharmaceutical company write papers for you.
Create your own journal and self-publish anything you couldn't
Re: (Score:2)
The underlying concept is faulty (Score:2)
The notion that a paper has merit because it has a lot of cites is intrinsically faulty. It's easy to posit a paper that is so advanced that no one cites it all all, since it obviates all existing research. This is more of the "closed shop" mentality working in academia, it's the same with the peer review process, which has shown time and again to be easily manipulated. This is just another and obviously flawed example of how easy it is to show "measurables" to non-technical people.
Google Scholar is Useless (Score:4, Informative)
Re: (Score:3)
I'm not sure the field you work in is relevant. It attributes at least one of my papers to someone with the same name in a different field, and it won't let me correct the data because I don't have an email address which matches its whitelist.
Fortunately no one relies solely on citation count (Score:2)
The system is not broken, except for the incompetent. No one relies solely on citation counts to hire or to give awards. It is a first-level weed-out mechanism. After that first step, the specific papers are examined, and the quality and impact of those papers will be considered along with other things.
In a way, the citation count is like a GPA. No one hires a student based solely on the basis of a 4.0 GPA. The 4.0 can lead to an initial interview, but it doesn't replace the interview.
Replication rates of ~50% (Score:2)
Can we trust this paper? (Score:1)