I recently set-up my google scholar profile, and I configured it to alert me when any new article cites one of my papers. I did for my personal statistics (you know that I dislike evaluation based only on statistics of citations), because I want to see who is citing my work, and if they are doing it properly.
A few days ago I received an alert reporting that this paper is citing one of my publications. This is a rather strange paper: I think it is a good example of what happens in some academic area, so I decided to share my thoughts in public.
First the facts.
The paper is titled “Performance Analysis of IES Journals using Internet and Text Processing Robots“, and has been presented at The IEEE 37-th Annual Industrial Electronics Conference IECON. The conference is listed in IEEE Xplore, and it is sponsored by the Industrial Electronic Society (IES) of the IEEE. The paper is mainly a collection of data extracted from IES journals, how many paper, how many citations, impact factor, etc. The interesting part, however, comes when you look at the References section: it lists 59 references, and except for “The Perl Black book“, and “Learning Perl“, the remaining ones are citations of other papers that have been published in IES journals, magazines or conference proceedings . Interestingly, most of these references are not cited in the text: indeed one may wonder why references:
 K.T. Chau, C.C. Chan, Chunhua Liu, “Overview of Permanent-Magnet Brushless Drives for Electric and Hybrid Electric Vehicles,” IEEE Trans. on Industrial Electronics, vol. 55, no. 6, pp. 2246-2257, June 2008.
 T. Cucinotta, A. Mancina, G.F. Anastasi, G. Lipari, L. Mangeruca, R. Checcozzo, F. Rusina, “A Real-Time Service-Oriented Architecture for Industrial Automation ,” IEEE Trans. on Industrial Informatics, vol. 5, no. 3, pp. , Aug 2009.
are reported here, since they have nothing to do with the content of the paper. Also, one might wonder why these 57 papers have been selected among the many others published by IES. I think only the authors can answer our curiosity.
It is worth to note that one of the authors of the paper is Editor in Chief of the IEEE Transactions in Industrial Informatics, and was former Editor in Chief of the IEEE Transactions on Industrial Electronics. Both journals are published by the Industrial Electronics Society.
A few weeks ago, we got a paper rejected from the IEEE Transaction on Industrial Informatics of the IES. Ok, probably the paper was not so great after all. However, one of the anonymous referees wrote this review (which I report entirely, typos included, I only emphasised one sentence in bold):
This manuscript already received rejection and the revised version is stilt not on the IEEE Trans. level.
It seems that your manuscript is weak on the current state-of-the-art description, and it does not have enough current journal references. You have placed your findings in the content of conference papers instead of journal papers, which is OK but only for work published on conferences but not in journals. Notice, that out of 31 references only 3 are to the journal papers and this reason alone should be a good reason to reject the manuscript. These 3 journals:
Journal of Systems Architecture
ACM SIGPLAN Notices
are also of questionable quality with low Eigenfactor Score, Article Influence Score, or Impact Factors
If authors are not able to connect their findings to recent journal publications of other authors it could mean:
(1) there is not much recent interest in the subject
(2) authors are not following recent journal literature
Both are good reasons for rejecting the manuscript.
You are probably wondering what our paper was about. Well, it does not matter, since the referee did not bother to write any technical comment to reinforce his suggestion to reject the paper. In fact, this referee plainly suggests that the paper ought to be rejected simply and solely because we did not reference the right journals.
Please, pay attention: he did not say which papers we should cite. Also, he is saying that there is no interest in the subject because we did not reference the important journals. Or that we are not following recent journal literature, because we did not cite the important journals. Actually, he does not even know if the topics is uninteresting or if we are just stupid: apparently, it does not matter to him.
So far I just reported the facts, and I believe that the facts speak for themselves. Everybody at this point can make his opinion on what is going on in the IES. However, for those of you that are not aware of how academic publishing works, I think it is worth to spend some more words of personal comments.
In my opinion, it is quite evident that both Fact 1 and Fact 2 are just two “tricks” used to increase the number of citations to journals of the IES. In the first case, by “publishing” a paper whose only purpose seems to be to list references to papers into publications of the IES. In the second case, by “suggesting” future potential submitters to IEEE TII to cite more “good journals”, and implicitly, journals of IES.
The not-so-hidden goal of these tricks is to increase the Impact Factor. The IF index is a measure of a journal performance: the more citations to papers of a journal, the higher is the IF. It is quite clear that one of the goals of the Editor in Chief of any journal is to increase its IF. I can just visualise in my mind the EICs of all the IEEE Transactions sit around a table during their annual meeting, playing the “my IF is bigger then yours” game.
Is the IF a good measure of the quality of a journal? The matter has been debated for a long time. In biology and medical sciences, it is well known that in the past (and still now) the quality of a researcher was measured by the IF of the journals where his papers were published. Certainly, if the IF is built using a lot of tricks like the ones I described above, the correlation between IF and quality becomes weaker.
Are those tricks “legal”? Well, yes, there is nothing illegal going on here. Unethical maybe, but not illegal. However, in the long run, these tricks are potentially devastating for the academic community at large.
In the short run, the situation is win-win for the authors and for the editors. For example, in the first case I should be happy that the authors bothered citing my paper: this citation will contribute to my h-index, and no human being will ever check the more than one thousand citations to my scientific production one by one. This citation has become just one additional number in my batch, and my h-index will go up, thanks to the authors of the strange paper of Fact 1.
As for Fact 2: yes my paper got rejected. But the message is that by randomly citing some additional paper in the right journal, maybe in the future my paper will be accepted, thus contributing to the journal IF; and once it is published there, the more the IF goes up, the better it is for me. As for the Editorial board, they will see the IF increase, and they can go to the annual meeting and show good numbers to their colleagues. It looks like a good deal, after all.
Then, if everybody is happy, why that gut feeling?
If academic world is like this, it is not the place I want to be. I decided to pursue an academic career for a good reason: it was not for money (my Italian friends know better), and not for glory. It was for fun. I have fun doing research and teaching, and I am paid for that. But I don’t want to work in an environment where these tricks are commonplace, because that takes away part of the fun, and in the long run will take away all the fun.
Therefore, these are my decisions:
- I shall not review any paper neither for IEEE Transaction on Industrial Informatics, nor for any other publications of IES.
- I shall not submit any paper to any journal of IES.
And, of course, if I convinced you of my reasons, I invite you to do the same.
One final note.
Someone may think I wrote this post as a revenge for the rejection of my paper. Actually, I did not think of taking any action until I discovered Fact 1. Also, I already have enough publications in my CV, and I am in no rush to publish.
Therefore, we just resubmitted our paper to another journal, which has a much lower IF, but whose EIC does not play any “trick”.