How to increase the Impact Factor of a Journal

I recently set-up my google scholar profile, and I configured it to alert me when any new article cites one of my papers. I did for my personal statistics (you know that I dislike evaluation based only on statistics of citations), because I want to see who is citing my work, and if they are doing it properly.

A few days ago I received an alert reporting that this paper is citing one of my publications. This is a rather strange paper: I think it is a good example of what happens in some academic area, so I decided to share my thoughts in public.

First the facts.

Fact 1

The paper is titled “Performance Analysis of IES Journals using Internet and Text Processing Robots“, and has been presented at The IEEE 37-th Annual Industrial Electronics Conference IECON. The conference is listed in IEEE Xplore, and it is sponsored by the Industrial Electronic Society (IES) of the IEEE.  The paper is mainly a collection of data extracted from IES journals, how many paper, how many citations, impact factor, etc.  The interesting part, however, comes when you look at the References section: it lists 59 references, and except for “The Perl Black book“, and “Learning Perl“, the remaining ones are citations of other papers that have been published in IES journals, magazines or conference proceedings .  Interestingly, most of these references are not cited in the text: indeed one may wonder why references:

[35] K.T. Chau, C.C. Chan, Chunhua Liu, “Overview of Permanent-Magnet Brushless Drives for Electric and Hybrid Electric Vehicles,” IEEE Trans. on Industrial Electronics, vol. 55, no. 6, pp. 2246-2257, June 2008.

or

[53] T. Cucinotta, A. Mancina, G.F. Anastasi, G. Lipari, L. Mangeruca, R. Checcozzo, F. Rusina, “A Real-Time Service-Oriented Architecture for Industrial Automation ,” IEEE Trans. on Industrial Informatics, vol. 5, no. 3, pp. , Aug 2009.

are reported here, since they have nothing to do with the content of the paper. Also, one might wonder why these 57 papers have been selected among the many others published by IES. I think only the authors can answer our curiosity.

It is worth to note that one of the authors of the paper is Editor in Chief of the IEEE Transactions in Industrial Informatics, and was former Editor in Chief of the IEEE Transactions on Industrial Electronics. Both journals are published by the Industrial Electronics Society.

Fact 2

A few weeks ago, we got a paper rejected from the IEEE Transaction on Industrial Informatics of the IES. Ok, probably the paper was not so great after all. However, one of the anonymous referees wrote this review (which I report entirely, typos included, I only emphasised one sentence in bold):

This manuscript already received rejection and the revised version is stilt not on the IEEE Trans. level.

It seems that your manuscript is weak on the current state-of-the-art description, and it does not have enough current journal references. You have placed your findings in the content of conference papers instead of journal papers, which is OK but only for work published on conferences but not in journals. Notice, that out of 31 references only 3 are to the journal papers and this reason alone should be a good reason to reject the manuscript.  These 3 journals:
Real-time Systems
Journal of Systems Architecture
ACM SIGPLAN Notices
are also of questionable quality with low Eigenfactor Score, Article Influence Score, or Impact Factors

If authors are not able to connect their findings to recent journal publications of other authors it could mean:
(1) there is not much recent interest in the subject
(2) authors are not following recent journal literature
Both are good reasons for rejecting the manuscript.

You are probably wondering what our paper was about. Well, it does not matter, since the referee did not bother to write any technical comment to reinforce his suggestion to reject the paper. In fact, this referee plainly suggests that the paper ought to be rejected simply and solely because we did not reference the right journals.

Please, pay attention: he did not say which papers we should cite.  Also, he is saying that there is no interest in the subject because we did not reference the important journals. Or that we are not following recent journal literature, because we did not cite the important journals. Actually, he does not even know if the topics is uninteresting or if we are just stupid: apparently, it does not matter to him.

Comments

So far I just reported the facts, and I believe that the facts speak for themselves. Everybody at this point can make his opinion on what is going on in the IES. However, for those of you that are not aware of how academic publishing works, I think it is worth to spend some more words of personal comments.

In my opinion, it is quite evident that both Fact 1 and Fact 2 are just two “tricks” used to increase the number of citations to journals of the IES. In the first case, by “publishing” a paper whose only purpose seems to be to list references to papers into publications of the IES. In the second case, by “suggesting” future potential submitters to IEEE TII to cite more “good journals”, and implicitly, journals of IES.

The not-so-hidden goal of these tricks is to increase the Impact Factor. The IF index is a measure of a journal performance: the more citations to papers of a journal, the higher is the IF. It is quite clear that one of the goals of the Editor in Chief of any journal is to increase its IF. I can just visualise in my mind the EICs of all the IEEE Transactions sit around a table during their annual meeting, playing the “my IF is bigger then yours” game.

Is the IF a good measure of the quality of a journal? The matter has been debated for a long time. In biology and medical sciences, it is well known that in the past (and still now) the quality of a researcher was measured by the IF of the journals where his papers were published. Certainly, if the IF is built using a lot of tricks like the ones I described above, the correlation between IF and quality becomes weaker.

Are those tricks “legal”? Well, yes, there is nothing illegal going on here. Unethical maybe, but not illegal. However, in the long run, these tricks are potentially devastating for the academic community at large.

In the short run, the situation is win-win for the authors and for the editors. For example, in the first case I should be happy that the authors bothered citing my paper: this citation will contribute to my h-index, and no human being will ever check the more than one thousand citations to my scientific production one by one. This citation has become just one additional number in my batch, and my h-index will go up, thanks to the authors of the strange paper of Fact 1.

As for Fact 2: yes my paper got rejected. But the message is that by randomly citing some additional paper in the right journal, maybe in the future my paper will be accepted, thus contributing to the journal IF; and once it is published there, the more the IF goes up, the better it is for me. As for the Editorial board, they will see the IF increase, and they can go to the annual meeting and show good numbers to their colleagues. It looks like a good deal, after all.

Then, if everybody is happy, why that gut feeling?

If academic world is like this, it is not the place I want to be. I decided to pursue an academic career for a good reason: it was not for money (my Italian friends know better), and not for glory. It was for fun. I have fun doing research and teaching, and I am paid for that. But I don’t want to work in an environment where these tricks are commonplace, because that takes away part of the fun, and in the long run will take away all the fun.

Therefore, these are my decisions:

I shall not review any paper neither for IEEE Transaction on Industrial Informatics, nor for any other publications of IES.

I shall not submit any paper to any journal of IES.

And, of course, if I convinced you of my reasons, I invite you to do the same.

One final note.

Someone may think I wrote this post as a revenge for the rejection of my paper. Actually, I did not think of taking any action until I discovered Fact 1. Also, I already have enough publications in my CV, and I am in no rush to publish.

Therefore, we just resubmitted our paper to another journal, which has a much lower IF, but whose EIC does not play any “trick”.

Advertisements

12 thoughts on “How to increase the Impact Factor of a Journal

  1. For your amusement… It is not only the IES to try tricks to increase the impact factor…
    Here is some interesting exchange about a paper that I submitted to Trans. on CAS I
    This is a quote from the Journal editor

    >> As a personal comment, I noticed that the list of references is not
    >> particularly updated as far as the papers which recently appeared in
    >> the IEEE TCAS are concerned. I would greatly appreciate it if you
    >> could do your best to put your manuscript in perspective for the
    >> TCAS-I readership. To do so, please scan the recent journal
    >> literature (say since 2009 for TCAS-I and TCAS-II and including those
    >> listed in the pre-publication link on
    >> IEEEXplore) and add some of the references that you may find relevant
    >> for your manuscript.

    My comment to this comment:

    >> Out of curiosity:
    >>
    >> “To do so, please scan the recent journal literature (say since 2009
    >> for TCAS-I and TCAS-II and including those listed in the
    >> pre-publication link on
    >> IEEEXplore) and add some of the references that you may find relevant
    >> for your manuscript.”
    >>
    >> I would have expected a comment to the extent that our reference list
    >> may not be exhaustive but the particular attention to papers
    >> published in the Transaction themselves is odd…. Is this to beef up
    >> the impact factor of the TCAS transactions?
    >> Alberto

    And the response!!!!!!!!!!!!!

    >It is my impression and of the
    > associate editor who handled the manuscript that you probably have
    > overlooked some recent contributions in IEEE TCAS-I and/or -II that may help in putting your manuscript in the interest of the TCAS-I readership better.
    > I don’t think it is my duty to provide you with the exact references,
    > but yours.

    Note the last sentence!!!! I as Giuseppe do not think I need to have one more paper (albeit in the end it was accepted for publications) and I felt very sad for the general situation of the IEEE Societies and their journals. I have not seen Trans. CAD, JSSC, Trans on Computers, Trans on Automatic Control and the Proc. of the IEEE doing anything like this and I am hopeful this behavior will change. I am considering a letter to the President of the IEEE. If you are interested to sign a letter that deplores this situation please let me know!
    Alberto Sangiovanni Vincentelli

    Reply
    • Alberto,

      of course I agree with youm and I am willing to sign the letter! I cannot believe that this kind behaviour of behaviour is already so widespread inside IEEE. It is true that other transactions (like TC, TAC, etc.) are not using these tricks: however, I feel we have to stop this as soon as it is possible, or it will soon be extended to other publications.

      Reply
    • I have been the editor-in-chief of IEEE TCAS-I who handled the paper of Alberto Sangiovanni Vincentelli. Though I welcome this discussion on IF and its possible misuse, there are a few facts in relation to Alberto Sangiovanni Vincentelli’s paper that he “forgot” to mention:
      1. His submission to IEEE TCAS-I was a resubmission of a previously rejected submission to IEEE JSSC.
      2. The authors did not make a serious effort to put their first manuscript in the perspective of the TCAS-I readership. 24 citations were to JSSC or a SSCS related conference, only one to TCAS-I.
      3. As TCAS-I publishes papers of a more design methodological and theoretical nature, a scope which is different from that of JSSC, I believe it is only logical that I asked the author to put his manuscript more in the perspective of the TCAS-I readership. It is not the duty of the editor-in-chief to suggest the relevant papers and help rewriting the paper, but of the author to make a serious effort to relate and refer to all relevant prior art.
      4. Alberto Sangiovanni Vincentelli accepted my point of view and even apologized for having brought the matter up. He understood and accepted that TCAS-I was not trying to “beef” up the impact factor, which is correct.

      Wouter Serdijn
      Editor-in-Chief (2010-2011)
      IEEE Transactions on Circuits and Systems-I: Regular Papers

      Reply
  2. Peppe,
    Needless to say I agree with your viewpoint (and with Alberto’s), to a point where
    I can sign any petition to the IEEE to discourage these dismal practices.
    Let me just add one additional piece of information.
    Other, more reputable, IEEE transactions have recently played a little bit with the semantics of the word “major changes required”. To make a long story short, associate editors decide more and more frequently to recommend “reject and re-submit” instead of “major revision required”. From the perspective of the authors, very little changes. You have a relatively short deadline to re-submit the paper, you are required to provide answers to the reviewers, and you work is processed by the same reviewers as the previous submission. So, why the Hack do we need this change? The short answer is that from the statistical point of view a “reject and resubmit” is considered as a brand new submission. This way, the average time–to–print decreases dramatically and the editor in chief becomes the white horse man whom came to town to secure fresh material to the journal. In fact, I believe this is bad, for the simple reasons that it is sweeping under the carpet. The real fact is that many reputable journals have a very long turn–around–time, and this ultimately decreases the interest of the journal and the motivation of the authors to submit excellent papers there (many of my excellent colleagues have decided to submit only conference papers).
    To wrap up, I think the time is ripe to get rid of all these cheap tricks and focus on how to make the most of our communities and of our publications.

    Reply
    • Luigi,
      I confirm the practice you describe. Indeed, I experimented it during my terms as associated editor of IEEE TC. It adds to the increasingly long list of malpractices in our journals.

      One might start to wonder why these practices are being applied more and more frequently…

      Reply
  3. From my relatively younger perspective, the above mentioned comment

    “If authors are not able to connect their findings to recent journal publications of other authors it could mean:
    (1) there is not much recent interest in the subject
    (2) authors are not following recent journal literature
    Both are good reasons for rejecting the manuscript.”

    contains two really disruptive and discouraging points. First, should we only pursue research areas where “there is much interest” ? Assume I’m proposing a novel mechanism that will trigger interest in a new area that was deemed as unfeasible or non-convenient so far … but now with my new toy it will change the world … still such a logic would imply my work deserves to be rejected solely because “it didn’t attract much recent interest” ? Think of what happened recently with the advent of cloud computing and the rediscovery of virtualization.

    Second, after my relatively short experience with both IEEE and ACM journals (just for the sake of fairness), where I experimented a delay between 2 and 3 years from the very first submission till the final publication of the papers, can somebody explain me how am I supposed to consider journal papers as “recent literature” ? In the world of Information Technology, and particularly Embedded Systems, 3 years is a lot. It really changes the perspective. Sometimes, an outstanding engineering effort to decrease complexity and/or resource requirements of certain solutions may become largely obsolete after a new generation of devices exhibiting increased memory, computation and battery life capabilities. Unfortunately, I’m tempted to consider journal papers in these areas, especially coming from the transactions series, as very well written and very thoroughly reviewed papers, but at the same time with quite old and outdated contents. When searching for the latest recent findings, I dig into conference proceedings. Feel free to blame me for that.

    And, for what it matters, why do we keep distinguishing so much between journal papers and conference proceedings papers, in an era in which rarely does a paper get delivered and printed on paper, whilst it is 99% times downloaded from the Internet and read on the screen ?

    My 2 cents,

    Tommaso

    Reply
  4. I’m glad to hear that eventually someone is taking a stand against these practices. I strongly encourage you guys to write a letter to the IEEE, and to publicize it in our communities. If I write it, nothing would change. But if it is someone like Alberto, well… the impact is different.

    If it can be of any help, this is the last review I received for a Transactions on Industrial Informatics paper (that has been accepted):

    “Manuscript has to be updated. By the time of publications the most current journal reference would be already couple years old and this may hurts the journal ranking. Notice that the world community is ranking journals based on two years old journal references. Therefore, this paper, if it accepted, will not be even considered in journal ranking analysis. If authors are looking for recent publication within the Society scope they may consider visiting the recently developed web page with the access to papers published at Society’s journals and conferences (http://tii.ieee-ies.org/IESpub.htm)”

    Good luck!

    Marko

    Reply
  5. Let me add my contribution to this funny (sad) story about IEEE TII.
    Our paper was already accepted and we were about to send them the source files. As the new guy became Editor, he sent us a quite strong email saying the paper could not be published anymore. The reason was quite difficult to decrypt to me. Master Giorgio then translated the obscure comments into “he wants us to cite some of his journal papers”. I was honestly surprised (at that time) of the interpretation. However it really worked that way: we added two references to TII and that’s it “Congratulations (once again) your fine work…”

    Enrico

    PS: I’m available to sign anything. Peppe, did you get any feedback from some of our US friends? Sanjoy, Jim, maybe Stankovic, Sha?

    Reply
  6. Citing references from high IF Journals for researchers of developing countries is difficult. Simply because most of these journals are no OPEN ACCESS and most University libraries in these developing countries do not have subscription for these journals. Hence, good researches from developing countries may not have a place in high IF Journals. I wish to submit these high IF journals should be open accessed especially to developing countries.

    Reply
  7. I think many IEEE journals play these tricks. This is a standard boilerplate used by the IEEE Transactions on Systems, Man, and Cybernetics Part C:

    ==

    Since many of the papers published in the IEEE SMC Transactions (see Xplore: http://ieeexplore.ieee.org) have been influential in their respective topics, we ask you to find and potentially cite recent, relevant papers in your revision.

    ==

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s