Dealing with Conflicts of Interest (COIs)

Dear readers,

In this post, we will share with you a letter  by Aurélien Max about conflict of interests (COIs).  While Min and I haven’t submitted to ACL to avoid COIs,  many area chairs did submit to the conference. In fact, for many Area Chairs (ACs) the ability to submit was their precondition for accepting the position.  This request does not contradict ACL COI policies, which detail how to handle such papers.  Specifically, they were curated by another area chair, who assigns reviewers and who make recommendation about the acceptance/rejection of these submissions. In the reviewing system, ACs cannot see their own papers or papers with whom they declared COIs.

 In the letter below, Aurélien expresses his concerns about the current policy.  Let us know if you share the same concerns.

Screen Shot 2017-03-16 at 6.02.57 AM.pngHi Min and Regina,

I must first thank you for the very significant transparency effort that you are making as ACL’17 PC chairs. I would like to echo some concerns that were raised among colleagues on the complex issue of conflicts of interest. Following the interesting call for area chairs, you clarified rules for COIs for that particular role, stating that it would be unfair to not allow area chairs to submit papers in their areas:

From: https://chairs-blog.acl2017.org/2016/12/26/changes-to-the-reviewing-process/

“Also, to clarify, area chairs are allowed to submit to the conference and to the same area.  It would be unfair otherwise, as many of you have students or junior colleagues whom you are mentoring that may be submitting to the area.  As per previous ACL events, any conflicts of interest (COIs) for a submission where one or more area chair are involved need to be declared, so that another (non-COI) chair within the area can be assigned to manage the submission. In the case that all area chairs have a COI, the paper will be routed to a different area.”

Some of us (that would include me), possibly after witnessing instances of COIs at various levels and the impressive amount of regulations that attempt to fight this (most notably in the research funding area), believe that some roles are so sensitive that COIs should be completely avoided if possible. This would include people in a position where they may end up being both judge and interested party at the same time, as e.g. eliminating competing papers can have some positive effect on other papers. It is true that this is already the case for reviewers, but this is somehow dealt with using several reviewers per paper and allocating few papers per reviewer, author response periods, and the scrutiny of area chairs. Area chairs play a very important role at the very core of paper selection (I assume that a large proportion of papers at ACL venues are the object of very tough decisions), which is undoubtely a very important and time consuming service to the community. But one may argue that it is also a very rewarding position, as is certainly shown by the significant number of candidates for this year’s ACL (and, in particular, self-nominating candidates — I should maybe indicate that none of us were candidates for this role at ACL’17).

So, our question, which does not directly applies to ACL’17, is: why not make area chairing an even more openly disinterested service to the community by making it a rule that area chairs cannot submit papers to their area? The fact that you had to clarify the position for ACL’17 on the blog made us wonder whether there was a large consensus or people not agreeing to the opposite position. Considering that we may be moving towards recurrent calls for area chairs at ACL venues, stating such a rule at the time of application should not pose any problems and should not have any significant effect on the number of candidates.

Your arguments for allowing area chairs to submit to their area were of course convincing, but there are several ACL venues each year, and in fact some people decide not to submit to some conferences for various reasons, including because they do not have access to sufficient funding.

To be honest, our discussion started by one of us spotting the fact that Regina said her group was not submitting to ACL this year (https://chairs-blog.acl2017.org/2017/02/02/last-minute-writing-advice/). Although there may be many reasons for this, some assumed that this was maybe a way of avoiding any COIs — given your transparency effort we would of course never say that this should be necessary (and again the actual reasons may not have anything to do with this), but as a message and potential future rule we would like to see things going in that direction. In our country, we have the example of a national elected professor sub-committee whose members committed to not submitting applications for promotion that would have to be dealt with by the same sub-committee for the time of their 4-year service. This not only instills trust, this also assigns a new type of value to community service and makes rotation for such duties a desirable effect for all.

Many thanks again for the time you are devoting to ACL this year, and thank you for reading.

Best,
Aurélien Max, Univ. Paris-Sud

 

3 thoughts on “Dealing with Conflicts of Interest (COIs)

  1. Disallowing submission by area chairs does not eliminate the COI problem, because area chairs have lots of other conflicts. While my group did not submit to the dialogue area at ACL this year, there are submissions from other groups in my (fairly large) university — papers that I’m not involved with at all, but still pose a conflict of interest for me. I think it is unreasonable to ask that all these people not submit to ACL. So area chair (and program chair) COIs will arise, and there need to be policies to handle these.

    Like

  2. These are good comments. It is hard to set a uniform policy for a large group of experts to obey, since their circumstances greatly differ. We have 61 ACs this year (Ron is one), and many of them abstained from submitting to their areas for exactly the reasons as stated. In certain cases, this may be easier (for smaller niche areas where ACL is not the primary conference for those practitioners), and in others, possibly very difficult (IE, NLP and QA, and semantics are huge areas, almost functioning as conferences in their own right; and other areas don’t have other annually re-occurring premier venues that would garner as much prestige as ACL).

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s