Abstract: "Back It Up" Policy for Answers
In this post, I propose updating the policy on Economics.SE for deletion of low-quality answers. While there has been some recent discussion regarding our site’s policy regarding deletion of low-quality answers, this question has come up on this site multiple times (reference provided below). A recent post has demonstrated that there is significant community support to develop more detailed guidelines for deletion of posts. Here, I review the policies adopted by other Stack Exchange sites and the rationale behind these policies and review criteria for how these policies might be successfully adopted on this site. In the end, I propose a stricter set of guidelines specifically for the deletion of answers. Specifically, I propose that we adopt the “back it up” principle adopted by over a dozen other stack exchange sites: a non-mathematical answer (or similar) must be backed up by citations or is subject to deletion. Here is an example. This policy reduces the amount of subjective judgement needed to determine whether a post needs to be deleted while also significantly increases the quality of posts on the site.
- The question portion of this post reviews the policies of other sites and discusses the rationale for them. Upvote this question if you support a revision of our policy.
- The answer portion of this post will contain specific policies that we might adopt. I post a specific policy based on an assessment of other sites’ policies. Upvote my answer if you support the specific policy that I propose. Also, feel free to submit alternative proposals.
Reviewing Answer-Deletion Policies Across Sites
On this site, we have discussed the problems associated with low-quality answers several times. I list these here.
- Bad in my opinion vs. low quality
- Post Removal and Rules Reminder
- What to do about heterodox answers in low-frequency questions?
- Preemptive protection
- Wikipedia Referencing and Content Rules - Is this time for me?
A common thread among these posts is the tension arising from the inherent subjectivity of determining whether an answer is of low quality. Fortunately, this problem has been already been solved (or at least mitigated) on many other Stack Exchange sites through the adoption of the so-called “back it up” principle. For most sites, this means that an answer must be backed up with a citation of some sort. On our site, it might mean something slightly different. For example, an answer to a question of a mathematical nature must be backed up with mathematical argumentation (obviously). A question of a more “wordy” nature must be backed up with a reference or citation to, say, a peer-reviewed journal. As argued on another site,
“For anyone who is trying to do actual research, answers without sources are useless. I'm not even going to talk about how answers without sources are often inaccurate, but I've noticed that people who know where they got their information from usually don't misremember things or post incorrect facts.”
The rationale, summarized in an oft-cited blog post by Robert Carataino is that “opinion, by itself, is noise.” If an answer is not backed up, it should be subject to deletion. See here for a list of sites that have adopted a policy of this nature: How widespread is the "back it up" principle?
Criteria for a Successful “Back It Up” Policy on Economics.SE
The idea of “back it up” is universal. However, since each site on Stack Exchange is different, the particulars of implementing such a policy are subtly different. Here I review some criteria that such a policy should have to be as successful as possible.
- Because Economics.SE contains mathematical and computational questions, as well as discourse-based “wordy” questions, simply requiring all answers to contain citations is not a sufficient policy. However, the principle of “back it up” should be universal---since mathematical answers, say, can be backed up by mathematical argumentation. A successful policy on our site needs to articulate this.
- The policy should be effective! That is, it should reduce the visibility of unsourced answers and other answers of low-quality. One way to do this is to simply encourage users to downvote such answers. Given that this site attracts a lot of laypeople who are very willing to upvote unsourced answers, I believe that the policy should be to delete such answers.
- Such a policy should put as little burden on moderators as possible. Deletion of answers can only be done by moderators, with the exception of answers with a negative score. Given this, the community should be vigilant to downvote answers that do not satisfy this policy. (A downvote can be undone if the author fixes the answer via edits.) When such answers do not get the necessary downvotes, enforcement of the policy rests with the moderators. Thus, the policy should be such that a moderator can decide whether or not an answer should be deleted almost instantly and without effort. To accomplish this, there should be a simple, nearly programmatic rule for this. An example that might arise, for example, is that a moderator wants to allow for some sort of grace period for the author to revise the answer. In this case, a programmatic rule would be that an unsource answer MUST be deleted if it is older than 3 days, for example. By tying the hands of the moderator in this way, we in fact make their job easier.
- How should this policy be applied to very old answers? There are a lot of old, unsourced answers on this site. A successful policy must describe how to deal with these. My take is that they will either be fixed by the community slowly over time or that they will be eventually deleted. For example, over time community members edit them or will flag them to bring them to the attention of moderators and they will slowly be resolved. If we’ve tied the hands of the moderators, as described above, the solution will be simple. Remember, “opinion, by itself, is noise.”
- The policy should be short and simple. Short and simple policies are the most effective.