1
$\begingroup$

As can be seen when clicking on my name here I founded the ReplicationWiki. Replication is my field of expertise. I came across stack exchange because I saw someone had linked to the ReplicationWiki in the Quant Stack Exchange and someone had found this helpful. The information provided there was incomplete so I explained in more detail.

I then looked what other questions are already asked here and saw someone asked for journals that publish replication of experiments, someone else wanted to know what software is used most in economics and I saw a question for examples of code for instrumental variables. All this information is readily available in the wiki, that's why I provided it here and posted the links to show how to find it. I searched for further questions that are already answered in the wiki and found one in the quant stack exchange on replication of a specific study and several questions on datasets in the open data stack exchange that I could answer with the help of the wiki and several users voted that they found the information useful: a, b, c.

I see that this can be seen as a way of self-promotion, and someone already complained about this. I am open about the fact that I founded the wiki, and I am grateful for advice how to deal with this appropriately. I already found the feedback useful that I should not edit answers of others with links to the project because even though it is shown that I edited them it is not directly clear that I as founder of the project added that information and that does not mean the original author of the answer endorses it.

There are two points that I would like to add here and that I need some advice for, a tag for replication related questions and the question how to identify studies that the community thinks should be replicated.

For questions that are related to replication of empirical studies I think it would be nice to have a tag and I created one. So far in the economics stack exchange there is only one question where I clearly see it should fit, the one on journals that publish replications of experiments. For a tag creation one is asked to provide relevant links. The most relevant links in this case as far as I would say are the web project that I founded myself and the literature list that I compiled in a different project. I am however obviously biased and would like to know what you think about it. I have worked on the topic for several years and already posted much of the information I have on the internet, so I thought I should also share it here. Unavoidably that can be seen as a conflict of interest and I would like to have your advice. On the one hand I would like to encourage the stack exchange community to ask their questions on replication, on the other hand I don't want to harm the project (and in the end myself) by attaching too much importance to my own work.

The second point is my question how to find a mechanism to identify studies that should be replicated. In the economics stack it was only partly answered in some comments and then after complaints about self-promotion some voted it down and I removed it. The same question in the quant stack exchange already got a relevant answer, maybe because there I could already refer to an example question someone has asked on a specific study they wanted to replicate. I have not yet understood why such questions are not asked here more often. Strangely I found three such questions on economics studies in the stats stack (d, e, f) and none here, so I asked there, too and it is upvoted there. The point is that I wonder why in psychology voting on such studies works whereas in economics we do not yet have anything alike that is used much. Could you help me to ask this in a way appropriate for this site?

$\endgroup$
  • 3
    $\begingroup$ I'm not sure how to answer your question right now, but I'd thought that I'd mention that I think the Replication Wiki is a great idea. It is something that is very much needed. I'd love to contribute to it some time. $\endgroup$ – jmbejara Feb 11 '16 at 5:04
2
$\begingroup$

I think that if you were to ask the question now it (with the disclaimer) it would not be closed. But then again I did not vote to close it earlier either so perhaps I am wrong.

The wording of the question on the psychology site is very nice:

What are important studies that your field of Psychology gives credence to, but which--as far as you know--have not been replicated in any published follow-up work?

A main component of the question is finding studies that are accepted, "given credence to" by the community. I don't know if there are empirically proven theories that are widely accepted in economics. There seem to be a lot of paradigm shifts in macro and international economics, where theory and data meet. Perhaps by now people are skeptical about all encompassing theories. If this is true than nominating such studies would be difficult.

Perhaps there should be two questions:

  1. Are there important/general empirically testable economic theories that are currently accepted?
  2. In your opinion which of these can and should be replicated?
$\endgroup$
0
$\begingroup$

The thing is, this is a Question and Answer site, not a forum. The best answers get voted to the top, and the one that meets the asker's requirements, gets accepted by the asker as the answer.

That's the system. And it depends on there being a right answer.

Big-list questions such as the one you propose are a poor fit to Q&A sites such as this. They are suitable for forums; this is not a forum.

Just pointing to the existence of some big-list questions elsewhere does not make them alright here.

So the correct way to ask a big-list question here, is to not ask it. Such questions should be closed and deleted. You can help keep this site useful, by flagging or voting to close such questions as too broad.

$\endgroup$
  • $\begingroup$ With "big list question" I assume you mean a question like "please list studies that should be replicated". I agree that this would not be a question to ask here, and I do not ask it here. What I asked in the main forum is how to find a mechanism to identify such studies. That is not a big list question. It may however be a question that should getter be asked in a discussion forum, so a good answer might be to recommend one. Here in meta I asked a meta question, how to add information on a topic on which you already provide information elsewhere online. Do you have an answer to that? $\endgroup$ – Jan Höffler Feb 18 '16 at 11:19
  • $\begingroup$ Ok, I've tried again to understand what it is you're actually asking, and have posted another answer that tries to interpret what you're saying. As it stands, you've got a very long question with a bunch of irrelevant links, and your actual question remains unclear. $\endgroup$ – EnergyNumbers Feb 18 '16 at 11:48
  • $\begingroup$ I agree that that the question is too long and has many aspects. $\endgroup$ – Jan Höffler Feb 18 '16 at 12:19
  • $\begingroup$ @JanHöffler then cut out everything irrelevant (the entire body text), start with your title, and just elucidate on the title as you've done in the comment above ^ $\endgroup$ – EnergyNumbers Feb 19 '16 at 6:34
0
$\begingroup$

It's very unclear what you are asking, because you seem to be asking "how do I write an answer", which is trivial. You type the words into the box. It makes no difference whether you've provided the information elsewhere or not, the answer is the same: type the information into the box, then press "Post Your Answer".

Just posting a link to content elsewhere is unacceptable. We're building a body of content here. This is not a link farm, and must not be used as such. So if you've the answer to a question, put it in an answer here. Type it into the answer box and submit it. Do not merely link to the information elsewhere. Links rot. Tens of thousands of wikis like yours have been set up and abandoned over the last twenty years, and links to them just suffer link rot and become useless. We're trying to build a library of content here that will have longevity.

$\endgroup$
  • $\begingroup$ In some questions people ask for content that cannot be provided here like "how can I find data on...". A short explanation with a link to a list of such datasets then feels appropriate to me. And no, there is no other wiki with the specific information that we collected and that helped me to answer several questions here. $\endgroup$ – Jan Höffler Feb 18 '16 at 12:27
  • $\begingroup$ Then post the list of datasets in the answer. Merely linking to a list elsewhere is useless, and is always going to look like spamming. $\endgroup$ – EnergyNumbers Feb 19 '16 at 6:33
  • $\begingroup$ In the case of the instrumental variable datasets that would have meant a full screen of datasets. I don't think anyone would want that here, and it is common practice to point to external websites where the information asked for can be found, of course together with a short explanation. $\endgroup$ – Jan Höffler May 8 '16 at 16:44

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .