10
$\begingroup$

I say "5 days" because it has been 5 24-hour periods after I received notification from SE to join the private Beta.

I have no experience from other private Betas, and certainly Q&A subjects differ in popularity (which affects also their private Beta experience). But as in Public Beta, there must be some accumulated experience as to whether the statistics of the Private Beta indicate a site that has better or worse options for a successful Public Beta and then on to eternity.

If there are users here that have participated in other private betas (and even better, in the private Beta of the previous failed attempt for the Economics site), or also, SE community managers that have a wider picture, it would be useful to share some thoughts: I am not asking for an oracle ("you will/you will not succeed"), just an indication as to where we stand at the moment.

Just please, marketing-speak to the effect "it is too early to tell", "keep on trying" etc is not suitable here -we are economists. I am specifically asking "is it satisfactory up to know", without saying that the fate has been sealed.

Me, I will provide the data, and some comments related to benchmarks for Public Beta (of course by the time you will read this, the running data may be a bit different).

A) Number of questions asked (using the time stamp of the site) \begin{array}{| r | r | } \hline \text {day}& \text {Q's} \\ \hline \hline \text {Nov-18} & \text {13} \\ \hline \text {Nov-19} & \text {38} \\ \hline \text {Nov-20} & \text {11} \\ \hline \text {Nov-21} & \text {16} \\ \hline \text {Nov-22} & \text {7} \\ \hline \text {Nov-23} & \text {11} \\ \hline \text {Total Q} & \text {96} \\ \hline \hline \end{array}

"Healthy Public Beta" is "15 questions per day".

B) Answers \begin{array}{| r | r | } \hline \text {# of A's}& \text {# of Q's} \\ \hline \hline \text {Unanswered on Hold} & \text {8} \\ \hline \text {Unanswered active} & \text {14} \\ \hline \text {1} & \text {36} \\ \hline \text {2} & \text {19} \\ \hline \text {3} & \text {11} \\ \hline \text {4} & \text {4} \\ \hline \text {5} & \text {2} \\ \hline \text {6} & \text {1} \\ \hline \text {10} & \text {1} \\ \hline \text {Total Q} & \text {96} \\ \hline \hline \end{array}

Two statistics here:
a) Questions Answered (excluding the "On hold Unanswered") is $74/88 = 84$%. Public Beta benchmark is $90$%.
b) Weighted "Answer ratio" = $149/96 = 1.55$. The benchmark for Public Beta is $2.5$.

C) Accepted Answers (Green checks) \begin{array}{| r | r | r| r|} \hline \text {# of A's}& \text {with Green} & \text {Without Green}& \text {Totals}\\ \hline \hline \text {1 answer} & \text {8} & \text {28} & \text {36} \\ \hline \text {2+ answer} & \text {16} & \text {22} & \text {38}\\ \hline \hline \text {Totals } & \text {24} & \text {48} & \text {74}\\ \hline \hline \end{array}

There is no relevant benchmark value for Public Beta. It is obvious that having 2 or more answers increases the chance of the OP finding something useful. Still, 22 questions with $2+$ answers remain without an accepted answer. Just in case OPs are neglecting to act on the matter, please do.

D) Users

In "No rep points earned" are those users with rep $101$ or $1$.

\begin{array}{| r | r | } \hline \text {rep points Status}& \text {# of Users} \\ \hline \hline \text {rep points earned} & \text {64} \\ \hline \text {NO rep points earned} & \text {154} \\ \hline \text {Total Users} & \text {218} \\ \hline \hline \end{array}

This is an indication of how many people are... "actively active" on the site. Of course from the 154 users with no rep earned here, there may be those that silently browse the site. Still, the "actively active" users are the engine of growth -let's just not put our hopes in the existence of a Solow residual.

$\endgroup$
13
  • 4
    $\begingroup$ Nice summary. Of all the metrics, I don't care at all about the accepted answers. I feel that leaving them unchecked invites more for new answers, and none of them was so satisfactory answered that no new answer could dominate them. I think the biggest lack are in questions, so I'll try to work on that again soonishly. $\endgroup$
    – FooBar
    Nov 23, 2014 at 21:09
  • $\begingroup$ @FooBar Thanks. Regarding the Green check, I am not so much seeing it as an "incentive mechanism" to attract answerers, but rather, as a possible signal to newcomers as to whether they will find "useful answers" here or not to their questions. But indeed, it is the least important metric. $\endgroup$ Nov 23, 2014 at 21:13
  • $\begingroup$ Thanks for the summary. Just curious, how did you find this data? $\endgroup$
    – jmbejara
    Nov 26, 2014 at 2:45
  • 1
    $\begingroup$ @jmbejara The very old fashioned way. Scrolled the list of questions, recording date of posting, number of answers and Green checks, and also scrolled the list of users and counting the total as well as those with the rep points mentioned. With 96 questions and 218 users, it wasn't worth searching the various queries available on SE instead. $\endgroup$ Nov 26, 2014 at 2:48
  • $\begingroup$ Oh, ok. Yeah. Thanks for doing that!! $\endgroup$
    – jmbejara
    Nov 26, 2014 at 3:06
  • $\begingroup$ This question appears to be off-topic because it is about the private beta. Similar questions would be welcome about the public beta now. $\endgroup$ Dec 10, 2014 at 15:50
  • $\begingroup$ @JasonNichols How can the history of the site be off-topic? You should find some other way to close this question. Putting it "on hold" is inaccurate, and worse, it creates the wrong impressions (about the OP, no less), since, whether we want it or not, the "on hold" in the title of the question says "this question is flawed" -and this question is not flawed. $\endgroup$ Dec 10, 2014 at 16:58
  • 1
    $\begingroup$ I switched it to locked for historical significance, is that better? $\endgroup$ Dec 10, 2014 at 17:58
  • $\begingroup$ honestly, feeling out the controls here, so please provide feedback. $\endgroup$ Dec 10, 2014 at 17:58
  • 1
    $\begingroup$ @JasonNichols I absolutely agree with you that a) This question should be closed and remain "frozen" as a trace of our beginnings and b) that analogous questions regarding the public beta would be useful. So as for a) "Locked for historical significance" -it may even be a first for the SE world, so absolutely, yes, go ahead. $\endgroup$ Dec 10, 2014 at 19:41
  • 1
    $\begingroup$ @JasonNichols I think historical lock is fine for all the private beta questions. If in doubt about how to use the mod tools, just ask in the mod-only teachers' lounge. NB you will have to make your Economics account into the "parent user" of your chat account, to get access. At the moment, your Area51 account is the parent user of your chat account - you can change this on your chat profile. $\endgroup$
    – 410 gone
    Dec 10, 2014 at 20:37
  • $\begingroup$ @JasonNichols This thread appears to remain unlocked. What happened to the history laurels?! $\endgroup$ Dec 13, 2014 at 22:53
  • $\begingroup$ Odd, I swear I locked it. I just locked it again, @AlecosPapadopoulos. Let me know if it gets unlocked again, please. $\endgroup$ Dec 15, 2014 at 1:20

1 Answer 1

5
$\begingroup$

I wouldn't worry too much about those Area 51 benchmarks - they were set out years ago, and haven't been updated with what's been learnt in the intervening few years.

users

64 active users isn't a huge amount: we could do with sending out more private beta invitations (see the box on the right, in the main site, labelled "Invite Fellow Experts".)

enter image description here

expertise

It's much easier to attract non-expert questions later, once we're known as a community of experts, than it is to attract expert answerers later, if the site has the appearance of being occupied by a bunch of novices.

Expert questions are getting posted, and generally getting answered, and that's great. It's OK to have some unanswered questions too: one of the grounds for sending a private beta out, is to link to an unanswered question, and ask the invitee if they're able to answer it; an interesting, challenging question can be a great way to hook new experts in.

We haven't done very well at getting the non-expert stuff closed and removed, which is a problem. We have some users that persist in posting non-expert content. I genuinely don't believe their intentions are of sabotage; perhaps they're just trying to hoover easy rep by asking foundational questions; or they're just so excited that they can't wait; but the result is detrimental.

activity in meta and chat

This meta is reasonably active, and that's a very good thing. Chat is active too - please do pop over and introduce yourself if you haven't already.

returning contributors

The really important metric, is do our experts contribute, and keep returning. There isn't a simple measure of that - it's more a question of looking at the content being generated, and looking at who's generated it.

patience

One of the SE Inc Community Team will be along shortly to review things, and give us some guidance on whether we're close to going to public beta or not.

I've seen private betas extended by a week or two.

It's not wise to hurry it. While we're in private beta, we've all got more privileges. It's easy to create tags. We can build a core community. We've got a protected space where we can invite experts to give questions and answers. Don't be in too much of a hurry to leave that behind, because if this site doesn't take off, the next opportunity for an Economics.SE private beta could be years away ...

$\endgroup$
7
  • $\begingroup$ So the private Beta period can be extended? That's certainly a good thing - I was under the impression that private Betas last only for 7 days, and that's it. $\endgroup$ Nov 23, 2014 at 21:37
  • $\begingroup$ Yes, I've seen private betas extended by a week or two. More than that, and it starts to look like a site won't make it out of private beta ... $\endgroup$
    – 410 gone
    Nov 23, 2014 at 21:38
  • $\begingroup$ So we got, let's say at most 10 days to improve things. 10 days that changed the -ok, not the world. $\endgroup$ Nov 23, 2014 at 21:39
  • $\begingroup$ The Powers That Be may come by and love what they see, and send us public straight away. They've got WAY more experience than me of private betas - probably about 20-40 times as much - and it's their job to know this stuff. I expect that they also have access to loads of dev info and maybe customised internal stats. We'll find out what the word from the owners is in a day or three, I expect. $\endgroup$
    – 410 gone
    Nov 23, 2014 at 21:44
  • 1
    $\begingroup$ In Experience We Trust. $\endgroup$ Nov 23, 2014 at 21:46
  • $\begingroup$ Yup, I trust their judgement. I'm here because I consider them to be the experts at running Q&A sites. (oh, and the two weeks I mentioned, was an additional two weeks on top of the nominal minimum of a 1-week private beta. So, I reckon we've got up to 16 more days to go public, but hopefully we'll do it in two days) $\endgroup$
    – 410 gone
    Nov 23, 2014 at 21:46
  • $\begingroup$ Just to note that apparently it's a public holiday in the USA, where most of the Community Team are sited, so any announcement about the arrival of, or the delay to, public beta status may not come until the start of next week. $\endgroup$
    – 410 gone
    Nov 27, 2014 at 14:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .