Open vs closed betas. Thoughts?

[Eiwynn]

There are good points and bad to both open and closed betas. Don’t forget you can also have multiple phases of testing.

Here is one of the strongest proponents speaking of how much open betas help them:

@Havenstone

Belatedly: I just wanted to say how much I value the feedback I’ve had over the years on the XoR forum thread.

It would be a much weaker game without the steady flow of feedback there, and I intend to keep writing future games in the series with ongoing testing on an open forum thread.

I can understand that it won’t be for all authors; but I’d much rather have feedback at a stage when I can still change the game to accommodate it than only offer it up for testing once it’s pragmatically too late to deal with significant changes.

Now I will state what I once said as well:

@Eiwynn

A multi-phase testing period with different goals during each phase is important. Choosing a testing team is hard as well. That is why developing relationship with testers is a very important but very underappreciated task that authors and developers need to do.

I’ve tested games for a very long time. I’ve known, as friends, developers for years. When a developer starts a project they know me, know what I am good at testing and what I’m not very good at testing. So, when someone I know says: “Hey, I have this project I’m working on,” both of us know what the other is bringing to the table.

This may not be viable at this time, but the WiP forum is a very valuable screening tool -if it is utilized by the author. You’re absolutely correct, time, effort and energy is going to be expended by everyone to get that done - but if you want a product tested right - it is going to need to be done.

I know some people are gun-shy about public or group feedback and if you provide an email or PM source where they can give the private feedback then you can have them in your group testing as well.

As far as wanting the same people - maybe. It depends on everyone’s strengths. Someone might be very good at picking up continuity errors and logical break-downs and so you want them in on the testing early. Another may be really good at misspellings and grammar and so bringing them in later might be a good idea.

Also, if your story is a historical setting and deep in politics or military jumk, then observe the forum users in Cataphrak or Goshman’s threads. Both of those authors have a hard-core following that would be wonderful for your project… if you want more fanciful, bright 18th century type of story telling, look at jeantown or Fiogan’s threads… their fans love the good story set in mystery and mystic.

There are also special circumstances - as with Mara’s current WiP - the end-result is I’d prefer a closed controlled beta process if I were her.

Another special circumstance might be the contest offered here. Do you really want everyone seeing your entry before you submit it?

So, in conclusion there is no hard-and-fast rule on “better” or “worse” only what fits your needs as a developer and author.