Yes–the tradeoff is tricky here. Getting the direct conversation with testers is energizing, but it can eat time in a serious way.
This would actually make a great #choice.
2 Likes
Just out of curiosity did doing a closed beta at the end of ZE wind up being more helpful to you vs. the open one you were doing before?
I really don’t think author needs to be part of conversations daily or anything. But maybe one hour a week or something like that Where betas could asking stuff or just talking could be a good middle ground. Or just and update. in that closed thread like version x it changes Y character chapter 4.
That alone could be better. I know time is scarce, so no more than that is necessary. Also a thread lounge to talking between betas could be great.
And a pre form Thanks for participating could be TONS BETTER THAN SILENT WALL… Now I least know how I suppose to ended the betas to apply for other beta. I have lost almost all this year just for that detail. lol. That shows how terrible communication is.
3 Likes
would it not be helpful to highlight to the beta tester issues that have been found. It’s no good send a email list 37 different errors within a game when 36 of them have already been found. It not only wastes the beta testers time but the writers time as well because he/she will have to read the email and check if the error has been addressed.
If the writer wants the beta testers help in making his game better then surely they can find the time to ensure that the beta testers are kept in the loop, even if it is only with standard replies. Apart from anything else it’s just being polite.
The other thing I have just thought of (while having a bath) is do the writer know who is beta testing there game?
3 Likes
Yeah, that’s why a post with updates and known errors could shine. And that doesn’t need author interaction with betas or anything. Just a list with known errors and updates.
2 Likes
I doubt I will ever do another open beta. I receive tons of feedback that boils down to requests for customizations (like @Mary_Duffy mentioned in the other thread). I also received reviews stating they were upset there was not much more content in the published game than the beta (50k words isn’t much, I guess).
I still run a core closed beta group while I write and will run short-term betas at milestones. Otherwise, open betas are a thing of the past.
11 Likes
Correct me if I am wrong (which I normally am) but in beta testing arn’t the betas looking for errors in code / game play and spelling mistakes rather than giving there opinion on how the character should behave or this should happen in that scene etc?
1 Like
Here is what Jason prefaced my beta test with–I didn’t check to see if this is standard:
"I’m looking for “high level” and “low level” feedback. Not mid-level feedback.
"Low-level = typos and continuity errors. A continuity error is when a character’s gender flips, or someone comes back from the dead, or you run into a plotline that just doesn’t make sense (because it’s probably a coding error).
“'High level” feedback has to do with things like plot, pacing, and characters. “Scene A didn’t work for me because x, y, and z,” is useful feedback. “B character was entirely unsympathetic, because u, w, and v,” is also useful feedback."
I’m not sure where “how a character should behave” falls on the mid-level/high-level spectrum. Probably it has to do with how broad the feedback is.
That’s awful. 
I’m really sorry to read that.
I hate looking for spelling mistakes and generally avoid doing so. Yes I’m the annoying person who will say “there are multiple spelling mistakes” and then not point them out, but I figure they can ask someone else to point them out, or look for them themselves. I’m not going to do the part of beta-testing I hate the most.
Code errors are generally picked up by the quicktest tools (although there are sometimes errors still in the game.) I do make note of any I come across, and read the code once I’ve finished playing to pick up anymore errors I see in that. There’s a few common errors that the testing tools won’t pick up, but it’s easy enough to spot as a beta-tester. Pronoun problems is one of the most common of those.
I keep an eye out for any unexpected stat changes too. If stat changes don’t seem to make sense, or if the stat bars are moving in the wrong direction. Also missed brackets on variables.
I generally gave feedback on everything I could think of. How consistent characters acted, if there seemed to be minor choices missing. I’ll usually comment on inclusivity stuff as well. That’s me though.
1 Like
During our Monday call, we resolved to do the following:
- When a game goes to copyedit, we’ll send an “the beta is now closed; thank you for participating” email.
- We’ll encourage authors to respond to beta feedback: either ask clarifying questions, or send thank yous as they see fit.
- We’ll invite authors to a closed thread where they can discuss a game with beta testers that have submitted feedback. We’ll see how that goes.
- We’ll encourage authors to submit a (brief) changelog with beta updates.
@Mara I’m sorry you misunderstood the “one beta at a time” rule. As soon as you submit feedback on a game, you’re free to apply to other betas. You don’t have to wait for a beta to close. You just have to submit feedback. It’s only if you apply to two betas simultaneously, or apply for a beta when you haven’t provided feedback on a previous one that is still open that you are denied. And, as long as a beta is open, you’re free to keep playing and providing more feedback, even if you’ve joined a new one.
17 Likes
Jim, I have many fond memories of your original open beta. So it’s sad to see you moving away from them. Nevertheless I completely understand.
3 Likes
My attempt to communicate succinctly failed; I apologize. I’ll attempt to clarify my statement and leave it at that.
“schedule” in this regard, is not a work schedule given to direct the tester but rather a communication of “projected deadlines and milestones to be met and achieved”; the purpose of which is to help the tester focus on areas of which the author/developer is currently working on. Feedback given after the work has been finalized is an example of what this communication tool is geared towards preventing.
“testing needs” is not CoG directing the tester on “how to play the game” which in of itself is both counter-productive and results in skewed testing. Rather, it is the communication to the tester on the order of the “We need high level feedback and low level feedback”
The author/developer can often get specific feedback on areas of particular concern by just stating what is of concern. Something akin to: “I am having particular trouble with this particular scene; how is it working for you guys? Are there too many repetitions, or does the repeating make sense? Is the lore provided here in context working or is it overwhelming you as a reader?”
This focus on communication would increase the efficiency and the quality of the testing and testers themselves would not feel like they were spinning their wheels in the same rut over and over. More importantly, this would give the author better feedback which was more actionable and on point to their specific needs.
Thank you for taking the time to listening to me - I hope the process and experience will be more fun and productive for everyone involved.
4 Likes
“projected deadlines and milestones to be met and achieved” is exactly what I mean by a “schedule.” We can’t give deadlines or milestones to volunteers.
I don’t know what you mean by “how to play the game.” But asking for “specific feedback on areas of particular concern” is exactly what I mean by “direction.”
You examples are precisely what I mean when I say “we can’t give schedules or direction to volunteers,” because that would be violating (my understanding of) labor law.
2 Likes
Well In Spanish law that was no contemplate so I didn’t know That was a issue in America. Here if you firm or even accept a basic form you could basically do the same as a paid professional except the consequences of the break it.
However a way to direct us without affecting that law is simply a Update description that’s perfectly legal. For what I have check on internet .
Something like.
12/16/16 Update 2.12 Changed Fred dialogue adding new content in x path.Added new ending.
That’s legal and we as testers could totally freely say oh maybe I should check out that… Or not… So you have not violate any law. You didn’t change my neutrality.
After playing MUDs for 20 years and being a part of various processes in both player & admin roles I’d say you have your solution.
Create a private forum thread with topics inside of it the list of games currently being tested.
Then have each thread password protected and only accessible to the author and beta testers.
That allows an inclusive testing process rather then a top down structure where testers feel their feed back is going into a black hole akin to letters to congress.
I’m new to Choicescript but for the last year I’ve bought and played nearly every choice game on the market that appealed to me.
If testing feels like work then you will have to add appropriate reinforcement in order for people to continue with the behavior. Free access to a few games doesn’t seem to be cutting it, but access and feeling valued certainly will.
1 Like
Belatedly: I just wanted to say how much I value the feedback I’ve had over the years on the XoR forum thread.
It would be a much weaker game without the steady flow of feedback there, and I intend to keep writing future games in the series with ongoing testing on an open forum thread.
I can understand that it won’t be for all authors; but I’d much rather have feedback at a stage when I can still change the game to accommodate it than only offer it up for testing once it’s pragmatically too late to deal with significant changes.
13 Likes
So, I was just talking a little about this process with some people, and I had an idea which I was wondering if it’d be useful 
When I decide if I want to buy a CoG, I always use the preview of the first few chapters to see if it interests me. I’m wondering if it might be useful to include a similar preview for official beta tests as well. I think I would be more likely to join a beta test if I could see the the writing of the game intrigues me, rather than going off of the description. I’d imagine that many other potential testers might feel similarly, and I’d think it could be a way to garner more enthusiastic and specifically-interested testers as well.
I can also understand if some authors might not want a preview of their early draft available to anyone, but if that’s the case, I’d suppose it could be the author’s decision, yes?
(Sorry if I shouldn’t be bumping such an old thread, but I thought it’d be preferable to starting a new one, yes? )
5 Likes
I think the issue with having a preview for beta is “early feedback.”
I mean, you have those previews posted. Ppl can comment and criticize those previews while not giving feedback for the story as a whole.
But ofc, the solution for that can be as simple as “ignore the early feedback.”
1 Like
I’m not sure I understand why that would necessarily be a negative I’d think that people who’d comment on the preview but not the full game wouldn’t give any feedback at all if no preview is provided, so I don’t think CoG would be losing any feedback
2 Likes
That’s essentially already covered. Join the beta and, if you stop reading part way through, provide feedback on what you did read.
1 Like