All About Beta Testing

General
22

@Havenstone, Wow! Thank you. Those links are awesome. It’s clear that I’m diving into an established conversation here. It’s especially helpful that you’ve linked to @Lucid as well, it’s great to see my sentiments echoed by an established HG author.

Thanks @Szaal, that’s definitely the approach I’m familiar with.

Hahaha, @Jacic, I might have shot myself in the foot there. I’m one of the people who introduced that rule. We based it off the Brandon Sanderson lectures available on YouTube. The idea isn’t that you discount reader feedback, but more that you take everything they say with a grain of salt. If you keep getting the response that a particular scene is boring–you should add in more action–you go away with the understanding that ‘people don’t like this scene. It could be boring (or there could be some other problem at the root of that feedback), and it may be that adding some action would help (but I should also consider other options).’

It’s more or less the same process, but we kind of mentally reduce feedback to a thumbs up or a thumbs down. ‘Characterisation? Thumbs up! Pacing? Thumbs doooown.’ That way we still get targeted feedback, but there’s not struggle to retain the creative vision in a sea of 'you should’s.

So I’m thinking I might chuck in some ‘tester friendly’ infrastructure (frequent save points comes to mind) and then put it up for the open beta in a day or so. It’s a very experimental piece, and definitely not the usual CoG style, so I’ll be interested to see what y’all think. (Sorry for the clunky Americanism. I’m an Aussie.)

5 Likes
23

I think authors has to switch the perspective from conventional novels. In a novel is your characters and your vision. Yes, you could listen feedback. But in inner stance is your story. Here there is the main elephant in the room interactivity aka choice. Story has to be a team between a player (not reader, player that’s a key to understanding the situation) If you don’t give enough room to player agency and decision. You could be done a literary masterpiece and still a terrible Cog , a souless cog. Because what makes Cog a cog are Choices and could creating something with the author.
So if many people asking why I can’t choosing here in that scene or why my character has to be with settled personality. That normally is good idea to hear. Because is what mayor part of people expect. If they can’t choose certain amount of things. Casuals never will bought and read past the demo. Anyway , just my two cents

1 Like
24

Glad it was helpful. :slight_smile: Let me also pull out from one of those threads a note by another of our most prominent HG authors:

By contrast with Lucid (and Jim’s future intentions), we of course have Cataphrak, whose “open betas” produce threads that are the stuff of legend. Rambling, almost-impossible-to-keep-up-with, forum-breaking legend.

https://forum.choiceofgames.com/t/guns-of-infinity/2656/39896

And I just want to give another shout out to my writing community on the XoR thread, who over almost four years of open alpha/beta testing have strengthened my game immeasurably – even though I’ve also taken many things with a healthy grain of salt.

11 Likes
25

You hear me out lol, Even of your first intention was launch me out of your face. But I think you did a great job dealing with feedback. And with me. I was really rude back then, but I think I was in the right track. Is something lot of authors forget, A reader or tester could be an ass , and still giving you the best advice for your own work. People look more at forms, that what they really are saying.

5 Likes
26

No problems, so am I lol

3 Likes
27

Two questions for you:

  1. Have you had choice game fans test your game? A reading group/workshop is a different set of readers.
  2. Do you have a following outside of this community?

For a game to be " good", it needs to be tested with scrutiny. Not just spelling/grammar. I’m talking continuity, impact of choices, player agency – all that good stuff that makes choice games the bee’s knees.

Part of the requirement for open testing is to gather feedback. Most new game writers don’t have a dedicated group of choice-game fans to call on to test their work. Since there are numerous WIPs at any one time, there is competition for testers. Therefore, open betas provide the most exposure to testers.

People like me, Cataphrak, Lucid, and Allen Gies all have success in the Hosted Games arena and we all did open betas until we assembled our Avengers of testing teams. That’s not to say our way is right but it’s right.

17 Likes
28

The catch of having groups like that (where you know each other and are friends, I guess) is that they will tone down any criticism they have. What I mean is, if there’s something that sucks in your story (hypothetically speaking), your writing group may be more forgiving/lenient, but that won’t do you any favors once your game is out in the market. The good thing about having testers who don’t know you is that they will be 100% honest and tell you what they see wrong without watering it down.

8 Likes
29

Also, a writing group might not be testing for the same things, especially if it’s not an interactive fiction-specific group. Plus, beta testing on the forums lets our diverse community take a look at your game and see if it’s reaching as wide an audience as it could; I know I and several others usually push pretty hard for asexual/aromantic and nonbinary options, which are fairly simple to implement, but can make a big difference in who wants to buy your game.

7 Likes
30

I’ll help you out. I haven’t put my game out yet, so take it with a grain of salt.[quote=“Doug, post:1, topic:26284”]
2: I’m confused–doesn’t this poach the market?
[/quote]

As many people commented already, not really. The forum is the smaller slice of the pie when it comes to the readership you might get. Besides, having a beta test with seven or eight people hardly means the whole forum. Launching a game (a Hosted Game, I should add) that is well-received might get you some more purchases from here. And I wouldn’t count on people just playing the demo forever and not buying it, from my experience in the Guns of Infinity thread, the smashing amount of peple who tested it also bought it. Maybe it’s because it’s a series, but I’d disagree.

And if you ever have another game on your mind, you’ll already have some people willing to help you or who’d be intereste din giving your works a try (a few Hosted authors, like MultipleChoice and Cataphrak, have developed a fanbase-esque).

I think it’s mostly bug catching and language correction for spelling mistakes.

I tend to treat to see betas this way: go for it. Nine pairs of eyes work better than one to get a mood or a scene right. You can also get people with different tastes, from different walks of life to help you out, and I believe that’ll enrich your game, regardless of opnions on it.

2 Likes
31

The answer to #1 is: sort of. I’ve had IF fans play the game, and one of them was familiar with CoG. It’s probably worth noting that I’m talking about ‘playing’ (or ‘reading’) as opposed to testing. The game that I’d like to submit was written as an experimental creative piece to complement my honours thesis. It’s written in choicescript, but has little in common with most CS games. In particular, the total number of choices offered in the game is very few. As a result of that, I’ve been able to create stable builds for each round of beta playing. I’ve had readers come to me with continuity errors in the past, but that’s always been when I jumped a production cycle and gave them a not-yet-proofread draft.

The answer to #2 is ‘oh man, that’d be awesome.’ Most of what I’ve written in the last year or so has been to test genre or digital literacy theories. They’re usually half-formed skeletons of projects that get abandoned as soon as I’ve learned what I needed. This particular piece needed to be assessed as a creative work as well as an experimental piece, so it got to live to see completion.

Mmmm, that’s one of the challenges we’ve struggled with. We’re a fairly large group (I run a student society at my university), so you can often find someone you don’t know pretty well to give you some honest feedback. I personally like the James Portnow manoeuvre: ‘hey, I’m playing this game that a friend of mine made. I don’t think it’s really good, but I’d like a second opinion just to be sure. Can you tell me what you think?’

Of course you can only use that a few times until people start catching on, but it’s good while it lasts…

@hallofmirrors That’s certainly true, and I’m definitely interested in listening to that kind of feedback. Unfortunately I’m not sure if I’ll be able to accommodate any substantive changes. I’ve started my next creative project, and I’m kind of going by the finished, not perfect/fail faster mantra. I’m definitely going to polish (line and copy edit), but I want to avoid any substantive edits. I could probably create an asexual option fairly easily, but most other changes will probably just be put down as tips for next time.[quote=“Vertigo, post:30, topic:26284”]
I tend to treat to see betas this way: go for it. Nine pairs of eyes work better than one to get a mood or a scene right. You can also get people with different tastes, from different walks of life to help you out, and I believe that’ll enrich your game, regardless of opnions on it.
[/quote]

I can certainly understand that logic. It’s a very different approach to the one I’m used to taking, and I’ll be interested in seeing how it goes.

Barring any unexpected problems (I’m working on a small project due this evening, and I’m not sure if my last wave of edits left my piece in a stable build), my draft should be up sometime today (AEST).

2 Likes
32

This is proof enough to me the Open Beta is needed for your submission. Good luck.

3 Likes
33

@Doug, just a quick note of clarification that the open beta doesn’t have to be quite as open as Cataphrak’s and mine have been.

Quoting @jasonstevanhill here, the open beta COG requires for Hosted Games does not mean “you have to post it so any rando can play without contacting the author.” It means at a minimum that the beta has to be publicly announced on the forum, and forum members who express interest are allowed to join a private message thread where they’re given access to your draft.

That will give you a better sense of how many testers are getting free access to your beta and how many of them are actually giving feedback.

11 Likes
34

@Havenstone Oh! That’s good to hear, but the cat’s a little out of bag already… I’ll have a think, and look at the kind of responses I get. It may be that it’s more productive to have a closed beta.

Thanks for letting me know!

35

Well, at least ppl will know about the work and start giving the feedback (although I’m yet to click your thread :stuck_out_tongue: )

1 Like
36

To follow up to Havenstone, if you say “I did a beta!” but you mean, “my three best friends and my mom played it and they all told me it was awesome!” that doesn’t mean it was given a beta test. We want to see the community given a real opportunity to engage an author in a dialogue.

In particular, when we at Hosted Games are trying to decide if we should expend money/capital on a game (putting it on Steam, putting it in a mailer, tweeting about it), Rachel gives her input, and then we look at the beta thread to see how the community has responded. Thus, whether the game is freely accessible or not, letting us see the feedback of the beta testers helps us decide how we market it.

8 Likes
37

Say that someone has read a game and hypothetically speaking not talking about this project per se, believe it is far lower anything resembling a quality bar. If that member of forum say so probably would get flag or censorship for saying so. Could that negative feedback seen directly to Cog staff because I think author doesn’t gain from it. But if quality is terrible People should have some place to say it.

38

Um I just wanted to ask how long estimately would a game be released after beta testing was announced done? I’ll close this thread if the question is not allowed…

39

It depends on how long the publishing queue is. CoG has only a handful of personnel and a lot of CoG and HG titles to plow through so no estimate could be given.

2 Likes
40

Refer to here for the queue for current year.

2 Likes
41

It depends. Some take a few months, some take a year. It depends on how busy the staff are, how many games are in the queue, if they need anything extra done (like copyedit) and how long it takes authors to get the required paperwork back to COG.

2 Likes