All About Beta Testing

First of all, people seem to have missed a critical detail for you. If the game is written in choice script it can only be published under Hosted Games. All other “elsewhere” options must be free of charge unless you come to an arrangement with Choice of Games.

On point 3: There are some atrociously bad games on the hosted game brand - try out some of the earlier games on the hosted games page of the main site and you’ll see why public testing is a hard requirement. That said if you can afford professional proof-reading/editing knock yourself out. I think you’d have to be spectacularly lucky to recoup your costs on that though.

1 Like

You know how much think Translation or edit cost? Cost more of the profit I could make in ten years if lucky my friends. Around 80,000 words and I have already more is 6,000 words edition. A translation services could be double that and not a good one. You are saying 18,000 dollars in something that not going produce more than 2,000 in my end. I am broke. And no one work in base of a percentage of an author who never produced anything. And I don’t want writing in Spanish. Why I should? I hardly doubt I will learning English writing on other language. My problems with beta is not being shy at all. It was the way trolls and no one else try it.

2 Likes

Thanks @Eiwynn, I’ll check out Diamant Rose. (And I appreciate your perspective on the beta testing side of things.)

Don’t worry, @LordOfLA I know about that requirement. It’s extremely unlikely that I’d end up taking the game elsewhere (though ultimately up to the community and CoG, I suppose), but if I were to do so, I’d port the game to another language. I have yet to find any atrocious games, but I’ll admit that I’ve stayed on the established side of things (Broadsides, Robots, Everything Max Gladstone). I’d be curious to look at the earlier hosted gamebooks as you suggest to get some perspective.

2 Likes

I’m surprised they teach that to be honest. (That’s not what I’ve been taught which is more along the lines of have your own ideas of how the story is to progress but listen to feedback and decide whether to act on it. It’s not to say you shouldn’t have your own work and style, but there’s nothing wrong with listening to what people want. You’re talking about making it into a commercial product after all rather than just a personal piece. If the advice doesn’t fit your story line, you’re under no compulsion to fix it in that exact way, but you still know it needs to be changed. That’s especially true if you have multiple people telling you the same thing, you should definitely listen. If that’s the case you need to justify why you can ignore the advice rather than the other way around. It can sometimes also give you some great ideas for story lines and character interactions. Sometimes people will see my characters or the MC in quite a different but still very valid light to me.

I find IF to be quite different to standard novels where you can get away with less feedback in my opinion because they’re your characters, acting how you decide the story is going to play out. IF is interactive, people want to be able to have a particular amount of freedom which testers can pick up for you. For example you may have decided the story needs to go in a particular direction but a lot of readers hate that words are being put into mouth so to speak. A few extra choices or a bit of rewording to show why something has to be that way will result in a much happier audience long term.

Anyway, good luck with it :slight_smile:

5 Likes

TL;DR @Doug
Listen to any comments, suggestions, and criticisms. But you as the writer/developer still hold the wheel to decide how you want your story goes.

1 Like

Just wanted to affirm that your concerns are commonsense ones, and this isn’t the first time they’ve been raised. Which is probably why so many people are quick to whip out the things we’ve said before, especially me, Eric, and Eiwynn. :slight_smile:

We had a similar conversation a couple of years ago, when the requirement was still new and the open/closed issue more negotiable:

And since then:

There’s some on it here:

This thread had some derails, but here’s a post from an established HG author who doesn’t like open beta tests:

And finally, regarding CoG “crowdsourcing” HG proofreading and editing, here’s a recent post from Jason that I believe @Lys may have had in mind above:

8 Likes

@Havenstone, Wow! Thank you. Those links are awesome. It’s clear that I’m diving into an established conversation here. It’s especially helpful that you’ve linked to @Lucid as well, it’s great to see my sentiments echoed by an established HG author.

Thanks @Szaal, that’s definitely the approach I’m familiar with.

Hahaha, @Jacic, I might have shot myself in the foot there. I’m one of the people who introduced that rule. We based it off the Brandon Sanderson lectures available on YouTube. The idea isn’t that you discount reader feedback, but more that you take everything they say with a grain of salt. If you keep getting the response that a particular scene is boring–you should add in more action–you go away with the understanding that ‘people don’t like this scene. It could be boring (or there could be some other problem at the root of that feedback), and it may be that adding some action would help (but I should also consider other options).’

It’s more or less the same process, but we kind of mentally reduce feedback to a thumbs up or a thumbs down. ‘Characterisation? Thumbs up! Pacing? Thumbs doooown.’ That way we still get targeted feedback, but there’s not struggle to retain the creative vision in a sea of 'you should’s.

So I’m thinking I might chuck in some ‘tester friendly’ infrastructure (frequent save points comes to mind) and then put it up for the open beta in a day or so. It’s a very experimental piece, and definitely not the usual CoG style, so I’ll be interested to see what y’all think. (Sorry for the clunky Americanism. I’m an Aussie.)

5 Likes

I think authors has to switch the perspective from conventional novels. In a novel is your characters and your vision. Yes, you could listen feedback. But in inner stance is your story. Here there is the main elephant in the room interactivity aka choice. Story has to be a team between a player (not reader, player that’s a key to understanding the situation) If you don’t give enough room to player agency and decision. You could be done a literary masterpiece and still a terrible Cog , a souless cog. Because what makes Cog a cog are Choices and could creating something with the author.
So if many people asking why I can’t choosing here in that scene or why my character has to be with settled personality. That normally is good idea to hear. Because is what mayor part of people expect. If they can’t choose certain amount of things. Casuals never will bought and read past the demo. Anyway , just my two cents

1 Like

Glad it was helpful. :slight_smile: Let me also pull out from one of those threads a note by another of our most prominent HG authors:

By contrast with Lucid (and Jim’s future intentions), we of course have Cataphrak, whose “open betas” produce threads that are the stuff of legend. Rambling, almost-impossible-to-keep-up-with, forum-breaking legend.

And I just want to give another shout out to my writing community on the XoR thread, who over almost four years of open alpha/beta testing have strengthened my game immeasurably – even though I’ve also taken many things with a healthy grain of salt.

11 Likes

You hear me out lol, Even of your first intention was launch me out of your face. But I think you did a great job dealing with feedback. And with me. I was really rude back then, but I think I was in the right track. Is something lot of authors forget, A reader or tester could be an ass , and still giving you the best advice for your own work. People look more at forms, that what they really are saying.

5 Likes

No problems, so am I lol

3 Likes

Two questions for you:

  1. Have you had choice game fans test your game? A reading group/workshop is a different set of readers.
  2. Do you have a following outside of this community?

For a game to be " good", it needs to be tested with scrutiny. Not just spelling/grammar. I’m talking continuity, impact of choices, player agency – all that good stuff that makes choice games the bee’s knees.

Part of the requirement for open testing is to gather feedback. Most new game writers don’t have a dedicated group of choice-game fans to call on to test their work. Since there are numerous WIPs at any one time, there is competition for testers. Therefore, open betas provide the most exposure to testers.

People like me, Cataphrak, Lucid, and Allen Gies all have success in the Hosted Games arena and we all did open betas until we assembled our Avengers of testing teams. That’s not to say our way is right but it’s right.

16 Likes

The catch of having groups like that (where you know each other and are friends, I guess) is that they will tone down any criticism they have. What I mean is, if there’s something that sucks in your story (hypothetically speaking), your writing group may be more forgiving/lenient, but that won’t do you any favors once your game is out in the market. The good thing about having testers who don’t know you is that they will be 100% honest and tell you what they see wrong without watering it down.

7 Likes

Also, a writing group might not be testing for the same things, especially if it’s not an interactive fiction-specific group. Plus, beta testing on the forums lets our diverse community take a look at your game and see if it’s reaching as wide an audience as it could; I know I and several others usually push pretty hard for asexual/aromantic and nonbinary options, which are fairly simple to implement, but can make a big difference in who wants to buy your game.

7 Likes

I’ll help you out. I haven’t put my game out yet, so take it with a grain of salt.[quote=“Doug, post:1, topic:26284”]
2: I’m confused–doesn’t this poach the market?
[/quote]

As many people commented already, not really. The forum is the smaller slice of the pie when it comes to the readership you might get. Besides, having a beta test with seven or eight people hardly means the whole forum. Launching a game (a Hosted Game, I should add) that is well-received might get you some more purchases from here. And I wouldn’t count on people just playing the demo forever and not buying it, from my experience in the Guns of Infinity thread, the smashing amount of peple who tested it also bought it. Maybe it’s because it’s a series, but I’d disagree.

And if you ever have another game on your mind, you’ll already have some people willing to help you or who’d be intereste din giving your works a try (a few Hosted authors, like MultipleChoice and Cataphrak, have developed a fanbase-esque).

I think it’s mostly bug catching and language correction for spelling mistakes.

I tend to treat to see betas this way: go for it. Nine pairs of eyes work better than one to get a mood or a scene right. You can also get people with different tastes, from different walks of life to help you out, and I believe that’ll enrich your game, regardless of opnions on it.

2 Likes

The answer to #1 is: sort of. I’ve had IF fans play the game, and one of them was familiar with CoG. It’s probably worth noting that I’m talking about ‘playing’ (or ‘reading’) as opposed to testing. The game that I’d like to submit was written as an experimental creative piece to complement my honours thesis. It’s written in choicescript, but has little in common with most CS games. In particular, the total number of choices offered in the game is very few. As a result of that, I’ve been able to create stable builds for each round of beta playing. I’ve had readers come to me with continuity errors in the past, but that’s always been when I jumped a production cycle and gave them a not-yet-proofread draft.

The answer to #2 is ‘oh man, that’d be awesome.’ Most of what I’ve written in the last year or so has been to test genre or digital literacy theories. They’re usually half-formed skeletons of projects that get abandoned as soon as I’ve learned what I needed. This particular piece needed to be assessed as a creative work as well as an experimental piece, so it got to live to see completion.

Mmmm, that’s one of the challenges we’ve struggled with. We’re a fairly large group (I run a student society at my university), so you can often find someone you don’t know pretty well to give you some honest feedback. I personally like the James Portnow manoeuvre: ‘hey, I’m playing this game that a friend of mine made. I don’t think it’s really good, but I’d like a second opinion just to be sure. Can you tell me what you think?’

Of course you can only use that a few times until people start catching on, but it’s good while it lasts…

@hallofmirrors That’s certainly true, and I’m definitely interested in listening to that kind of feedback. Unfortunately I’m not sure if I’ll be able to accommodate any substantive changes. I’ve started my next creative project, and I’m kind of going by the finished, not perfect/fail faster mantra. I’m definitely going to polish (line and copy edit), but I want to avoid any substantive edits. I could probably create an asexual option fairly easily, but most other changes will probably just be put down as tips for next time.[quote=“Vertigo, post:30, topic:26284”]
I tend to treat to see betas this way: go for it. Nine pairs of eyes work better than one to get a mood or a scene right. You can also get people with different tastes, from different walks of life to help you out, and I believe that’ll enrich your game, regardless of opnions on it.
[/quote]

I can certainly understand that logic. It’s a very different approach to the one I’m used to taking, and I’ll be interested in seeing how it goes.

Barring any unexpected problems (I’m working on a small project due this evening, and I’m not sure if my last wave of edits left my piece in a stable build), my draft should be up sometime today (AEST).

2 Likes

This is proof enough to me the Open Beta is needed for your submission. Good luck.

3 Likes

@Doug, just a quick note of clarification that the open beta doesn’t have to be quite as open as Cataphrak’s and mine have been.

Quoting @jasonstevanhill here, the open beta COG requires for Hosted Games does not mean “you have to post it so any rando can play without contacting the author.” It means at a minimum that the beta has to be publicly announced on the forum, and forum members who express interest are allowed to join a private message thread where they’re given access to your draft.

That will give you a better sense of how many testers are getting free access to your beta and how many of them are actually giving feedback.

10 Likes

@Havenstone Oh! That’s good to hear, but the cat’s a little out of bag already… I’ll have a think, and look at the kind of responses I get. It may be that it’s more productive to have a closed beta.

Thanks for letting me know!

Well, at least ppl will know about the work and start giving the feedback (although I’m yet to click your thread :stuck_out_tongue: )

1 Like