On Beta Testing

To add structure, you mean? It might be really difficult to get a form to apply to the different genres and coding styles of games except in the most general way. A slightly art-nouveau mafia game with a a lot of violence and a touch of sci-fi and the occult will be very different to test than a spooky estate agent game, for instance, or a middle-school aimed game.

The link in my first post to CoG’s beta test instructions does have a general description of what kinds of feedback is useful, though! Did you mean something like that?

And then there are the different styles of beta testing. I hadn’t thought to mention it, but when I beta test I always use four categories, in this order: 1. Things I Liked; 2. Typos and Continuity [i.e., low-level feedback]; 3. Plot and Characters [i.e., high-level feedback]; 4. General Thoughts.

I go through and make a numbered list under each category, adding points as I notice things. Then at the end I give a few general reactions under that last header.

Depending on the length of the game, one play-through with that style of feedback can take me anywhere from 30 minutes for a very short game to seven hours for a long and complex one. (That’s my record so far . . . long game, lots of comments.) So I usually play through once per build, but I pay a lot of attention to the details. Next build, I make a new and different character, so I won’t necessarily see changes that only applied to my previous character build.

I just sort of made that up though so I’d have some structure. I have no idea what the other beta testers do, but I imagine we’ve each very different ways of approaching the beta process.

4 Likes

They’re* And like I said, optional. It’d be useful to both new testers and new writers.

What is some advice you have for striking the right tone in giving feedback?

I find that I sometimes come off a lot harsher then I mean when I give anything more than casual feedback.

When I interned for online news if I didn’t literally shout and threaten people they wouldn’t do anything outside of my mandatory AP style edits, and that’s bleed over into how I tend to give serious feedback.

It’s less that I intent to be terse with people so much as “short, direct, and to the point” are what is generally considered the standard in my area of the news business.

I don’t think that’s generally what the norm is in literary editing, and so I’m curious how people find a good balance between “professional editor” voice and “this is a friendly suggestion” voice.

For example, I can think of several times when I’ve basically told writers go completely overhaul news stories because of stuctural problems – changes that while they aren’t factual grammar edits they aren’t actually just my opinion either, they’re changes that the stylebook literally requires.

They weren’t “suggestions” they were errors, and whenever I ‘asked’ people to change them instead of ‘telling’ them the changes didn’t get made even when they weren’t optional.

While CoG doesn’t have too many absolute rules, there are times when there are things that aren’t a matter of opinion that need to be pointed out.

So I mean, I think what I’m trying to get at is does anybody have a good way to convey the difference between say “I think you should change the labels on the stat screen because this is my opinion” and “you need to change the labels the stats screen because you’re using a word incorrectly” with out basically stating every block of feedback if it’s opinion or you’re pointing out a factual error.

Edit: @Fiogan There’s a whole field of study dedicated to examining how internal biases effect the reviewing process. I can PM you some links if you’re intrested.

6 Likes

Piggybacking off of @Shockbolt’s post, but in the other direction.

I feel like I have the same problem with the concept of “tone”.

I mean, spelling and grammar errors are easy enough to correct because if they used the wrong “there/they’re/their” than its a matter of fact, but sometimes I feel like I’m over stepping my boundaries as a beta tester when I notice something is off context wise and comment on it.

How do you properly (and politely) draw attention to something that really needs closer examination, without sounding: “This is wrong. You did wrong” about it?

4 Likes

I didn’t know what CoG looks for in beta testers. So thank you for posting that.

Personally, I’ve never done closed beta testing, only open betas here on the forum. I search for bugs and errors first, but after that, my approach is to see how engaging, entertaining, and professional the game is. Basically, what the overall first impression is. If it’s lacking in any of those ares, I try to isolate what is negatively influencing the impression. If the first impression is good, then start playing different routes.

4 Likes

I really do not beta test much anymore (well except for the open WIP betas) for the reason I work better with a little feedback, this way you can work in tandem with author know what they are focusing on, and it makes it easier for me to notice other things while doing the normal low-high feedback.

One of the best authors I had the privileged to work with with was @Lucid he welcomes feedback and you can see the immediate responses in his work and comments. One that I more thank liked mucked up is Sergi’s and prolly made poor @jasonstevanhill pull out his hair and to beat me like a red-headed step-child. So I would suggest viewing this forum first for all prospective beta testers:

also

Look at the requirements for high and low level feedback this is what will be required of you, mid level is the what editors like @Fiogan get hired for, so know what you are getting into before you say pick me pick me! Also know being a beta tester for CoG is a privilege and a job your input makes all these games better. We all have seen a game or two in release that was buggy as all get go, that fault lies with the beta testers, bugs can not get squashed without being reported.

5 Likes

Well, for Slammed we did a beta that was very helpful. @FairyGodfeather can attest to the changes I put in based on feedback (and good to see you around again FG ^_^) We did one for a currently tabled game as well, and I am going to badly need one for Love and Lasers just because of how different it is from the usual CoG. What would you like to know from a game author’s perspective?

4 Likes

…what I’m gathering from this thread is that I probably shouldn’t volunteer to beta test a CoG because I can’t not jump on “mid level stuff.”

Have any of you found an effective way of… repressing, I suppose, that sort of inclination?

Probably a silly question, but is there such a thing as too much feedback?

2 Likes

Only if it means you don’t pass on the feedback by the deadline. Even if you write reams and reams of comments, no one is going to force the author to read through all of them. Some of them will, some will pick and choose.

That being said, being a good beta reader means trying to give proper weight to things. If the author is struggling with pacing, focus on that, rather than, I don’t know, writing a treatise on British vs American spelling conventions. :slight_smile:

9 Likes

When I beta I usually start with my general feeling what emotions give me the game.
Second If I found the flow and the plot with sense.
Then I explain the type of character I play, and if the game left room from customitation and some role playing stuff. if story felt railroad or not, characters…
Finally I give my ideas on how improve the game or just adding minor stuff to give flavor.
I can’t give grammar advice or edition stuff so I focused on the story itself and stats if are balanced or not.
Authors normally are very nice and easy to talk with.

3 Likes

Very good to know! I appreciate this. I’ve been wanting to help and do a bit of this, but life and kids tend to keep me bogged down. My forte is definitely more editing. I always wondered what sort of feedback authors found most helpful in their work.

2 Likes

Both Editing and plot feedback are important. Each of us should focus on what we are best. I being not native is futile attempt to edit something, it would end worse lol.

2 Likes

I’ve beta tested a few of the CoG, and I think they have internal readers to thoroughly edit for tone and theme. What I think they need when they ask the forum seems to be a larger number of people to comb through the game and try to break it with regard to continuity from choices and flush out remaining pesky typos. Whenever you release something to the public, the glitches will become immediately apparent (Like the recent Darksouls having a glitch at the very first checkpoint) so that is what they are trying to simulate by having 20-50 (?) extra people test it. They don’t need quality or artistic feedback at that point unless it’s something completely wrong…such as a low possibility branch that mentions a character has three eyes where it’s not been mentioned anywhere else.

3 Likes

Oh, no, I get that. I’m curious as to how people who have the same tendency as I to jump on everything are able to ignore that urge and focus solely on the low/high level stuff that CoG is looking for.

Usually I get wrapped up like I’m just reading the story and don’t have many comments - until the thing crashes or does something completely wrong. The CoG ones are usually quite polished by the time they get to forum beta testers. I actually never made it to the end of Hollywood Visionary because I couldn’t get past a certain junction.

When I have the time, I’ll try to play through WIPs. I’ll point out a few specific things, but mainly just be encouraging and move on. Beta testing is different - I’ll send in documents full of typos, errors, suggestions, etc., with only the occasional compliment to remind the potentially-harried author I wouldn’t be sinking in so much time if I didn’t believe in the project.

I haven’t gotten to a full beta yet for my WIP, but the play testing as I write has been hugely helpful. It’s a story with a lot of choices and a focus on immersion, so if a section is boring, impossible to navigate, or not realistic, I can get feedback before it’s too late to change it. Plus it helps with the really bizarre bugs that quicktest doesn’t get, like the one with Unknown potentially showing up in your bathtub.

One thing I don’t really like during testing feedback is using persuasive arguments. If something makes you uncomfortable, or you love/hate a character, or a choice isn’t working for you, tell the author. You should trust that they’re listening, and you don’t have to write a speech to get their attention. In particular, debating to convince other testers to support an opinion of how the game should be changed seems counter-productive.

9 Likes

@Sashira

I think your first point is what I find myself doing.

When I read WiPs I’m not generally looking to nitpick people, so I comment on a few small or personal issues (Like wanting to pick six favorite flavors of ice cream!) rather than writing up huge long lists of typos and errors.

But i dunno if that’s actually what people want from readers. I feel like I see more authors saying “please point out typos” than I do anything else. But, given my news background, I also know that for many people there is nothing more demoralizing than a list of technically errors that scrolls on for pages and pages.

4 Likes

@Shockbolt As to striking a good tone in giving feedback - I may not be the best person to answer this, but I try to keep in mind that I’m making suggestions. My typo list doesn’t have much of a tone, I don’t think. It just looks like this (and much like a lot of others’ typo lists on forum WiPs):

(5). The sparkling pink vampire ponies leapt across teh* fast-moving crystal river.
- *the

[N.B. not a real line, nor from an actual beta test.]

When making a high-level suggestion about plot, characters, or mechanics, I try to use a question format when I can: “Do you think a ‘maybe’ option would be possible here, instead of just shades of ‘yes’ and ‘no’? I would have liked to take a more neutral position.”

I can only suggest what I think, and sometimes I do include a brief reason why. “I would have liked an option to tell Captain Thatfellow that I don’t mind if he sticks around, because I want to maintain a relationship without either alienating him or jumping him.”

With editing, I do address grammar of course, which is not looked for in Choice of Games beta tests (WiPs on the forums vary in this regard, I gather?). @Samuel_H_Young and @Nocturnal_Stillness, the only two ChoiceScript authors whose works I have edited, could probably answer questions about niceness levels with editing vs. better than I could.

Even with editing, though, I try to make a point of saying, ‘I would recommend this but style guides vary’ instead of just basic corrections without notation, if an issue could really go either way.

And any narrative or character choice is going to be a suggestion, to my mind. If I feel strongly about an issue when editing, I’ll say so. If I feel strongly about it when beta testing, I’ll just say something like, ‘This section broke my immersion because of Y and Z reasons’. Then the author can decide what to do about it, or whether to do anything at all. Does that sort of answer your question?

@WaterOracle Even the ‘this is wrong’ issues are still at the discretion of the author, though. Beta testers do not control the game. An editor or the publisher would be the one to take a stand on things, to my mind, and not beta testers. I liked @Sashira’s comment about not trying to convince the author with a paragraph of arguments as to why one is right.

@P_Chikiamco I really appreciate your point about weighting issues properly; that’s helpful. Thanks! Could you share an item or two of especially helpful feedback from Slammed!, if you don’t mind? And why was it exceptionally helpful - what changed, how it improved the game overall, anything like that?

@OScott That’s hard for me too because I have to not-edit! I always look at issues I spot and think, "Is this an ACTUAL TYPO that even my word processing program would spit back out, or something like swapped genders? *if (yes), *goto make a note. *if (no - there’s any amount of room for debate), *goto skipit. I think twice I’ve given actual grammar feedback, and only because I found it terribly distracting and I wasn’t sure it was intentional.

Sometimes ‘bad’ grammar IS intentional, too, or a result of house preferences (don’t get me started on ellipses). So reporting grammar errors isn’t helpful because beta testers would be talking over each other, and then the editor, house, and author are making the style decisions anyhow.

Besides, editors don’t have trouble catching poor grammar. Certain typos or continuity errors, though . . . I don’t know how other ChoiceScript editors work, but I look at the entire manuscript, code and all. I’m not playing the game, so I might not notice if a sentence fragment in line 120 needs the word ‘and’ to properly meld with the other half of the sentence down in line 258.

Another issue is that the writer and the editor sees ALL of the plot and character information. I’ve seen a few games critiqued because readers felt like the plot had holes, or the characters were shallow. The information was there in the story, somewhere, but the reader never saw it, and wasn’t inclined to do a replay after an unsatisfying game. The editor and the writer have already seen every single line, so it’s harder to tell if key information is missing from certain routes. Beta testers would notice, though!

(@Shockbolt if I DO have a huge list of typos and errors lurking in my prosey halls, please at least tell me they’re there so I can try to hunt them down . . .)

I also don’t think it’s any one beta tester’s job, as @P_Tigras pointed out on another thread, to ‘fix’ games. Lots of beta testers noticing different small things is the way to go - especially in a game, where the beta testers may only be seeing 30% of the game per run, and all shuffled together in different ways.

11 Likes

With beta testing I want readers to be honest tell me what works and what doesn’t. Which characters you like or dislike and tell me why. Critiques are important to help improve a writers ability.

11 Likes

Hm, it’s been a while. One thing I do remember changing because of @FairyGodfeather was allowing the player to tell Ecstasy to wait when Ecstasy confesses. FG pointed out that in some play throughs Ecstasy would have done the exact same thing to the PC, and it would make sense from both a game standpoint and a real life stand point if the PC had the option to do that as well.

6 Likes