Feedback about the Beta Process


#1

Continuing the discussion from Empyrean: Take down your father with his own secret airship!:

Not sure I’m doing this right, @FairyGodfeather but here is a new topic to discuss the beta process.


Thoughts on Beta Testing?
Empyrean Post-Release Thread
#2

Regarding your earlier comment about people kinda half-assing participation in the betas, I think it’s easy to forget that beta testers are doing this for fun. From the company perspective I’m sure you guys think that’s selfish. You’re giving us free early access after all. Thing is I don’t really participate in the forums just to read WIPs. I do it to socialize with people who have similar interests and discuss the WIPs. The one way communication of the email beta testing model takes a lot of the fun out of it. If it isn’t fun it starts to feel like work, and I already got plenty of that to do.


#3

I didn’t want to make the topic myself in case I was crossing some boundaries. We’ve also got the ability to split off posts from one topic to start a new topic, but not having been a part of the start of the discussion I wasn’t certain what, if anything should be split off. This works fine though.


#4

Yes.

I’m accustomed to getting feedback on my feedback from the author directly, and talking about the game with them. That’s what makes the job rewarding, not free access to a five dollar game. For example if the testing I did for @Cataphrak’s games had just been the equivalent of sending radio signals into space I wouldn’t have bothered.

Pretty much the reason I don’t make a habit of beta testing official games.


#5

I really participate in Beta for the very selfish situation of being my style of playing represented . I role-playing mostly evil characters. For me is important give advice on it , because it is normally something authors don’t look after allow machiavelian choices. They are logically focused in hero stuff, and all heroic part of any story. And that’s normal because it is what most of people want to play. However, what make me love choice of games is the possibility of be evil even a little. Other companies in the genre No need give names, don’t give real choice only a small illusion of choice , in a same preset personality… Cog is different or at least it was. So I felt in love .

Then being a tester become a totally different matter, a way to genuinely try to helping even if the game is not my cup of tea. I still playing the beta saying maybe I could help making it better . …

But then a wall of silence is casted, nobody to talking about t their game. And with then the doubt Is my feedback good? Did I something wrong? Am I boring them with a tons of feedback they don’t want? And just stopped sending each game… Then just become an habit, send 3 email with feedback. When you really want say more things, give more opinions… Explore with something if that stuff maybe is a plot hole or just a misunderstanding or even just a code error misplaced… There is no one to talking with just a cold wall of silence in a email. …

Sometimes a miracle appear in the form of the author asking for anything and then IT COMES THE LIGHT ALLELLUYAH . Then beta is fun again, you know what you could do to helping, you feel useful and all become better and interesting and author gets all he wants .

Lately, official cog beta testing is more and more a wall of silence … To the point you have no clue if the beta has been closed or not. Nobody asking to you and you feeling each time more and more unwanted and hated … Discarded . We already have our nobility testers… Please, go away you don’t deserved it . Be happy we let try a game at once, looser.

I don’t ask for access for playing free I BOUGHT ALMOST ALL GAMES I don’t want steal you money playing free. I just wanted to make games better.

If there were something to talking other betas it could be better at least with someone to talk about it new ideas and more issues would appear… But I don’t know … I am just a trash beta that is not considered well enough to have the beta title and was stripped of any reward I had . So my feedback would be 100% ignored but at least feel great express my feelings on it.


#6

I do not think our testers half-ass it at all! At all! And yes, I totally recognize people do this for fun.

I just did want to highlight that, not unlike our normal COG submission process, we ask people “email this address if you want to participate” and . . . people email the wrong address. Similarly, in the editorial process for COG we ask people to begin by sending us their CV, and often …people do not. They send an email describing themselves or something. I even created a new webpage describing what a CV is. So.


#7

I mean, cool, but. The thing is, do you want people’s feedback, or do you want to feel self-satisfied in dismissing that feedback? Like, the idea that people email the wrong address is one thing, but you seem to be implying that if people email the wrong address, their feedback is by definition invalid.

Which is fine I guess! But it seems to me that you have two options. You could either say to yourself

  1. This person made a mistake that to me is ridiculous, therefore their feedback is invalid

or

  1. this person made a mistake that to me is obvious, and also this person didn’t like this aspect of our game, which means that people who don’t think like me dislike this thing about our game. So if I want to please the aspect of our audience that doesn’t think like me, I should address this criticism.

Does that make sense?


#8

I think you’re missing the instance I’m talking about. I’m not talking about beta feedback. Not current beta folks, but new people when we put out a call for beta testers. “Would you like to beta for us? If so, join by emailing X.” Then people email Y address. I think that’s a simple thing to do, and for me it is indicative when folks are applying for a job or a volunteer gig that you follow instructions. So it’s not that we’re discounting feedback we’re receiving, or something.


#9

Have you considered the possibility that error might be caused by the device you are mailing from. I am on a kindle and it is forever changing words even when you type them in the way they should be. Great for spell checking but a pain for email addresses and passwords.

Also I suffer from dyslexia so it can be difficult to spot miss types in what I write although I can easily spot miss types in other peoples writing and code. It also makes me good at spotting plot holes.

So would I be discounted as a beta tester just because I mistyped a email address?


#10

After a few observations, It seems that the general complaint from beta-testers, in my opinion, is that they give feedback, and yet the feedback remains unused by the author, so they feel that their feedback are not valued enough?

I am also going to assume most of these feedback are the “high level” ones? The ones about pacing, characters, etc?

I am sure people remember that while giving feedback is very much appreciated, it is ultimately the author’s prerogative and choice whether to accept or reject the feedback? If 10 people give feedback, you will have 10 opinions that can widely vary from one another. It is unrealistic to say that every feedback will be equal and/or accepted, when they can greatly alter an author’s story.

So to the beta-testers, since one of the complaint is the lack of communication, and that you are not given feedback for your feedback, do you want the author (or person in charge) to personally contact you to say whether your feedback will be accepted or rejected? Do note that authors may reject one feedback, only to accept it later after further considerations. This is probably why no feedback is given to the feedback, because they are always in consideration, and are not immediately accepted or rejected.

Also, I think one of the main problems is that when the feedback given remains unused, the beta-testers feel that they are not rewarded for their efforts.

(Feel free to correct me if I am wrong, we may get further insight on how to improve this beta-testing process :slight_smile:)


#11

I’m not clear on what you’re asking, then, Mary. In the previous thread, the one which led to the creation of this one, CoG expressed frustration at the lack of participation in open betas. People then responded that forum threads/discussion felt more natural to them than the email process. My comment was a response to that: maybe you would get more/better feedback if you changed your requirements for beta participation.

You are of course welcome to dismiss my suggestion! However, it would seem that CoG’s habit of dismissing people is part of the problem you are (purportedly) trying to solve!


#12

No No No… I am not an entitled rat like you are suggesting. I don’t care if they used my feedback or not …What I want is some sort of communication between both parts. We want this or that … We had changed this or that. Or a simple we have received your feedback, thanks for participating .Or Could you clarify this? Or we don’t want change that …

COMMUNICATION. Or at least say the beta is closed … You could twisted all and say all of you are wasted children that have tantrums. I don’t read anyone saying AUTHOR HAS TO DO WHAT I SAID. We want communication and collaboration to make games better. Not a wall of silence that even don’t say if beta has been closed


#13

It has been said that there is no feedback to beta tester, is it literally no feedback, nothing, nada, or is it a standard thank you for your mail?


#14

Macro level stuff is hard to incorporate by the time the game hits closed beta. If several beta testers had said, “I would love to have more time with that possible love interest,” even if, after reflection, I agreed, it would have been hard to make the structural and pacing changes that would require.

If, on the other hand, there were a multi part beta, 33% and 66% through the process, and then again at the end, that would allow for more macro level stuff to be discussed and considered and taken into account.

However, that would also be logistically really hard, possibly impossible. You would want the same people doing all of the parts of the beta, ideally. And I would guess some people would have a harder time showing their work in that unfinished state than others.

I wonder whether it would, in the end create a better product. I’m very much of two minds about it.


#15

NADA NOTHING you receive game link and the password for it Nothing else. Maybe you win the lottery and an author somehow asked you something or give you the thanks… But is rare as win the lotto. They even don’t tell you if beta has ending …NIENTE. .
and that’s all. I have seen my feedback used in some so is not an entitled You are that why because author don’t used your info. They use it BUT THERE IS NO COMMUNICATION


#16

I beta tested Mid summer not the last one Empyrean. If I would had beta that believed me I would say loud and clear how broken it is and how bad stats work. You could have used the feedback or not. That’s not my issue at all. But games gain from people looking for problems and continuity issues . That’s for sure.


#17

That is quite bad, with no feedback at all you don’t even know if the email has been received. It must be very frustrating to spend hours working through other people errors to not even get a “your email is in a queue it will be answered shortly” I can understand why people are not interested in beta testing.


#18

I’m not asking anything. (Sorry, I think I’ve lost the thread of discussion.) I mean, I think we’re agreed that the suggestion from beta testers right now seems to be: run the betas on the forum, not over email. No one is dismissing that. We’re listening. We have this thread, and we’re reading it! I was explaining that (in the past) we have had a difficult time even getting people to you know, send their request to join the beta to the email address it’s supposed to go to. I guess you were saying “you should let people who do that join anyway.” If we just run them on the forum in the future that problem (people not following directions) may be obviated.


#19

Yeah, you get it that’s main issue people has. Why bother with it if they totally don’t care to even say hey it’s over thanks.


#20

It’s very difficult sending feedback into the void as I used to call it.

Spending hours upon hours of playing a game thoroughly, sometimes time after time, poring over the code, writing down extremely detailed feedback, and offering suggestions, only to not even receive a response. That makes me feel unappreciated. Even more-so if once the game is released it contains issues that I specifically pointed out during my own testing of the game. I wonder what’s the point.

Now, not all testing’s like that. Personally I’m not that interested in a back and forth with the author anymore. My feedback is my gift to them and they can do with it what they want. I don’t want to have them defend their game to me or explain their reasons. I’d just like an acknowledgement, and a thank you, and if they want any further clarification on any points I’ve made that’s fine too.

I’m not good at the providing feedback in a group setting. I don’t like other forum members questioning my feedback either. I guess I don’t mind the silence.

I do think some better communication would be appreciated though. If it’s marked on the beta test threads when a beta test is closed. I’d like to see some change logs of what gets changed because otherwise we’re going in completely blind and I might just be repeating work I’ve already done. It’s also valuable to know whether I should run through the game again or not.

The lack of change logs is really, really frustrating, and limits me in whether I can test a game multiple times or not.

I do understand the frustration of people not following instructions. We’d frequently have people in the beta test threads posting things that showed they’d not read them. But hey, that’s sometimes how it goes. They see one person do something and everyone else assumes that’s the correct way and they jump onboard. But sending to the wrong email address could be the result of an email auto-completing the wrong email.

That leads me to believe that beta-testers are important. So an acknowledgement and a thank you would be nice.

I’ve been doing online beta-testing for various games for over 15 years now. The ones that win my loyalty, where I’m actually extremely passionate about the games, are those who actually respond to my feedback. Even if it’s just a personal note and a thank you. When they don’t answer with so much as an acknowledgement of the time I’ve spent trying to help them that leaves me disappointed.

Not that I’ve had the focus to test anything as of late, which is extremely frustrating.

I’d suggest give the forum testing a trial run. We could probably create private locked forum categories for every single separate beta-test game. That way people can still be limited to one test at a time if that’s expected. And it doesn’t have the same limitations as a group PM would.

I’d say if testing is done on the forum then please also continue to allow for email feedback. For the people like me who sometimes can’t provide public feedback. I can’t count the games I’ve actually paid for beta access to and then not actually taken advantage of it because they wanted the discussions on a forum instead of via email. :frowning: