Under the Hosted Games submission guidelines, it’s listed that the game must pass Quicktest and Randomtest.
Now obviously I have gaps in code, but I have areas that are completely fine and that still somehow error out (for example, having my save/load system implemented with hidden choices if a slot hasn’t been used which somehow fails quicktest despite working completely fine in the game and appearing to be rock-solid).
My concern is that at the end of my process, this is going to be a multi-million word RPG with tons of repeatable gameplay loops that have very intensive scripts ran through them. If everything works completely fine in testing and in gameplay, but both of these still fail, am I screwed on getting published from your experience?
I know I can email Choice of Games on this matter, and that’s something that I’ve considered; but am posting here instead at the moment as I’m still a long ways out from a publishing standpoint.
I don’t have an answer specifically about the hosting (although I suspect the answer is “it has to pass them both”) but from my personal experience I definitely recommend being proactive with running both checks.
I got into the habit of running each at the end of every writing session. It’s a bit frustrating, but way better than leaving it until the end, when it’s like death by a thousand cuts to deal with all the little errors that have cropped up!
My concern is that at least in VSCode that I’m running with the extension, I am experiencing errors on both that do not appear in the actual game environment.
An example is the identification system for my Salvaging Loop. RandomTest claims that there’s an error tied to it that a *return fails because invalid return, we’ve already returned from the last gosub.
However, in testing when that specific part is called (or any of the 11 parts to identify for that matter), all 11 are identified correctly and return to the main loop without issue or crashing.
So RandomTest in this instance is claiming there’s an issue, when there actually isn’t.
Same with QuickTest, it claims restore checkpoint for slot slot_1 could fail; we might not have reached *save_checkpoint slot_1. Use *if choice_saved_checkpoint_slot_1 which that choice for slot 1 cannot be picked unless my variable slot_1 is set to true and that is done the very first time that you use the save system and pick slot 1. So of course it’s erroring on something that cannot logically occur in the first place. I also ensured that when slot_1 is set to true, it is done so before *save_checkpoint slot_1 is initiated, ensuring that a load keeps that variable in check.
All of this to say that I’m getting errors for issues that in production testing do not or cannot occur.
EDIT: I also want to ensure I clarify that I have tested that identification loop without fail 19 times merely to humor myself and ensure that there’s not some logical blip in the universe in which it would fail. However RandomTest fails on iteration one every time.
Note that Quicktest doesn’t play your game “properly” so it can definitely flag false positives. There are ways and means around avoiding those or guarding certain parts of code from Quicktest.
Random test on the other I believe pretty much plays your game as a player would, so I’d be dubious if it was crashing somewhere. Note that it might not just be which choice causes a crash but what other choices were chosen before it. That’s not to say it’s impossible that you’ve found a random test issue, but I’d imagine it’s either a). A bug in random test or b). A bug in your game. I don’t think random test typically exhibits false positives (someone with more game writing experience, please correct me if I’m wrong in this).
That can happen even without extentions. Quicktest is weird and can fail to undestand even “core” ChoiceScript features at times.
Personally, I choose to work my way around that and modify my code until it passes both tests. Sending HG a game that doesn’t pass because I’m sure no errors will occur during playthrough sounds like asking for trouble. Just bite the bullet and restructure.
Quick test executes every line even if it’s inside an *if command. If you have a piece of code you’re 100% sure is fine but QT is throwing an error, use an always false *if to skip the code:
*if (false)
*goto skip_problematic_code
For random test, it’s like @CJW said, it plays the game like a player would. So, even if you can’t see the bug, make an effort to find it. If I recall correctly, in Vscode you can configure to print out the text and even the choices selected, so you can follow the same path the test is doing. In CSIDE you can even track step by step the problematic piece of code.
But if you are 100% sure there’s no bug, there’s a special variable choice_randomtest you can use like in the quick test example. You don’t have to define this variable anywhere, it’s defined by the engine itself.
You probably know you can have RT display the choices selected. You can also make it display relevant stats at a given point by including a choice that only RT will ever see:
*if choice_randomtest
*choice
#At this point, we have ${followers} followers, with ${nonkids} nonkids and ${kids} kids. Arms ${arms}, wealth ${wealth}.
*goto postchoice10
Then when you run RT, you’ll see the stats displayed at that point.
For RT crashes, if you do a playthrough following all the choices RT chose on the crash path and also make sure any randomized stats are set to the same values they have at the crash point (CSIDE is good for this), and you don’t replicate the crash, then maybe you’re safe.
But like others have said, trusting RT’s bug-catching is usually the right idea, and *choice_randomtest is better used to skip over potential code loops rather than to do an end run around RT bug reports.
My guess is that HG will not greenlight any game that doesn’t pass QT and RT – however fervently the author and/or their beta testers assure them they never came across any bugs on their playthroughs. Adjusting your code to pass would be wise.
Pretty sure you HAVE to have it pass both to submit. I spent ages tracking down what are frequently non-issues and incorrectly pointed randomtest bugs in one project in particular that was a bit of an experimental project.
If you’re running scripts, be aware these can cause the testers to bug out (and I’m pretty sure aren’t allowed in published projects anyway.)
If you’re not running the testers through the server version then do so. I’ve had a lot of issues with the web based testers.
Be aware that the random test may not point to the correct line that the error is located.
Loops will 100% crash the testers and can easily be missed on playthroughs unless people pick exactly the wrong sequence of choices.
I will look to make some modifications, but may post for clarification/help in runarounds. The *return error RandomTest is encountering in the salvage loop does have me worried as if it skips core gameplay loops not sure how it’s going to be able to get ‘out’ if it initializes the job.
@Jacic sorry I should’ve clarified! By scripts, I meant long subroutines haha. Should’ve made that distinction. All of it is in native CS and not using JS or the *script command.
Again, you’ve all been a ton of help. It’s going to be a… pain to get this all modified, but I’ll do so rather than risk publishing failure.
That moment when you do bugfixing on an unfinished file and then realize the error is simply because it runs down towards another area from being unfinished and randomtest was in another way pointing it out… My identification system was solid, it was the unfinished part I got yelled at haha.
While I appreciate the nod, I’m more than willing to put in the work to be a Hosted Games creator, as I’m passionate about this community and IF in general, and would rather go through official channels than negotiate a CS license for the language.
