How to deal with arrays

Game Development
1

TLDR Can we please have better array support? Would you take code contributions?

Hi all. I am amazed by how vibrant this community is. Kudos to @dfabulich and the team for creating a tool that got so much creative energy going. It’s really interesting to read the discussions, and see the several ways people have pushed features like the array syntax, or multi-replace, to the limits.

An issue I find often discussed is the lack of support for arrays. I wanted to summarize here what I have learned so far by reading the forum and hacking around CS. I hope this is helpful to everybody struggling, like me, with arrays, and I’ll be happy to share code with all those that are interested.

I’ll write the story of my quest in second person, like many of the games we like.

You are writing your epic RPG saga, and decide to create a log of all the amazing
quests your hero will complete. On this log, you will run stats and show them to the player.

Or you decide to record all the choices taken by the player, so you can replay, test
or debug your game (Wouldn’t that be a great tool? I believe it’s doable with some *script).

Awesome! What’s the log? Just a long list of entries you can scan, append to, and possibly modify. Let’s implement that in ChoiceScript. You will:

*choice #1 - do it all by hand

At the top of your startup.txt, you manually add a bunch of entries

*create log_1
*create log_2
…
*create log_100

Well, let’s hope 100 is enough. Luckily, we can use the convenient array syntax to loop (*) over the log and do what we need.

On day ${i} you did: ${log[i]}

(*) Oh, yes, you have to simulate the loop with goto and labels.

PROS: pure ChoiceScript.
CONS: what happens on day 101?

*choice #2 - use a code generator

You write up some Python / C++ / other language to generate the code for you. This tool will write the sequence of *create commands you need. It may also implement the loop for you. Nice!

Hey, you could even use ChoiceScript to write ChoiceScript for you! [Tool] ChoiceScript Helper (Currently Unreleased) - #2 by RETowers @RETowers

PROS: less tedious, the tool can be reused in multiple occasions.
CONS: you maintain your own tool and nobody else can take advantage of it.

*choice #3 - use strings

It turns out Choicescript already has an infinite data structure: strings! You can append the daily log to a large log string, and it can grow as much as your computer memory will allow. If you want to parse it back and breaking it up, you’ll need a separator among entries and you will have to write a little parsing routine.

Also, strings can’t be modified in place. If you want to change some data, you’ll have to copy the data over to a new variable and adjust the data while you do that. (You could implement that with *script for performance.)

Still, strings can be a useful tool if the log entry is simple and you don’t have to modify it - eg. you can log your travels with one character: N W S E.

PROS: unlimited growth.
CONS: slow performance, complex code for the parser, very hard to modify the data.

*choice #4 - use *script to create variables

You decide the go all out and understand what happens when you create a variable. Now, in scene.js, Scene.prototype.create simply does this.stats[varname] = value (also updates this.created). Ok, maybe with Javascript we can create variables as needed.

You need some trick as you can’t directly pass a variable to *script, but that’s easy to solve if you read from this.stats. Once the new variable is created, you can use it easily in ChoiceScript.

*create day 101
*create entry “You mastered a new spell”
*script this.stats[“log_”+this.stats.day] = this.stats.entry

Your log for day ${day}: ${log[day]}

*set log[101] “Sorry, my bad, you didn’t master the spell yet”

This is rather simple and readable. Now you can programmatically create variables anywhere in your code (not just at top - this would be great for scoping or recursion). Still, you remember that unfortunately this is not a real array where you can get the length, insert, remove, append or sort.

PROS: very simple, allows for unlimited growth, no need to pre-create variables.
CONS: *script is a little like cheating, and your game won’t be hosted on COG.

*choice #5 - Use JS arrays directly

Now that you have discovered Javascript, you decide to try to interface Choicescript to a real Javascript array. Again, you’ll need some workaround to pass data forth and back between the two layers but, wow, this looks like fun.

  1. You need to define JS functions to create, read, and write the array. You need a persistent namespace to store the functions, so you use document (you guessed that this.stats would also work, but this doesn’t seem to be the case). These functions will read their arguments from shared variables via this.stats, like you did at choice #4.

  2. Every time you create / read / write, you’ll have to use *script to invoke the functions.

  3. If you built some preprocessor at choice #2, you can extend it with some shortcuts for this.

  4. Oh, yes, you need to pass or bind ‘this’, because… Javascript.

You try to read data from an array.

*script document.js_array_read = function(x){
x.stats.js_value = x.stats[x.stats.js_varname][parseInt(x.stats.js_pos)];
}

*comment This will read log_array_js[day] into entry
*js_read log_array_js day entry

Your tool will expand the last line to:

*set js_varname “log_array_js”
*set js_pos day
*script document.js_array_read(this);
*set entry js_value
Your log for day ${day}: ${entry}

It’s a little tricky to set up but, with some tooling, the code (before expansion) is very easy to understand. Also, in your experiments, it runs REALLY fast, faster than a collection of pre-created variables. And, it’s a real array that you can potentially sort, map, filter etc. using some *script.

PROS: unbounded data structure, dynamic creation, and IT’S FAST.
CONS: hard to setup, and again it’s using *script.

*choice #6 - Not selectable - Modify Choicescript

You decide to just hack Choicescript, add your array commands in scene.js and tweak them the way you like. Now in your new CS+, you can *create_array *slice_array *array_replace and all the magic that other languages have. As you are on it, you also change strings so they are 0-based, instead of 1-based, for expressions like string#pos.

PROS: the sky is the limit.
CONS: well, now your game can’t be hosted anywhere.

Thanks all for reading this far. What choice did you take?

In my experiments, #5 seemed to be very powerful, but probably #4 strikes the best balance: it requires only one simple *script command to get the job done.

Now, if *create got extended to accept an expression for the variable name (a limited change in code, it would reuse a lot of code from Scene.prototype.set), this would make a lot of interesting tricks possible. The often requested *create_array could be emulated with a loop of *create varname_{id}.

Oh, I didn’t speak about matrices and objects, but the problems and solutions are similar to these.

So, do you have similar tricks to share? Let me have your thoughts!

5 Likes
2

Just did a quick read and simply wanted to point out this

You can host the game either on CoG/HG. You just need to notify them first so the modification could be commited to the CS main branch.

It simply requires an… intense communication between you and the company.

3 Likes
3

I think the CS community would benefit from real support to arrays. I would be OK with arrays of limited sizes if at least I had proper search and update funtions. However, I’m not sure how many authors would actually make use of these features.

To be honest, for me, a more urgent feature is dedicated syntax for loops since they are very predominant in every CS project!

For example, to traverse an “array” something like this would be a godsend:

*create stat_0 "strength"
*create stat_1 "dexterity"
*create stat_2 "wisdom"
*create stat_3 "intelligence"
*create stat_4 "carisma"
*create stat_5 "constitution"
----------------------

*label calculate_highest_stat

*temp i 0
*temp highest (0-1000)
*repeat 6   ← << number of time to be repeated >>
      if {stat[i]} > highest
            *set highest {stat[i]}
      *set i +1
*return

With proper support for arrays:

*label calculate_highest_stat

*temp highest (0-1000)
*for i in stat
      if {i} > highest
            *set highest {i}
*return

To do the same thing currently, I have to do this:

*label calculate_highest_stat

*temp i 0
*temp highest (0-1000)
*label begin_for_loop
*if {stat[i]} > highest
      *set highest {stat[i]}
      *goto continue_for_loop
*label continue_for_loop
*set i +1
*if i <= 5
      *goto begin_for_loop
*return

It’s less readable, takes more space and is more prone to errors.

4 Likes
4

Yes, loops would be another important addition to the language. I agree with you that writing loops in CS can be error-prone - nesting in particular can be tough. I go around that with some tooling.

// will substitute any occurrence of XX with each value from 0 to 99
*unroll XX 0 100    
     *create array_XX 0
*for n 0 100   // will create unique label, test, increment, goto
    Entry ${n} is equal to ${array[n]}
1 Like
5

@choicehacker Interesting. I like improvements which make my coding/writing cleaner, easier to read, easier to debug, and easier to maintain. Thanks for spelling out these choices!

2 Likes
6

Thank you for bumping this @LAM, it made me realise there’s actually some more options for the toolkit available now (though still no ideal one).

This isn’t targeted at you specifically, I just wanted to add it for anyone else who comes looking :slightly_smiling_face:

So below follows a couple more array related options.

Array Creation Patch

I wrote a patch adding *create_array and *temp_array commands to ChoiceScript. It's not merged (unfortunately) but those more comfortable with CS and git could use it to patch their own copies.

See: Add *create_array, *temp_array, *delete_array by CareyJWilliams · Pull Request #136 · dfabulich/choicescript · GitHub.

Pros

  • Keeps to ChoiceScript (mostly)
  • Sticks to standard array semantics (in CS)
  • Reduces effort of initialising arrays

Cons

  • Unsupported by CoG (at the time of writing)
  • No variable length arrays

The ChoiceScript Library (CSLIB)

See: 🧰 [TOOL] CSLIB - ChoiceScript Library

Contains support for both ‘arrays’ and ‘loops’ to some extent.

For loops there is a routine called ‘repeat’ which takes a scene+label ref to a routine, and repeatedly passes it a number up to a limit. You can use the following idiom (NEVER is a constant variable always set to false) to make your CS look reasonably close to a standard for loop:

*gosub_scene cslib_util repeat "startup" "print_numbers" 5
*if (NEVER)
    *label print_numbers
    *params n
    ${n}
    *return

Arrays are still a pain to create (unless you use them in conjunction with the patch above), but they are a lot easier to work with in CSLIB:

*comment create empty array
*create player_name_1 ""
*create player_name_2 ""
*create player_name_3 ""
*create player_name_4 ""
*create player_name_5 ""
*create player_name_6 ""
*create player_name_7 ""
*create player_name_8 ""
*create player_name_9 ""
*create player_name_10 ""
*create player_name_count 0
*create player_name_max 10

*comment populate
*gosub_scene cslib_array set_from "player_name" 1 "Jack" "Janet" "Jericho" "Joseph"
*gosub_scene cslib_array push "player_name" "Jack"
*gosub_scene cslib_array push "player_name" "Jemma"

*comment merge to a sring and print
*gosub_scene cslib_array join "player_name" ", "
${player_name_count} player names: ${cslib_ret}

Pros

  • 100% ChoiceScript
  • You get array methods for free: push, pop, filter, etc.
  • Helps with code structure / readability
  • CSLIB is well-tested, whereas custom code wouldn’t necessarily be
  • Variable length arrays (kinda: they have a max, but this can be set as high as you need)

Cons

  • Arrays are still a pain to define (unless you combine with above)
  • Requires CSLIB / an external dependency
  • Less performant
5 Likes
7

*raises hand*

1 Like
8

First, I want to say THANK YOU to everyone who works on ChoiceScript. It has a lot of potential and I am enjoying exploring it.

I’m grateful (hey, it IS that time of year!) it exists and has people actively working on it.

Yup, arrays are a pain to create BUT as @choicehacker noted in “#2 (use a code generator)” above, it’s easy to generate *create statements for a one-dimensional array with ChoiceScript itself. It’s just annoying having to

  1. copy and paste all this into startup.txt and
  2. read and work with the cumbersome startup.txt contents

When imagining only reading a single statement such as *create_array player_name[50] rather than this mess, well… Quite frankly, I’ve started looking at other interactive fiction tools to use.

I generated 501 ChoiceScript lines - abbreviated below:

*create player_name_length 500
*create player_name_1 ""
*create player_name_2 ""
...
*create player_name_499 ""
*create player_name_500 ""

Not that I would ever have 500 players, of course. That’d be loco. Anyway, I used the following ChoiceScript code, which asks for the array name and length (upto 10,000 should be well beyond the limit of one anyone would ever need):

*temp len 10
*temp index 1
*temp ar_name ""
*temp cmd "*create"

What's the array's name?
*input_text ar_name
What's the array's length?
*input_number len 1 10000
${cmd} ${ar_name}_length ${len}
*line_break
*label loop
${cmd} ${ar_name}_${index} ""
*line_break
*set index +1
*if (index > len)
    *goto after_loop
*else
    *goto loop
*label after_loop
*return

I was thinking about two-dimensional arrays. They would be nice. It seems natural to have:
player_name_1
player_wisdom_1
player_species_1
player_strength_1
player_magic_level_1
(and a bunch of other attributes for each player)
then
player_name_2
player_wisdom_2
player_species_2
player_strength_2
player_magic_level_2
etc…

Anyway, I’m definitely going to have to grab cslib and cside and look at what you’ve written with them. @cjw @choicehacker @cup_half_empty

Thank you, again!

-LAM

4 Likes
9

Glad to hear you’re enjoying it! Despite its frustrations, it is indeed very fun to just mess around with. I think that’s why most of us have stuck around like we have. The community is a huge part of that as well.

I really hope we can get *create_array. I don’t think it’s a controversial addition (it’s not adding new functionality or trying to push CS in a different direction), and would make the lives of authors so much easier.

2 dimensional arrays do actually work exactly as you’d expect out of the box, which is neat:

*create attribute "strength"
*create current_player 1
*create player_1_strength 5

${player[current_player][attribute]}
4 Likes
10

@LAM I’m happy this is still useful, one year later. As mentioned, CSLIB is a neat project that can really help with arrays, I invite you to join the effort.

For my projects (car racing, mazes, Master Mind etc) I have relied heavily on a Python code generator, which compiles the following commands to CS code: function, for loop, array, matrix, insert, unroll, scriptblock, trackedchoice, js_set, js_get and more. I’d be happy to share the code if you are interested.

1 Like