Translation and Internationalisation

Today we're going to tell you about our unique solutions to the problem of effective translation and internationalisation in our games.

Internationalisation, and language translation in general, is a topic that doesn't often get considered early on in a game's development cycle. Ignoring it at the start of development can make it much harder to effectively add it later on, which is a lesson we learnt from our last release AdvertCity. But it's a very important feature to add for a game that wants to succeed.

The problem: We want our game to be accessible as possible, which means it should be playable by anyone who wants to, wherever they are in the world. However, not everyone understands English, and even in a text-light action game like sphereFACE, it's necessary to be able to understand menus, success and death messages to have a good experience. So how do you identify what language someone wants to speak, and how do you then present all your game elements in their ideal language?

Let's break this down into its components:

  • Problem 1: How to identify what language someone wants to speak?
  • Problem 2: How to store in-game text data in such a way as to enable it to be retrieved at runtime not just for the correct language, with the correct grammar?
  • Problem 3: How to get your game text translated into as many languages as possible?

Let's address each of these in turn.

1. How to identify what language someone wants to speak?

This part is relatively simple. In most cases, you can trust the user's computer's "locale" setting to tell you what language they have their operating system set to. To do this we use the open source Boost::Locale library which can tell us the ISO-639 code ("en") and the ISO-3199 country code ("US"). We can also provide an override to players as a menu option, to let them choose a different language from a dropdown of available languages.

The list of languages sphereFACE alpha currently supports

2. How to store in-game text data in such a way as to enable it to be retrieved at runtime not just for the correct language, with the correct grammar?

This is the interesting part. This is a challenge thousands of developers have faced over the decades, and there have been many different solutions for this. So how is ours different?

Most translation systems start by separating the text from the rest of the code (which is a very good idea, for reasons we'll explain in section 3). Say you have a button in a game that lets you borrow a certain amount of cash from the bank, that looks like this:
Yep, that currency is Dogecoin

In a non-internationalised game, the code for that button might look like:
The text is constructed in-place, amidst the game logic

The parts in yellow make up the "string" of text that is shown to the player, and the number is inserted automatically based on a value only known at runtime, so can't be hardcoded. A simple translation system might then extract the string for "Borrow", add a space, and extract the string for "XDG" (the currency marker). It would then have both strings imported using a function that chooses one of several standalone files, each of which contains all the strings in that language.

This system has some limitations that become obvious very quickly. What if you're translating into a language that would put the amount borrowed before the word "borrow", instead of after? What if you're translating into a language where spaces don't separate words?

This is usually solved by having a meta-data system in the translation file; in this case the string would look something like "Borrow %sXDG" where %s marks the point the numerical value should be inserted. This works for many strings, and is a very common solution - for instance, Gnu Gettext uses this kind of scheme, and in turn is widely used in all sorts of software, some of which you might be running right now on whatever device your'e reading this on. One downside of this solution is that you're parsing text at runtime, which can be computationally expensive; for something like a word processor this might not matter very much, but for a game, every shred of CPU power is vital to producing as smooth an experience as possible.

This method does still have some other limitations, too. In a more complex case, what if you want to differentiate between singular and plural values - "1 asteroid" vs "2 asteroids"? The in-place substitution scheme above isn't sufficiently complex to cope with that, and most developers will work around this by having their game logic take up the slack by just choosing between a "singular" and "plural" version of the text string to get. The translator has to do more work for these cases by translating the entire sentence twice just to change a single word, or in our English case, a single letter. But even this might not be enough. Some languages might not have a separate pluralisation for 1 item vs 2 or more items; or they might have a separate pluralisation grammar for 2 of something vs 3 of something. Or they might translate zero as a singular, while in English we call zero a plural, i.e. "there are no asteroids". In many cases, developers just give up and settle for a less-than-perfect translation for those "problem" languages.

sphereFACE death message in (Brazilian) Portuguese

Obviously a more flexible solution is required, and ideally one that doesn't have the performance cost of realtime string parsing.

The way we solve this problem is by separating the strings along with their logic from the bulk of the code, but still keeping them in C++ header files with the very minimum amount of syntactic glue necessary to hold them together, and import them into code at runtime. Each translation phrase is constructed as a polymorphic virtual function:
The simple localisation code for a "close" button

This has a mix of advantages and disadvantages, so there's a tradeoff involved. On the one hand, this is a lot more verbose than the method above, and it also requires the translator to understand which parts need to be changed and which left alone - in this case, only those bits in quotes should be translated. On the other hand, it gives you the advantage of being able to add comments about context in a clear way next to the text to be translated, which can really help with clarity of translation. Secondly, it improves performance because you're not parsing text from an external file at runtime, instead generating your output programmatically - and the compiler can optimise this a lot, to improve performance considerably. Most importantly, it also allows for far more powerful language structures:
Translation code for (the English version of) the "borrow" button seen above

Here we specify the amount borrowed as a parameter passed to the function. Here the code constructs and returns a string containing the relevant parts of the phrase, and incorporating the amount in the right place - as well as expressing it in fixed-point rather than scientific notation. We can also comment the context, and add more comments in-line explaining different parts of the code. Bear in mind this is still kept separate in a language file specific to the language, in this case, English. The actual button code (compare to the original code above) now looks like this:
This is much simpler, because we've moved the text generation logic into the language-specific function.

Now we never have to change the game logic to account for languages with more complexity, such as different styles of pluralisation, and different layouts. The tradeoff here is that your translators have to have some basic understanding of how the code in your language function is laid out and what it does, and this gets more complicated with more sophisticated text constructs. But that complexity would still be there if using a templated "%s" style substitution, and that would also still be coupled to game logic to work out plurals versus singulars and so on - so in practice the overall complexity is actually slightly lower. It's also possible to construct far more elaborate structures, such as:

This complicated function from AdvertCity describes how a building is situated relative to a road in the city, based on various parameters. An example of the text it produces is: "On a busy one way boulevard, 200m long, from which you can see the ocean."

A translator doesn't have to understand the code here - they can simply replace all the text inside the double quotes to produce a naive translation. But if we're lucky enough to have a translator with some programming skill for a given language, they can rearrange the structures, reorder the strings and set it up to produce something that flows like a natural sentence in their language. And being able to add detailed comments here, like the section dividers, and little explanatory notes about unit lengths, gives our translators all the tools they need to work with this to get great results - even if they aren't themselves confident with code, this sort of structure aims to be as self-explanatory as possible.

A final benefit of doing this in polymorphic code is that it allows us to specialise only some parts of a language; rather than having to duplicate all English strings to localise into both British and American English, we can have a base English for which most text will be the same in common between the two, and we just specialise spellings such as "color" vs "colour". This makes the overall maintenance much simpler and reduces duplicated effort considerably. And reducing effort is really the name of the game when you're on a tight budget - which brings us on to...

3. How to get your game text translated into as many languages as possible?

This is a trick question really; the real answer is "you pay as many translators as you can!" But in a game like this, where budget is very limited, we have to try to minimise how much we spend on translation while getting the best result we can. That's why having polymorphic systems that allow you to specialise only some parts of a language make a huge difference. We mentioned earlier that separating translatable text from code is a good idea - you don't want to force your translators to sift through your entire codebase to find those strings that need to be translated. Our system of having one file with just the language functions for that language means each translator for each language only ever needs to deal with one file.

It makes translating pluralisations and other variations of a string a lot simpler, too - the sentence-generator above would have needed one version for every possible permutation of 6, 2, and 3 different parameters to translate the old-fashioned way accurately - that would be 36 variations on the same sentence in an traditional translation file to get the same result!

One more advantage of this system to productivity is that the heavy use of context alongside the text to be translated allows for fewer mistakes or inappropriate translations for the context, which means fewer corrections are needed later and the overall translation quality is much higher.

The Steam Greenlight page for sphereFACE showing supported languages

Finally, building the system in from the very beginning means we can get some localisation done on a voluntary basis from helpers within our network - we couldn't ask a volunteer to translate a thousand words for free, but it's easy to get a sentence at a time translated here and there; this is how we've already managed to get sphereFACE fully localised into several languages even at this early Alpha phase.

Hopefully this has shed some light on our approach to solving one of the more common problems games face in the international market!