Proposal to introduce accessibility in Adventure Game Studio

[gendiskan]

Hi, my name is Sergio. Despite losing my sight in 2005, I am passionate about video games, especially graphic adventures.

Based on my training and experience in the field of inclusion as an educational advisor, I developed the project 'Videociegos'. This innovative project focuses on analyzing, promoting, and advising on accessibility in video games.

Currently, I collaborate with well-known companies in the sector like PlayStation and Xbox, as well as prominent media outlets, including Hobbyconsolas, one of the most-read Spanish websites about video games. On this platform, I have a pioneering section on an international level. You can find more information at:

https://www.hobbyconsolas.com/reportajes/especial-videociegos-mejores-videojuegos-accesibles-jugadores-ciegos-1282418 .

As my favorite genre is graphic and narrative adventures, I've also launched the project 'Choose Your Own Video Adventure' to connect readers and gamers on Zenda, the prestigious literary magazine by renowned writer Arturo Pérez Reverte. You can learn more about this project at:

https://www.zendalibros.com/elige-tu-propia-videoaventura-un-proyecto-para-animar-a-la-lectura-y-el-ocio-digital/

In collaboration with the cultural center of the U.S. Embassy, I've organized the first International Inclusive Video Game Fair 'We All Play Here,' which took place from January 18 to 20, 2024. This was the first event of its kind worldwide. Among many other activities, like awareness workshops, conferences, and tournaments, it included the 'Choose Your Own Inclusive Video Adventure' award for the most accessible adventure of the year, which was won by 'Brok the Investigator.'

For all these reasons, I would love to collaborate voluntarily to introduce accessibility measures in graphic adventures developed with the 'Adventure Game Studio' engine, similar to what exists for visual novels developed with 'Ren Py.' I always say in my articles that adventures seem to be the ideal genre to introduce accessibility in video games and the easiest to adapt for the more than 400 million gamers with disabilities worldwide, according to Microsoft.

Of course, I would also commit to spreading the word in the media I collaborate with to encourage more developers to use this engine.

Regarding this, I would like to get in touch with the people responsible for 'Adventure Game Studio,' or any developer interested in implementing accessibility in their adventures, either through messaging on these forums or via my profile on 'X,' @videociegos.

Thank you very much for your attention, I look forward to your responses!

[Alan v.Drake]

Unfortunately, we don't have the manpower to implement and maintain such a feature. Accessibility works best when the engine is actually designed for it, and Adventure Game Studio was originally developed for DOS, a legacy we haven't yet managed to fully sever.

I don't know how in practice one would go to turn a graphical game to be accessible to those who cannot see, but perhaps it could be achieved by a plugin exposing the necessary API to meet those requirements.

My only experience with accessibility is restricted to the web, where all the necessary facilities are already exposed in modern browsers, but I know very little what these integrations would mean for the engine.


- Alan

[Snarky]

I feel accessibility is something we should strive for. I haven't followed and read the links yet, so maybe all this is covered there, but what I think would be useful for AGS is to get some guidance on what would be considered the highest priorities for accessibility, and what good practice would be.

For example, are we talking about improving the AGS editor so that it is more accessible for game creators, or primarily about making the games more accessible? Or is the priority accessibility on the website and forums?

If it's mainly about the games, what are the major pain points? Perhaps there are things that could be built into the engine that would make all AGS games more accessible out of the box, without too much work. (I'm imagining this might be things like ensuring compatibility with screen magnification software or to adjust graphics to be higher-contrast, compatibility with screen reader APIs on the OS level, or perhaps the ability to remap keys via a configuration file.)

Or is it going to come down to giving game creators the tools necessary to be able to make more accessible games, with the responsibility falling on them? Perhaps some things could be added to the default templates. (For example, there could be a built-in keyboard alternative to the point-and-click UI, with buttons to move the cursor and cycle through options, if that would be useful. I would quite like having a way to cycle through all interactive things on screen without having to manually move the cursor, actually.) Or maybe an accessibility template, complete with a short tutorial on accessibility, would be more useful. (For example, it wouldn't be too hard to add something like an "alt tag" to a room, with text descriptions of what the screen shows, but I can't imagine most game developers would actually use it. Though perhaps if it was a standard part of the AGS room format and could be read from an external file, others could create accessibility "annotations" for games.)

[heltenjon]

He says that Brok the InvestiGator by @COWCAT won the award for being the most accessible game. You get an idea why on this section of the game's home page. It looks well deserved, too. COWCAT is on the forum occasionally, so it would be interesting if he could share some thoughts about what kind of tools would be helpful. Even if Alan's ultimately right, I guess I agree with Snarky that it would be nice to know more.

[Snarky]

Some good stuff in the Brok documentation! I didn't realize sending text to the clipboard could be a way to interact with screen readers.

It occurs to me that it wouldn't be hard to make a module with custom functions to log all dialog and descriptions, which could then be replayed. I've seen similar things in other engines. (This would be significantly simpler than the ability to actually rewind the game, as Ren'Py allows.)

Perhaps there could be a page on the Wiki or in the manual about ways to make your game more accessible.

[Babar]

I was always interested in making accessible games, and tried a couple of times to implement a keyboard alternative system like what Snarky mentioned, but laziness and frustrations with AGS (at the time) got in the way.

Still, there are some things we as a community could do that could help.

As I understand it, the way AGS renders text, it is pick-uppable by text-to-speech systems, so that's already a plus. Beyond that, it just rests on the developer to describe everything, maybe in a "Visually impaired" mode. Might also be nice to have the names of hotspots/characters/objects on screen somehow accessible to these systems? So like, tapping a certain key would list them all off.

Would also be nice to implement a dyslexic-friendly text system. There are a couple of open source dyslexic friendly fonts available online, but it takes some effort to optimise them to work in AGS. Beyond that, the difficulty dyslexic people have with reading text isn't just down to the font, so some system that highlighted text on screen would be nice to implement in-engine.

Talking about fonts, I don't know if this changed, or if I misremember how outlines work, but it would be nice if the colour of the outlines was also configurable (instead of just being black?).

[Danvzare]

Would also be nice to implement a dyslexic-friendly text system. There are a couple of open source dyslexic friendly fonts available online, but it takes some effort to optimise them to work in AGS. Beyond that, the difficulty dyslexic people have with reading text isn't just down to the font, so some system that highlighted text on screen would be nice to implement in-engine.

My sister is dyslexic and I can tell you that those dyslexic fonts are bullshit. Typically, they're harder to read than a normal font for dyslexics.

Also any font that helps dyslexic people read (like Comic Sans), helps everyone to read. If I recall correctly, Courier New is the easiest to read font. Also, what people consider easy to read and what actually is easy to read, are two different things. As an example, people tend to prefer to read black on white, but blue on pale yellow is easier to read.
I know this because I read a few studies on it. In them they gave people various texts to read with different formatting while tracking their eye movements to see how fast they were reading, then giving them surveys to full out.

If you want to make the engine dyslexic friendly, introduce an option in the setup to alter the text speed or make it so the text doesn't move forward unless you press a button, overriding any ingame setting. Also a text-to-voice function would be very useful as well.

Talking about fonts, I don't know if this changed, or if I misremember how outlines work, but it would be nice if the colour of the outlines was also configurable (instead of just being black?).

You can change outline colour.
It's "game.text_shadow_color" that you're looking for to do that.

[gendiskan]

Hi again, and thanks a lot for your interest.

Let me explain: my goal is for adventures developed with AGS to be accessible to as many players as possible. Ideally, and I'm not sure if this is possible because I don't know how the engine works, but based on what I've seen with the visual novels engine Ren'Py, I would like as many accessibility measures as possible to be applied by default to all adventures. Some measures would still need to be applied by each developer individually.

Many of these have already been discussed, but I'll summarize them here. I'll start with the measures that, hopefully, could be incorporated into the engine so that all adventures developed from now on include them. And if we dare to dream, it would be amazing if they could somehow be applied to already published games (I'd love to play those from Wadjet Eye Games, for example):

   

• Adjustable text size, color, and opacity, including fonts for players with dyslexia.
•     High contrast filters to highlight characters and interactive elements.
•     Filters for major types of color blindness: Protanopia, Deuteranopia, and Tritanopia.
•     Screen reader compatibility to verbalize menus, texts, and subtitles.
•     Sound adjustments (mono or stereo) and volume settings for effects, music, and voices, if any.
•     Analog control, allowing players to navigate menus and interactive elements with keys or buttons.
•     Compatibility with keyboard, gamepad, and Xbox Adaptive Controller to connect specific controllers for all types of motor impairments.

These would be the main measures that could be generally introduced. Then, each game should include other design-specific measures:

   

• Text translation.
•     Audio description: descriptions of non-verbal information, environments, and actions during cutscenes and gameplay.
•     Closed captions: Subtitles for sound effects.
•     Tutorials, hints, and reminders of objectives and major events in the game.
•     Adapting or omitting board puzzles or similar.

Again, I'm not sure if this would be possible, but it would be ideal. So, if there are any AGS engine developers reading this, please let me know if it would be possible to introduce these general measures into the engine.

Thanks so much, and let's keep in touch!

Best,
Sergio

PS: Sorry if I take a while to respond, but a little adventurer entered my life two and a half months ago and needs constant supervision.

[Crimson Wizard]

Strictly from the technical point of view (not considering potential design or configuration issues),

* Adjustable text size, color, and opacity, including fonts for players with dyslexia.
* High contrast filters to highlight characters and interactive elements.
* Filters for major types of color blindness: Protanopia, Deuteranopia, and Tritanopia.

* Sound adjustments (mono or stereo) and volume settings for effects, music, and voices, if any.

These should be relatively straightforward to do.
Speaking of fonts, in AGS currently fonts may be replaced even by placing another font file into the game directory. That is how additional translations are done (including "unofficial" translations).
But anything that changes text size may conflict with the game elements, and I don't know if that's even possible to resolve without participation from the game developer.

* Analog control, allowing players to navigate menus and interactive elements with keys or buttons.

I think this is theoretically possible, but will require implementing whole new control system within the engine from scratch, possibly with external input configuration, because keys or control buttons may be also used for certain purposes within the game (script).

* Screen reader compatibility to verbalize menus, texts, and subtitles.

I have a guess that this is easier to do using an external tool, which works over the game window with final game image.
In AGS this may be complicated to do, because it does not have distinct "text element", and texts are baked onto other graphical elements. Some game objects have "text" property, but not all, so they won't be able to tell what is drawn on them. And there's also a "raw drawing" feature that lets paint text on sprites.

* Compatibility with keyboard, gamepad, and Xbox Adaptive Controller to connect specific controllers for all types of motor impairments.

I suppose this is a task for the backend library we're using (which is currently SDL2).


PS.
As an additional note, I would also explore if above tasks may be accomplished by a engine plugin.
Plugins are convenient in the sense that one may experiment with them without adding maintenance burden to the core engine.
But, on the other hand, this may require expanding plugin API in the engine too.

[Snarky]

Thanks @gendiskan, I think that list makes it a lot easier to see what is and is not possible in AGS. FYI, Crimson Wizard is one of the main engine devs.

You should be aware that compared to Ren'Py, AGS is a more "primitive" engine in a couple of ways. First, because it is older. And second, because it is more low-level in several areas, not providing ready-made solutions but only commands in its programming language that allow game developers to build their own solutions. That means many things often end up being hardcoded in each game, and can make it more difficult to offer built-in engine-wide accessibility features.

Strictly from the technical point of view (not considering potential design or configuration issues),

* Adjustable text size, color, and opacity, including fonts for players with dyslexia.
[...]
* Sound adjustments (mono or stereo) and volume settings for effects, music, and voices, if any.

These should be relatively straightforward to do.

In terms of how this would actually work, Crimson Wizard, are you thinking that players would configure it in winsetup.exe (or by editing the cfg file), and it would adjust the default color and volume values set in the editor?

Because as we know, AGS games don't come with any default in-game configuration UI.

Also, this would only work if the game developer never uses the script API to overrule the starting values set in the editor, because then they become hardcoded. My intuition is that most games do set these values in script at some point, and so this solution would have quite limited effectiveness, but I might be wrong about that.

It would be possible to create a module (or even an engine API) that would define text styles that could be used throughout the game, which could then be configured/overruled by an external style sheet (or configured in an in-game menu), but you would need to convince game makers to actually use it (or deprecate all the commands to hardcode text appearance).

(BTW, the SpeechBubble 2.0 module that I've been working on off-and-on, still very much incomplete, has this notion of "styles" to configure text appearance, and it would be quite possible to read them from a style sheet. I'm also building an in-game UI to configure it: my experience is that this is a massive amount of work, but a lot of it has to do with configuring the appearance of the bubble, rather than the text itself.)

For audio type volume, the ideal is probably that all games have in-game volume controls for the different audio types, but since game devs have to build the GUIs, it's hard to guarantee. It could be part of the standard templates, at least.

* Analog control, allowing players to navigate menus and interactive elements with keys or buttons.

I think this is theoretically possible, but will require implementing whole new control system within the engine from scratch, possibly with external input configuration, because keys or control buttons may be also used for certain purposes within the game (script).

Interesting. It doesn't merely require new keyboard handling, but a notion of UI focus that AGS doesn't currently have. (To give an example: currently, if there are multiple text fields on-screen at the same time and you type something, it fills in every field simultaneously.)

In the near-term, it might be possible to create a module to provide keyboard control to GUIs, and add it to the default templates.

* Screen reader compatibility to verbalize menus, texts, and subtitles.

I have a guess that this is easier to do using an external tool, which works over the game window with final game image.
In AGS this may be complicated to do, because it does not have distinct "text element", and texts are baked onto other graphical elements. Some game objects have "text" property, but not all, so they won't be able to tell what is drawn on them. And there's also a "raw drawing" feature that lets paint text on sprites.

Yeah, and I'd add to the list of challenges that the way text is done in AGS, you can't always tell whether text has changed or not. For example, there could be a label with text that is set again every game cycle whether or not it is empty, the same as before, or has new content. (A mouseover tooltip is the obvious example, but depending on how things are scripted this could apply to a lot of UI text, e.g. status messages.) And again you run into the problem that AGS doesn't have a notion of UI focus, or even of a UI hierarchy.

On the other hand, I would expect it to be fairly easy to provide screen reader support for  character speech or dialog, since that does have this notion of "focus" (one character speaks at a time, except for background speech) and the content of each line is clearly defined.

However, if there was an API for screen reader integration built into AGS, it might make it easier for game makers to offer screen reader support in their games. (I'm not sure exactly what such integration requires. I had a hard time finding documentation.)

[Crimson Wizard]

In terms of how this would actually work, Crimson Wizard, are you thinking that players would configure it in winsetup.exe (or by editing the cfg file), and it would adjust the default color and volume values set in the editor?

Because as we know, AGS games don't come with any default in-game configuration UI.

Also, this would only work if the game developer never uses the script API to overrule the starting values set in the editor, because then they become hardcoded. My intuition is that most games do set these values in script at some point, and so this solution would have quite limited effectiveness, but I might be wrong about that.

The way I look at this, there are two different scenarios:
1. When the accessibility is implemented by the author of the game.
2. When the accessibility is not implemented by the author of the game.

For the first situation, the engine should provide at least a minimal necessary API for making this possible; and then the rest is doable with the help of script modules, and adjusted on per game basis.

For the second situation, the engine should provide a way to override certain game behavior, such as text looks, or audio controls. How this is configured is a problem, but at the very least there is config file.

I imagine that, for the fine tuning, the configuration would include a substitution table: font overrides, color overrides etc. Then, whenever engine draws a text, it would take the current game option for a font, or text color (whether it is coming from the global options or "drawtext" function arguments), and use that as a key in this table, where the overriding font or color is a return value.
This might also be not a precise color, but a rule, that sets minimal color levels, such as brightness, saturation etc.

Similar approach may be used for other things.

I would definitely throw in an ability to configure skipping options externally (text skipping, video skipping, cutscene skipping etc), because some games may simply not implement these or implement inconveniently.

EDIT:
The latest version of the engine supports several "overrides" in config (not exposed to winsetup), suggested to be used in case of emergency. For example, there's a way to set save/restore hotkeys that bring up default builtin save/restore dialogs. I introduced this after couple of games were released with the broken save/load functionality. Visually this looks out of place of course, but at least lets to save your progress in such game.

[Babar]

Has anyone tried any of the more popular screen reader softwares? I am sure the topic came up here before (but I can't find where with the current search functionality  ), and it was mentioned that they COULD pick up the text from AGS, and I'm pretty sure this was years ago.

[gendiskan]

Thank you so much for your interest in accessibility.

I'm not familiar with the difficulties that might come with implementing each measure. Maybe we can stay in touch via email to discuss the matter. Mine is: sergioveravalencia@gmail.com

Regarding screen readers, the most used one on Windows is called JAWS. However, the operating system itself includes another one called Narrator, which is commonly used by compatible PC games like Firebird or 1428: Shadows over Silesia.

I've asked a blind programmer who has just developed an accessible mobile game called Galadia if he could lend a hand.

Hugs, thank you very much, and let me know:

Sergio

[Snarky]

@gendiskan, taking it to email is not ideal because this isn't really a one-to-one conversation. (It should be possible to "watch" the thread so you get an email alert whenever there is a post, but unfortunately this feature doesn't seem to work correctly.)

I'm not an engine dev, but I wouldn't mind taking a crack at a module to do keyboard navigation instead of moving the mouse for GUIs and other interactive elements (hotspots, objects, characters), and getting that into the default templates.

[eri0o]

@Snarky I made this module: https://www.adventuregamestudio.co.uk/forums/index.php?topic=57379.0

[Snarky]

Well, then my work here is done. 

[Cassiebsg]

A little off topic, but not entirely...

CaptainD made this game some time ago: https://www.adventuregamestudio.co.uk/site/games/game/2298-blindsweeper/

It's basicly MineSweeper, but blind... the game "graphics" is just a black screen, and you play by sound alone. Maybe worth a try?

[Snarky]

Another thread just brought up text speed (or ability to turn off automatic text advancement entirely) as another option that it would be useful for players to override.

[Danvzare]

Another thread just brought up text speed (or ability to turn off automatic text advancement entirely) as another option that it would be useful for players to override.

I think adding this as an option would be a brilliant idea.
While it could potentially break some things in a few games, I think that's something that the user themselves should be able to handle on a case by case basis, and is a small price to pay for the extra accessibility it would add.

[Crimson Wizard]

I'm adding this for the next version of AGS: