[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Gambas-bugtracker] Bug #3188: Making Gambas GTK apps testable via AT-SPI: patch to add accessible names for Label/ComboBox/buttons

http://gambaswiki.org/bugtracker/edit?object=BUG.3188&from=L21haW4-

André ROTHE reported a new bug.

Summary
-------

Making Gambas GTK apps testable via AT-SPI: patch to add accessible names for Label/ComboBox/buttons

Type             : Request
Priority         : Medium
Gambas version   : Master
Product          : GTK+3 component


Description
-----------

Hi,

I'm using Gambas to build a complex application (a client UI for REST services), and I want to cover it with end-to-end (E2E) tests. The tests are written by Claude Code, which is surprisingly good at Gambas. The idea is to let Claude drive the running Gambas application — following my instructions — to test functionality.

This works because a lot of information about the GTK widgets is available through the AT-SPI accessibility tree (the same one screen readers use, exposed via ATK). However, some custom-drawn widgets expose an empty AT-SPI name/description, so Claude had to fall back to locating them by pixel geometry. To avoid that brittle workaround, I patched my local Gambas to fill in those missing names and descriptions. The widgets I found so far are Label, ComboBox (read-only) and the push/toggle/tool buttons — there may well be more.

I'm attaching the patch and a demo app (UI in Gambas, tests in Python) so you can check the changes. It's quite cool to watch an automated Gambas application run on your desktop :-)

About the demo: test_widgets.py compiles and starts the Gambas demo app, then selects some buttons etc. It builds on e2e_uidriver.py, which is almost generic (it works with any Gambas GTK2/GTK3 app) and provides the methods to find and operate widgets in the Gambas component tree. Claude Code can add new test steps on top of e2e_uidriver.py.

How the patch works (for review):

Since Gambas 3.16 many widgets in gb.gui.base (Label, TextLabel, ComboBox, …) are UserControls that paint their own text, so the underlying GTK widget has no GtkLabel child and ATK derives an empty accessible name. The same is true for the custom-drawn push/toggle/tool buttons in gb.gtk (their text goes through a GtkCellRendererText, not a label child).

The patch fixes this in two layers:

gb.gtk (C++): I added a hidden Control._AccessibleName property (CWidget.cpp) backed by a new gControl::setAccessibleName() (gcontrol.cpp/.h), which calls atk_object_set_name(gtk_widget_get_accessible(border), …). In gControl::setTooltip() I now also set the ATK description explicitly from the tooltip (markup-stripped via pango_parse_markup), because GTK only derives it lazily, so it reads empty/racy right after launch. In gButton::setText() the custom-drawn button branch now sets the ATK name from the plain (markup-stripped) text.
gb.gui.base (Gambas): the self-painting widgets push their displayed text into that property — e.g. Label.class calls Me._AccessibleName = $sText whenever the text changes, and ComboBox.class publishes the selected value (so a read-only combo, whose value isn't shown in a child widget, becomes readable). The call is wrapped in Try, so on a toolkit that doesn't implement _AccessibleName it's simply a no-op.
Because gb.gtk3/src/*.cpp are symlinks to gb.gtk/src/, the C++ part covers both GTK2 and GTK3. Qt is not covered — gb.qt* has no _AccessibleName, so there the Try does nothing; a QAccessible-based equivalent would be the analogous fix.

All names/descriptions added are derived from text the user already set (Text/Caption, tooltip, selected value), so they should be correct by construction. These are the widgets I hit during testing; there are very likely more UserControl-based widgets that deserve the same one-line _AccessibleName treatment.

Best regards,
André


System information
------------------

[System]
Gambas=3.21.99 51912e1da (master)
OperatingSystem=Linux
Distribution=openSUSE Leap 16.0
Kernel=6.12.0-160000.32-default
Architecture=x86_64
Cores=12
Memory=31712M
Language=de_DE.utf8
Platform=x11
Desktop=MATE
DesktopResolution=96
DesktopScale=7
WidgetTheme=green-submarine
Font=Sans,10
DarkTheme=False
[Programs]
gcc=gcc (SUSE Linux) 15.2.0
git=git version 2.51.0
msgmerge=msgmerge (GNU gettext-tools) 0.22.5
pngquant=2.18.0 (January 2023)
rpmbuild=RPM version 4.20.1
[Libraries]
Cairo=libcairo.so.2.11804.4
Curl=libcurl.so.4.8.0
DBus=libdbus-1.so.3.32.4
GDK2=libgdk-x11-2.0.so.0.2400.33
GDK3=libgdk-3.so.0.2418.32
GStreamer=libgstreamer-1.0.so.0.2607.0
GTK+2=libgtk-x11-2.0.so.0.2400.33
GTK+3=libgtk-3.so.0.2418.32
OpenGL=libGL.so.1.7.0
Poppler=libpoppler.so.148.0.0
QT5=libQt5Core.so.5.15.17
QT6=libQt6Core.so.6.9.1
RSvg=librsvg-2.so.2.60.2
SDL=libSDL2-2.0.so.0.3000.7
SQLite3=libsqlite3.so.3.51.3
[Environment]
AUDIODRIVER=pulseaudio
CLUTTER_BACKEND=x11,*
CONFIG_SITE=/usr/share/site/x86_64-pc-linux-gnu
CPU=x86_64
CSHEDIT=emacs
CVS_RSH=ssh
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
DEBUGINFOD_URLS=https://debuginfod.opensuse.org/ 
DESKTOP_SESSION=mate
DISPLAY=:0
EDITOR=pluma
FROM_HEADER=
GB_GUI=gb.gtk3
GDMFLEXISERVER=/usr/libexec/lightdm/gdmflexiserver
GDMSESSION=mate
GDM_LANG=de_DE.utf8
GIO_LAUNCHED_DESKTOP_FILE_PID=2284193
GPG_TTY=kein Terminal
GTK_IM_MODULE=ibus
GTK_MODULES=canberra-gtk-module:canberra-gtk-module
GTK_OVERLAY_SCROLLING=0
GUESTFISH_INIT=\e[1;34m
GUESTFISH_OUTPUT=\e[0m
GUESTFISH_PS1=\[\e[1;32m\]><fs>\[\e[0;31m\] 
GUESTFISH_RESTORE=\e[0m
G_BROKEN_FILENAMES=1
G_FILENAME_ENCODING=@locale,UTF-8,ISO-8859-15,CP1252
HISTSIZE=1000
HOME=<home>
HOST=<hostname>
HOSTNAME=<hostname>
HOSTTYPE=x86_64
INPUT_METHOD=ibus
JAVA_BINDIR=/usr/lib64/jvm/java-21-openjdk-21/bin
JAVA_HOME=/usr/lib64/jvm/java-21-openjdk-21
JDK_HOME=/usr/lib64/jvm/java-21-openjdk-21
JRE_HOME=/usr/lib64/jvm/java-21-openjdk-21
LANG=de_DE.utf8
LANGUAGE=de_DE.utf8
LD_LIBRARY_PATH=/usr/lib/oracle/21/client64/lib:
LESS=-M -I -R
LESSCLOSE=lessclose.sh %s %s
LESSKEY=/usr/etc/lesskey.bin
LESSOPEN=lessopen.sh %s
LESS_ADVANCED_PREPROCESSOR=no
LOGNAME=<user>
MACHTYPE=x86_64-suse-linux
MAIL=/var/mail/<user>
MANPATH=<home>/.local/share/man:/usr/local/man:/usr/local/share/man:/usr/share/man
MANPATHISSET=yes
MATE_DESKTOP_SESSION_ID=this-is-deprecated
MINICOM=-c on
MORE=-sl
MOZ_GMP_PATH=/usr/lib64/mozilla/plugins/gmp-gmpopenh264/system-installed
MULE_HOME=/opt/mule-standalone-4.6.0
NLS_LANG=GERMAN_GERMANY.UTF8
OLDPWD=<home>
OSTYPE=linux
PAGER=less
PATH=<home>/.npm-global/bin:<home>/.local/bin:<home>/bin:/usr/local/bin:/usr/bin:/bin:<home>/.dotnet/tools:/snap/bin:/var/lib/snapd/snap/bin
PROFILEREAD=true
PWD=<home>
QEMU_AUDIO_DRV=pa
QT4_IM_MODULE=xim
QT_FONT_DPI=96
QT_IM_MODULE=ibus
QT_IM_SWITCHER=imsw-multi
QT_LOGGING_RULES=*.debug=false
QT_QPA_PLATFORMTHEME=gtk2
QT_SCALE_FACTOR=1
SDK_HOME=/usr/lib64/jvm/java-21-openjdk-21
SESSION_MANAGER=local/<hostname>:@/tmp/.ICE-unix/3010,unix/<hostname>:/tmp/.ICE-unix/3010
SHELL=/bin/bash
SHLVL=0
SQLPATH=/usr/lib/oracle/21/client64/lib:
SSH_AGENT_PID=3103
SSH_ASKPASS=/usr/libexec/ssh/ssh-askpass
SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
TERM=xterm
TNS_ADMIN=/opt/iclient
TZ=:/etc/localtime
USER=<user>
VDPAU_DRIVER=va_gl
VENDOR=suse
WINDOWMANAGER=mate-session
XAUTHLOCALHOSTNAME=<hostname>
XAUTHORITY=<home>/.Xauthority
XDG_ACTIVATION_TOKEN=mate-panel-3172-<hostname>-gambas3-8_TIME2955605005
XDG_CACHE_HOME=<home>/.cache
XDG_CONFIG_DIRS=/etc/xdg:/usr/local/etc/xdg:/usr/etc/xdg
XDG_CONFIG_HOME=<home>/.config
XDG_CURRENT_DESKTOP=MATE
XDG_DATA_DIRS=<home>/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/share/mate:/usr/local/share/:/usr/share/:/var/lib/snapd/desktop
XDG_DATA_HOME=<home>/.local/share
XDG_GREETER_DATA_DIR=/var/lib/lightdm-data/<user>
XDG_RUNTIME_DIR=/run/user/1000
XDG_SEAT=seat0
XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0
XDG_SESSION_CLASS=user
XDG_SESSION_DESKTOP=mate
XDG_SESSION_ID=3
XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session0
XDG_SESSION_TYPE=x11
XDG_STATE_HOME=<home>/.local/state
XDG_VTNR=7
XKEYSYMDB=/usr/X11R6/lib/X11/XKeysymDB
XMODIFIERS=@im=ibus
XNLSPATH=/usr/share/X11/nls
XSESSION_IS_UP=yes



----[ Gambas bugtracker-list is hosted by https://www.hostsharing.net ]----