[Gambas-user] [Gambas Bug Tracker] Bug #920: Optimize JSON.Encode for big data structures, lowers execution time from seconds to milliseconds.

Sat Apr 16 12:35:26 CEST 2016

http://gambaswiki.org/bugtracker/edit?object=BUG.920&from=L21haW4-

Alexander KUIPER reported a new bug.

Summary
-------

Optimize JSON.Encode for big data structures, lowers execution time from seconds to milliseconds.

Type             : Request
Priority         : Medium
Gambas version   : 3.8
Product          : Unknown

Description
-----------

The JSON.Encode works fine for small data structures, but when I was testing with a big dat structure (5000 entries), the performance of the JSON.Encode takes 4-10 seconds. This looks to be caused by the string appending call "&=" (which is called recursive).

By replacing the "&=" concatenation with a stream and using the latest "Open String" feature, it goes from seconds to milliseconds.

The Gambas3 JSON.Encode call takes +/- 8 seconds
START - JSON.Encode - 04/16/2016 12:33:34.645
STOP  - JSON.Encode - 04/16/2016 12:33:42.037

Same call with using Streams - less then 120 milliseconds:
START - JEncode - 04/16/2016 12:33:34.528
STOP  - JEncode - 04/16/2016 12:33:34.637

BTW: See the attached example project, with the JEncode function

System information
------------------

[System]
Gambas=3.8.4
OperatingSystem=Linux
Kernel=3.13.0-79-generic
Architecture=x86_64
Distribution=Ubuntu 14.04.4 LTS
Desktop=GNOME
Theme=Gtk
Language=en_US.UTF-8
Memory=980M

[Libraries]
Cairo=libcairo.so.2.11301.0
Curl=libcurl.so.4.3.0
DBus=libdbus-1.so.3.7.6
GStreamer=libgstreamer-0.10.so.0.30.0
GStreamer=libgstreamer-1.0.so.0.204.0
GTK+2=libgtk-x11-2.0.so.0.2400.27
GTK+3=libgtk-3.so.0.1000.8
OpenGL=libGL.so.1.2.0
Poppler=libpoppler.so.44.0.0
QT4=libQtCore.so.4.8.6
QT5=libQt5Core.so.5.2.1
SDL=libSDL-1.2.so.0.11.4

[Environment]
CLUTTER_IM_MODULE=xim
COLORTERM=gnome-terminal
COMPIZ_BIN_PATH=/usr/bin/
COMPIZ_CONFIG_PROFILE=ubuntu
DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-CKsI05Pf4P
DEFAULTS_PATH=/usr/share/gconf/ubuntu.default.path
DESKTOP_SESSION=ubuntu
DISPLAY=:0
GB_GUI=gb.qt4
GDMSESSION=ubuntu
GDM_LANG=en_US
GNOME_DESKTOP_SESSION_ID=this-is-deprecated
GNOME_KEYRING_CONTROL=/run/user/1000/keyring-GGg4gU
GNOME_KEYRING_PID=2842
GPG_AGENT_INFO=/run/user/1000/keyring-GGg4gU/gpg:0:1
GTK_IM_MODULE=ibus
GTK_MODULES=overlay-scrollbar:unity-gtk-module
HOME=<home>
IM_CONFIG_PHASE=1
INSTANCE=
JOB=dbus
LANG=en_US.UTF-8
LANGUAGE=en_US
LC_ADDRESS=nl_NL.UTF-8
LC_IDENTIFICATION=nl_NL.UTF-8
LC_MEASUREMENT=nl_NL.UTF-8
LC_MONETARY=nl_NL.UTF-8
LC_NAME=nl_NL.UTF-8
LC_NUMERIC=nl_NL.UTF-8
LC_PAPER=nl_NL.UTF-8
LC_TELEPHONE=nl_NL.UTF-8
LC_TIME=nl_NL.UTF-8
LESSCLOSE=/usr/bin/lesspipe %s %s
LESSOPEN=| /usr/bin/lesspipe %s
LOGNAME=<user>
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36:
MANDATORY_PATH=/usr/share/gconf/ubuntu.mandatory.path
OLDPWD=<home>/JSONEncodeImprovement/.src
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
PWD=<home>
QT4_IM_MODULE=xim
QT_IM_MODULE=ibus
QT_QPA_PLATFORMTHEME=appmenu-qt5
SELINUX_INIT=YES
SESSION=ubuntu
SESSIONTYPE=gnome-session
SESSION_MANAGER=local/<hostname>:@/tmp/.ICE-unix/2943,unix/<hostname>:/tmp/.ICE-unix/2943
SHELL=/bin/bash
SHLVL=1
SSH_AUTH_SOCK=/run/user/1000/keyring-GGg4gU/ssh
TERM=xterm
TEXTDOMAIN=im-config
TEXTDOMAINDIR=/usr/share/locale/
TZ=:/etc/localtime
UPSTART_SESSION=unix:abstract=/com/ubuntu/upstart-session/1000/2613
USER=<user>
VTE_VERSION=3409
WINDOWID=58720267
XAUTHORITY=<home>/.Xauthority
XDG_CONFIG_DIRS=/etc/xdg/xdg-ubuntu:/usr/share/upstart/xdg:/etc/xdg
XDG_CURRENT_DESKTOP=Unity
XDG_DATA_DIRS=/usr/share/ubuntu:/usr/share/gnome:/usr/local/share/:/usr/share/
XDG_GREETER_DATA_DIR=/var/lib/lightdm-data/<user>
XDG_MENU_PREFIX=gnome-
XDG_RUNTIME_DIR=/run/user/1000
XDG_SEAT=seat0
XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0
XDG_SESSION_ID=c1
XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session0
XDG_VTNR=7
XMODIFIERS=@...3498...=ibus
_=/usr/bin/gambas3