This paper was presented at IEEE RCIS'2013 conference. It addresses the problem of modernizing graphical user interfaces of interactive applications by re-engineering their resource files in four phases: resource decompilation, which extracts resource files from the executable code of an interactive application; resource recovery, which transforms extracted resources into a Concrete User Interface Model based on static analysis and derivation rules; model editing, which ap-plies requested changes on the model, and resource recompilation, which produces a new interactive application or generation, which re-generates a new graphical user interface. The paper motivates and details this re-engineering approach by focusing on methods and algorithms implemented in UsiResourcer, a software tool that reverse engineers MS Windows resource files into a Concrete User Interface Model for further process. A discussion on the generalization of the approach is also provided
Re-Engineering Graphical User Interfaces from their Resource Files with UsiResourcer
1. IEEE International Conference on Research Challenges in Information Science RCIS'2013 (Paris, May 29-31 2013) 1
Óscar Sánchez Ramón1,2, JeanVanderdonckt1, and Jesús García Molina2
1UniversitéCatholique de Louvain (Belgium), Louvain School of Management
{oscar.sanchez,jean.vanderdonckt}@uclouvain.be
2University of Murcia, Faculty of Informatics
{osanchez,jmolina}@um.es
2. - Context and motivation
- Approach for reverse engineering GUIs from
resource files
- Limitations of the solution
- Generalisation of the solution
- Conclusions and future work
2IEEE International Conference on Research Challenges in Information Science RCIS'2013 (Paris, May 29-31 2013)
3. - Context and motivation
- Approach for reverse engineering GUIs from
resource files
- Limitations of the solution
- Generalisation of the solution
- Conclusions and future work
3IEEE International Conference on Research Challenges in Information Science RCIS'2013 (Paris, May 29-31 2013)
4. - Interactive applications evolve over time
- New platforms, devices appear with new opportunities
- New situations and usage also appear
- Users want more dynamic interaction
- Two main situations occur
- Specifications and/or models exist
- In this case, new requirements should be brought to these
specifications and/or models to produce a new Graphical User
Interface (GUI)
- Specifications and/or models are no longer accessible
- Source code exists
- Source code no longer exists (no documentation, nothing!)
4IEEE International Conference on Research Challenges in Information Science RCIS'2013 (Paris, May 29-31 2013)
5. GOAL: Reverse engineer resources (binary files)
to get UsiXML CUI models
CUI
model
UsiResourcer GrafiXML
6. - Context and motivation
- Approach for reverse engineering GUIs from
resource files
- Limitations of the solution
- Generalisation of the solution
- Conclusions and future work
6IEEE International Conference on Research Challenges in Information Science RCIS'2013 (Paris, May 29-31 2013)
7. - GUI reengineering is not new
- Different starting points: source code (programming/markup
languages,…), interaction traces, screenshots,…
- Different ending points: other source code, a model, a series
of models
- Different techniques: static and dynamic analysis, pattern
matching, slicing, …
- Use Cameleon Reference Framework (CRF) to
express starting and ending points
7IEEE International Conference on Research Challenges in Information Science RCIS'2013 (Paris, May 29-31 2013)
8. Task & Domain (T&D)
Cameleon Reference Framework in UsiXML
Final User Interface (FUI)
WindowWindow
Concrete User Interface
(CUI)
AIC
facet=control
AbstractIndividual
Container
Abstract User Interface (AUI)
textInputtextInput buttonbutton buttonbutton
AIC
facet=control
AIC
facet=control
9. Final User Interface (FUI)
Concrete User Interface
(CUI)
Abstract User Interface (AUI)
Task & Domain (T&D)Context of use
User
Platform
Environment
Context of use = (user, platform, environment)
May inform any level of abstraction
10. - Reengineering with Cameleon
- 1) Reverse engineering: Windows Resources CUI model
- 2) Restructuring: adapt CUI model
- 3) Forward engineering: regenerate code in another language
14IEEE International Conference on Research Challenges in Information Science RCIS'2013 (Paris, May 29-31 2013)
Final User Interface
(FUI)
Concrete User Interface
(CUI)
Reverse
engineering
Final User Interface
(FUI)
Concrete User Interface
(CUI)
Forward
engineering
Restructuring
11. 16IEEE International Conference on Research Challenges in Information Science RCIS'2013 (Paris, May 29-31 2013)
Executable
Interactive
application
FinalUIlevelConcreteUIlevel
Resource
decompilation
UsiResourcer:
Resource
recovery
Resource
files
CUI
model
New Re-
source
file
GrafiXML:
CUI Model
editing
New
Executable
Interactive
application
Resource
recompilatio
n
NewCUI
model
GrafiXML:
GUI
generation
NewGUI
Phase 2:
Modeling the source GUI
Phase 1: Resource decompilation
resource
model
Phase 3: Resource to CUI
M2M transformation
Phase 4: Forward engineering
12. 17IEEE International Conference on Research Challenges in Information Science RCIS'2013 (Paris, May 29-31 2013)
Executable
Interactive
application
Resource
decompilation Resource
files
Phase 1: Resource decompilation
13. 18IEEE International Conference on Research Challenges in Information Science RCIS'2013 (Paris, May 29-31 2013)
Executable
Interactive
application
Resource
decompilation
UsiResourcer:
Resource
recovery
Resource
files
Phase 2:
Modeling the source GUI
Phase 1: Resource decompilation
resource
model
15. 20IEEE International Conference on Research Challenges in Information Science RCIS'2013 (Paris, May 29-31 2013)
Executable
Interactive
application
Resource
decompilation
UsiResourcer:
Resource
recovery
Resource
files
CUI
model
Phase 2:
Modeling the source GUI
Phase 1: Resource decompilation
resource
model
Phase 3: Resource to CUI
M2M transformation
17. 22IEEE International Conference on Research Challenges in Information Science RCIS'2013 (Paris, May 29-31 2013)
DIALOG window
graphicalCio isVisible = true
box type = vertical
dlgID = n cio name = n
Width = w graphicalContainer width =
w*4/xChar + 2*border width
(where xChar is the average width of
the dialog box font character in pixel)
box width = w*4/xChar
Height = h graphicalContainer height =
h*8/yChar + title bar height
+ bottom border width (where yChar is
the average height of the dialog box
font character in pixel)
box height = h*8/yChar
Text = t and Caption
= true and t ≠ null
cio defaultContent = t
FontName = n and
SetFont = true
graphicalIndividualComponent
textFont = n
FontSize = s and
(SetFont = true or
ShellFont = true)
graphicalIndividualComponent
textSize = s
Extended = true and
(SetFont = true or
ShellFont = true) and
Weight ≥ 550
graphicalIndividualComponent
isBold = true
Disabled = true graphicalCio isEnabled = false
TopMost = true graphicalContainer isAlwaysOnTop =
true
18. 23IEEE International Conference on Research Challenges in Information Science RCIS'2013 (Paris, May 29-31 2013)
Box detection algorithm
- Search for separators iteratively:
- Vertical separators(first)
- Horizontal separators
- Stop when:
- All the elements in a box are
vertical or horizontally arranged
- No separators can be found
- Frames are considered as boxes
1
2
3
4
19. 25IEEE International Conference on Research Challenges in Information Science RCIS'2013 (Paris, May 29-31 2013)
Executable
Interactive
application
Resource
decompilation
UsiResourcer:
Resource
recovery
Resource
files
CUI
model
Phase 2:
Modeling the source GUI
Phase 1: Resource decompilation
resource
model
Phase 3: Resource to CUI
M2M transformation
20. 27IEEE International Conference on Research Challenges in Information Science RCIS'2013 (Paris, May 29-31 2013)
Executable
Interactive
application
FinalUIlevelConcreteUIlevel
Resource
decompilation
UsiResourcer:
Resource
recovery
Resource
files
CUI
model
New Re-
source
file
GrafiXML:
CUI Model
editing
New
Executable
Interactive
application
Resource
recompilatio
n
NewCUI
model
GrafiXML:
GUI
generation
NewGUI
Phase 2:
Modeling the source GUI
Phase 1: Resource decompilation
resource
model
Phase 3: Resource to CUI
M2M transformation
Phase 4: Forward engineering
21. - Context and motivation
- Approach for reverse engineering GUIs from
resource files
- Limitations of the solution
- Generalisation of the solution
- Conclusions and future work
30IEEE International Conference on Research Challenges in Information Science RCIS'2013 (Paris, May 29-31 2013)
24. - Context and motivation
- Approach for reverse engineering GUIs from
resource files
- Limitations of the solution
- Generalisation of the solution
- Conclusions and future work
33IEEE International Conference on Research Challenges in Information Science RCIS'2013 (Paris, May 29-31 2013)
25. Change resource type (operating system)
create resource metamodel, parser and transformation
Change resource file version
update resource metamodel and parser
Change target UIDL
rewrite resources2cui transformation
26. - Context and motivation
- Approach for reverse engineering GUIs from
resource files
- Limitations of the solution
- Generalisation of the solution
- Conclusions and future work
35IEEE International Conference on Research Challenges in Information Science RCIS'2013 (Paris, May 29-31 2013)
27. - UsiResourcer: Windows 7 resource file UsiXML
- The approach can be used with other resource
types
- The UsiXML CUI makes the approach
independent of the target platform/technologies
28. - Many other formats for resource files:
- XUL, IBM3270 terminal, Minitel screen definition,
- CICS with Screen Definition Facility (SDF): panel, fields, attributes, stored
objects, panel jumping, …
- MS Windows resource file
- Macintosh OS resource file (binary)
- Platform S390: BMS (CICS), MFS (IMS), SDF
- Platform AS400: SDF
29. 38
Thank you very much for your attention!
Any question?
See more at:
www.usixml.org, www.usixml.eu
www.lilab.eu, www.lilab.be
IEEE International Conference on Research Challenges in Information Science RCIS'2013 (Paris, May 29-31 2013)
Editor's Notes
The first line gives a name to the dialog box (in this case, 400). The name is followed by the keyword DIALOG and four numbers. The first two numbers are the x- and y-coordinates of the dialog box when the dialog box is invoked by the program. The second two numbers are the width and height of the dialog box. These coordinates and sizes are not pixels, but are based on the size of a system font character (in this case, an 8-point MS Shell Dlg font): x-coordinates and width are expressed in units of 1/4 of an average character width and y-coordinates and height are expressed in units of 1/8 of a character height. Because system font characters are often approximately twice as high as they are wide, the units on both the x- and y-axes are about the same. The second line defines some style for the dialog box (a combination of window style and dialog style). Within the left and right brackets are defined the child window controls that will appear in the dialog box. This dialog box uses six types of child window controls: a left-justified text, a combo box, a group box, radio buttons and push buttons. The first number is a value that the child window uses to identify itself when communicating to its parent (the dialog box window). The next four numbers set the position of the child window control (relative to the upper left corner of the dialog box's client area) and the size. Some style flag can also follow to define more precisely the appearance and functionality of the control.Notice that if the dialog boxe has a caption bar (which is the case here), these measurements concern the dialog box's client area, and the caption bar will be shown above the y-coordinate.In fact, MS Shell Dlg is not a physical font, rather a face name of a nonexistant font. It ensure the previous Windows operating system compatibility. It can be specified in either the Setup file during the installation process or when customizing a local system by double-clicking Control Panel's Regional Options icon. Because the font generated using MS Shell Dlg is different on different versions of Windows, the dialog box can look different depending on the version.This allows using coordinates and sizes that will retain the general dimensions and look of the dialog box regardless of the resolution of the video display.Because the text control does not send messages back to its parent, this value is set to -1.