Job mediation services can assist job seekers in finding suitable employment through a personalised approach. Consultation or mediation sessions, supported by personal profile data of the job seeker, help job mediators understand personal situation and requests. Prediction and recommendation systems can directly provide job seekers with possible job vacancies. However, incorrect or unrealistic suggestions, and bad interpretations can result in bad decisions or demotivation of the job seeker. This paper explores how an interactive dashboard visualising prediction and recommendation output can help support the dialogue between job mediator and job seeker, by increasing the "explainability" and providing mediators with control over the information that is shown to job seekers.

1. Supporting job mediator
and job seeker through an
actionable dashboard
Francisco Gutiérrez, Sven Charleer & Katrien Verbert
https://augment.cs.kuleuven.be/
1

2. [background]
!2

3. [background]
!3

4. [background]
!4
Job Seeker Mediation Sessions “Dream Job”

5. Services:
Register in the service.
Update profile.
Advising job seekers who struggle to
find a job.
5
[context]
VDAB employs job mediators all
over the country to support job
seekers.
Mediation sessions can last
from 15 minutes to two hours.

6. [context]
!6
RECOMMENDER SYSTEM
Black Box (?) Dream Job
MEDIATORJOB SEEKER

7. 7
[context]
The Mediation service wants
to improve advice sessions
through a new recommender
system!
Three years of data,
700 000 job seekers.
Predicts the chance of
finding a job within 120 days.

8. 8
[context]
Original dashboard for mediators
Age
Studies
Experience and competences
Languages
Education
Increase opportunities
Teacher
Chance of finding the job within
120 days

9. 9
[context]
Original dashboard for mediators

10. 10
[goal]
Design a tool that attempts to
increase:
Trust
Control and justification of predictions
Acceptance of recommendations
Supportive and collaborative
-> (during mediation sessions)

11. Support the dialogue between
“expert” and “laymen”
“… the expert can become the
intermediary between the [system] and
the [end-user] in order to avoid
misinterpreta4on and incorrect decisions
on behalf of the data… “
Sven Charleer, Andrew Vande Moere, Joris Klerkx, Katrien Verbert, and Tinne
De Laet. 2017. Learning Analytics Dashboards to Support Adviser-Student
Dialogue. IEEE Transactions on Learning Technologies (2017), 1–12.
11
[goal]
Learning analytics, student
counseling

12. 12
[preliminary study]
Costumer journey approach
During one day workshop to gain insight into typical job
seeker-mediator session (n = 5)
Observation of hands-on time
With the original dashboard
Observations of individual mediation sessions
Between mediators and job-seekers (Nm = 3, Njs= 6) 15-30 min
5-Likert Scale Questionnaire
Perceptions of dashboard and predictions

13. 13
[findings]
Contextual Information
Lack of data might lead to unrealistic recommendations.
RS can take a [supportive role] during the sessions.
Transparency
Mediators aware of the “blackbox”.
Provide further information through visualisation.
Controllability
Mediator needs to remain in control.
Recommender-output <- [mediator] - > Job Seeker (motivation)
Justification
Provide visual support to convey the message. [reality check]
Job-seeker visual literacy plays an important factor.

14. 14
[design goals]
[DG1] Control the message
The mediator filters the information flow to convey a message,
and avoid potential demotivation.
[DG2] Clarify the recommendations
The mediator requires further details about predictions
[DG3] Support the mediator
The dashboard assists the mediator during the session

15. 15
[working prototype]
The dashboard suggests four jobs. …architect (?)

16. 16
Age was removed to avoid demotivation. Days
unemployed was left as “eye opener”.
[working prototype]

17. 17
Different ways to present parameter data. (age)
[design & development]
Forest plot Circles chart Barchart

18. Qualitative evaluation with expert users:
(N = 12, 10f, age: M= 40.7, SD = 9.4)
18
[evaluation]
Years of experience: (M = 9, SD = 4.3)
Six mediators dealt only with higher education job seekers.
Four with secondary to higher education.
Two with job seekers without technical/professional education.
Semi-structured interviews
1) Feedback on parameter visuals.
2) Interaction feedback with the working prototype dashboard.

19. Feedback on parameter visuals
Circles chart, barchart. forest plot
out-loud interpretation
answer a 5-likert scale regarding clarity of visuals
19
[evaluation]
Interaction feedback with prototype
Screen recorded, data included errors on purpose.
Task:
“prepare the dashboard for a costumer that is recently unemployed,
experienced in cleaning and wishes to change careers if possible.”

20. 20
[results]
[DG1] Control the message
* Five mediators used negative parameters to support their message.
* Two mediators removed negative parameters to avoid demotivation.
Two themes
(1) Customization
“Incorrect predictions must go” “age can be demotivating” “too much
information might be difficult to process” “would like to see an overview of
everything” “depends on the job seeker”
(2) Importance of the human factor
Data is only part of the information on the job seeker and misses external
context gathered during the session.

21. 21
[results]
[DG2] Clarify recommendations
Two themes
(1) Understanding the visualisation
Circles was considered the most clear representation to
mediators. Too much information might be hard to interpret.
“Can be used as an eye-opener” “If all parameters are negative
dashboard should not be shown”
(2) Convincing power
Certain data might be confrontational and demotivating,
customization helps the mediator filter the message.
higher education
lower education

22. 22
[results]
[DG3] Support the mediator
The way of using the dashboard is highly dependent on the
mediator and the situation.
Useful cases:
Orientation cases
The job seeker does not have a good idea of possible career
paths. The mediator can “help orientate” them.
Mediator is stuck
Can be used as a “starting point”. provide support to guide long
conversations. Deal with “problem cases”, “does not know what he/
she wants and is being resistant.

23. 23
[conclusions]
With our tool experts become “gatekeepers” of the data
Dashboard carries the message, mediator “moulds” the message.
RS with increased “explainability” increase user’s trust
and provides them with actionable insights
Customization is used beyond data filtering
Visualization supports a dialogue creating “reality checks” by revealing
negative factors and assist in motivation.

24. 24
[conclusions]
Job mediators face a broad job-seeker audience.
Different educational backgrounds, social situations, personalities.
Incompleteness and lack of profile information
Makes visual job search and suggestions “as-is” difficult.
We attempt to clarify recommendations to both
mediators and job seekers.
Help mediators to control the message they wish to convey
depending on the context.

25. 25
[future work]
Deploy the dashboard in realistic
settings!
Gain further insight into the impact of the tool in
mediation sessions.
Labor market exploration.
Explore tools to enable job seekers to explore the labor market in a
personalized way.

26. 26
thanks! questions… (?)
Sven Charleer
Phd. Computer Science
Freelance UX researcher/designer
sven.charleer@gmail.com
@SvenCharleer | Twitter
SvenCharleer.com
Francisco Gutiérrez
Phd. Student
Augment HCI, KU Leuven
francisco.gutierrez@cs.kuleuven.be
@FranciscoGhz | Twitter
augment.cs.kuleuven.be