SlideShare una empresa de Scribd logo
1 de 6
Descargar para leer sin conexión
Abstract—Exam is an evaluation tool to measure teaching
and learning outcomes of educators and their learners
respectively. Nowadays, an automated exam question set
generator is a must have to reduce educator’s time on
preparation of exam question set and increase the quality of
exam question set. This paper proposes an Automated Exam
Question Set Generator (AEQSG) using Utility Based Agent
(UBA) and Learning Agent (LA). Furthermore, AEQSG applies
Bloom Taxonomy (BT) scaling to automate Bloom’s Taxonomy
(BT) difficulty level distribution and Genetic Algorithm (GA) to
optimize the generation of exam question set and generate high
quality exam question set that follow educational institution’s
guide-lines. The purpose of utility based agent in AEQSG is to
give the user an option to choose actions based on a user’s
preference (utility) for each generation state. Meanwhile, the
purpose of learning agent in AEQSG is to learn from its past
exam results (past generation experiences).
Index Terms—Automated exam question generator, bloom’s
taxonomy scaling, genetic algorithm, learning agent, utility
based agent.
I. INTRODUCTION
Exams are implemented in educational institutions to
monitor the progress of educators’ teaching outcome and
learners’ learning outcome. The manual preparation process
of exam question set requires educators’ great commitments
and is time-consuming. In order to reduce educators’
commitments in preparing exam question set, we have
proposed an Automated Exam Question Set Generator
(AEQSG) using Utility Based Agent (UBA), Learning Agent
(LA), Bloom Taxonomy (BT) Scaling and Genetic
Algorithm (GA).
UBA chooses actions based on a preference (utility) for
each state that maximizes the expected utility. Meanwhile,
LA starts to act with basic knowledge (question mark) and
then are able to act and adapt automatically (average past
question result mark) through learning. In our proposed
system, UBA and LA will be applying BT scaling to achieve
a desirable result. BT scaling automate the distribution of
Bloom’s Taxonomy difficulty level and automate learning
experience using Bell Curve Analysis based on the average
past exam question result mark to get optimal result for exam
question set. GA optimizes the generation of exam question
Manuscript received September 25, 2019; revised December 22, 2019
Tengku Nurulhuda Tengku Abd Rahim, Ma. Stella Tabora Domingo,
Mohamed Farid Noor Batcha are with the MIMOS Berhad, Technology Park
Malaysia, Kuala Lumpur 57000 Malaysia (e-mail: huda.rahim@mimos.my,
stella.domingo@mimos.my, farid.batcha@mimos.my).
Zalilah Abd Aziz is with the Faculty of Computer and Mathematical
Sciences, Universiti Teknologi MARA, Boulder, 40450 Shah Alam,
Selangor, Malaysia (e-mail: zalilah@tmsk.uitm.edu.my).
set based on chosen preference (utility) in AEQSG.
Bloom’s Taxonomy (BT) and GA will not be detailed
since this paper is an extended paper from the Automated
Exam Question Generator using Genetic Algorithm paper
[1].
II. INTELLIGENT AGENT
A. Utility Based Agent (UBA)
Utility based agent is almost similar to the goal based agent
that acts based on goals but it also finds the optimum way to
achieve the goal. It is beneficial when there are multiple
possible alternatives and an agent has to choose before
performing the best action. The utility function maps each
state to a real number to evaluate the efficiency of each action
in achieving the goals [2].
UBA as in Fig. 1 uses a world model together with a utility
function that measures its preferences among states and then
chooses the action that leads to the best expected utility. By
averaging over all possible outcome states, weighted by the
probability of the outcome; expected utility is computed [3].
Fig. 1. A model-based, utility-based agent [3].
B. Learning Agent (LA)
A general intelligent agent known as a learning agent is the
preferred method for creating state-of-the-art systems in
Artificial Intelligence where any type of agents can be
generalized into a learning agent to generate better actions [4].
A learning agent has the ability to learn from its past
experiences even though it starts with basic knowledge and
then adapts automatically through learning [2].
There are four main conceptual components in a learning
agent as shown in Fig. 2. The learning element is responsible
on improvements based on learning from the environment.
The learning element takes feedback from the critic.
Automated Exam Question Set Generator Using Utility
Based Agent and Learning Agent
Tengku Nurulhuda Tengku Abd Rahim, Ma. Stella Tabora Domingo, Mohamed Farid Noor Batcha, and
Zalilah Abd Aziz
International Journal of Machine Learning and Computing, Vol. 10, No. 1, January 2020
164
doi: 10.18178/ijmlc.2020.10.1.914
Meanwhile the critic describes how well the agent is doing
with respect to a fixed performance standard. The
performance element is responsible for external action
selection. The problem generator is responsible for actions
suggestion that will lead to new and informative experiences.
Therefore, a learning agent is able to learn, analyze
performance and look for new ways to optimize its
performance [2].
Fig. 2. A general learning agent [3].
III. DEEP REINFORCEMENT LEARNING
Deep Reinforcement Learning (DRL) is the combination
of Reinforcement Learning (RL) and Deep Learning (DL);
whereas RL with function approximation by deep neural
networks [5]. RL and DL are both systems that learn
autonomously. The difference between RL and DL is RL
learns dynamically by adjusting actions using continuous
feedback in order to optimize the reward, while DL learns
from a training set and then applies that learning to a new data
set [6].
Meanwhile, the difference between RL and DRL is RL is
dynamically learning with a trial and error method to
maximize the outcome, while DRL is learning from existing
knowledge and applying it to a new data set [7]. Unlike
supervised and unsupervised learning algorithms, RL is
about training an agent to interact with an environment and
maximize its reward [8].
RL model as shown in Fig. 3 consist of three basic
concepts: state, action, and reward. The state describes the
current situation of the agent. The action describes the action
taken by agent at each state. The reward describes the
feedback from the environment [8].
Fig. 3. Reinforcement learning model [8].
IV. RELATED WORK
Automated Test Paper Generation [9] is a web-based
application system prototype implementing utility-based
agent to cater exam difficulty level, and shuffling algorithm
to design question set randomly without repetition or
duplication. On the basis of utility value, two sets of question
items are selected. The first question items set consist of the
highest utility value, whereas the second question items set
consist of the second highest utility value of the test paper.
Accumulative difficulty level of the test paper is calculated
based on the question items’ utility value.
An intelligent agent is introduced to Automatic Question
Paper Setting (AQPS) [10] to automate the generation of
question paper using random number generation technique
and backtracking algorithm known as Automatic Question
Paper Generator Agent (AQPA). Basic components of AQPS
are Hash Table (HT) database, Repeated Question Checker
Module (RQCM) and Question Paper Generation Module
(QPGM). The HT database is used to store questions with
their keys with optimum space; in order to find required
question very quickly. RQCM ensures that a new retrieved
question does not match with any question in selected
question database. QPGM generates the question paper based
on selected pre-defined question format and selected question
database.
The proposed framework for Automatic Question
Generation (AQG) [11] is multi-agent which consists of
Document Processing Agent (DPA), Information
Classification Agent (ICA) and Question Generation Agent
(QGA). DPA will first start processing input text using Tree
Tagger tool to produce a one word per line form together with
its tag and lemma; the processed output of Tree Tagger is
ranked based on the frequency of each word occurrence; then
prefix and suffix of each word is removed with a stemming
process to get the root word. Bloom’s category is found in
ICA based on selected keywords after the stemming process
and word count from DPA by matching keyword with action
verb in the repository. The output from ICA will be used in
QGA to generate questions based on template that suit the
selected keywords according to the Bloom’s levels.
The two main components in Utility Based Test Paper
(UBTP) agent [12] are Knowledgebase Developer (KD) and
Test Paper Generator (TPG). KD is a component that stores
question items with its utility value with integer value in
range between 0 and 5 in file form. TPG is a component that
generate a test paper from selected question items (filtered by
its utility value) from knowledgebase based on overall
difficulty level (from 1 percent to 100 percent) entered by
examiner. Three sets of question items will be selected on the
basis of utility value in TPG algorithm. In other words, first
set has the highest utility value calculated, second set has one
less than highest utility value calculated and third set has 0
utility values.
Most of papers above implement Utility Based Agent
(UBA) only in their system but none of them are using
Learning Agent (LA) or Deep Reinforcement Learning
(DRL). In order to improve the quality of Exam Question Set
Generator, we propose multi agent with combination of two
agents; UBA and LA. We also suggest to apply DRL for LA.
V. PROPOSED SYSTEM
The overall architecture of the proposed system i.e.
Automated Exam Question Set Generator (AEQSG) using
Utility Based Agent (UBA) and Learning Agent (LA) is
shown in Fig. 4.
International Journal of Machine Learning and Computing, Vol. 10, No. 1, January 2020
165
Exam Question
Set
Exam Question
Set Generator
Question Set
Preference
(Utility)
Question
Bank
Fig. 4. Automated exam question set generator overall architecture.
Question collection workflow process as shown in Fig. 5
will be done before hand to provide the proposed system with
a collection of questions either new or previous question from
past exams. User can either key in or uploaded file for new
and previous questions into the system. Then the system will
automatically validate each uploaded file and validate each
questions keyed in or uploaded. After that, all validated
questions will be stored in the question bank.
Start
Initialize
Question List
(LQ)
Input
Question(s)
Details
Upload File(s)
For Each
Question (q)
Is q Valid?
Yes
Has More q?
Add q to LQ
No
For Each File (f)
Is f Valid?
Yes
No
Has More f?
No
No
End
Save LQ Question
Bank
Yes
Yes
Fig. 5. Question collection workflow process.
All uncategorised questions either new question or
previous question from past exam that does not have any BT
category or chapter category assigned when the questions are
keyed in or uploaded using file; the proposed system will
automatically categorise based on BT categories (as shown in
Table I) and chapter during question categorisation workflow
process (as shown in Fig. 6).
Therefore the person in charge to key in or upload
questions to proposed system can be anyone other than
Subject Matter Expert (SME). Thus, in the same time will
helps to reduce educators’ commitments and reduce
significant amount of time in preparing exam question set. In
addition to above benefits, the question categories would be
more appropriated to course syllabus guideline since it not
only categorised BT category but also chapter category for
the specific question saved into question bank.
TABLE I: BLOOM’S TAXONOMY CLASSIFICATION [1]
ID Category Level
C1 Knowledge
Easy
C2 Comprehension
C3 Application
Medium
C4 Analysis
C5 Synthesis
Hard
C6 Evaluation
Question categorisation workflow process will be started
with pre-processing of uncategorised questions using Natural
Language Processing pre-processing (NLP pre-processing)
included lower case standardization, remove non alphabet
character, tokenization, Part of Speech (POS) tagging, noun
extraction, verb extraction and stemming verbs before
Bloom’s Taxonomy classification (BT classification) and
chapter classification parallel workflow process begin.
Start
End
Lower Case
Standardization
Remove Non-
Alphabet
Character
Question
Bank
Tokenization
Stemming Verb
WordNet
Bloom s
Taxonomy
Classification
Verb Extraction
(LV)
Retrieve
Uncategorised
Question
Noun Extraction
(LN)
Chapter
Classification
Part of Speech
(POS) Tagging
Fig. 6. Question categorisation workflow process.
Start End
Retrieve Verb
List (LV)
For Each Verb
(v)
BT Verbs
Found CBT? Yes
LBT != Empty?
Identify BT
Category of v
(CBT(v))
No
No
Add CBT(v) into
LBT
Has More v?
Yes
No
Select CBT with
Highest Occurrence
(Max(CBT))
Assign CBT to
Question
Question
Bank
Update
Question
Select The Most
Relevant CBT
based on Rules
Yes
Count Occurrence
for Each CBT
(Sum(CBT))
No
Yes
Has > 1
Max(CBT)?
Initialize BT
Category List
(LBT)
Fig. 7. Bloom’s taxonomy classification workflow process.
In BT classification workflow process (as shown in Fig. 7)
each verb in verb list extracted after NLP pre-processing
International Journal of Machine Learning and Computing, Vol. 10, No. 1, January 2020
166
done; will be identified with respective BT categories based
on Bloom’s Taxonomy (BT) verbs. Then respective BT
categories with highest occurrence will be selected after the
summation of each BT categories done. If more than one
highest occurrence found, the most relevant BT category will
be selected based on rules. The selected BT category will be
assigned to the question and the question in question bank
will be updated with selected BT category.
End
Convert LN to
N-gram List
(LNG)
For Each N-
gram (ng)
Chapters
/ Nouns
Found C?
Yes
LC != empty?
Identify Chapter
of ng (C)
No
No
Add C into LC
Has More
ng?
Yes
No
Select C with
Highest Occurrence
(Max(C))
Assign C to
Question
Question
Bank
Update
Question
Select The Most
Relevant C
based on Rules
Yes
Count
Occurrence of
Each C (Sum(C))
No
Yes
Have > 1
Max(C)?
Start
Initialize
Chapter List (LC)
Retrieve Noun
List (LN)
Fig. 8. Chapter classification workflow process.
Meanwhile in chapter classification workflow process (as
shown in Fig. 8), the noun list extracted from previous NLP
pre-processing will be converted into n-gram list. After that
each n-gram in n-gram list will be identified with respective
chapter categories based on Chapters / Nouns reference table.
Then, each chapter categories summation done, the
respective chapter categories with highest occurrence will be
selected. The most relevant chapter category will be selected
based on rules, if more than one highest occurrence found.
Later, the question in question bank will be assigned and
updated with selected chapter category.
Utility Based Agent
Genetic
Algorithm (GA)
Learning Agent
Bloom s
Taxonomy
Scaling
Question Set
Preference
Exam Question
Set
User
Fig. 9. Proposed model for automated exam question set generator.
Our proposed system, AEQSG will be implemented using
both Utility Based Agent (UBA) and Learning Agent (LA). It
also implement Bloom’s Taxonomy (BT) Scaling and
Genetic Algorithm (GA) to optimize the process and enhance
the quality of the new exam question set. The proposed model
for AEQSG using UBA and LA is shown in Fig. 9.
The UBA and LA are chosen as a model for the proposed
system to achieve our AEQSG goals while optimizing exam
question set generation using GA. Moreover, LA is chosen to
increase the quality of exam question set using Bell Curve
Analysis based on the average past exam question result mark
in BT Scaling. Furthermore, BT scaling also provides the
mechanism to automate the distribution of Bloom’s
Taxonomy difficulty.
AEQSG will generated the new exam question set from
question bank based on user’s question preference. The user
is given the option to select excluded year range, academic
year, semester, education level, exam duration, exam
difficulty, subject, chapter selection, total question and
question margin for question set preference (utility function)
before the new exam question set is generated. User’s
question preference workflow process as shown in Fig. 10
will be executed before GA workflow process begin.
Start
Input Question
Set Preference
(P)
Is TQ > 2
& CS > 0
& TQ > CS?
Save P, MBT and
MQ
Yes
No
Question Set
Preference
Excluded Year
Range
Reference (EY)
Academic Year
Reference (AY)
Semester
Reference
(SM)
Subject
Reference (SJ)
Chapter
Selection
Reference (CS)
Education
Level
Reference (LV)
Exam Duration
Reference (DR)
Exam Difficulty
Reference (DF)
Total Question
Reference (TQ)
Question
Margin
Reference
(QM)
Calculate Max
BT Category Per
Chapter (MBT)
Calculate Max
Question Per BT
Category (MQ)
Initialize Fittest
Question Set
(QSF)
Question
Bank
A
Fig. 10. User’s question preference workflow process.
GA workflow process as shown in Fig 11, start with
initialize question sets based on user’s preferences question
set. After question set been initialized, the fitness value for
each question sets will be evaluated and question set is
selected using Roulette Wheel selection. Then, single
crossover and mutation will be performed based on selected
question set. After single crossover and mutation done,
fitness value for new question sets will be evaluated. Later,
International Journal of Machine Learning and Computing, Vol. 10, No. 1, January 2020
167
the best fitness will be selected from the current question sets
and pass to Bloom Taxonomy (BT) Scaling.
UBA chooses actions based on a preference (utility) for
each state that maximizes the expected utility. UBA using
GA to optimizes the generation of exam question set based on
chosen preference (utility) in AEQSG and generate high
quality exam question set that follow educational institution’s
guide-lines.
The fitness formula as shown in Fig 12, is used to calculate
the fitness value in GA (that resides in UBA) to get the best
fitness before the fitness value is pass to LA to proceed with
BT scaling process. W is represent exam question set quality
weightage percentage as shown in Table II. The lowest value
of the fitness value is the best fitness.
Initialize Question
Sets Based on P
(QSTQ)
End
For Each Iteration
(i)
Select QS using
Roulette Wheel
Selection
Perform Single
Crossover
Perform
Mutation
Save QSF
Has More i?
Yes
Question
Bank
Bloom s
Taxonomy
Scaling (BTS)
Evaluate Fitness
Value for Each
New QS
No
Evaluate Fitness
Value for Each
Question Set (QS)
Select Best
Fitness QS (QSB)
Evaluate New
Fitness Value
After BTS
Assign New
Fitness QS to
QSF
Yes
New Fitness
Value is Fittest?
No
A
Fig. 11. Genetic algorithm workflow process.
Start
Retrieve
Question Set
(QSF)
For Each
Question (q)
Retrieve AS
Question
Bank
Has Average
Score (AS)?
Yes
Has More q? No
Yes
No
AS = Question
Mark
Add AS into LS
Initialize Score
List (LS)
Scores
B
Fig. 12. BT scaling – average score workflow process.
LA starts to act with basic knowledge (question mark)
where there is no result mark average (as shown in Fig. 13);
and then it is able to act and adapt automatically (average past
question result mark) through learning. BT scaling automate
the distribution of Bloom’s Taxonomy difficulty level (as
shown in Fig. 14) and automate learning experience using
Bell Curve Analysis based on the average past exam question
result mark (as shown in Fig. 15) to get optimal result for
exam question set.
TABLE II: QUALITY OF EXAM WEIGHTAGE [1]
No. of
Level
No. of
Category
Example Weightage Quality
3
6 C1, C2, C3, C4, C5, C6 100
Good
5 C1, C2, C3, C4, C6 90
4 C1, C2, C4, C5 80
3 C2, C4, C5 70
4 C3, C4, C5, C6 60
2 3 C1, C3, C4 50 Average
2 C4, C5 40
1
2 C3, C4 30
Bad
1 C5 20
Start
Retrieve
Question Set
(QSF)
For Each
Question (q)
Retrieve AS
Question
Bank
Has Average
Score (AS)?
Yes
Has More q? No
Yes
No
AS = Question
Mark
Add AS into LS
Initialize Score
List (LS)
Scores
B
Fig. 13. BT scaling – average score workflow process.
Calculate Total
Question (TQ)
For Each BT
Category (c)
Has More c?
No
Total
TQ > MQ?
Retrieve Max
Question Per BT
Category (MQ)
Remove q from
QSF
Yes
Yes
Calculate
Current Total
Question (TCQ)
No
Retrieve Initial
Total
Question (TQ)
TQ > TCQ?
Yes
No
Select BT Category
with Min
Occurrence (NBT)
Select New
Question Based
on NBT (QBT)
Add QBT into
QSF
Recalculate TCQ
Question
Bank
Question
Bank
No
B C
Fig. 14. BT scaling – average score workflow process.
By adopting Deep Reinforcement Learning (DRL) in LA,
we will beneficial from both Reinforcement Learning (RL)
and Deep Learning (DL) advantages. RL training an agent to
interact with an environment and maximize its reward while
DL complement RL by providing RL with function
approximation by deep neural networks. RL is dynamically
learning with a trial and error method to maximize the
outcome but DRL learns dynamically by adjusting actions
using continuous feedback in order to optimize the reward
and applying it to a new data set.
Since we are in the midst of exploring DRL for this
proposed system, we will not discussed DRL in further
details. What kind of algorithm or model or deep neural
network that will be used in future for the proposed system
will not been described in this paper.
Scores
Bell Curve Analysis
Based on LS
No
Yes
Expected Bell
Curve?
End
Perform Mutation
on QNS with Same
BT Category
Update AS in LS
Select New
Question with Min
Average Score (QNS)
Retrieve AS
Has AS?
Yes
No
AS = Question
Mark
Question
Bank
No
C
Fig. 15. BT scaling – average score workflow process.
VI. CONCLUSION AND FUTURE WORK
This paper present a novel and feasible approach for
Automated Exam Question Generator using Utility Based
Agent (UBA) and Learning Agent (LA) by implementing
Genetic Algorithm (GA) and Bloom’s Taxonomy (BT)
scaling which includes automated Bloom’s Taxonomy (BT)
difficulty distribution in exam question set and Bell Curve
Analysis.
In future, a few researches can be done based on LA to
explored more on Deep Reinforcement Learning (DRL) with
the combination of other type of algorithms or model or deep
neural networks that will be used to improvise current
proposed system.
CONFLICT OF INTEREST
The authors declare no conflict of interest.
AUTHOR CONTRIBUTIONS
All works in this paper have been done amongst authors.
REFERENCES
[1] T. N. Tengku Abd Rahim, Z. Abd Aziz, R. H. Ab Rauf and N.
Shamsudin, “Automated Exam Question Generator using Genetic
Algorithm,” in Proc. 2017 IEEE Conference on e-Learning,
e-Management and e-Services (IC3e), Shah Alam, 2017.
[2] Types of AI Agents. Java T Point. [Online]. Available:
https://www.javatpoint.com/types-of-ai-agents
[3] S. Russell and P. Norvig, Artificial Intelligence, A Modern Approach,
Third Edition, New Jersey: Pearson Education, 2010.
[4] P. Gupta, Rational Agents for Artificial Intelligence, Hackernoon,
September 2017.
[5] Funstematics.ai. 2019. [Online]. Available:
https://www.funstematics.ai/ai-case-studies
[6] M. Rouse, “deep learning,” SearchEnterpriseAI, TechTarget, October
2019.
[7] T. Williams. (September 2019). Reinforcement learning Vs. deep
reinforcement learning: What’s the difference?” Techopedia Inc.
[Online]. Available:
https://www.techopedia.com/reinforcement-learning-vs-deep-reinforc
ement-learning-whats-the-difference/2/34039
[8] Jay, “Machines that learn with reinforcement learning,” Adverai, July
2018.
[9] S. A El_Rahman and A. H. Zolait, “Automated test paper generation
using utility based agent and shuffling algorithm,” International
Journal of Web-Based Learning and Teaching Technologies, pp. 69-83,
2019.
[10] K. H. Pinjani, R. Y. Raut, and P. S. Yedey, “Developing an intelligent
agent for Automatic Question Paper setting,” in Proc. National
Conference on Advanced Technologies in Computing and
Networking-ATCON-2015, Amravati, 2015.
[11] S. Pandey and K. C. Rajeswari, “Automatic question generation using
software agents for technical institutions,” International Journal of
Advanced Computer Research, vol. 3, no. 13, pp. 307-311, 2013.
[12] M. J. Arshad, M. Naz, Y. Saleem, A. Farooq and K. H. Asif, “Modeling
an agent for paper generation system using utility based approach,”
Journal of Faculty of Engineering & Technology, pp. 109-124, 2012.
Copyright © 2020 by the authors. This is an open access article distributed
under the Creative Commons Attribution License which permits unrestricted
use, distribution, and reproduction in any medium, provided the original
work is properly cited (CC BY 4.0).
Tengku Nurulhuda Tengku Abd Rahim received
the B.I.T. degree in computer system technology from
University of Malaysia Sarawak (UNIMAS), Kota
Samarahan, Sarawak, Malaysia in 2000 and the
M.C.S. degree in computer science from MARA
University of Technology (UiTM), Shah Alam,
Selangor, Malaysia in 2016. She is currently a senior
engineer at Artificial Intelligence Department,
MIMOS Berhad, Kuala Lumpur, Malaysia.
Ma. Stella Tabora Domingo graduated with BSCoE
in the field of computer engineering from University
of Baguio, Philippines in 2005
She works currently as a staff engineer at the
Artificial Intelligence Department, MIMOS Berhad,
Kuala Lumpur, Malaysia.
Mohamed Farid Noor Batcha graduated with BSEE
in the field of communications from University of
Kentucky, United States in 2001 and obtained his
masters of engineering (electrical) from University
Technology Malaysia in 2010.
He is currently a staff engineer at Artificial
Intelligence Department, MIMOS Berhad, Kuala
Lumpur, Malaysia.
Zalilah Abd Aziz received the BSc (Hons) degree in
computer science from ITM/UKM, Shah Alam,
Selangor, Malaysia in 1996, the MSc in computer
science (software engineering) from University Putra
Malaysia (UPM) in 2003, Serdang, Selangor,
Malaysia and PhD in Computer science (artificial
intelligence) from University of Nottingham,
Nottingham, United Kingdom in 2013.
She is currently a senior lecturer at the Faculty of
Computer and Mathematical Sciences, MARA
University of Technology (UiTM), Shah Alam, Selangor, Malaysia since
1997. Her primary research include software quality, software engineering,
artificial intelligence, combinatorial optimization problems, metaheuristics
and programming.
o

Más contenido relacionado

La actualidad más candente

IRJET - Neural Network based Leaf Disease Detection and Remedy Recommenda...
IRJET -  	  Neural Network based Leaf Disease Detection and Remedy Recommenda...IRJET -  	  Neural Network based Leaf Disease Detection and Remedy Recommenda...
IRJET - Neural Network based Leaf Disease Detection and Remedy Recommenda...IRJET Journal
 
Test Case Optimization and Redundancy Reduction Using GA and Neural Networks
Test Case Optimization and Redundancy Reduction Using GA and Neural Networks Test Case Optimization and Redundancy Reduction Using GA and Neural Networks
Test Case Optimization and Redundancy Reduction Using GA and Neural Networks IJECEIAES
 
Experiences in shift left test approach
Experiences in shift left test approachExperiences in shift left test approach
Experiences in shift left test approachJournal Papers
 
SOFTWARE TESTING: ISSUES AND CHALLENGES OF ARTIFICIAL INTELLIGENCE & MACHINE ...
SOFTWARE TESTING: ISSUES AND CHALLENGES OF ARTIFICIAL INTELLIGENCE & MACHINE ...SOFTWARE TESTING: ISSUES AND CHALLENGES OF ARTIFICIAL INTELLIGENCE & MACHINE ...
SOFTWARE TESTING: ISSUES AND CHALLENGES OF ARTIFICIAL INTELLIGENCE & MACHINE ...ijaia
 
Abstract.doc
Abstract.docAbstract.doc
Abstract.docbutest
 
A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...
A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...
A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...Editor IJCATR
 
Promise 2011: "An Iterative Semi-supervised Approach to Software Fault Predic...
Promise 2011: "An Iterative Semi-supervised Approach to Software Fault Predic...Promise 2011: "An Iterative Semi-supervised Approach to Software Fault Predic...
Promise 2011: "An Iterative Semi-supervised Approach to Software Fault Predic...CS, NcState
 
Determination of Software Release Instant of Three-Tier Client Server Softwar...
Determination of Software Release Instant of Three-Tier Client Server Softwar...Determination of Software Release Instant of Three-Tier Client Server Softwar...
Determination of Software Release Instant of Three-Tier Client Server Softwar...Waqas Tariq
 
COMPARATIVE STUDY OF SOFTWARE ESTIMATION TECHNIQUES
COMPARATIVE STUDY OF SOFTWARE ESTIMATION TECHNIQUES COMPARATIVE STUDY OF SOFTWARE ESTIMATION TECHNIQUES
COMPARATIVE STUDY OF SOFTWARE ESTIMATION TECHNIQUES ijseajournal
 
IRJET - A Novel Approach for Software Defect Prediction based on Dimensio...
IRJET -  	  A Novel Approach for Software Defect Prediction based on Dimensio...IRJET -  	  A Novel Approach for Software Defect Prediction based on Dimensio...
IRJET - A Novel Approach for Software Defect Prediction based on Dimensio...IRJET Journal
 
TOWARDS PREDICTING SOFTWARE DEFECTS WITH CLUSTERING TECHNIQUES
TOWARDS PREDICTING SOFTWARE DEFECTS WITH CLUSTERING TECHNIQUESTOWARDS PREDICTING SOFTWARE DEFECTS WITH CLUSTERING TECHNIQUES
TOWARDS PREDICTING SOFTWARE DEFECTS WITH CLUSTERING TECHNIQUESijaia
 
IRJET- Software Bug Prediction using Machine Learning Approach
IRJET- Software Bug Prediction using Machine Learning ApproachIRJET- Software Bug Prediction using Machine Learning Approach
IRJET- Software Bug Prediction using Machine Learning ApproachIRJET Journal
 
Genetic algorithm based approach for
Genetic algorithm based approach forGenetic algorithm based approach for
Genetic algorithm based approach forIJCSES Journal
 

La actualidad más candente (19)

IRJET - Neural Network based Leaf Disease Detection and Remedy Recommenda...
IRJET -  	  Neural Network based Leaf Disease Detection and Remedy Recommenda...IRJET -  	  Neural Network based Leaf Disease Detection and Remedy Recommenda...
IRJET - Neural Network based Leaf Disease Detection and Remedy Recommenda...
 
Test Case Optimization and Redundancy Reduction Using GA and Neural Networks
Test Case Optimization and Redundancy Reduction Using GA and Neural Networks Test Case Optimization and Redundancy Reduction Using GA and Neural Networks
Test Case Optimization and Redundancy Reduction Using GA and Neural Networks
 
Experiences in shift left test approach
Experiences in shift left test approachExperiences in shift left test approach
Experiences in shift left test approach
 
Dc35579583
Dc35579583Dc35579583
Dc35579583
 
SOFTWARE TESTING: ISSUES AND CHALLENGES OF ARTIFICIAL INTELLIGENCE & MACHINE ...
SOFTWARE TESTING: ISSUES AND CHALLENGES OF ARTIFICIAL INTELLIGENCE & MACHINE ...SOFTWARE TESTING: ISSUES AND CHALLENGES OF ARTIFICIAL INTELLIGENCE & MACHINE ...
SOFTWARE TESTING: ISSUES AND CHALLENGES OF ARTIFICIAL INTELLIGENCE & MACHINE ...
 
M018147883
M018147883M018147883
M018147883
 
Abstract.doc
Abstract.docAbstract.doc
Abstract.doc
 
A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...
A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...
A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...
 
Kumar2021
Kumar2021Kumar2021
Kumar2021
 
Promise 2011: "An Iterative Semi-supervised Approach to Software Fault Predic...
Promise 2011: "An Iterative Semi-supervised Approach to Software Fault Predic...Promise 2011: "An Iterative Semi-supervised Approach to Software Fault Predic...
Promise 2011: "An Iterative Semi-supervised Approach to Software Fault Predic...
 
Determination of Software Release Instant of Three-Tier Client Server Softwar...
Determination of Software Release Instant of Three-Tier Client Server Softwar...Determination of Software Release Instant of Three-Tier Client Server Softwar...
Determination of Software Release Instant of Three-Tier Client Server Softwar...
 
COMPARATIVE STUDY OF SOFTWARE ESTIMATION TECHNIQUES
COMPARATIVE STUDY OF SOFTWARE ESTIMATION TECHNIQUES COMPARATIVE STUDY OF SOFTWARE ESTIMATION TECHNIQUES
COMPARATIVE STUDY OF SOFTWARE ESTIMATION TECHNIQUES
 
06522405
0652240506522405
06522405
 
Ijcatr04051006
Ijcatr04051006Ijcatr04051006
Ijcatr04051006
 
Bd36334337
Bd36334337Bd36334337
Bd36334337
 
IRJET - A Novel Approach for Software Defect Prediction based on Dimensio...
IRJET -  	  A Novel Approach for Software Defect Prediction based on Dimensio...IRJET -  	  A Novel Approach for Software Defect Prediction based on Dimensio...
IRJET - A Novel Approach for Software Defect Prediction based on Dimensio...
 
TOWARDS PREDICTING SOFTWARE DEFECTS WITH CLUSTERING TECHNIQUES
TOWARDS PREDICTING SOFTWARE DEFECTS WITH CLUSTERING TECHNIQUESTOWARDS PREDICTING SOFTWARE DEFECTS WITH CLUSTERING TECHNIQUES
TOWARDS PREDICTING SOFTWARE DEFECTS WITH CLUSTERING TECHNIQUES
 
IRJET- Software Bug Prediction using Machine Learning Approach
IRJET- Software Bug Prediction using Machine Learning ApproachIRJET- Software Bug Prediction using Machine Learning Approach
IRJET- Software Bug Prediction using Machine Learning Approach
 
Genetic algorithm based approach for
Genetic algorithm based approach forGenetic algorithm based approach for
Genetic algorithm based approach for
 

Similar a Automated exam question set generator using utility based agent and learning agent

final report (ppt)
final report (ppt)final report (ppt)
final report (ppt)butest
 
IRJET- Analysis of Question and Answering Recommendation System
IRJET-  	  Analysis of Question and Answering Recommendation SystemIRJET-  	  Analysis of Question and Answering Recommendation System
IRJET- Analysis of Question and Answering Recommendation SystemIRJET Journal
 
Student Selection Based On Academic Achievement System using k-mean Algorithm
Student Selection Based On Academic Achievement System using k-mean AlgorithmStudent Selection Based On Academic Achievement System using k-mean Algorithm
Student Selection Based On Academic Achievement System using k-mean AlgorithmNik Ridhuan
 
Online Examination and Evaluation System
Online Examination and Evaluation SystemOnline Examination and Evaluation System
Online Examination and Evaluation SystemIRJET Journal
 
IRJET- Automated Exam Question Generator using Genetic Algorithm
IRJET-  	  Automated Exam Question Generator using Genetic AlgorithmIRJET-  	  Automated Exam Question Generator using Genetic Algorithm
IRJET- Automated Exam Question Generator using Genetic AlgorithmIRJET Journal
 
Automated Question Paper Generator And Answer Checker Using Information Retri...
Automated Question Paper Generator And Answer Checker Using Information Retri...Automated Question Paper Generator And Answer Checker Using Information Retri...
Automated Question Paper Generator And Answer Checker Using Information Retri...Sheila Sinclair
 
A Study on Machine Learning and Its Working
A Study on Machine Learning and Its WorkingA Study on Machine Learning and Its Working
A Study on Machine Learning and Its WorkingIJMTST Journal
 
IRJET- Evaluation Technique of Student Performance in various Courses
IRJET- Evaluation Technique of Student Performance in various CoursesIRJET- Evaluation Technique of Student Performance in various Courses
IRJET- Evaluation Technique of Student Performance in various CoursesIRJET Journal
 
IRJET- An Automated Approach to Conduct Pune University’s In-Sem Examination
IRJET- An Automated Approach to Conduct Pune University’s In-Sem ExaminationIRJET- An Automated Approach to Conduct Pune University’s In-Sem Examination
IRJET- An Automated Approach to Conduct Pune University’s In-Sem ExaminationIRJET Journal
 
An unsupervised feature selection algorithm with feature ranking for maximizi...
An unsupervised feature selection algorithm with feature ranking for maximizi...An unsupervised feature selection algorithm with feature ranking for maximizi...
An unsupervised feature selection algorithm with feature ranking for maximizi...Asir Singh
 
ASSOCIATION RULE DISCOVERY FOR STUDENT PERFORMANCE PREDICTION USING METAHEURI...
ASSOCIATION RULE DISCOVERY FOR STUDENT PERFORMANCE PREDICTION USING METAHEURI...ASSOCIATION RULE DISCOVERY FOR STUDENT PERFORMANCE PREDICTION USING METAHEURI...
ASSOCIATION RULE DISCOVERY FOR STUDENT PERFORMANCE PREDICTION USING METAHEURI...cscpconf
 
Association rule discovery for student performance prediction using metaheuri...
Association rule discovery for student performance prediction using metaheuri...Association rule discovery for student performance prediction using metaheuri...
Association rule discovery for student performance prediction using metaheuri...csandit
 
Supervised learning techniques and applications
Supervised learning techniques and applicationsSupervised learning techniques and applications
Supervised learning techniques and applicationsBenjaminlapid1
 
IRJET- Prediction of Crime Rate Analysis using Supervised Classification Mach...
IRJET- Prediction of Crime Rate Analysis using Supervised Classification Mach...IRJET- Prediction of Crime Rate Analysis using Supervised Classification Mach...
IRJET- Prediction of Crime Rate Analysis using Supervised Classification Mach...IRJET Journal
 
trialFinal report7th sem.pdf
trialFinal report7th sem.pdftrialFinal report7th sem.pdf
trialFinal report7th sem.pdfUMAPATEL34
 
IRJET- Tracking and Predicting Student Performance using Machine Learning
IRJET- Tracking and Predicting Student Performance using Machine LearningIRJET- Tracking and Predicting Student Performance using Machine Learning
IRJET- Tracking and Predicting Student Performance using Machine LearningIRJET Journal
 
Introduction to Machine Learning
Introduction to Machine LearningIntroduction to Machine Learning
Introduction to Machine LearningSujith Jayaprakash
 

Similar a Automated exam question set generator using utility based agent and learning agent (20)

final report (ppt)
final report (ppt)final report (ppt)
final report (ppt)
 
IRJET- Analysis of Question and Answering Recommendation System
IRJET-  	  Analysis of Question and Answering Recommendation SystemIRJET-  	  Analysis of Question and Answering Recommendation System
IRJET- Analysis of Question and Answering Recommendation System
 
Student Selection Based On Academic Achievement System using k-mean Algorithm
Student Selection Based On Academic Achievement System using k-mean AlgorithmStudent Selection Based On Academic Achievement System using k-mean Algorithm
Student Selection Based On Academic Achievement System using k-mean Algorithm
 
final
finalfinal
final
 
Online Examination and Evaluation System
Online Examination and Evaluation SystemOnline Examination and Evaluation System
Online Examination and Evaluation System
 
IRJET- Automated Exam Question Generator using Genetic Algorithm
IRJET-  	  Automated Exam Question Generator using Genetic AlgorithmIRJET-  	  Automated Exam Question Generator using Genetic Algorithm
IRJET- Automated Exam Question Generator using Genetic Algorithm
 
Automated Question Paper Generator And Answer Checker Using Information Retri...
Automated Question Paper Generator And Answer Checker Using Information Retri...Automated Question Paper Generator And Answer Checker Using Information Retri...
Automated Question Paper Generator And Answer Checker Using Information Retri...
 
A Study on Machine Learning and Its Working
A Study on Machine Learning and Its WorkingA Study on Machine Learning and Its Working
A Study on Machine Learning and Its Working
 
IRJET- Evaluation Technique of Student Performance in various Courses
IRJET- Evaluation Technique of Student Performance in various CoursesIRJET- Evaluation Technique of Student Performance in various Courses
IRJET- Evaluation Technique of Student Performance in various Courses
 
IRJET- An Automated Approach to Conduct Pune University’s In-Sem Examination
IRJET- An Automated Approach to Conduct Pune University’s In-Sem ExaminationIRJET- An Automated Approach to Conduct Pune University’s In-Sem Examination
IRJET- An Automated Approach to Conduct Pune University’s In-Sem Examination
 
An unsupervised feature selection algorithm with feature ranking for maximizi...
An unsupervised feature selection algorithm with feature ranking for maximizi...An unsupervised feature selection algorithm with feature ranking for maximizi...
An unsupervised feature selection algorithm with feature ranking for maximizi...
 
ASSOCIATION RULE DISCOVERY FOR STUDENT PERFORMANCE PREDICTION USING METAHEURI...
ASSOCIATION RULE DISCOVERY FOR STUDENT PERFORMANCE PREDICTION USING METAHEURI...ASSOCIATION RULE DISCOVERY FOR STUDENT PERFORMANCE PREDICTION USING METAHEURI...
ASSOCIATION RULE DISCOVERY FOR STUDENT PERFORMANCE PREDICTION USING METAHEURI...
 
Association rule discovery for student performance prediction using metaheuri...
Association rule discovery for student performance prediction using metaheuri...Association rule discovery for student performance prediction using metaheuri...
Association rule discovery for student performance prediction using metaheuri...
 
Supervised learning techniques and applications
Supervised learning techniques and applicationsSupervised learning techniques and applications
Supervised learning techniques and applications
 
IRJET- Prediction of Crime Rate Analysis using Supervised Classification Mach...
IRJET- Prediction of Crime Rate Analysis using Supervised Classification Mach...IRJET- Prediction of Crime Rate Analysis using Supervised Classification Mach...
IRJET- Prediction of Crime Rate Analysis using Supervised Classification Mach...
 
Machine Learning by Rj
Machine Learning by RjMachine Learning by Rj
Machine Learning by Rj
 
B05110409
B05110409B05110409
B05110409
 
trialFinal report7th sem.pdf
trialFinal report7th sem.pdftrialFinal report7th sem.pdf
trialFinal report7th sem.pdf
 
IRJET- Tracking and Predicting Student Performance using Machine Learning
IRJET- Tracking and Predicting Student Performance using Machine LearningIRJET- Tracking and Predicting Student Performance using Machine Learning
IRJET- Tracking and Predicting Student Performance using Machine Learning
 
Introduction to Machine Learning
Introduction to Machine LearningIntroduction to Machine Learning
Introduction to Machine Learning
 

Más de Journal Papers

Graphene field effect transistor simulation with tcad on top-gate dielectric ...
Graphene field effect transistor simulation with tcad on top-gate dielectric ...Graphene field effect transistor simulation with tcad on top-gate dielectric ...
Graphene field effect transistor simulation with tcad on top-gate dielectric ...Journal Papers
 
Electrochemically reduced graphene oxide (ergo) as humidity sensor effect o...
Electrochemically reduced graphene oxide (ergo) as humidity sensor   effect o...Electrochemically reduced graphene oxide (ergo) as humidity sensor   effect o...
Electrochemically reduced graphene oxide (ergo) as humidity sensor effect o...Journal Papers
 
Electrical bistabilities behaviour of all solution-processed non-volatile mem...
Electrical bistabilities behaviour of all solution-processed non-volatile mem...Electrical bistabilities behaviour of all solution-processed non-volatile mem...
Electrical bistabilities behaviour of all solution-processed non-volatile mem...Journal Papers
 
Electrical transportation mechanisms of molybdenum disulfide flakes graphene ...
Electrical transportation mechanisms of molybdenum disulfide flakes graphene ...Electrical transportation mechanisms of molybdenum disulfide flakes graphene ...
Electrical transportation mechanisms of molybdenum disulfide flakes graphene ...Journal Papers
 
A real time aggressive human behaviour detection system in cage environment a...
A real time aggressive human behaviour detection system in cage environment a...A real time aggressive human behaviour detection system in cage environment a...
A real time aggressive human behaviour detection system in cage environment a...Journal Papers
 
A numerical analysis of various p h level for fiber optic ph sensor based on ...
A numerical analysis of various p h level for fiber optic ph sensor based on ...A numerical analysis of various p h level for fiber optic ph sensor based on ...
A numerical analysis of various p h level for fiber optic ph sensor based on ...Journal Papers
 
A novel character segmentation reconstruction approach for license plate reco...
A novel character segmentation reconstruction approach for license plate reco...A novel character segmentation reconstruction approach for license plate reco...
A novel character segmentation reconstruction approach for license plate reco...Journal Papers
 
A hybrid model based on constraint oselm, adaptive weighted src and knn for l...
A hybrid model based on constraint oselm, adaptive weighted src and knn for l...A hybrid model based on constraint oselm, adaptive weighted src and knn for l...
A hybrid model based on constraint oselm, adaptive weighted src and knn for l...Journal Papers
 
Wafer scale fabrication of nitrogen-doped reduced graphene oxide with enhance...
Wafer scale fabrication of nitrogen-doped reduced graphene oxide with enhance...Wafer scale fabrication of nitrogen-doped reduced graphene oxide with enhance...
Wafer scale fabrication of nitrogen-doped reduced graphene oxide with enhance...Journal Papers
 
Ultrasonic atomization of graphene derivatives for heat spreader thin film de...
Ultrasonic atomization of graphene derivatives for heat spreader thin film de...Ultrasonic atomization of graphene derivatives for heat spreader thin film de...
Ultrasonic atomization of graphene derivatives for heat spreader thin film de...Journal Papers
 
Preliminary study of poly (tetrahydrofurturyl acrylate) thin film as a potent...
Preliminary study of poly (tetrahydrofurturyl acrylate) thin film as a potent...Preliminary study of poly (tetrahydrofurturyl acrylate) thin film as a potent...
Preliminary study of poly (tetrahydrofurturyl acrylate) thin film as a potent...Journal Papers
 
New weight function for adapting handover margin level over contiguous carrie...
New weight function for adapting handover margin level over contiguous carrie...New weight function for adapting handover margin level over contiguous carrie...
New weight function for adapting handover margin level over contiguous carrie...Journal Papers
 
Implementation of embedded real time monitoring temperature and humidity system
Implementation of embedded real time monitoring temperature and humidity systemImplementation of embedded real time monitoring temperature and humidity system
Implementation of embedded real time monitoring temperature and humidity systemJournal Papers
 
High voltage graphene nanowall trench mos barrier schottky diode characteriza...
High voltage graphene nanowall trench mos barrier schottky diode characteriza...High voltage graphene nanowall trench mos barrier schottky diode characteriza...
High voltage graphene nanowall trench mos barrier schottky diode characteriza...Journal Papers
 
High precision location tracking technology in ir4.0
High precision location tracking technology in ir4.0High precision location tracking technology in ir4.0
High precision location tracking technology in ir4.0Journal Papers
 
Modeling of dirac voltage for highly p doped graphene field effect transistor...
Modeling of dirac voltage for highly p doped graphene field effect transistor...Modeling of dirac voltage for highly p doped graphene field effect transistor...
Modeling of dirac voltage for highly p doped graphene field effect transistor...Journal Papers
 
Implementation of vehicle ventilation system using node mcu esp8266 for remot...
Implementation of vehicle ventilation system using node mcu esp8266 for remot...Implementation of vehicle ventilation system using node mcu esp8266 for remot...
Implementation of vehicle ventilation system using node mcu esp8266 for remot...Journal Papers
 
Energy level tuning of cd se colloidal quantum dots in ternary 0d 2d-2d cdse ...
Energy level tuning of cd se colloidal quantum dots in ternary 0d 2d-2d cdse ...Energy level tuning of cd se colloidal quantum dots in ternary 0d 2d-2d cdse ...
Energy level tuning of cd se colloidal quantum dots in ternary 0d 2d-2d cdse ...Journal Papers
 
A review on graphene based light emitting functional devices
A review on graphene based light emitting functional devicesA review on graphene based light emitting functional devices
A review on graphene based light emitting functional devicesJournal Papers
 
Uncertainty on virtualization and internet of things (iot) the connectivity...
Uncertainty on virtualization and internet of things (iot)   the connectivity...Uncertainty on virtualization and internet of things (iot)   the connectivity...
Uncertainty on virtualization and internet of things (iot) the connectivity...Journal Papers
 

Más de Journal Papers (20)

Graphene field effect transistor simulation with tcad on top-gate dielectric ...
Graphene field effect transistor simulation with tcad on top-gate dielectric ...Graphene field effect transistor simulation with tcad on top-gate dielectric ...
Graphene field effect transistor simulation with tcad on top-gate dielectric ...
 
Electrochemically reduced graphene oxide (ergo) as humidity sensor effect o...
Electrochemically reduced graphene oxide (ergo) as humidity sensor   effect o...Electrochemically reduced graphene oxide (ergo) as humidity sensor   effect o...
Electrochemically reduced graphene oxide (ergo) as humidity sensor effect o...
 
Electrical bistabilities behaviour of all solution-processed non-volatile mem...
Electrical bistabilities behaviour of all solution-processed non-volatile mem...Electrical bistabilities behaviour of all solution-processed non-volatile mem...
Electrical bistabilities behaviour of all solution-processed non-volatile mem...
 
Electrical transportation mechanisms of molybdenum disulfide flakes graphene ...
Electrical transportation mechanisms of molybdenum disulfide flakes graphene ...Electrical transportation mechanisms of molybdenum disulfide flakes graphene ...
Electrical transportation mechanisms of molybdenum disulfide flakes graphene ...
 
A real time aggressive human behaviour detection system in cage environment a...
A real time aggressive human behaviour detection system in cage environment a...A real time aggressive human behaviour detection system in cage environment a...
A real time aggressive human behaviour detection system in cage environment a...
 
A numerical analysis of various p h level for fiber optic ph sensor based on ...
A numerical analysis of various p h level for fiber optic ph sensor based on ...A numerical analysis of various p h level for fiber optic ph sensor based on ...
A numerical analysis of various p h level for fiber optic ph sensor based on ...
 
A novel character segmentation reconstruction approach for license plate reco...
A novel character segmentation reconstruction approach for license plate reco...A novel character segmentation reconstruction approach for license plate reco...
A novel character segmentation reconstruction approach for license plate reco...
 
A hybrid model based on constraint oselm, adaptive weighted src and knn for l...
A hybrid model based on constraint oselm, adaptive weighted src and knn for l...A hybrid model based on constraint oselm, adaptive weighted src and knn for l...
A hybrid model based on constraint oselm, adaptive weighted src and knn for l...
 
Wafer scale fabrication of nitrogen-doped reduced graphene oxide with enhance...
Wafer scale fabrication of nitrogen-doped reduced graphene oxide with enhance...Wafer scale fabrication of nitrogen-doped reduced graphene oxide with enhance...
Wafer scale fabrication of nitrogen-doped reduced graphene oxide with enhance...
 
Ultrasonic atomization of graphene derivatives for heat spreader thin film de...
Ultrasonic atomization of graphene derivatives for heat spreader thin film de...Ultrasonic atomization of graphene derivatives for heat spreader thin film de...
Ultrasonic atomization of graphene derivatives for heat spreader thin film de...
 
Preliminary study of poly (tetrahydrofurturyl acrylate) thin film as a potent...
Preliminary study of poly (tetrahydrofurturyl acrylate) thin film as a potent...Preliminary study of poly (tetrahydrofurturyl acrylate) thin film as a potent...
Preliminary study of poly (tetrahydrofurturyl acrylate) thin film as a potent...
 
New weight function for adapting handover margin level over contiguous carrie...
New weight function for adapting handover margin level over contiguous carrie...New weight function for adapting handover margin level over contiguous carrie...
New weight function for adapting handover margin level over contiguous carrie...
 
Implementation of embedded real time monitoring temperature and humidity system
Implementation of embedded real time monitoring temperature and humidity systemImplementation of embedded real time monitoring temperature and humidity system
Implementation of embedded real time monitoring temperature and humidity system
 
High voltage graphene nanowall trench mos barrier schottky diode characteriza...
High voltage graphene nanowall trench mos barrier schottky diode characteriza...High voltage graphene nanowall trench mos barrier schottky diode characteriza...
High voltage graphene nanowall trench mos barrier schottky diode characteriza...
 
High precision location tracking technology in ir4.0
High precision location tracking technology in ir4.0High precision location tracking technology in ir4.0
High precision location tracking technology in ir4.0
 
Modeling of dirac voltage for highly p doped graphene field effect transistor...
Modeling of dirac voltage for highly p doped graphene field effect transistor...Modeling of dirac voltage for highly p doped graphene field effect transistor...
Modeling of dirac voltage for highly p doped graphene field effect transistor...
 
Implementation of vehicle ventilation system using node mcu esp8266 for remot...
Implementation of vehicle ventilation system using node mcu esp8266 for remot...Implementation of vehicle ventilation system using node mcu esp8266 for remot...
Implementation of vehicle ventilation system using node mcu esp8266 for remot...
 
Energy level tuning of cd se colloidal quantum dots in ternary 0d 2d-2d cdse ...
Energy level tuning of cd se colloidal quantum dots in ternary 0d 2d-2d cdse ...Energy level tuning of cd se colloidal quantum dots in ternary 0d 2d-2d cdse ...
Energy level tuning of cd se colloidal quantum dots in ternary 0d 2d-2d cdse ...
 
A review on graphene based light emitting functional devices
A review on graphene based light emitting functional devicesA review on graphene based light emitting functional devices
A review on graphene based light emitting functional devices
 
Uncertainty on virtualization and internet of things (iot) the connectivity...
Uncertainty on virtualization and internet of things (iot)   the connectivity...Uncertainty on virtualization and internet of things (iot)   the connectivity...
Uncertainty on virtualization and internet of things (iot) the connectivity...
 

Último

Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024The Digital Insurer
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024Results
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxKatpro Technologies
 
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsRoshan Dwivedi
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
Developing An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of BrazilDeveloping An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of BrazilV3cube
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityPrincipled Technologies
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure servicePooja Nehwal
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 

Último (20)

Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Developing An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of BrazilDeveloping An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of Brazil
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 

Automated exam question set generator using utility based agent and learning agent

  • 1. Abstract—Exam is an evaluation tool to measure teaching and learning outcomes of educators and their learners respectively. Nowadays, an automated exam question set generator is a must have to reduce educator’s time on preparation of exam question set and increase the quality of exam question set. This paper proposes an Automated Exam Question Set Generator (AEQSG) using Utility Based Agent (UBA) and Learning Agent (LA). Furthermore, AEQSG applies Bloom Taxonomy (BT) scaling to automate Bloom’s Taxonomy (BT) difficulty level distribution and Genetic Algorithm (GA) to optimize the generation of exam question set and generate high quality exam question set that follow educational institution’s guide-lines. The purpose of utility based agent in AEQSG is to give the user an option to choose actions based on a user’s preference (utility) for each generation state. Meanwhile, the purpose of learning agent in AEQSG is to learn from its past exam results (past generation experiences). Index Terms—Automated exam question generator, bloom’s taxonomy scaling, genetic algorithm, learning agent, utility based agent. I. INTRODUCTION Exams are implemented in educational institutions to monitor the progress of educators’ teaching outcome and learners’ learning outcome. The manual preparation process of exam question set requires educators’ great commitments and is time-consuming. In order to reduce educators’ commitments in preparing exam question set, we have proposed an Automated Exam Question Set Generator (AEQSG) using Utility Based Agent (UBA), Learning Agent (LA), Bloom Taxonomy (BT) Scaling and Genetic Algorithm (GA). UBA chooses actions based on a preference (utility) for each state that maximizes the expected utility. Meanwhile, LA starts to act with basic knowledge (question mark) and then are able to act and adapt automatically (average past question result mark) through learning. In our proposed system, UBA and LA will be applying BT scaling to achieve a desirable result. BT scaling automate the distribution of Bloom’s Taxonomy difficulty level and automate learning experience using Bell Curve Analysis based on the average past exam question result mark to get optimal result for exam question set. GA optimizes the generation of exam question Manuscript received September 25, 2019; revised December 22, 2019 Tengku Nurulhuda Tengku Abd Rahim, Ma. Stella Tabora Domingo, Mohamed Farid Noor Batcha are with the MIMOS Berhad, Technology Park Malaysia, Kuala Lumpur 57000 Malaysia (e-mail: huda.rahim@mimos.my, stella.domingo@mimos.my, farid.batcha@mimos.my). Zalilah Abd Aziz is with the Faculty of Computer and Mathematical Sciences, Universiti Teknologi MARA, Boulder, 40450 Shah Alam, Selangor, Malaysia (e-mail: zalilah@tmsk.uitm.edu.my). set based on chosen preference (utility) in AEQSG. Bloom’s Taxonomy (BT) and GA will not be detailed since this paper is an extended paper from the Automated Exam Question Generator using Genetic Algorithm paper [1]. II. INTELLIGENT AGENT A. Utility Based Agent (UBA) Utility based agent is almost similar to the goal based agent that acts based on goals but it also finds the optimum way to achieve the goal. It is beneficial when there are multiple possible alternatives and an agent has to choose before performing the best action. The utility function maps each state to a real number to evaluate the efficiency of each action in achieving the goals [2]. UBA as in Fig. 1 uses a world model together with a utility function that measures its preferences among states and then chooses the action that leads to the best expected utility. By averaging over all possible outcome states, weighted by the probability of the outcome; expected utility is computed [3]. Fig. 1. A model-based, utility-based agent [3]. B. Learning Agent (LA) A general intelligent agent known as a learning agent is the preferred method for creating state-of-the-art systems in Artificial Intelligence where any type of agents can be generalized into a learning agent to generate better actions [4]. A learning agent has the ability to learn from its past experiences even though it starts with basic knowledge and then adapts automatically through learning [2]. There are four main conceptual components in a learning agent as shown in Fig. 2. The learning element is responsible on improvements based on learning from the environment. The learning element takes feedback from the critic. Automated Exam Question Set Generator Using Utility Based Agent and Learning Agent Tengku Nurulhuda Tengku Abd Rahim, Ma. Stella Tabora Domingo, Mohamed Farid Noor Batcha, and Zalilah Abd Aziz International Journal of Machine Learning and Computing, Vol. 10, No. 1, January 2020 164 doi: 10.18178/ijmlc.2020.10.1.914
  • 2. Meanwhile the critic describes how well the agent is doing with respect to a fixed performance standard. The performance element is responsible for external action selection. The problem generator is responsible for actions suggestion that will lead to new and informative experiences. Therefore, a learning agent is able to learn, analyze performance and look for new ways to optimize its performance [2]. Fig. 2. A general learning agent [3]. III. DEEP REINFORCEMENT LEARNING Deep Reinforcement Learning (DRL) is the combination of Reinforcement Learning (RL) and Deep Learning (DL); whereas RL with function approximation by deep neural networks [5]. RL and DL are both systems that learn autonomously. The difference between RL and DL is RL learns dynamically by adjusting actions using continuous feedback in order to optimize the reward, while DL learns from a training set and then applies that learning to a new data set [6]. Meanwhile, the difference between RL and DRL is RL is dynamically learning with a trial and error method to maximize the outcome, while DRL is learning from existing knowledge and applying it to a new data set [7]. Unlike supervised and unsupervised learning algorithms, RL is about training an agent to interact with an environment and maximize its reward [8]. RL model as shown in Fig. 3 consist of three basic concepts: state, action, and reward. The state describes the current situation of the agent. The action describes the action taken by agent at each state. The reward describes the feedback from the environment [8]. Fig. 3. Reinforcement learning model [8]. IV. RELATED WORK Automated Test Paper Generation [9] is a web-based application system prototype implementing utility-based agent to cater exam difficulty level, and shuffling algorithm to design question set randomly without repetition or duplication. On the basis of utility value, two sets of question items are selected. The first question items set consist of the highest utility value, whereas the second question items set consist of the second highest utility value of the test paper. Accumulative difficulty level of the test paper is calculated based on the question items’ utility value. An intelligent agent is introduced to Automatic Question Paper Setting (AQPS) [10] to automate the generation of question paper using random number generation technique and backtracking algorithm known as Automatic Question Paper Generator Agent (AQPA). Basic components of AQPS are Hash Table (HT) database, Repeated Question Checker Module (RQCM) and Question Paper Generation Module (QPGM). The HT database is used to store questions with their keys with optimum space; in order to find required question very quickly. RQCM ensures that a new retrieved question does not match with any question in selected question database. QPGM generates the question paper based on selected pre-defined question format and selected question database. The proposed framework for Automatic Question Generation (AQG) [11] is multi-agent which consists of Document Processing Agent (DPA), Information Classification Agent (ICA) and Question Generation Agent (QGA). DPA will first start processing input text using Tree Tagger tool to produce a one word per line form together with its tag and lemma; the processed output of Tree Tagger is ranked based on the frequency of each word occurrence; then prefix and suffix of each word is removed with a stemming process to get the root word. Bloom’s category is found in ICA based on selected keywords after the stemming process and word count from DPA by matching keyword with action verb in the repository. The output from ICA will be used in QGA to generate questions based on template that suit the selected keywords according to the Bloom’s levels. The two main components in Utility Based Test Paper (UBTP) agent [12] are Knowledgebase Developer (KD) and Test Paper Generator (TPG). KD is a component that stores question items with its utility value with integer value in range between 0 and 5 in file form. TPG is a component that generate a test paper from selected question items (filtered by its utility value) from knowledgebase based on overall difficulty level (from 1 percent to 100 percent) entered by examiner. Three sets of question items will be selected on the basis of utility value in TPG algorithm. In other words, first set has the highest utility value calculated, second set has one less than highest utility value calculated and third set has 0 utility values. Most of papers above implement Utility Based Agent (UBA) only in their system but none of them are using Learning Agent (LA) or Deep Reinforcement Learning (DRL). In order to improve the quality of Exam Question Set Generator, we propose multi agent with combination of two agents; UBA and LA. We also suggest to apply DRL for LA. V. PROPOSED SYSTEM The overall architecture of the proposed system i.e. Automated Exam Question Set Generator (AEQSG) using Utility Based Agent (UBA) and Learning Agent (LA) is shown in Fig. 4. International Journal of Machine Learning and Computing, Vol. 10, No. 1, January 2020 165
  • 3. Exam Question Set Exam Question Set Generator Question Set Preference (Utility) Question Bank Fig. 4. Automated exam question set generator overall architecture. Question collection workflow process as shown in Fig. 5 will be done before hand to provide the proposed system with a collection of questions either new or previous question from past exams. User can either key in or uploaded file for new and previous questions into the system. Then the system will automatically validate each uploaded file and validate each questions keyed in or uploaded. After that, all validated questions will be stored in the question bank. Start Initialize Question List (LQ) Input Question(s) Details Upload File(s) For Each Question (q) Is q Valid? Yes Has More q? Add q to LQ No For Each File (f) Is f Valid? Yes No Has More f? No No End Save LQ Question Bank Yes Yes Fig. 5. Question collection workflow process. All uncategorised questions either new question or previous question from past exam that does not have any BT category or chapter category assigned when the questions are keyed in or uploaded using file; the proposed system will automatically categorise based on BT categories (as shown in Table I) and chapter during question categorisation workflow process (as shown in Fig. 6). Therefore the person in charge to key in or upload questions to proposed system can be anyone other than Subject Matter Expert (SME). Thus, in the same time will helps to reduce educators’ commitments and reduce significant amount of time in preparing exam question set. In addition to above benefits, the question categories would be more appropriated to course syllabus guideline since it not only categorised BT category but also chapter category for the specific question saved into question bank. TABLE I: BLOOM’S TAXONOMY CLASSIFICATION [1] ID Category Level C1 Knowledge Easy C2 Comprehension C3 Application Medium C4 Analysis C5 Synthesis Hard C6 Evaluation Question categorisation workflow process will be started with pre-processing of uncategorised questions using Natural Language Processing pre-processing (NLP pre-processing) included lower case standardization, remove non alphabet character, tokenization, Part of Speech (POS) tagging, noun extraction, verb extraction and stemming verbs before Bloom’s Taxonomy classification (BT classification) and chapter classification parallel workflow process begin. Start End Lower Case Standardization Remove Non- Alphabet Character Question Bank Tokenization Stemming Verb WordNet Bloom s Taxonomy Classification Verb Extraction (LV) Retrieve Uncategorised Question Noun Extraction (LN) Chapter Classification Part of Speech (POS) Tagging Fig. 6. Question categorisation workflow process. Start End Retrieve Verb List (LV) For Each Verb (v) BT Verbs Found CBT? Yes LBT != Empty? Identify BT Category of v (CBT(v)) No No Add CBT(v) into LBT Has More v? Yes No Select CBT with Highest Occurrence (Max(CBT)) Assign CBT to Question Question Bank Update Question Select The Most Relevant CBT based on Rules Yes Count Occurrence for Each CBT (Sum(CBT)) No Yes Has > 1 Max(CBT)? Initialize BT Category List (LBT) Fig. 7. Bloom’s taxonomy classification workflow process. In BT classification workflow process (as shown in Fig. 7) each verb in verb list extracted after NLP pre-processing International Journal of Machine Learning and Computing, Vol. 10, No. 1, January 2020 166
  • 4. done; will be identified with respective BT categories based on Bloom’s Taxonomy (BT) verbs. Then respective BT categories with highest occurrence will be selected after the summation of each BT categories done. If more than one highest occurrence found, the most relevant BT category will be selected based on rules. The selected BT category will be assigned to the question and the question in question bank will be updated with selected BT category. End Convert LN to N-gram List (LNG) For Each N- gram (ng) Chapters / Nouns Found C? Yes LC != empty? Identify Chapter of ng (C) No No Add C into LC Has More ng? Yes No Select C with Highest Occurrence (Max(C)) Assign C to Question Question Bank Update Question Select The Most Relevant C based on Rules Yes Count Occurrence of Each C (Sum(C)) No Yes Have > 1 Max(C)? Start Initialize Chapter List (LC) Retrieve Noun List (LN) Fig. 8. Chapter classification workflow process. Meanwhile in chapter classification workflow process (as shown in Fig. 8), the noun list extracted from previous NLP pre-processing will be converted into n-gram list. After that each n-gram in n-gram list will be identified with respective chapter categories based on Chapters / Nouns reference table. Then, each chapter categories summation done, the respective chapter categories with highest occurrence will be selected. The most relevant chapter category will be selected based on rules, if more than one highest occurrence found. Later, the question in question bank will be assigned and updated with selected chapter category. Utility Based Agent Genetic Algorithm (GA) Learning Agent Bloom s Taxonomy Scaling Question Set Preference Exam Question Set User Fig. 9. Proposed model for automated exam question set generator. Our proposed system, AEQSG will be implemented using both Utility Based Agent (UBA) and Learning Agent (LA). It also implement Bloom’s Taxonomy (BT) Scaling and Genetic Algorithm (GA) to optimize the process and enhance the quality of the new exam question set. The proposed model for AEQSG using UBA and LA is shown in Fig. 9. The UBA and LA are chosen as a model for the proposed system to achieve our AEQSG goals while optimizing exam question set generation using GA. Moreover, LA is chosen to increase the quality of exam question set using Bell Curve Analysis based on the average past exam question result mark in BT Scaling. Furthermore, BT scaling also provides the mechanism to automate the distribution of Bloom’s Taxonomy difficulty. AEQSG will generated the new exam question set from question bank based on user’s question preference. The user is given the option to select excluded year range, academic year, semester, education level, exam duration, exam difficulty, subject, chapter selection, total question and question margin for question set preference (utility function) before the new exam question set is generated. User’s question preference workflow process as shown in Fig. 10 will be executed before GA workflow process begin. Start Input Question Set Preference (P) Is TQ > 2 & CS > 0 & TQ > CS? Save P, MBT and MQ Yes No Question Set Preference Excluded Year Range Reference (EY) Academic Year Reference (AY) Semester Reference (SM) Subject Reference (SJ) Chapter Selection Reference (CS) Education Level Reference (LV) Exam Duration Reference (DR) Exam Difficulty Reference (DF) Total Question Reference (TQ) Question Margin Reference (QM) Calculate Max BT Category Per Chapter (MBT) Calculate Max Question Per BT Category (MQ) Initialize Fittest Question Set (QSF) Question Bank A Fig. 10. User’s question preference workflow process. GA workflow process as shown in Fig 11, start with initialize question sets based on user’s preferences question set. After question set been initialized, the fitness value for each question sets will be evaluated and question set is selected using Roulette Wheel selection. Then, single crossover and mutation will be performed based on selected question set. After single crossover and mutation done, fitness value for new question sets will be evaluated. Later, International Journal of Machine Learning and Computing, Vol. 10, No. 1, January 2020 167
  • 5. the best fitness will be selected from the current question sets and pass to Bloom Taxonomy (BT) Scaling. UBA chooses actions based on a preference (utility) for each state that maximizes the expected utility. UBA using GA to optimizes the generation of exam question set based on chosen preference (utility) in AEQSG and generate high quality exam question set that follow educational institution’s guide-lines. The fitness formula as shown in Fig 12, is used to calculate the fitness value in GA (that resides in UBA) to get the best fitness before the fitness value is pass to LA to proceed with BT scaling process. W is represent exam question set quality weightage percentage as shown in Table II. The lowest value of the fitness value is the best fitness. Initialize Question Sets Based on P (QSTQ) End For Each Iteration (i) Select QS using Roulette Wheel Selection Perform Single Crossover Perform Mutation Save QSF Has More i? Yes Question Bank Bloom s Taxonomy Scaling (BTS) Evaluate Fitness Value for Each New QS No Evaluate Fitness Value for Each Question Set (QS) Select Best Fitness QS (QSB) Evaluate New Fitness Value After BTS Assign New Fitness QS to QSF Yes New Fitness Value is Fittest? No A Fig. 11. Genetic algorithm workflow process. Start Retrieve Question Set (QSF) For Each Question (q) Retrieve AS Question Bank Has Average Score (AS)? Yes Has More q? No Yes No AS = Question Mark Add AS into LS Initialize Score List (LS) Scores B Fig. 12. BT scaling – average score workflow process. LA starts to act with basic knowledge (question mark) where there is no result mark average (as shown in Fig. 13); and then it is able to act and adapt automatically (average past question result mark) through learning. BT scaling automate the distribution of Bloom’s Taxonomy difficulty level (as shown in Fig. 14) and automate learning experience using Bell Curve Analysis based on the average past exam question result mark (as shown in Fig. 15) to get optimal result for exam question set. TABLE II: QUALITY OF EXAM WEIGHTAGE [1] No. of Level No. of Category Example Weightage Quality 3 6 C1, C2, C3, C4, C5, C6 100 Good 5 C1, C2, C3, C4, C6 90 4 C1, C2, C4, C5 80 3 C2, C4, C5 70 4 C3, C4, C5, C6 60 2 3 C1, C3, C4 50 Average 2 C4, C5 40 1 2 C3, C4 30 Bad 1 C5 20 Start Retrieve Question Set (QSF) For Each Question (q) Retrieve AS Question Bank Has Average Score (AS)? Yes Has More q? No Yes No AS = Question Mark Add AS into LS Initialize Score List (LS) Scores B Fig. 13. BT scaling – average score workflow process. Calculate Total Question (TQ) For Each BT Category (c) Has More c? No Total TQ > MQ? Retrieve Max Question Per BT Category (MQ) Remove q from QSF Yes Yes Calculate Current Total Question (TCQ) No Retrieve Initial Total Question (TQ) TQ > TCQ? Yes No Select BT Category with Min Occurrence (NBT) Select New Question Based on NBT (QBT) Add QBT into QSF Recalculate TCQ Question Bank Question Bank No B C Fig. 14. BT scaling – average score workflow process. By adopting Deep Reinforcement Learning (DRL) in LA, we will beneficial from both Reinforcement Learning (RL) and Deep Learning (DL) advantages. RL training an agent to interact with an environment and maximize its reward while DL complement RL by providing RL with function approximation by deep neural networks. RL is dynamically learning with a trial and error method to maximize the outcome but DRL learns dynamically by adjusting actions
  • 6. using continuous feedback in order to optimize the reward and applying it to a new data set. Since we are in the midst of exploring DRL for this proposed system, we will not discussed DRL in further details. What kind of algorithm or model or deep neural network that will be used in future for the proposed system will not been described in this paper. Scores Bell Curve Analysis Based on LS No Yes Expected Bell Curve? End Perform Mutation on QNS with Same BT Category Update AS in LS Select New Question with Min Average Score (QNS) Retrieve AS Has AS? Yes No AS = Question Mark Question Bank No C Fig. 15. BT scaling – average score workflow process. VI. CONCLUSION AND FUTURE WORK This paper present a novel and feasible approach for Automated Exam Question Generator using Utility Based Agent (UBA) and Learning Agent (LA) by implementing Genetic Algorithm (GA) and Bloom’s Taxonomy (BT) scaling which includes automated Bloom’s Taxonomy (BT) difficulty distribution in exam question set and Bell Curve Analysis. In future, a few researches can be done based on LA to explored more on Deep Reinforcement Learning (DRL) with the combination of other type of algorithms or model or deep neural networks that will be used to improvise current proposed system. CONFLICT OF INTEREST The authors declare no conflict of interest. AUTHOR CONTRIBUTIONS All works in this paper have been done amongst authors. REFERENCES [1] T. N. Tengku Abd Rahim, Z. Abd Aziz, R. H. Ab Rauf and N. Shamsudin, “Automated Exam Question Generator using Genetic Algorithm,” in Proc. 2017 IEEE Conference on e-Learning, e-Management and e-Services (IC3e), Shah Alam, 2017. [2] Types of AI Agents. Java T Point. [Online]. Available: https://www.javatpoint.com/types-of-ai-agents [3] S. Russell and P. Norvig, Artificial Intelligence, A Modern Approach, Third Edition, New Jersey: Pearson Education, 2010. [4] P. Gupta, Rational Agents for Artificial Intelligence, Hackernoon, September 2017. [5] Funstematics.ai. 2019. [Online]. Available: https://www.funstematics.ai/ai-case-studies [6] M. Rouse, “deep learning,” SearchEnterpriseAI, TechTarget, October 2019. [7] T. Williams. (September 2019). Reinforcement learning Vs. deep reinforcement learning: What’s the difference?” Techopedia Inc. [Online]. Available: https://www.techopedia.com/reinforcement-learning-vs-deep-reinforc ement-learning-whats-the-difference/2/34039 [8] Jay, “Machines that learn with reinforcement learning,” Adverai, July 2018. [9] S. A El_Rahman and A. H. Zolait, “Automated test paper generation using utility based agent and shuffling algorithm,” International Journal of Web-Based Learning and Teaching Technologies, pp. 69-83, 2019. [10] K. H. Pinjani, R. Y. Raut, and P. S. Yedey, “Developing an intelligent agent for Automatic Question Paper setting,” in Proc. National Conference on Advanced Technologies in Computing and Networking-ATCON-2015, Amravati, 2015. [11] S. Pandey and K. C. Rajeswari, “Automatic question generation using software agents for technical institutions,” International Journal of Advanced Computer Research, vol. 3, no. 13, pp. 307-311, 2013. [12] M. J. Arshad, M. Naz, Y. Saleem, A. Farooq and K. H. Asif, “Modeling an agent for paper generation system using utility based approach,” Journal of Faculty of Engineering & Technology, pp. 109-124, 2012. Copyright © 2020 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0). Tengku Nurulhuda Tengku Abd Rahim received the B.I.T. degree in computer system technology from University of Malaysia Sarawak (UNIMAS), Kota Samarahan, Sarawak, Malaysia in 2000 and the M.C.S. degree in computer science from MARA University of Technology (UiTM), Shah Alam, Selangor, Malaysia in 2016. She is currently a senior engineer at Artificial Intelligence Department, MIMOS Berhad, Kuala Lumpur, Malaysia. Ma. Stella Tabora Domingo graduated with BSCoE in the field of computer engineering from University of Baguio, Philippines in 2005 She works currently as a staff engineer at the Artificial Intelligence Department, MIMOS Berhad, Kuala Lumpur, Malaysia. Mohamed Farid Noor Batcha graduated with BSEE in the field of communications from University of Kentucky, United States in 2001 and obtained his masters of engineering (electrical) from University Technology Malaysia in 2010. He is currently a staff engineer at Artificial Intelligence Department, MIMOS Berhad, Kuala Lumpur, Malaysia. Zalilah Abd Aziz received the BSc (Hons) degree in computer science from ITM/UKM, Shah Alam, Selangor, Malaysia in 1996, the MSc in computer science (software engineering) from University Putra Malaysia (UPM) in 2003, Serdang, Selangor, Malaysia and PhD in Computer science (artificial intelligence) from University of Nottingham, Nottingham, United Kingdom in 2013. She is currently a senior lecturer at the Faculty of Computer and Mathematical Sciences, MARA University of Technology (UiTM), Shah Alam, Selangor, Malaysia since 1997. Her primary research include software quality, software engineering, artificial intelligence, combinatorial optimization problems, metaheuristics and programming. o