Gary Boetticher and Nzim Lokhandwala

1. Assessing the Reliability of a Human Estimator http:// nas.cl.uh.edu/boetticher/publications.html The 3 rd International Predictor Models in Software Engineering (PROMISE) Workshop Gary D. Boetticher Nazim Lokhandwala Univ. of Houston - Clear Lake, Houston, TX, USA [email_address] [email_address]

10. Ecommerce: Competitive Procurement Buyer Admin Buyer 1 Buyer n ... Buyer Software Distribution Server Supplier 1 Supplier 2 Supplier n : Supplier Software http://nas.cl.uh.edu/boetticher/publications.html The 3 rd International Predictor Models in Software Engineering (PROMISE) Workshop

11. Sample Estimation Screenshots http://nas.cl.uh.edu/boetticher/publications.html The 3 rd International Predictor Models in Software Engineering (PROMISE) Workshop

12. Feedback to Users http://nas.cl.uh.edu/boetticher/publications.html The 3 rd International Predictor Models in Software Engineering (PROMISE) Workshop

14. User Demographics - 2 http://nas.cl.uh.edu/boetticher/publications.html The 3 rd International Predictor Models in Software Engineering (PROMISE) Workshop 5.3856 28 3.6629 Process Industry 4.4391 25 1.4382 Procurement & Billing Domain Experience 5.3856 28 3.6692 Software Projects 4.4390 25 1.4382 Hardware Projects No. of Projects estimated 2.4757 15 1.6967 Software Project Manager 3.0633 25 1.0169 Hardware Project Manager Years of Experience as a Std. Dev. Max. Ave. Years

16. Results: Under vs. Best http://nas.cl.uh.edu/boetticher/publications.html The 3 rd International Predictor Models in Software Engineering (PROMISE) Workshop Ave. Accuracy 48.22% 64% VFI 64% ThresSel 64% Logistic 68% J48 76% PART Accuracy Classifier Y Y Y Total Lang Exp. Y Y Y Total Workshops Y Total Conferences Y Tech Undergrad Courses Y Y Software Proj. Mgmt Exp. Y Y Level of College Y Y Y Y # of Hardware Proj. Est. Y Y Y Mgmt Undergrad Crses Y Mgmt Grad. Courses Y Y Y Y Hardware Project Management Exp. Y Y Y Domain Exp. VFI Thresh. PART Logistic J48 Demographic Evaluator Classifier 68% Logistic/ Logistic 70% VFI / VFI 74% PART/J48 74% J48/J48 74% LogitBoost/J48 74% Bagging/J48 76% ThresholdSel/ ThresholdSel 78% ADTree/Part Accuracy Class./Eval.

17. Under vs. Best: Attribute Reduction http://nas.cl.uh.edu/boetticher/publications.html The 3 rd International Predictor Models in Software Engineering (PROMISE) Workshop Y Y Total Lang Experience Y Y Total Workshops Y Y Total Conferences Y Y Tech Undergrad Crses Y Y Soft. Proj. Mgmt Exp. Y Y Y Level of College Y # of Software Proj. Est. Y Y Y Y # of Hardware Proj. Est. Y Y Y Mgmt Undergrad Crses Y Y Y Mgmt Grad. Courses Y Y Y Y Hardware Proj. Mgmt Exp. Y Y Y Y Domain Experience VFI Thresh PART Logistic J48 Demographic Evaluator Classifier 68% Logistic / Logistic 70% VFI / VFI 74% PART / J48 74% ADTree / J48 74% PART/ PART 74% J48/ PART 76% ADTree/ ThreshSel Accuracy Class / Eval

18. Under vs. Best: Attribute Reduction http://nas.cl.uh.edu/boetticher/publications.html The 3 rd International Predictor Models in Software Engineering (PROMISE) Workshop Domain Exp <= 3 | No Of Hardware Proj Estimated <= 4 | | Hardware Proj Mgmt Exp <= 1 | | | MgmtUGCourses <= 0: BEST (23.0/8.0) | | | MgmtUGCourses > 0: UNDER (13.0/1.0) | | Hard. Proj Mgmt Exp > 1: BEST (5.0) | No Of Hard. Proj Est. > 4: UNDER (5.0) Domain Exp > 3: BEST (4.0) J48 Rule: 74% Accuracy BEST UNDER <-- classified as 21 4 | BEST 9 16 | UNDER

19. Results: Best vs. Over http://nas.cl.uh.edu/boetticher/publications.html The 3 rd International Predictor Models in Software Engineering (PROMISE) Workshop Ave. Accuracy 42.86% 60% Ridor 60% ThresholdSel 60% RandComm 62% Decorate 66% RndTree Accuracy Classifier Y Y Total Lang Experience Y Total Workshops Y Y Total Conferences Y Tech Undergrad Courses Y Y Soft. Proj. Mgmt Exp. Y # of Software Proj. Est. Y Mgmt Undergrad Crses Y Y Y Mgmt Grad. Courses Y Y Y Hard. Proj Mgmt Exp. Threshold Selector Ridor Rnd Comm Demographic 62% ADTree / ThresholdSel 66% ThresholdSel / ThreshSel 72% Rand. Comm./ RandComm 80% IB1 / Ridor Accuracy Class/ Eval

20. Experiment: Best vs. Over http://nas.cl.uh.edu/boetticher/publications.html The 3 rd International Predictor Models in Software Engineering (PROMISE) Workshop 62% Ridor Ridor 62% ThresholdSel Ridor 64% Ridor ThresholdSel 66% ThresholdSel NNge 72% Decorate PART 72% Decorate NNge 72% Decorate RndComm 74% Decorate RandomForest 74% Decorate IBk 74% Decorate IB1 80% RndComm RandomTree 80% RndComm RndComm Accuracy Evaluator Classifier Y Y Y Total Lang Experience Y Total Workshops Y Tech Undergrad Courses Y Y Tech Grad Courses Y Software Proj. Mgmt Exp. Y Y Procurement Industry Exp Y Level of College Y # of Hardware Proj. Est. Y Y Mgmt Undergrad Courses Y Mgmt Grad. Courses Y Y Y Hard. Proj Mgmt Exp Y Domain Experience Thresh Ridor Rand Comm. Decorate Demographic

21. Experiment: Best vs. Over TechUGCourses < 45.5 | Hardware Proj Mgmt Exp < 6 | | No Of Hardware Proj Estimated < 4.5 | | | No Of Hardware Proj Estimated < 3 | | | | TechUGCourses < 23 | | | | | Hardware Proj Mgmt Exp < 0.75 | | | | | | TechUGCourses < 18 | | | | | | | Hardware Proj Mgmt Exp < 0.13 | | | | | | | | TechUGCourses < 0.5 | | | | | | | | | TechUGCourses < -1 : F (1/0) | | | | | | | | | TechUGCourses >= -1 | | | | | | | | | | Degree < 3.5 : A (4/0) | | | | | | | | | | Degree >= 3.5 : A (5/2) | | | | | | | | TechUGCourses >= 0.5 | | | | | | | | | TechUGCourses < 5.5 | | | | | | | | | | Degree < 3.5 : F (5/0) | | | | | | | | | | Degree >= 3.5 | | | | | | | | | | | TechUGCrses < 2 : A (1/0) | | | | | | | | | | | TechUGCrses >= 2 : F (1/0) | | | | | | | | | TechUGCrses >= 5.5 | | | | | | | | | | Degree < 3.5 | | | | | | | | | | | TechUGCrs < 10.5 : A (3/0) | | | | | | | | | | | TechUGCrses >= 10.5 | | | | | | | | | | | | TechUGCrs<12.5 : F (3/0) | | | | | | | | | | | | TechUGCrses >= 12.5 | | | | | | | | | | | | | TechUGCrs<16: A (2/0) | | | | | | | | | | | | | TechUGCrs>15 : A (2/1) | | | | | | | | | | Degree >= 3.5 : F (1/0) | | | | | | | HardProjMgmt Exp >= 0.13 : A (2/0) | | | | | | TechUGCourses >= 18 : A (2/0) | | | | | Hard Proj Mgmt Exp >= 0.75 : F (1/0) | | | | TechUGCourses >= 23 : F (5/0) | | | No Of Hardware Proj Est >= 3 : F (1/0) | | No Of Hardware Proj Est >= 4.5 : A (5/0) | Hardware Proj Mgmt Exp >= 6 : F (4/0) TechUGCrses >= 45.5 : A (2/0) The 3 rd International Predictor Models in Software Engineering (PROMISE) Workshop BEST OVER <-- classified as 23 2 | BEST 8 17 | OVER

23. http://nas.cl.uh.edu/boetticher/publications.html Questions? The 3 rd International Predictor Models in Software Engineering (PROMISE) Workshop

24. http://nas.cl.uh.edu/boetticher/publications.html Thank You ! The 3 rd International Predictor Models in Software Engineering (PROMISE) Workshop