2. Learning Goals
At the end of this workshop, you will understand how to:
â˘Explain how user roles impacts item access
â˘Create and Administer Benchmark Assessments
â˘Explain how test categories and test dates impact test content
and results access
â˘Understand best practices for creating assessment items
2
3. 3
Where can I find helpful resources ?
⢠DPI and Pearson have loaded a variety of helpful materials
into Schoolnet. Search Home Base in Instructional Materials to
locate:
⢠Quick reference cards
⢠End user PowerPoint presentations
⢠Training scripts
⢠Schoolnet contains screen specific help on every screen ( to
access click the )
⢠For hardware and software questions click System
Requirements in the footer of any screen
4. 4
Pearson K12 Technology Customer Education
Self-Paced Distance Learning: When are we training?
⢠Self-Paced Distance Learning, accessed via PowerSource,
⢠Web-based âMastery in Minutesâ training includes short tutorials that are
topic-specific for easy answers to simple tasks
⢠Online distance learning consists of interactive courses approximately one
hour in duration. Designed for adult learners, this type of training includes
authentic assessment, interactive storylines, and hand-drawn graphics to
engage learners.
⢠Unlimited access! LEAs determine who can access the materials
Authentic Assessment to evaluate
understanding of concepts
Practice throughout
the course
5. 5
What Assessment Items are available?
DPI has loaded a total of 35,919 assessment
items (as of Sept 19th
2014)
⢠ELA â 6,352
⢠Math â 13,536
⢠Science â 12,834
⢠Social Studies â 2,432
⢠Additionally, there are 869 passages
More items/ Passages can be created and submitted for approval to the
state. Once approved they will be available for every educator in North
Carolina.
6. 6
What Instructional Materials are
available?
DPI has loaded a total of 38,026 instructional
materials (as of sept 19th
2014)
⢠Curricular Units - 396
⢠Instructional Unitsâ 1,545
⢠Lesson Plans â 12,212
⢠Assessments - 442
⢠Resources â 11,454
⢠External Resources â 11,977
More instructional materials can be created and submitted for approval to
the state. Once approved they will be available for every education in North
Carolina.
7. Item Types
TYPE Paper Online Clickers Notes
Multiple choice Y Y Y
T/F Y Y Some
Gridded Y Y Some
Open Response Y Y Some Can be associated with a
rubric
Inline Response Y Y N
Matching Y Y N
Hot spot -
Single
N Y N
Hot Spot â
Multiple
N Y N
Drag and Drop N Y N
Click Stick Drop N Y N
Tasks Some Y N A collection of items
7
8. Building the Item Bank
⢠Grouping multi-step items into reusable tasks
⢠Snip it tool for adding Items
⢠Paste from Word
⢠Using the html editor
⢠Use of express test to create placeholders
⢠Answer Key Only tests
⢠Bulk uploading Scores
8
9. Tasks
⢠A task is a
collection of
items that you
can reuse
⢠Each item can
have a different
standard and
point value
⢠Group items that
are part of a
project and canât
stand alone
9
10. 10
Activity:
⢠Create a Task item with at least 2 activities.
⢠Note the difference when saving in Leadership vs Teacher role.
11. Item Visibility Options
â˘Determine who should have access to each item
â˘Enable co-authoring to create items collaboratively
â˘Reserve some items for district benchmarking
11
12. Process for Submitting and Approving
⢠Any user can submit an item, passage, or rubric to
be shared
⢠Users who are permissioned to do so can approve
content at their default institution
⢠Once the content is approved, the submitter can no
longer edit the content
12
13. ⢠Naming conventions for assessments
For example:
⢠Preparing for testing
⢠Discuss who will do what
Including item creation, test creation, scheduling and
assigning tests, and monitoring the collection of results
13
Math 6 CDB 1 2013-2014
Math 6 CDB 1 M 2013-2014
Subject
Grade
Test title
School year
Modified
Rolling Out an Assessment Program
13
16. 16
Review Activity: (Creating a test)
⢠Create a District Benchmark containing 4-6 items (title it with
your initials). Choose 8th
Grade Math.
⢠Edit one item to be an Open Response item.
⢠Find the task item and add it to the test (hint â Advanced
Search).
17. Benchmark vs. Classroom Tests
Benchmark
â˘Results appear in School &
District Data and Classrooms
â˘Tests created at the district or
school level for the purpose of
institution-wide data collection
â˘Only the highest level category
used for KPI calculations
Classroom
â˘Results appear in only
Classrooms
â˘Tests created for the purpose of
classroom use
â˘Can be teacher-created or
selected from a pool of pre-
made tests
â˘My Classroom or Common
Classroom categories available
17
18. Test Category
What Type of Assessment am I creating
Test
Category
Use for Where to Find Results
My
Classroom
Check for Understandings, classroom
tests with scores for teacher use only
In Student Profile and
Classrooms
Shared
Classroom
Assessment
Assessment shared by Teachers
Tests that can be used by multiple
teachers of a course that donât need
review at aggregate level
In Student Profile and
Classrooms
School
Assessment
End of Unit Assessments
Tests taken by students in multiple
teachersâ classes that should be viewed
at the school level either by teacher or
aggregated
Same as above plus
throughout School & District
Data with the exception of
KPIs
District
Benchmark
Baseline/Benchmark Assessments
Tests taken by all students in a course
that should be available for all reports
Same as above plus in KPI if
this is the most recent test
18
19. Understanding Test Stages
Private Draft ⢠Visible only to the creator
Public Draft ⢠Visible and available for editing to all users with
permission to create and edit tests
⢠Does not apply to teacher-created classroom tests
Ready To
Schedule
⢠Test content is complete and cannot be edited
⢠Be sure to add any new items to Item Central before
scheduling
Scheduled ⢠The test is assigned, but has not reached the start date
⢠A test switches from Scheduled to In Progress
automatically when the start date is reached
In Progress ⢠Current time is between the start date and the end date
⢠Automatically switches to Complete when the number of
results exceeds a certain percent of eligible test takers
(default is 85%)
Complete ⢠The test is Complete when the current time is past the
end date or percentage hit
19
20. Scheduling at the School & District
Option Available to Description
Assign to
students
All Assign the test to a specific set of courses
Recommend
to teachers
District and
school test
administrators
Suggest this test to teachers based on
course taught. Teachers have option to
assign the test to students .
Recommend
to schools
District test
administrators
Turn over the assignment task to school
test administrators who can then assign
the test. Use this option when you do not
have sufficient knowledge of the courses
at each school to assign the tests.
School test administrators will see an
additional section titled Recommend Tests
on the Assessment Admin page. These
users must click Not Assigned to assign
these tests locally.
20
21. 21
Resume Activity
⢠Return to test.
⢠Go through test stages, Private, Public, Ready to Schedule.
⢠What do you need to do at Ready to Schedule stage?
⢠Copy test and replace the words (COPY) with âRecommendâ.
⢠Schedule the âRecommendâ test to start immediately and
recommend to schools/courses.
⢠Schedule the original test to start the next day and assign
directly to courses (or Quick Assignment)
24. Teacher Assigns Recommended Test
⢠Click âNoâ in the Assigned column of a
recommended test to assign it to sections
24
25. Preparing for Online Testing
⢠Students
- Understand available features, how to view
results, how results will be used
⢠Teacher or other proctors
- Access to Proctor Dashboard, online passcode,
usernames and passwords
⢠Computers
- Hardware and software
25
26. Printing
⢠Printing options available on Test Detail page
⢠If testing online, will some students with
accommodations test on paper?
Discuss who will print:
⢠Test booklet
⢠Answer sheets
⢠Answer key
⢠Scoring instructions or rubrics
26
29. 29
Resume Activity
⢠Return to original test (in progress and assigned to courses).
⢠Score open response items by sections
30. Track Completion Rates
Test administrators monitor the progress of scanning
by school and by section with Track Completion Rates.
30
31. Discussion: Implementing Assessments
⢠Building the item banks
- Submission/approval process
⢠Who will do what?
- Creating school assessments / district
benchmarks?
- Assigning / recommending?
- Scanning / printing
- Tracking data completion
37. Learning Goals
⢠School and District Data Overview
⢠Distinguish between the âwhoâ and the âwhatâ for reporting
⢠Define student sets
⢠Build custom reports
⢠Publish reports
⢠Tag key reports
⢠Run course or section reports
41. Drilling Down: Item Analysis Report
⢠Find a section that has district and local assessment
data
- Which skill and standard are you most concerned about for
your students?
- Which items were the most challenging for students and
what can you learn about their misunderstandings by
looking at the distractors?
- What evidence of strengths/weaknesses in curriculum,
instruction, or assessment are apparent in the data?
- What other data (student work, individual demographic
data) do you have that informs your knowledge about
student performance?
- What instructional actions will you take to differentiate
instruction based on the data?
42. Reports Available to Investigate Further
Access published reports in the Report Bank.
Districts may choose to published reports in the Report Bank.
Learn more about this in the Schoolnet II series
46. Functionality Review
Are you familiar with the following?
⢠Running reports from the Report Bank and
Benchmark Dashboard
⢠Creating a pre-formatted report
⢠Creating an analysis spreadsheet
⢠Saving a report and viewing a saved report
47. Creating a Report
First ask yourself:
⢠What is my goal for this report?
⢠Who do I want to know about?
⢠What do I want to know about these students?
⢠What do I want to compare?
48. Compare Reports
Report 1 Report 2
Who: Students who missed
multiple days of school last year
Who: Students who performed
poorly on a standardized test
What: Standardized Test report What: Attendance report
⢠How do these reports differ?
⢠Is it likely that the same students are in both reports?
⢠What questions do these reports help answer?
49. Making a Complete Report
Who: Student Set
+
What: (Report Parameters + Data Constraints)
⢠Pre-formatted reports: Choose from a limited selection
of student sets, report parameters, and data selections, all
available in one place
⢠Custom reports: Define student sets, report parameters,
and data constraints separately
⢠Analysis spreadsheet: Begin with a student set and
define columns of data
50. The âWhoâ
⢠The âwhoâ is your student set: a group of
students who meet pre-defined criteria.
⢠Example student sets include students who:
- Are enrolled in a particular class
- Are currently enrolled at your school
- Are enrolled in a special education program
- Were enrolled in 4th grade last year
- Performed poorly on a specific test
51. The âWhoâ
Defining a Student Set
⢠Create a student set:
- Manually, using a student filtering process
- Automatically, by clicking a graph or table
⢠The student set âboxâ appears in the following
locations:
- Analysis Spreadsheet
- Student Filter
- Custom Reports
52. The âWhoâ
Define the Student Set by Applying Filters
Filters are âandâ
criteria; students
must meet all filters.
53. ⢠Current Enrollment â Students enrolled as of today
⢠Total Enrollment â Students enrolled over time
Choose Total Enrollment to include all students at some
point during the school yearâto evaluate how the institution
performed over time. Also, use this option during summer.
The âWhoâ
Enrollment Filter
55. Try It Out
The âWhoâ â Define a Student Set
⢠Create a set of current students at a school
⢠Create a set of low-scoring students
⢠Save a student set
⢠Discuss enrollment selection (current vs. total)
56. The âWhatâ
Analysis Spreadsheet
⢠The âwhatâ is your data selection, the data points on which you
would like to report.
⢠Use your data selection to:
- Measure students across multiple assessments (such as
standardized vs. benchmark tests for same subject)
- View scores across multiple subjects and sort results to identify
students who perform better in some subjects than others
- Compare assessment data to attendance
- View demographic information (such as gender and ethnicity)
Analysis spreadsheet â The âwhatâ is displayed in the columns and
the âwhoâ in the rows. The student selection (âwhoâ) is the same
regardless of the column selection (âwhatâ).
57. The âWhatâ
Custom Reports
For custom reports, the âwhatâ is defined by report parameters and
data selections.
â˘Report parameters: Formatting (such as a table, graph, or pie chart)
and data elements (rows and columns, such as test section or score
group)
â˘Data constraints: Point in time (last yearâs test), school, and grade
level in which the students tested
58. The âWhatâ
Report Parameters: Grouping Data
⢠Use report parameters to group data using sub-rows and paging
⢠Use paging for both custom and pre-formatted reports
⢠For pre-formatted reports, page by subgroup category
Rows and sub-rows
grouped by subject
Paging by
student
subgroup
59. The âWhatâ
Custom Report Builder
Use custom
reports to define
parameters:
â˘Formatting
â˘Rows
â˘Columns
â˘Totals
60. Use of Row and Sub Row
The row and sub-row
selection alters the
report for the same data
62. Try It Out: Generate a Custom Report
⢠Generate custom reports
- Standardized test
- Independent practice
⢠Try some of the options in the Benchmark Item
Analysis report
⢠Save a few reports for other activities
63. Analysis Spreadsheet Custom Report
⢠For ~<200 students
⢠For specific students, such
as those who did well in
reading but not math
⢠To see student names on
the report
⢠If a pre-formatted report does
not meet your needs
⢠For a large selection of
students
⢠To find patterns
⢠To identify instructional areas
of concern for a group of
students
Choose a Report Type
64. Publishing Reports for Others
Report Manager â Users with the report manager role can publish
reports and manage published reports:
â˘Save the report
â˘Publish the report
-School-level report managers publish to the school bank
-District-level report managers publish to a single school, a district,
or multiple schools
â˘Edit and retract reports
â˘No approval workflow
65. Saving to the Report Bank
What can you save to the Report Bank?
⢠Pre-formatted reports
⢠Student sets
⢠Analysis spreadsheets with students
⢠Analysis spreadsheets without students*
⢠Custom report with students (focus for today)
⢠Custom report without students*
* When using these items, you are prompted to
select the missing component
66. Key Reports
⢠Display on Schoolnet home
page
⢠Highlight the especially
important reports for your
institution
⢠Display reports from your
institutionâs Report Bank only
⢠Tag reports as âKey Reportsâ
at the school level or across
all schools as part of the
publishing and contextualizing
process
67. Places to Find Reports
⢠Public Report Bank: Repository for published reports (published
reports are custom reports approved by an administrator for
district-wide or school-wide distribution)
⢠My Reports Bank: Repository for saved reports that you created
and published reports from the Report Bank that you
bookmarked
(Note: Only you can see the reports that you save.)
⢠Key Reports: Quick reference for published reports on the
Schoolnet home page that are tagged as âkeyâ reports
⢠Related Reports: Quick reference for published reports on the KPI
detail page that are related to a specific KPI
⢠Benchmark Dashboard: Quick reference for published reports
that include a single test administration
68. Try It Out: Publishing Reports
⢠Publish reports you want to make available to
other users
⢠Select a few reports to tag as Key Reports
⢠Retract any reports you donât want to remain
published
72. Understanding Roles and operations
within Home Base
⢠Roles for Schoolnet access
⢠Tools and support documentation available
⢠How to access user management in
Powerschool
⢠Cautions and scenarios
⢠Bulk export and import role assignments
72
73. Deployment of Schoolnet â Wrap up
Discussions
⢠Group Discussion for next steps for
deployment/ turn around training
⢠Home Base planning questions (utilizes
worksheet)
73
75. 75
Where can I find helpful resources ?
⢠DPI and Pearson have loaded a variety of helpful materials
into Schoolnet including (search Home Base in Instructional
Materials)
⢠Quick Reference cards
⢠End user PowerPoint's
⢠Training Scripts
⢠Schoolnet contains screen specific help on every screen ( to
access click the )
⢠For Hardware and Software questions click System
Requirements at the bottom of My Schoolnet (main start
screen)
76. Where do I find more information
⢠IIS webpage for further information:
http://www.ncpublicschools.org/homebase/
76
77. 77
Pearson K12 Technology Customer Education
Self-Paced Distance Learning: When are we Training?
⢠Self-Paced Distance Learning, accessed via PowerSource, 365/24/7
⢠Web-based âMastery in Minutesâ training includes short tutorials that are
topic-specific for easy answers to simple tasks.
⢠Online distance learning consists of interactive courses approximately one
hour in duration. Designed for adult learners, this type of training includes
authentic assessment, interactive storylines, and hand-drawn graphics to
engage learners.
⢠Unlimited access! LEAs determine who can access the materials
Authentic Assessment to evaluate
understanding of concepts
Practice throughout the course
80. Next Steps
⢠Wrap up
⢠Questions and answers
⢠Evaluation
⢠Explore on your own site
Editor's Notes
Today weâll focus on the basic for all tests so that you can create and administer a classroom test. Tomorrow weâll go into detail about rolling out a benchmark program and advanced features.
Show users how to find the Item bank -Hands On
Assessment admin â find an item â select subject â all items available now display
Speak to Items always being added to the system â through the submission process (other educators and DPI)
Show users how to find the Materials bank â Hands On
Classroomsâ Instructional Materials â select all subjects and grades â all Instructional Materials available now display
Speak to Instructional Materials always being added to the system â through the submission process (other educators and DPI)
Hands On
Note what Item are avabile for online vs. paper vs. clickers
If items are not available for your needs, itâs easy to build new items.
Attach any item type to a passage.
With gridded, can provide more columns that needed in order not to give a hint â student can answer flush left or right with any numerical equivalent
Clicker support depends on the model â reference QRC
Use of express test to create placeholders (this is not an assessment but a way to add items into SN)
Discuss
-Naming convention â Items for Upload
-Ready to schedule â add items to item central
Snip it tool for adding Items
- Easy tool for adding math equations (if already created)
If a screen shot is added it should be sized before being added to the item. (max 2mg recommended 25 kb)
Screen will NOT read picture
Paste from Word
Will copy all formatting over from Word to SN text editor
- Recommended for items/ Passages
Using the Html Editor
Embedding code from teacher tube/ Map Quest
Note: if embedding code the item CAN NOT be the last assessment Item on a test
Uploading MP3 (4 mg max)
Tasks are useful for multi-step project assignments that cover multiple skills.
Users who have permission to create items for other users will see this screen when they save an item. They have the option to choose who else will be able to see the item.
Discuss the use case for each option.
Co-authoring allows multiple users to collaborate. Can include a member with additional item rights to bypass the need for submit/approval. When the item is complete, anyone in the group with appropriate permissions can share the item to Item Central. User can select other users to co-author items with. List is restricted to users in the application that have the ability to create and edit items
Submission process in NC â Teacher â School â District - State
Contrast to co-authoring
Refer to QRCs.
[Trainer note:]
Emphasize how important it is to establish conventions before you start. If youâre making school-based tests, conventions are especially important, because you want the tests in the School & District Data menus to be grouped by school. Classroom-based tests should include the teacher name because they will show up in the student profile.
Naming conventions: Donât add the word âtest.â
Restrict Teacher access â stops a teacher from viewing assessment content before the start date of the testing window
New in version 16.1 â Restrict Content Access see next slide
New in version 16.1 â Restrict Content Access â NC not yet upgraded to 16.1
In Assessment Admin, you can create two kinds of tests. Benchmark is a generic term that covers all interim assessments not loaded as standardized tests. You might call them progress checks, baselines, minis, and the like.
If teachers are granted the âCreate and edit tests for an institutionâ operation, they can create and edit school benchmark category and common classroom tests.
A common classroom test can be created at the district or school level and behave just as those tests would. The exception is that their data does not appear in the School & District Data module. If teachers have rights to create common classroom tests at their institution, only other teachers at that institution and district users will see the test.
Note: Only the My Classroom test category can go to PowerTeacher Gradebook, if applicable.
My Classroom â Data visible for the section only
Common Classroom â Data visible for the section only (recommend)
School Benchmark â End of Unit Assessments â Allows for aggregate reporting on section/teacher comparison/ school. (assign)
District Benchmark â Baseline/Benchmark assessments that will be take across the district. (assign)
Recommend â more of less here is a test would you like your students to take it
Assign â Students will take this assessment
Note â Terminology may change (sys admin can add and rename test categories)
If creating Common assessment, school/District benchmark â it is recommended to move assessment into the public stage immediately to allow for collaboration.
Test admins can assign or recommend by course:
AssignâTeachers are required to administer the test
Recommend to teachersâSelect to recommend this test to students based on their course enrollment; teachers may assign the test to students, but are not required to do so
Recommend to schoolsâSelect to recommend this test to administrators who may assign the test to a teacher or section Use the option to recommend to schools when you do not have sufficient knowledge of the courses at each school to assign the tests to courses. You will need to set up a test admin at the school.
It is not recommended to use the Recommend option for Benchmark assessments. If students MUST take the assessment use the assign option
District Benchmarks should always be assigned to Students as you want them to take the test
District Benchmarks can not be Recommended to teachers only Schools
Recommend to teachers is used for School Benchmarks â this displays because your roll allows you to create and schedule both district and school benchmarks.
Select the specific courses from the left hand side of the screen using the check boxes and the select ADD SELECTED
Once all of the courses display on the right select Done, View summary
Demo how to get to proctor dashboard as a test admin
Will you print district tests centrally or at each school or classroom?
Even if testing fully online, may wish to print scorign rubrics
Omit if they are testing online only.
Answer sheets must be scanned on a scanner attached to a computer installed with ScanIt.
Reference scoring slide from yesterday if needed
Itâs useful to identify a school or section that is missing results.
Turn/Talk/Share:
Ask people to share a memory with the person next to them. In sharing, ask for one example from the group
What are the implications for this? What does it mean for your role in the school?
Why is this an important question to ask at the beginning of PD on instructional materials? Stress that this puts a positive look at lesson planning.
Reflects current students only
Defaults to your school
[Trainer note:]
Show how to switch to district or teacher view
Show how to drill into KPI details and add and remove filters
Current year tests only
Defaults to your institution
[Trainer note:]
Show filters
Expand to show details
Drill in to a few reports
A variety of published reports may be available to you
Benchmark reports that contain multiple tests are available here instead of the dashboard
Pre-formatted reports are a set of easy-to-create reports that allow some modifications
Some of these reports may already by available from the Benchmark Dashboard or Report Bank
[Trainer note:]
Show Demographic overview with current enrollment
Show a standardized test for math by all subtopics, and use it to discuss current vs. total enrollment; add paging by subgroup
Rather than recreate a report, you can save the parameters to run later
[Trainer note:]
Demonstrate how to save and access these kinds of reports
Should have at least one pre-made column set available; you can also add your own columns
In Classrooms, by default, student set is the selected sectionâcan override with student group
Maximum of 25 columns
Can export to Excel
[Trainer note:] Ask some review questions to check for understanding of prerequisites, then do a jigsaw activity.
Running a meaningful report takes more than just being able to navigate the site. For example:
The âwhoâ could be, for instance, all current 4th graders, students who did poorly on a math objective, or girls who missed a lot of school.
The âwhatâ could be, for example, how many students are in each score group of a test, which students are struggling in a specific skill, how many are in each special program
As you can see, it is not enough to say âI want to run a test report or an attendance report.â
A data constraint might be âwhere was the student when the test was taken?â
A report parameter would be a row or column definition.
You can combine these criteria to build custom student sets, such as all 10th-grade girls who missed a lot of school last year.
You will discover many more possibilities when you try out the student filter.
Define the student set by applying filters. The original student set represents all student records in the system. This is an âandâ filterâthe more filters you add, the smaller the student set gets. Once you get to zero, donât add more!
[Trainer note:]
Discuss what each letter meansâwho is included and who isnât?
A/B = includes students who were enrolled during the 11-12 school year and at some time in the past were in middle school
B = boys
Donât need specific school year for genderâdoes not matter
[Trainer note:]
More on this later.
Note how these reports have the same data, but the way the data is grouped changes the way you can interpret it.
With your custom group of students, you can also run a variety of customized reports. These reports show how many students meet the report parameters. For example, it may show how many students are in each score group for each school for a test. These reports do not display the actual scores in the table.
[Trainer note:]
Which one makes it easier to compare schools?
Which one makes it easier to find problem subjects by school?
Note that weâve used âpercent of rowâ instead of value since the schools are different sizes
[Trainer note:]
Show how to create a basic report that would be of interest to the group.
No approval workflow means that unlike lesson plans, reports do not need to go through an approval process before being published. However, you can set up a workflow for publishing reports at your school or district by using a school- or district-level report manager.
Can add URL here.
Under the Training tab search for Schoolnet Courses and Mastery in Minutes â