Administered Assessments – Crowdmark

Ready to grade assessments

maggie — Mon, 03 Jun 2024 13:28:32 +0000

With Crowdmark’s Ready to grade workflow, you can scan and upload existing assessments in any format and grade them using Crowdmark’s online grading tools. Scan your completed assessments using the instructions here.

If you’d like to create an assessment template using tools such as anonymized grading, multiple choice auto-grading or automatic student matching, check out our Crowdmark booklet workflows.

Creating a Ready to grade assessment

To begin, click Create assessment. Name your assessment, choose Administered (In-person) and select the Ready to grade option. Review your selections and click Done. You’ll be taken to the Assessment setup page.
In the Template section, upload a blank version of your assessment. Crowdmark will use this template to recognize and separate the assessment booklets when you upload them. Verify the template looks correct using the preview provided, and click Save and continue to move to the Questions page.
Note: because Ready to grade assessments do not require a dedicated cover page, all pages will be visible in the grading interface. Student anonymity is not guaranteed.
Define the questions on your assessment template using the instructions here. This is how Crowdmark will know where the questions are located for grading.

Uploading completed assessments

Scan your completed assessments according to the instructions here. If you’re used to scanning QR-coded assessments, make sure you are following the proper scanning workflow to get accurate results.
In the Manage scans tab, click to browse or drag and drop the PDF files of your scanned assessments. You’ll see Crowdmark processing the uploads into separate booklets based on your template.
Click Review to see and edit the booklets Crowdmark has recognized. Hovering over a page will allow you to zoom, rotate, delete or upload another page. Using the three dots menu on the far right, you can rotate, grade or delete the whole booklet. Whole number page numbers (1, 2, 3, etc.) are pages recognized as matching your template. Decimal page numbers (2.1, 2.2, 2.3, etc.) indicate extra pages that don’t match a template page.
If Crowdmark detects a missing page, you’ll see a red band and a warning with instructions. To fix a missing page, either upload a page or drag another page to the placeholder. You can drag template and extra pages within a booklet to reorder them or move them to other booklets.
To replace a template page, drag the new page on top of it. The label will show the page number being replaced.
To append a page, drag it to the side of the intended origin page. Watch for the page label to display which page number will be applied.

To maintain grading integrity, pages with evaluations cannot be moved. Removing evaluations from a page will restore mobility.
You can delete unwanted booklets at any time. When deleted, these booklets will be hidden and will not appear for grading. Deleted booklets can be restored from the Review scans page by clicking Restore booklet.
Pages can be replaced, or extra pages can be appended to a booklet by clicking the upload icon on the page it’s meant to follow. A modal will appear. Click to indicate whether you’re replacing a template page, or uploading an extra page, then click to browse or drag and drop the necessary files. Note: appendix pages must be uploaded in JPG format. If you do not have the option to export the necessary file type from your original scan, you can use an online conversion tool.
To preserve grading integrity, you will not be able to move pages that contain evaluations. Removing evaluations from a page will restore mobility.

Next steps:

Tool Tip session recordings

maggie — Fri, 05 Apr 2024 16:59:01 +0000

Thank you for your interest in Crowdmark. Now that you have access to the platform, here are some helpful tips to help you navigate and experiment with the features we offer.

1. Improving student success using Crowdmark

2. Printing, scanning and uploading best practices

3. Moving from online to hybrid assessment

4. Accessibility in Crowdmark

5. Troubleshooting and common errors

6. Managing grading teams using Crowdmark

7. Question randomization in Assigned assessments

8. Paper-based multiple choice bubble sheet exams

9. Setting student accommodations

10. Using Exam Matcher

Video how to – Administered Assessments

Riley Steinbach — Fri, 05 Apr 2024 16:55:38 +0000

Check out the videos below for help with Administered assessments. If you still have questions don’t hesitate to get in touch with us at support@crowdmark.com.

1. Creating an Administered assessment

2. Duplicating an assessment

3. Assigning specific students or team members

4. Updating assigned team members

5. Add and update question labels, points and bonus questions

6. Creating multiple choice questions

7. Fixing QR-code errors

8. Matching students manually

9. Fixing a matching mistake

10. Exam Matcher – Generating tokens

11. Exam Matcher – Matching students

Managing students in Administered assessments

Jessie — Mon, 21 Aug 2023 20:12:30 +0000

The Students page of an Administered Assessment contains a wealth of information available to Facilitators and Instructors.

What can you see on the Students page?

Name: This information includes either the student’s name, email address, or both, and corresponds to what is listed on your Course’s Students page.
Student ID #: If you have updated student information in your course roster to include student ID #’s, this unique identifier will be displayed in this column.
Section: If your course has sections, you can see which section each student belongs to.
Status: This column indicates whether email notification has been sent to students. Clicking on the Email status will provide more information on the student’s contact email address, as well as the number of emails that have been sent, and an option to resend an email.
Matching: This column indicates whether a student’s booklet has been successfully matched, or if it remains unmatched. Review a successful match by clicking on the Matched status chip.
Score: This column allows you to see student scores at a glance.

You can scroll side to side to view all columns in the student table.

Search

Find any student in seconds by using the integrated search bar. Search by any available student data including name, email, or student ID.

Filters

Use filters to find groups of students who have something in common, such as email status and matching status. To view all students again, simply click on the Clear filter button to remove any selections.

Custom views

Use this tool to customize which columns are visible. Toggle columns on or off based on your preferences or needs. Any changes you make will be saved for all student tables in Administered assessments.

On desktop devices, both the first and last columns are frozen to maintain context when viewing large tables. On mobile devices, only the last column is frozen. Frozen columns ensure that the student name, email, and the student menu remain visible at all times.

Pagination

Change the number of student records that appear on each page by clicking on the number label. Choose from 10 to 100 records per page. Any changes you make will be saved for all student tables in Administered assessments.

Status

Emails are sent to students when grades are released. The Status column offers insights into whether emails were sent successfully or if there are issues. Clicking an email status opens the Email history drawer. Here, you can review the email address used, the number of sent emails, and troubleshoot any failures. You’ll also find the option to resend an email.

Learn what each status indicator means:

Status	Description
None sent	Emails will be sent when you click the Send grades to students button
Sent	The email has been sent successfully
Failed	The email failed to send. You can try resending the email. If this doesn’t work, please contact Crowdmark Support.

Matching

The Matching column tells you whether a student has been successfully matched to a booklet. If a student has been matched, you can review the match by clicking on the Matched status chip.

Here’s what the different matching status indicators mean:

Status	Description
Matched	The student has been successfully matched to a booklet.
Unmatched	The student is yet to be matched to a booklet

Student menu

Click on the three-dot menu that appears at the end of each row to access the Student menu. This menu provides easy access to all student-related actions in a single location. For Administered assessments, you can perform the following actions:

Access student work in the grading interface
See the score page
Check email history for sent emails
Visit the student’s profile page

To ensure the best viewing experience, some menu items will open in a full-screen view or side drawer. To leave these views and return to the student table, simply click the close icon positioned in the top-left corner of your screen.

Including or excluding students

Access the student roster tool by clicking the Manage students button. The student roster will indicate which students have been included in the assessment. If sections are available in your course, you can use the tabs above the list of students to navigate between sections. Click checkboxes next to students to include or exclude them from the assessment. To select or deselect an entire section, use the checkbox at the top of the page to include or exclude all students in that tab. You can also use the search function to find a student in any section.

Note: When a student is matched to a booklet, they can’t be excluded from the assessment.

Note: Exam booklets for an assessment can only be generated once during setup. It is crucial to ensure that your student roster is current before you generate exam booklets for students. We recommend that you use the extra booklets feature during setup to ensure that you have additional booklets to accommodate any late enrollments to the assessment.

Inviting a proctor to administer a remote exam session

Jessie — Tue, 27 Jun 2023 21:29:35 +0000

If some students will be writing an exam remotely (with accessibility services or in a distance education scenario, for example), you can enable the remote session feature within an existing assessment. This feature provides a proctor with the resources to administer the exam and upload student work to Crowdmark. Creating a session and inviting a proctor will enable them to:

Sign in to Crowdmark
View the session details (date, location, participating students)
Download a QR-coded exam booklet for each student (a page is added to each booklet with the student and course information)
Scan and upload completed exam booklets to Crowdmark

You will be able to track the proctor’s progress as they complete these tasks. Since each student is assigned to a specific booklet, there is no need to match the proctored assessments to students after the pages are uploaded.

Enabling remote exam sessions

Remote sessions need to be enabled per-assessment.

Navigate to your assessment’s dashboard.
Click Assessment settings.
Click the Enable remote exam sessions toggle. A Sessions link will appear in the sidebar. This is where you can set up a remote session.

Creating a remote session

After the remote sessions feature is enabled, you may create a session.

Click Sessions in the sidebar. The sessions table will appear.
Click Add session. A window with setup options will appear.
Enter the Date and time your session will take place (if it doesn’t have a set date, you may choose a date range). Enter the Location (optional).
Click Add session. The Session details page will appear.

5. Under the Students section, click Add students.

6. Enter the name or email addresses of one of the students who will participate in the session. As you type, Crowdmark will narrow down the class list. Select the correct student.

7. Repeat for each student, then click Update to add the students.

Inviting a proctor

Once your session has been created and has students, you can invite the proctor to administer the exam.

Ensure that the session details page is open. Under the Proctor section, click Invite a proctor.
Enter the proctor’s email address, a note with any additional instructions you would like to include (optional), and your contact information (optional).
Click Invite proctor. An email will be sent to the proctor inviting them to sign in to Crowdmark to access the session details.

What will the proctor see?

The invitation email will include a button for the proctor to accept the invitation and view the session page. If the proctor doesn’t have a Crowdmark account, they will be prompted to sign up before accepting the invitation.
After accepting the invitation, the proctor will be directed to the session page.
Note: You can also access this page by navigating to the assessment’s Sessions page, then clicking the proctor’s name in the “View as proctor” column.
Each booklet the proctor downloads will have a QR code on each page (like normal Crowdmark exam booklets), but the booklet will be assigned and matched to a specific student. Crowdmark will add a page to the beginning of each booklet with student and course information to ensure that proctors distribute booklets to the correct students:

There is a link to documentation for proctors in the right sidebar of the session page: Proctoring a remote exam session.

Using automated matching

Jessie — Tue, 27 Jun 2023 20:51:54 +0000

When you use Crowdmark booklets for an assessment, can automatically match cover pages to students using Optical Character Recognition (OCR). You must leave a blank space on your cover page for Crowdmark to add a region for students to write their information. Your student data must include First Name, Last Name (or just Name) and Student ID columns. Student ID must be numeric. See Managing students in a course for more information about student data.

How it works

Leave a blank space on your template’s cover page. Leave a blank space between 9 cm (3.5 inches) and 18 cm (7 inches) down the cover page for this region in addition to the 3.8 cm (1.5 inch) space for the QR code.
See examples below of the Crowdmark-generated QR code and automated matching region (shown in blue). Click the images to enlarge.
Enable automated matching when setting up your assessment template. When you upload your template you will have the option to enable automated matching. Select this option. You may customize the Student ID label to be more meaningful to your students. Upload your template and finish the assessment setup process.
Administer the assessment and upload the completed booklets. Students can write their information in uppercase or lowercase writing or a combination of both, with one character per box. They should print each character clearly and within the boxes. If a student’s name has more characters than the boxes provided they should only write one character per box and continue writing their name to the right of the boxes.
Crowdmark will automatically match the cover pages that have been uploaded. You can monitor the progress on the Match booklets page.
Match any assessments that could not be matched automatically. Use the Next unmatched button on the Match booklets page to navigate through and match any remaining cover pages.

If you use an LMS (Blackboard, Canvas, Brightspace, Moodle, etc.)

In most cases, Crowdmark can match automatically based on the student information imported from your LMS. Sometimes, depending on your LMS and your institution’s configuration, Crowdmark may not be able to access student ID numbers. You can verify whether or not Crowdmark has access to the ID number by navigating to the Students page in your course and seeing if there is any data under the Student ID Number column.

If you use Blackboard or Moodle, the integration technology does not allow Crowdmark to import student numbers. If you use another LMS and the student number is not imported, please contact Crowdmark Support; we may be able to work with your institution’s IT staff to import student numbers in the future.

As a workaround, you may update student information manually with a CSV file. See Managing students in a course for more information.

Troubleshooting automated matching

Match rates on assessments are on average above 90%. This means 90% of student information was recognized by our OCR system and able to be matched automatically to students in your class. However, sometimes there are issues that cause match rates to be lower. Here are some reasons match rates might not be as high as expected:

Student handwriting issues. If students’ writing is unclear, too light on the page, if letters are scratched out, or if the writing crosses lines in the auto-matching info grid, the OCR system may not be able to match.
Alphanumeric student ID. Student IDs are required to be numerical in order to be recognized by OCR. If they contain letters, the possibility of matching is decreased significantly. If your student IDs contain letters, you may wish to import custom ID numbers students can use on Crowdmark assessments.
Lack of sparsity in student information. OCR requires sparsity to recognize information. This means that widely-varied characters are more easily recognized than sequential or similar characters. If student names or numbers are sequential, or contain repeating characters, matching is less likely.
Scan quality. Higher contrast is better, as it will be easier for the system to differentiate between the characters and the space around them. Mottled scanning, scanning in black and white, or low-contrast scanning will have a negative effect on match rates. See this document on scanning assessments for more details.
Incorrect scanned area. If the edges of scanned pages are not clear, or if there is additional space around the pages, OCR will not be able to complete matching. See this document on scanning assessments for more details.

Using multiple choice questions in Crowdmark booklets

Jessie — Tue, 27 Jun 2023 20:34:12 +0000

Crowdmark can automatically grade up to 200 multiple choice questions in an administered assessment. If you enable multiple choice on your assessment, Crowdmark will add up to two 100-question bubble sheets with responses A–E to the end of each student’s booklet. Multiple answers to a question is supported.

Benefits of using Crowdmark’s multiple choice bubble sheet workflow include:

Fully auto-graded
Easily edit student responses or errors
Compatible with standard printing and scanning
Compatible with standard 8.5×11 or A1 paper
QR-coded pages mean students are easily matched to their work

How it works

Create your assessment template. Crowdmark requires at least one page (a cover page) in the template. You can include the multiple choice question prompts in your template or leave them out and print them outside of Crowdmark. Crowdmark doesn’t currently offer a way to “link” the question prompts to the bubble sheet but we recommend including the questions in the template so students can reference them after grades are returned.
Enable multiple choice when setting up your assessment template. You will have the option to enable multiple choice, by choosing the number of bubble sheets you’d like to include. When the time comes to define the locations of your questions you do not need to label the multiple choice questions in your template.
Set default scoring options. From the assessment’s Dashboard, navigate to the Questions page in the sidebar. Click the Multiple choice settings button. Here, you have the option to:
- Edit the default points value for a correct response. By default students receive a score of 1 for a correct response.
- Edit the default points value for an incorrect response. By default, students receive a score of 0 for an incorrect response.
- Select how a question with multiple answers is interpreted. The options are: Students need to select ALL of the correct answers (e.g. A and B) and Students need to select ONE of the correct answers (e.g. A or B).
These settings apply to all multiple choice questions in this assessment. Question-specific changes in the answer key (step 4) will override these settings.
Complete the answer key and customize question points. On the Questions page, click on “Answer key” beside the MC questions. Click the bubbles to select the correct answer(s) to each question. You may select multiple answers to a question. If the page shows more questions than you have, leave the extras blank; Crowdmark will only grade questions you have selected answers for. You may use the input boxes beside each question to change the number of points students will receive for a correct answer.
Print, administer, scan, and upload the completed booklets. Students can use pen or pencil to fill in the bubbles. Use the edge detection setting on your scanner to ensure that no black or grey background is included around the edges of the scanned images.
Crowdmark will automatically grade the bubble sheets that have been uploaded. Students receive the default scores (set in step 3) for each correct and incorrect response. The MC question/page score is the total for all multiple choice questions on that page.
Resolve any unknown responses. If Crowdmark is unsure about a response on a page or a student has not answered a question, the page will be marked with “unknown response” (and ungraded). This may happen if a student has mistakenly filled in a bubble but has attempted to cross it out or fill in a different bubble.
On the Dashboard, under the Grading heading, you will see the MC question blocks. If there are unknown responses, click MC.
You’ll see the first unresolved page in the grading interface. The unknown response will be highlighted in yellow with a “?”. Clone the evaluation and click the bubble that the student selected to resolve the response.

What do students see?

Students will see a score beside each answer. The correct response to each question will be highlighted in green.

Troubleshooting bubble sheet issues

If you find there are errors with the bubble sheet scans or auto-grading, here are some things you can verify:

Errors can be caused by bad can quality or a visual artefact disrupting the detection of the multiple choice region (a line, shape, or writing/drawing).
It’s important to make sure the black triangles surrounding the bubble area are clear and present.
If a line is appearing, your scanner may need to be cleaned.
If the area where the staples were cut off is scanned in black it may cause a problem with detection. To solve this issue, ensure that your scanner makes any extra space around the scan white.

If the multiple choice area cannot be detected at all, Crowdmark treats the page as any other page in the template and you may grade it manually. You can enter a score in the sidebar and add annotations and comments to the page. Please contact Crowdmark Support with any other questions.

Creating a QR coded Administered assessment

Riley Steinbach — Thu, 25 May 2023 18:57:36 +0000

Video transcript

Crowdmark’s Administered assessment allows an instructor to create in-person paper exams, tests or assignments that are handwritten by students and then scanned into Crowdmark for evaluation. This video will show you how to create an Administered assessment and upload student booklets for grading.

From your Course page, click Create assessment, name your assessment and then select next. Crowdmark is fully scalable and can accommodate any number of students writing exams, whether it’s a single student or a class of thousands. Select the Administered assessment option. Select the status of your assessment: upcoming or ready for grading. For an upcoming assessment Crowdmark will place QR codes on each page of the assessment. Confirm your selections and then click Done. All students enrolled in the course are automatically assigned to the assessment. This can be updated or customized at any point in the left hand sidebar or from the Assessment setup dashboard by clicking Customize. Use the checkboxes to indicate which students or sections will write this assessment. You can also view or manage your Comment library from the assessment setup dashboard for quicker more effective grading. Once assessments have been completed or uploaded, use the settings icon in the left hand sidebar to manage your assessment name, comment library grading preferences, the Exam Matcher app or invite a proctor. You can also lock or delete your assessment here. Follow the step by step outline to manage and successfully finish setting up your assessment.

Step one: set up your assessment template as a PDF file and upload the template so that Crowdmark can place a QR code on every single page. Set up your template according to Crowdmark’s unique specifications to ensure the placement of QR codes on every single page of the assessment along with a section for automated matching. Once you have uploaded your file to Crowdmark, begin to configure your assessment template by selecting the Page layout. Double sided printing is recommended to avoid students placing work on pages without QR codes. If you’re using Crowdmark’s automated matching feature our system will use optical character recognition to match your students booklets to their names and ID numbers within Crowdmark. This requires special formatting which you can view in our Help Center. Multiple choice selection will allow Crowdmark to auto-grade multiple choice questions. Administered assessments can have up to 200 multiple choice questions which are answered on a bubble sheet. Choose whether to enable multiple choice here. Choose the number of extra booklets that you would like to generate. We always recommend adding in extra booklets as they cannot be added later on. Generate a preview of your assessment to ensure that the formatting is correct. Save your template and then generate your booklets. The downloaded assessment booklets are now ready for printing and distribution to students.

Step two: define the location of each question in your assessment template so that Crowdmark can separate them for grading. You can set a questions label and maximum points as well as indicate whether the question is a bonus which means that it won’t count towards the assessment’s total point value. Save and continue once you have defined the location of each question in your assessment template. If you’re using bubble sheets for multiple choice questions, you will need to select Edit answer key to fill out the multiple choice answer key. This is what our system will use for auto-grading. Select the correct answers and then choose a point value on the left hand side. More options can be found by selecting Multiple choice settings in the Questions page.

Step three: manage scans of student work. Once students have written the assessment the completed booklets will need to be scanned and uploaded as a PDF for grading. Drag and drop or browse for your PDF file. Crowdmark will process the QR codes to ensure that each page goes to the correct location for grading. If the exam pages are out of order, you don’t need to worry: Crowdmark’s QR codes will keep them organized for you.

Step four: Match booklets. If Crowdmark’s OCR is unable to match a booklet or if you’re not using automated matching, click to match booklets to the students manually. Search for booklet information then click the check box to select a student to match. This can be done before during or after grading to search for student who’s already been matched. Type their name and click to go to their booklet. Select Grading from the left hand sidebar to begin grading students booklets.

For more information, check out the Crowdmark Help Centre, or get in touch with Crowdmark Support. We’re always happy to hear from you! Thanks for watching

Duplicating Administered assessments

Paul Mitchell — Mon, 30 Jan 2023 15:00:00 +0000

An instructor or facilitator can duplicate any assessment that they have created in Crowdmark, from any course. This makes it easy to re-use and re-purpose assessments from previous courses. To duplicate assessments,

Use the Search function at the top of the dashboard to locate the assessment you wish to duplicate. In your My Courses page you can search for a course, and in the course dashboard you can search for an assessment.

Select the assessment you wish to duplicate, and click the three dots in the top right of the assessment tile to Duplicate.

3. Type the new assessment name. Use the checkbox to choose whether to include a customized roster and the Comment library and click Duplicate assessment

Managing sections

If you wish to administer an exam to multiple sections while keeping the student lists, grading and administration separate, we recommend the following workflow (this example is for two sections but can be modified to accommodate any number):

Create two Crowdmark assessments, one for each section.
If the assessment template and questions are the same you can duplicate the first assessment using the workflow above.
Include only the first section in the first assessment and only the second section in the second assessment.
Administer the exam, then scan the pages.
After scanning, you can save time by uploading all of the pages to one of the assessments (Section 1, for example). Pages belonging to the other version (Section 2) will automatically be routed to Section 2.
Note: You will need to have a role with upload permissions in each assessment.

Managing versions

If you wish to create multiple versions of an exam, we recommend the following workflow (this example is for two versions but can be modified to accommodate any number):

Create two versions of the template file.
Create two new Crowdmark assessments and include all of your students in both versions.
Since you included all of your students in both versions, Crowdmark will generate twice the number of booklets that you will need. Print half of the generated exam booklets from each version (plus any extras you might need).
Administer the exam, then scan the pages.
After scanning, you can save time by uploading all of the pages to one of the assessments (Version A, for example). Pages belonging to the other version (Version B) will automatically be routed to Version B.
Note: You will need to have a role with upload permissions in each assessment.

The assessment settings page

maggie — Mon, 30 Jan 2023 15:00:00 +0000

The Assessment settings page contains high level controls for your assessment. On this page you can find and edit:

Assessment title. Click the pencil icon to change the title of your assessment.
Shareable link (Assigned assessments only). Click the link icon to copy and paste a link to the assessment for students.
Comment library. Click Manage comment library to find your import and export tools.
Grading team management. Use the toggle to allow graders to see each other’s evaluations.
Exam Matcher (Administered assessments only). Click to generate tokens for proctors using the Exam Matcher.
Proctored sessions (Administered assessments only). Use the toggle to enable students writing exams in separate locations with outside proctors.
Lock assessment. Click here to make the assessment read-only to all users. This setting does not remove students’ access to their Assigned submissions. Assessments must be unlocked for grading.
Delete assessment. Click here to delete your assessment.

Matching booklets to students

maggie — Mon, 30 Jan 2023 15:00:00 +0000

After your assessments are uploaded, you must match their cover pages to students. In Crowdmark, this can be done anytime after uploading student work: before, during, or after grading.

Crowdmark booklet workflows require a separate cover page for matching, which will not appear in the grading interface. Ready to grade assessments do not require a separate cover page, and all pages appear in the grading interface. In either case, a booklet must be matched to a student in order for that student to receive their grades and feedback.

Crowdmark booklets offer an automated matching option that uses Optical Character Recognition (OCR) to scan your students’ cover pages and match them with existing student information. For more info on this process, see our document on using automated matching.

Crowdmark also offers Exam Matcher, an app which enables you to match students while taking attendance at the beginning of an exam that’s using QR-coded Crowdmark booklets. Read more about Exam Matcher.

Matching assessments to students

1. There are two ways to navigate when beginning the matching process: you can select Match booklets from the side bar menu, or click the tile on the Assessment setup page.
2. On this page you will see an overview of the matching progress including the percentage of booklets automatically matched, manually matched and unmatched.
3. If you’ve opted to use automated matching, the process will begin in the background, or you can click Run automated matching to restart the OCR matching process. Depending on your class size, this may take a few minutes.
4. If you’re not using automated matching, or if you have some students who could not be matched by OCR, select manually match booklets. A cover page and search field will appear. Locate the student’s information on the cover page and begin typing it into the search field. The list of students will be filtered as you type.
5. Once you’ve found the student, click on their email address to match them. The page will automatically move to the next unmatched booklet. To stop the page from moving foward, you can turn off the Auto-advance to next booklet toggle. You may also use the Next unmatched button to skip a booklet or move to the next unmatched booklet manually. Change the filter to All to search for a specific booklet number, return to the previous page or move to the next booklet sequentially. Changing the filter back to Unmatched will allow you to continue matching in sequence. Repeat until all cover pages are matched.

Matching labels

Each match will be labelled with the student’s name and email address. You’ll also be able to see whether the page was matched automatically, manually, or using Exam Matcher. If the page was matched using Exam Matcher, you’ll be able to see the room number and proctor information. You can filter by how a page was matched using the dropdown menu.

Labelling blank or extra booklets

Below the matching search field you’ll see a checkbox that allows you to mark booklets as Extra. Use the checkbox to indicate if an exam booklet does not need to be matched. Extra booklets will be skipped over when you click Next ungraded, and will be labelled as Extra in the Overview grid.

Fixing a matching mistake

If a student has been incorrectly matched, set the filter to All and use the Search to find that student’s booklet by name or booklet number. Click the trash can icon to un-match that student. The page will stay on the unmatched booklet, allowing you to match another student. Changing the filter back to Unmatched will allow you to continue matching.

Keyboard shortcuts

You can save time by using keyboard shortcuts. While matching, a list of shortcuts can be found by clicking the Keyboard shortcuts link below the search bar.

Matching multiple students to an assessment

Some assessments involve student collaboration: two stage exams, lab reports, and group projects are some examples.

It is possible to match any number of students to a cover page by turning off the Auto-advance to next booklet toggle. This will prevent the page from automatically switching after you’ve matched a student. Once you’ve matched all of the students to the page, turn the toggle back on, or click Next unmatched to advance to the next booklet.

Each student matched to an assessment will receive their own copy of the graded assessment once grades are returned.

Managing team members

maggie — Mon, 30 Jan 2023 15:00:00 +0000

Team members include co-instructors, TAs, graders, anyone who will help with printing, scanning or grading assessments in your course. A description of each role can be found here: What is the difference between team member roles?

Adding team members

From the Course team page, team members can be added with their email addresses. After you enter the email addresses you can select which role each team member should have.

To add team members:

In your course, click Course Team in the sidebar.
Click the Add by email button in the upper right corner. A modal window will appear.
Type or paste your team members’ email addresses in the field. Click the Continue button.
You will see a list of the team members you have entered. Under the Role column, select the role each team member should have. To remove a team member, click the x. For more information, see What is the difference between team member roles?
Click the Add team members button. Your team members will be added to the course and will each receive an invitation email.

What do team members see?

Course invitation email:

After you add a team member, they will receive an email from Crowdmark. The Go to course button will take the grader directly to the course. If the team member is a new user, they will first be prompted to set up an account.

Assessment invitation email:

Depending on the team member’s role, they may have access to all assessments (Instructors, Facilitators) or they may only have access to assessments where they have been invited (Graders, Uploaders). In order for a Grader or Uploader to have access to an assessment, they must be included in that assessment’s grading team. You can see the grading team settings in the assessment by clicking Team, then Manage team from the side bar, or by selecting Customize under Team from the Roster panel on the Assessment setup page. This will open the assessment team roster.

Clicking the checkboxes allows you to select specific team members to include/exclude or use the tabs include/exclude anyone with a specific role. You can also use the top left checkbox to include or exclude all team members in that tab, or use the search function to find a team member. Once you have made your selections, click Save.

Once added, team members will receive a similar notification email each time they are included in an assessment.

Removing team members from the course

From the Course Team page, a team member can be removed by clicking the x beside their name. This will remove their access to the course and its assessments but all of their past grading and data on Crowdmark will be preserved.

Updating a team member’s role

If you would like to update a team member’s role so they have different permissions, navigate to the Course Team page, search for the relevant team member and use the drop down arrow beside their current role to select a new one.

2024 Training Webinars With Maggie and Martin

Riley Steinbach — Fri, 06 Jan 2023 21:27:21 +0000

Maggie Wood, Customer Success Manager

Martin Pham, Customer Success Manager

In class, online or hybrid

Our training webinars are designed to help you navigate through Crowdmark for both in-person and online assessments.

Engaging in our training program will help you overcome any hurdles you encounter related to assessment set-up, the grading interface, or student success.

Tuesday Tool Tips

Every Tuesday during the academic year we will shine a spotlight on one of Crowdmark’s features to assist you in optimizing your workflows through concise 15-20 minute sessions. If you would like to register for the Fall 2024 Tool Tips sessions, click here to see the schedule.

Training Sessions

These 30 minute training sessions are designed for users who would like to learn how to set up administered and assigned assessments and improve their overall grading using Crowdmark. The Crowdmark team will guide you through a live demonstration and answer any questions to ensure you are set up for success. During these sessions you can expect our team to:

Help you get familiar with each assessment workflows
Support you to create better and more engaging assessments
Help you manage the grading of assessments

Administered Training

Instructors who are interested in learning more about the in-person, paper-based exam workflows will want to attend the Administered Assessment training. Check out the schedule and register now. In each session we will cover:

How to create your course
How to set up an in-person, paper-based assessment
How to add students and team members
An overview of our grading tools
How to return results to students

Assigned Training

Instructors who are interested in learning more about online assessments, join the Assigned Assessment training. Check out the schedule and register now. In each session we will cover:

How to create your course
How to set up an online assessment
How to add students and team members
An overview of our grading tools
How to return results to students

Custom Training

To request a custom training session for you or a group, click here or email us at support@crowdmark.com. Please email us if you have any questions, concerns, or require assistance with troubleshooting.

Printing and scanning Administered assessments

maggie — Mon, 05 Dec 2022 21:57:45 +0000

The Crowdmark booklet workflows are currently designed for flexible page sizes (including 8.5 x 11, legal, A4, portrait or landscape format), printed double or single-sided and stapled at the upper left corner. See Creating a Crowdmark booklet template for more information. You can also find a complete list of tips and tricks for QR coded assessments here.

Ready to grade assessments have no requirements for print formatting, but they do have specific needs when scanning. Follow the instructions below for best practices.

Printing QR coded Crowdmark booklets

When creating a QR coded assessment, Crowdmark recommends printing double-sided booklets to avoid students writing on pages with no QR codes. If using stapled booklets, we also recommend using ‘subset stapling’ when available. This commonly available printer/copier feature will place a staple after a pre-defined amount of pages, creating stapled booklets for you.

If you’re not sure your printer has a subset stapling setting available, we suggest checking in with the local machine administrator, manufacturer tech support, or looking up the printer model online.

Please make sure to double check that you are printing on the same sized paper as your template (for example: If you create a template in A4 and print in 8.5 x 11, it will cause errors when scanning.

Scanning assessments

When scanning completed booklets, there are some optimal settings to use:

Greyscale or color. At this time Crowdmark is not fully compatible with black and white images, and scanning in black and white will lead to QR code issues on upload.
200 DPI scanning resolution in multi-page PDF format. This should result in images with dimensions 1700×2200 pixels.
It is recommended to enable a “darker” setting if your scanner allows it. This helps light pencil marks to be seen more clearly.
If you are using multiple choice bubble sheets, ensure that your scanner makes any extra space around the scan white. If the area where the staples were cut off is scanned in black it may interfere with the multiple choice autograder.
Setting your scanner to auto-detect page borders will be helpful to avoid extra white space around the page and ensure proper grahpical positioning.
Each file must be under 500 MB in size. It is not necessary to upload a single PDF file containing all the assessment images. The uploading can be done in batches, we recommend under 2000 scanned pages per batch. If scanned images are uploaded a second time, the new images will replace the old images. Existing grading and annotation is not affected by uploading new scans

Considerations for QR-coded booklets:

Crowdmark reads QR code data on the assessment pages to route the scanned images to the right place for grading. It is important that the QR codes and other page-positioning graphics are not cut off or obscured in the printing and scanning steps, and that they are properly aligned on the scanned file.
If there are pages with no QR codes included in the scans, they will not be uploaded to the assessment, and will not be gradeable in Crowdmark. To add gradeable, QR-coded scratch pages for your students to use, include blank pages in your assessment template.

Considerations for non-QR-coded and Ready to grade assessments:

Booklets must be scanned with all pages together and ideally in order. Crowdmark’s booklet detection is based on your template, and if booklet pages are mixed up Crowdmark has no way of organizing them. If necessary, pages can be manually re-ordered within a booklet in the Review scans interface.
Unlike the QR coded workflow, uploading the same scans more than once will result in new booklets being created. Scans from different assessments will not be rerouted.
Pages outside of the template such as scratch work can be included with the upload. They will automatically be appended to the previous booklet for grading. Extra pages can be dragged and dropped between booklets when reviewing scans. If an extra page or part of a booklet is included in a different batch of scans, it will need to be manually uploaded to the correct batch or booklet. See uploading completed assessments for more details
When clicking a page to upload a replacement or add an extra page after the initial upload, the file must be in JPG format.

Inviting a printer or uploader to your team

Riley Steinbach — Mon, 05 Dec 2022 20:27:31 +0000

A Printer-Uploader is a team member who prints blank assessments and/or performs the scanning and uploading of pages from an administered assessment. This can be a person at a print shop, a staff person in the department, or another instructional assistant. Printer-Uploaders are allowed to download blank assessments and upload scanned images to the assessment on Crowdmark. They do not have access to grading interfaces, results, or lists of students and team members.

Instructors and Facilitators always have permission to download and upload assessments.

Note: If you are inviting someone to upload assessments but printing is being handled elsewhere, you may wish to include this person in the assessment after the assessment has been written. This will ensure that the person uploading will not have access to test questions in advance.

Inviting a Printer-Uploader

Printer-Uploaders can be invited to the course the same way as any other team member (see Managing team members in a course).

Uploading completed Crowdmark booklets

Riley Steinbach — Mon, 05 Dec 2022 20:24:30 +0000

Once your students have written a Crowdmark booklet assessment, the booklets need to be scanned and uploaded to Crowdmark for matching and grading. If you’re uploading Ready to grade assessments, follow the instructions here. For more details on how to scan and format your batches of booklets, click here. If you need to add a specific team member to upload files, you can find instructions here.

Uploading QR-coded booklets to Crowdmark

Click into the Manage scans tab of your assessment. Here you’ll see a box where you can drag and drop, or click to browse and upload your booklets. Crowdmark will process the QR codes and any errors will be listed below for fixing. QR coded booklets uploaded to a different assessment will always be routed to the correct assessment in Crowdmark, as long as the uploader has permissions in both places.

QR code errors in assessment upload

While uploading your exams, you may encounter an upload or processing error message.

Here are some common errors and solutions:

Unprocessable file: Crowdmark could not identify a QR code on the page. It may have been tampered with, cut off, or otherwise obscured. If the text beside the QR code is still readable, you can fix this manually by clicking the Fix button. See “Fixing errors” below.
The exam image must be a JPG or PDF: Crowdmark currently supports JPG and PDF file formats. If your files are in an unsupported format, you will have to rescan the images or convert them into the proper format.
The exam must have a density of 150 DPI or more: The quality of your scans is not high enough. The system is calibrated for 200 DPI, so scans with less or more than this setting may appear blurred.
QR code must be in the right place: Errors can happen if pages are scanned without proper borders, ratios or formatting. Extra white space around the page will cause the QR code to be in a different location than where it can be read.

Fixing errors

Click the All toggle button next to History limit the list to only show Errors.
Click the Fix button beside the error message. A modal window will appear.
Locate the UUID near the top of the image, beside the QR code. You may have to scroll sideways to find it. This is a combination of letters and numbers that uniquely identifies the assessment.
Click Select or type in the UUID and begin typing the UUID. Once you’ve located a match in the list, click on it to select it.
Click Save. Click outside of the modal window to return to the Uploads list.

Uploading duplicate QR codes

If a booklet is scanned incorrectly, poor quality, or otherwise needs to be replaced, you can simply upload a new scanned file. Crowdmark will recognize the QR code and replace the image. Uploading new QR-coded files does not affect existing matching or grading.

If you have students who have inadvertently written on duplicate booklets, they will not both be able to be uploaded to the same assessment, as the images will only be replaced. Duplicate booklets may be uploaded and graded by creating a separate Ready to grade assessment. Contact Crowdmark support if you have further questions or concerns.

Uploading non-QR coded booklets to Crowdmark

In the Manage scans tab, click to browse or drag and drop the PDF files of your scanned assessments. You’ll see Crowdmark processing the uploads into separate booklets based on your template. Note: Unlike QR-coded booklets, uploading a booklet more than once will create a new booklet for grading.
Click Review to see and edit the booklets Crowdmark has recognized. Hovering over a page will allow you to zoom, rotate, delete or upload another page. Using the three dots menu on the far right, you can rotate, grade or delete the whole booklet.
If Crowdmark detects a missing page, you’ll see a red band and a warning with instructions. To fix a missing page, either upload a page or drag another page to the placeholder. You can drag template and extra pages within a booklet to reorder them or move them to other booklets.
To replace a template page, drag the new page on top of it. The label will show the page number being replaced.
To append a page, drag it to the side of the intended origin page. Watch for the page label to display which page number will be applied.

To maintain grading integrity, pages with evaluations cannot be moved. Removing evaluations from a page will restore mobility.
If a booklet is deleted, it will be labelled as such in the Review scans interface, and will not appear for grading. If a booklet was deleted by mistake, you can click Restore booklet.
Pages can be replaced, or extra pages can be appended to a booklet by clicking the upload icon on the page it’s meant to follow. A modal will appear. Click to indicate whether you’re replacing a template page, or uploading an extra page, then click to browse or drag and drop the necessary files. Note: this function requires images to be in JPG format. If you do not have the option to export the necessary file type from your original scan, you can use an online conversion tool.

To preserve grading integrity, you will not be able to move pages that contain evaluations. Removing evaluations from a page will restore mobility.

Defining questions

Riley Steinbach — Mon, 05 Dec 2022 18:52:25 +0000

After you have uploaded your template, Crowdmark will ask you to define the location and other information about each question. Note that you will be able to make changes anytime after the assessment has been set up.

For each question you will need to specify the:

Label: This is generally the question number and should be consistent with the labeling on the template. It can be up to 8 characters long.
Max. points: This is what the question is out of. While grading, graders may enter any non-negative score, but will see a warning if the score is over the max. points specified (entering bonus points are possible).
Location: By default, Crowdmark places one question label (along with its max. points) at the top left of each page (for example, Q1 [10]). These labels indicate where each question is located and this is where students will see their scores for each question.

Defining questions during assessment setup

By default, Crowdmark will place one question label (with max. points) at the top left of each page as an example. To define your questions:

Add question labels to correspond to each question and sub-question in your template by clicking the Add question button. New question labels will appear on your template.
Drag and drop the question labels to the locations of the questions on your template. The Q1 label should be dragged to the location of the first question, for example. The question label and score will appear in the same location during grading so it is helpful to place these directly on top of your question content (or in the side margins of the page) so that it will not cover any student work. Question labels define the location where annotations will be available for grading this question. The gradeable area will continue to where the next question label is placed, even across page breaks.
Rename the questions to match the naming in your template using the Label fields on the right side of the screen.
Specify the max. points (what the questions are out of) for each question in the Max. points fields.
If you want to delete a question, click the red “x” beside the Label field of the question.

You should not label multiple choice questions here. They will be automatically labelled and graded on the bubble sheets. The answer key and points for multiple choice questions can be customized after you have finished setting up your assessment. See Using multiple choice questions for more information.

Defining questions from the Dashboard

From your assessment’s Dashboard, click Questions in the sidebar. A list of questions will appear.
Click the Edit questions button in the upper right corner. You will see the same screen as in the setup.
Here, you can move question labels, rename them, change their max. points, and delete questions. Note that if you have started grading a question you will not be able to delete that particular question until all scores and feedback have been removed.

QR-coded assessment cheat sheet

maggie — Fri, 02 Dec 2022 00:05:24 +0000

QR-coded assessments are an efficient and easy-to-organize workflow for in-person testing. We’ve compiled this simplified checklist of tips, tricks and best practices for instructors using QR-coded assessments to make sure all your settings are complete for the best possible experience. If you are unfamiliar with QR-coded assessments and need more detailed instructions on how to create one, please see the Administered Assessments section of our Help Center.

Your institution may have specific requirements for QR-coded assessment creation, scanning and uploading. For more details please contact your local administrative office.

Assessment setup

Template

Has a cover page that contains no information required in the grading interface and space for auto-matching, if required
Has adequate space or extra pages for student scratch work
Is set up for double-sided printing

Click here for more on Assessment templates.

Creation

Double-sided printing is selected to ensure QR codes on every possible page. If a student writes on a page with no QR code, that page will not be available for grading
Student IDs in the course are numerical only (if using auto-matching). Alphanumeric IDs cannot be read accurately by auto-match technology and significantly reduce the match rate. If this is not the case, see this help video for instructions on replacing existing student IDs
MCQ bubble sheet(s) are selected (if required)
QR codes are clear and unobscured by text on the page
All pages are present and accounted for

Defining questions

Question labels are placed on each question. Applicable annotation area will extend across page breaks to the next label
Question points and totals are accurate
MCQ questions are NOT labeled (if applicable)

Note: question labels, points and MCQ answer key can be adjusted at any time before, during and after grading

Generating booklets

Assessment roster is up to date. One booklet will be generated for each student in the current list
Required number of extra booklets are added. Booklets cannot be photocopied because each QR code on each page is unique. Generating extra booklets will make sure you have spares
PDF of all booklets is generated and stored in an accessible location. Note the file can be re-downloaded from your Administration panel at any time

Multiple choice answer key

MCQ answer key is filled out
MCQ settings have been adjusted. For more on default point settings, see #3 in this doc.

Printing Your institution may have specific requirements for printing. For more details please contact your local administrative office.

Necessary team members have been invited. See this doc for more details on inviting a printer to your team
Collated printing and/or subset stapling are enabled. If you’re unsure whether your printer has these settings available, we suggest checking in with your local admin office, contacting manufacturer support, or looking up your printer model number online
Double-sided printing is enabled
Printed area occupies the full boundaries of the printed page. Any reduction in size ratio will affect QR code legibility
Printed booklets have strong, full blacks with clearly-visible QR codes and MCQ bubbles

Congratulations, you’re ready for students to write the assessment!

Scanning, uploading, and matching

Scanning Your institution may have specific requirements for scanning. For more details please contact your local administrative office.

Cutting staples from booklets must not affect QR codes or MCQ bubble sheet content
Scanning settings are correct according to this help doc. If scan settings are not correct, QR code, MCQ and matching errors will occur
Pages are scanned in colour or greyscale
Scans have strong, full blacks with no white space. White space or ‘noise’ in an area that’s supposed to be black will cause QR code and MCQ page errors. If scans are too light, a ‘darker’ scan setting is recommended
Student work and auto-match details (if applicable) are clearly legible
Scans are limited to batches under 2000 pages
Scanning settings are consistent across batches

Note: booklets from different assessments or versions do not need to be sorted prior to scanning. QR codes will route booklets to the right assessment on upload

Note: if scanning errors occur, re-scanning and re-uploading will replace any existing files in Crowdmark

Uploading Your institution may have specific requirements for uploading. For more details please contact your local administrative office.

Necessary team members have been invited. See this doc for more details on inviting an uploader to your team
All scanned batches are present and accounted for
QR code errors have been fixed. If you have errors that are unable to be fixed, or unexpected upload results, please contact support@crowdmark.com including a link to the assessment you’re working on, email address of the account(s) having trouble, and any relevant PDF files
Scans from different assessments or versions have routed to the proper place. If uploads are not appearing in the relevant assessment and all uploaders are confirmed to have the correct permissions, please contact support@crowdmark.com including a link to the assessment you’re working on, email address of the account(s) having trouble, and any relevant PDF files

Matching

Student roster has been synced to ensure all relevant students are available to match
Student information is complete in the course. Students will not be able to be matched if the required information is not available. See this doc for more details on automated matching, and how to adjust student info
Run automated matching has been clicked (if applicable)
Un-matched booklets have been matched manually
Extra or blank booklets have been labelled

Note: matching is fully separate from grading, and can take place before, during or after grading

Congratulations, you’re ready for grading!

Creating a Crowdmark booklet template

Riley Steinbach — Wed, 30 Nov 2022 20:34:43 +0000

For in-person Crowdmark booklet assessments, an assessment template must be uploaded to Crowdmark for formatting. This creates booklets for printing or photocopying to distribute to students. After you upload your template, select which Crowdmark booklet workflow you would like to use:

QR coded booklets are unique, coded, numbered and created individually for each student by Crowdmark.

Non-QR coded booklets are based on a single photocopiable booklet formatted in Crowdmark.

To learn more about the differences between the two types of booklets, see details here.

The pages can be any size — including 8.5″ x 11″, legal, A4, portrait or landscape format. If you’re using QR coded booklets, Crowdmark will add a unique QR code to each page and generate a printable PDF file containing a booklet for each student. If you choose Non-QR coded booklets, Crowdmark will generate one photocopiable booklet with the assessment information on each page along with any additional formatting options you have added.

Download an example template

These documents are formatted so you can simply replace the content with your own. The formatting will work with both booklet types. Don’t forget to save to PDF before uploading:

This example template is in PDF format and can be uploaded directly to Crowdmark for a test run:

Example template (PDF)

Template specifications

To be compatible with Crowdmark booklets, your template must meet the following criteria:

PDF format. The template must be exported to PDF.
A cover page. The first page is where a student will write their name and other personal data. You may include instructions or other information, but note that you will not be able to grade this page. To retain anonymity, it’s important that this is the only page containing identifiable student information. If you’d like to use Auto-matching, see the instructions below for required spacing.
A white space region on the top of each page for QR code or assessment info placement. A simple way to proceed: leave at least 3.8 cm (1.5 inches) blank at the top of each page.

QR coded booklets only: pages must be the same size and orientation. Crowdmark determines QR code placement based on the center coordinates of the first page. If the following pages are a different size or orientation (landscape or portrait), the QR codes and header information may be superimposed over your assessment content.

Formatting your template for automated matching

If you would like to use Crowdmark’s automated matching feature, you will need to add extra space to the layout of your cover page. Crowdmark will add a region for students to write first name, last name, and student ID number. Please leave a blank space between 9 cm (3.5 inches) and 18 cm (7 inches) down the cover page for this region.

The fields will be matched with student data after you upload student work. To read more about this process, see our document on using automated matching.

See examples below of the Crowdmark-generated QR code and automated matching region (shown in blue). Click the images to enlarge.

Formatting your template for multiple choice

If you would like to include some multiple choice questions in your assessment, see Using multiple choice questions. You may include up to 200 questions with responses A–E in your template. Crowdmark will automatically add bubble sheets to the end of your template file after you enable multiple choice.

Including scratch pages in your template

QR coded booklets: Pages must contain QR codes to be graded in Crowdmark. If students write on pages with no QR codes, they will not be uploaded to Crowdmark and will not be present for grading. If you would like to grade your students’ scratch pages, or have them available to view while grading, include extra blank pages in your template. These QR-coded pages will scan and upload to Crowdmark along with the rest of the student’s work.

Non-QR coded booklets: Extra pages can be added after the assessment is formatted and written. They can be scanned and uploaded with the booklets, or they can be appended individually by uploading to a specific booklet in the Review scans page.

More details: