From the Assessment setup page or the Settings page, you may export the assessment’s entire comment library or comments from a specific question to a CSV file. This file can be used for analysis or imported to other assessments in Crowdmark.

To import and export comments from a CSV navigate to the Assessment setup page, or the Settings page.

Instructors and Facilitators can create rubrics in advance of grading by creating shared comments with points in the Comment library. When a comment is shared, it will appear in the Comment libraries and comment suggestions of all Graders, Facilitators, and Instructors on the team. Team members can then drag and drop the comments onto student work to apply the feedback and points to the question.

Setting up a rubric

You have two options for setting up a rubric in the comment library: you may set it up directly within the library in the grading interface, or you may fill out and import a CSV of comments to the library.

To set up a rubric within the comment library

1. Open the Comment library.
2. Click the New comment button in the upper right corner. A text box will appear.
3. Click the Share with team toggle above the question content. The comment will be shared.
4. Enter the comment text and points and click Save.
5. Repeat with each rubric item.

If you notice that a page of student work has been uploaded in the wrong orientation it can be easily fixed during grading.

1. Click the sidebar icon in the lower left corner. This will open a drawer with a thumbnail view of the pages.

    

2. Click the Rotate icons on the page you wish to rotate.

   

Tags provide a flexible way to organize and bookmark student work and evaluations with your team members. While grading, you can use a tags to bookmark exemplary answers, share questions that require another look, or to indicate suspected cheating. Students cannot see tags.

Adding a tag

1. Navigate to a question in an assessment as you would if you were going to grade it.
2. Click the Tag button in the sidebar (or enter the keyboard shortcut “t"). The tag input box will appear.
3. Enter a name for the tag and click the + button. The new tag will appear below.

Viewing evaluations with specific tags

You can view all evaluations with a specific tag or group of tags using the filter tool. See Filtering evaluations to learn more. Tagged booklets require an evaluation on the page to be included in the filter.

Comments can be managed through the comment library tool, which can be accessed through the book icon in the grading toolbar.

The comment library shows all comments you have left on a question. Graders can see their own comments and shared comments, and Instructors and Facilitators can see everyone’s comments. Using the comment library, you can view comments filtered by version, as well as view all shared comments for a question.

Benefits of using the comment library include:

• Once a comment is in the library, it can be used to grade that question through the entire assessment.
• If you select a comment then edit the content, it will save as a new comment.
• Comments from randomized questions are labelled by version, and they are available to be used across any version of that question.
• Comments can be edited once to change every instance used across the assessment
• Instructors and Facilitators can also set up comments in the comment library or set a rubric using comments before grading starts.
• A comment can be shared with the grading team by turning on its Share with team toggle. Shared comments will appear in every grader’s comment suggestions.

Searching for comments

To find a comment in the comment library, use the Search bar in the top left. All comments containing that keyword will appear, along with labels for which question and version they belong to.

You can also search for a comment while grading: when you begin to type comment content, auto-suggestions from your comment library will appear below. If you’re grading a question with versions, comments that apply to the version you are grading will appear first, followed by comments from other versions.

Comments from all versions will be available to be used. Duplicate comments across versions will display in each version, and will be linked automatically in the library to preserve bulk-editing.

Filtering evaluations by comment

Above each comment you will see the number of times it has been used. Grading teams can click on this number to show all evaluations using the comment using the filter tool. If the toggle to Allow graders to see each other’s evaluations is turned off, only Instructors and Facilitators will have access to this comment filter.

Bulk editing comments

If a comment has been used multiple times, you can make changes to all instances of the comment by using the comment library. Graders can only see and make changes to their own comments, Instructors and Facilitators can see and make changes to everyone’s comments. Duplicate comments automatically revert to the existing comment in the library, so you won’t need to worry about having to edit more than one. It will be important, however, to keep track of minor edits that have saved as a new comment. In an assessment with question randomization where there are linked duplicate comments across versions, you will have the option of editing the comment for the current version, or for all versions.

1. Select the comment you wish to edit.
2. Click the Edit buttton below the question content.
3. Make your changes and click Save. All instances of the comment will be updated.

Reordering comments

You can reorder comments to customize how they are displayed in the comment library as well as in the grading toolbar. To do this, drag a comment up or down the list of comments in the left sidebar of the comment library.

Adding a comment before grading

If you have a rubric or example feedback prepared in advance, you can create comments and associate them with point values without grading student work.

1. 1. Select the question or version you’re working on, and click the New comment button in the upper right corner of the comment library. A blank comment will appear.
   2. Enter your comment text, point value (optional) and click Save. Your comment will be stored in the comment library and will appear in your list of frequently used comments, visible in your comment toolbar. Comments for multiple versions of a question will be labelled as This version and Other versions. Comments from all versions will be available to use on any version, and duplicate comments across versions will be linked to preserve bulk-editing.

1. Optional: If you are an Instructor or Facilitator and wish to share your comments with the grading team, see the next section, Sharing a comment with your team.

Sharing a comment with your team (Instructors and Facilitators)

Instructors and Facilitators can choose to share comments with the grading team. When a comment is shared, it will appear in the comment libraries and comment suggestions of all Graders, Facilitators, and Instructors on the team. The comment is read-only to Graders.

Also see: Using a rubric

1. Select the comment you wish to share.
2. Click the Share with team toggle above the comment content. The comment will be shared.

An evaluation is the collection of annotations, comments, and score left on a question by a grader. When viewing student work, you’ll see a list of evaluations in the right sidebar.

• If nobody has graded the question, you’ll see Your name (new) indicating that you’re looking at your own blank evaluation that has not yet been saved. Once you add an annotation, comment, or score, your evaluation will be saved.
• If another grader has graded the question, you will see their evaluation first. You will also see an Add evaluation button, and if you’re an Instructor or Facilitator, a Clone/Edit evaluation button.

Primary evaluations

Students will only see one grader’s evaluation when their grades are returned. This evaluation is called the “primary” evaluation. While grading, the primary evaluation is indicated by the checkmark icon to the right of the grader’s name. By default, the first evaluation left on the question is marked as primary. Instructors and Facilitators can override this by adding their own evaluation or by selecting another evaluation to be primary.

Cloning evaluations left by others

Instructors and Facilitators have the option to clone another grader’s evaluation. This is useful in situations where an Instructor or Facilitator is reviewing a grader’s work and needs to make changes before the evaluation is shown to the student. Cloning leaves the grader’s work untouched while the Instructor or Facilitator makes changes to their own copy, ensuring that ownership of each piece of feedback is maintained.

Video transcript

Crowdmark makes grading remote exams easy and seamless! This video will show you the remote exam grading workflow.

On your Assessment screen, you’ll see submissisions organized by question. This allows graders to grade individual questions for consistency. You can also have individual graders grading different questions simultaneously. You’re able to see which questions have been submitted, and how many have been graded. Multiple Choice questions are automatically graded. For Image/PDF responses and Text entry responses, click into a question to begin grading.

Once you’re on the grading screen, you’ll see toolbars on the left and right hand side of your screen. On an Image/PDF response, you can use the pencil to hand-draw, place checks, “x”s or question marks, select an area to highlight, or change the colour of your entry by clicking on the droplet icon. Use the garbage can icon to delete anything you’ve added to the page.

Leave comments in your students’ work by clicking on the speech bubble icon. Every time you create a comment, it is automatically saved — creating your Comment library. Comments support markdown and LaTeX, images, or links to additional resources. If you choose to assign point values to a comment, points are automatically applied or removed as you grade. You can pre-populate a question’s Comment library using a CSV file, making it visible to all graders. This allows you to use the Comment library as a rubric. When students receive their grades, they will see all the feedback on their work.

On the right, you can input points using the built in keypad, or the numbers on your keyboard. You can Tag evaluations, and Filter assessments by question, team member, tag or point value. These tools help you get an overview of your grading, and can be a great way to keep track of trends. You can use the Keyboard shortcuts to move through more quickly. You can also view grades and comments from other graders, move to the next ungraded question, and click to move between student booklets. You can also use the Overview Grid to click through to a specific student or question.

Scroll up or down to move through the assessment you’re currently working on, and click on the question labels to grade specific questions.

When grading a Text entry question, your grading tools are designed for text. Use the cursor to select a block to be highlighted or struck-out. Each of these options allows you to attach a comment. Click the icon with the comment symbol to minimize the boxes as you work. Text responses also include the option for students to place links, or downloadable files within their responses.

You’re also able to view and override grading on your multiple choice questions.

Click the name of your assessment to exit the grading screen, then click on the Results tab to see students grades. Here you can distribute the grades to the students, or export the grades to a CSV file. If you’re using one of our LMS integrations, you may also have the option to export grades directly to the LMS.

For more information, check out the Crowdmark Help Centre, or get in touch with Crowdmark Support. We’re always happy to hear from you! Thanks for watching!

To start grading, click a question on your assessment’s Dashboard (Q1, for example). You will see the first student’s Q1. If grading has already started, you’ll be taken to a student’s Q1 that has not been graded.

The grading screen shows the current student’s pages arranged vertically. You can scroll to see all of the student’s responses. On the right is the grading sidebar, where you can see the booklet number and question you are currently grading.

• To go to the next Q1 that has not been graded: Click the Next ungraded button at the bottom of the grading sidebar or type Enter on your keyboard as a shortcut. Please note that “Next ungraded” won’t always navigate to the next student in sequence, especially if others are grading. It is designed to skip booklets that other graders are looking at to avoid collisions.

• To navigate between student papers: Use the arrows beside the Booklet # title or the keyboard shortcuts: l or shift + → to go to the next student, and h or shift + ← to go the previous student. You can also click the Booklet # title to go to a specific booklet.

• To navigate between questions: Click on the blue or green question label (Q2 [Not graded], for example) on the student’s page or use a keyboard shortcut: j or shift + ↓ to go to the next question, and k or shift + ↑ to go to the previous question. You can also go back to the assessment’s Dashboard and select a different question.

You can see an overview of grading by clicking the Overview grid button on your assessment’s Dashboard. Clicking on a grid tile will take you directly to that student’s response.

Instructors and Facilitators can view evaluations with specific tags or a combination of tags, a score or range of scores, or graded by a specific Grader by using the filter tool. You can also use the filter tool on a specific comment using the comment library (see Using the comment library).

Applying filters

1. 1. While grading, click Filter evaluations in the top right corner.
   2. You can filter by question version, team member, tag or score. Numbers beside the filter checkboxes indicate how many evaluations match that filter.

1. Click Apply.

If multiple filters are selected, only evaluations matching ALL filters will be shown.

Navigating filtered evaluations

A bar will appear on the bottom of the page showing which filters you have selected. The grading sidebar navigation will change to allow you to cycle through only evaluations with the selected filter(s). The Next filtered button loops through all pages that match the active filters. Clicking the Back button in your browser will take you to the last page you viewed.