Within ArbiterSports, we offer a powerful Evaluation tool that Assignors can use to gather information about its official’s performance. Officials, Assignors, and/or AD/Coaches may be set up to fill a form out about the performance of an official that was assigned to a game. *To obtain the Evaluations feature, please contact our Renewals team (801-702-8028). *
Before we set up a new evaluation, we will need to make sure of the following.
- Permission to set up Evaluations.
- Positions within the sport are marked as Evaluators.
- Relationships to the correct AD/Coach have been made with the corresponding team.
To do this, you’ll need to be the Primary Assignor
- Go to the Settings tab and click on the Advance blue link
- On the section for User Types, select Group Admin
- Inside the category of “By Permission set or by Group Admin”, use the second box down and select your name or the sub assignor name if being done for a sub assignor.
- Scroll down and find the permission called “View Evaluations Only”
- The box will need to be set to No.
- Scroll back up and click on update next to your name inside the By Permission set or by Official.
It is recommended to add an Evaluator/Observer position to know which position is the evaluation (Add Sports/Levels article)
- Go to the Assigning tab and click on Sports/Levels
- Click on the pencil next to the Sport that the Evaluation will be for.
- Click the pencil next to the Evaluator/Observer position
- Click on the box for Evaluator. This will open up the next two boxes on the right.
- If you would like for the official to know that an Evaluator is assigned to the game, mark the box under the Visible To Official.
- If you would like for the Contact to know that an Evaluator is assigned to the game, mark the box under the Visible To Contact.
- Click on the blue floppy disk icon and proceed to hit Save.
After that has been finished, proceed to add the position under the level that you would like the evaluator to evaluate for. (Positions Article)
To create a relationship you must:
- Click the Users
- Click the sub-tab Contacts.
- Click Relationshipsin the upper-left side under Utilities.
- Select a contact and check Show All on the right-hand side.
- Check the corresponding Team and click Save.
Upon completion of the prerequisites, we may now proceed to set up Evaluations
In this section, you’ll learn how to set up a new evaluation, know the function of the settings, build criteria.
- Go to the Users tab and click on Evaluations located at the left-hand side of the screen.
- Click on Add New Evaluation Report
- Proceed to give the Evaluation a name.
- You may select Use ArbiterSports template if you’d like to use one of our most commonly used evaluations (You’ll need to edit the evaluation name if selected). Copy one of your previously used evaluations or Start from Scratch. Upon making a decision, click on Next.
You’ll find yourself to be in the Edit Evaluation Report. This is where basic settings are set.
- Evaluation Report Name: Give a customized name to the specific report
- Score Range: The score range you want your report to have
- Require Comment (Low Score): If the evaluator gives a specific low score, they’ll be required to give an explanation as to why the score was given
- Require Comment (High Score): If the evaluator gives a specific high score, they’ll be required to give an explanation about the score given.
- Publish Responses #: After the number of responses submitted reaches this number, the officials and sub-admins may view them
- Sports & Levels: Select the sport and level this evaluation will apply to.
- Positions: Select the positions that are going to be evaluated.
- Send Reminder After ‘X’ Days: If someone is responsible for doing an evaluation, this setting will send out a reminder to fill out the evaluation after the number entered.
- Start Date: When the Evaluation will be available to be filled out. Usually this date is the first day of the season
- End Date: The last day the evaluation will be open to evaluators
- Use Sub-Categories: Each of the ratings criteria can be separated into sub-categories. This allows the assigner to group common criteria and thus determine weights better for specific areas.
Next, decide who can complete the evaluation report.
- Admin: Allow admins to complete evaluations. If scope is set, it will only let the admins evaluate games according to the parameters set inside Scope.
- Evaluators: The person assigned to the Evaluator/Observer position. Evaluators may not be evaluated.
- Lead Officials: The lead official is defined by the preference currently set. It can be the official in the first position listed or the highest ranked official. As an admin, you may edit the slot and mark the lead official position.
- All Officials: This allows every official assigned to the game to evaluate each other. They may not evaluate themselves.
- Team Contacts: Coaches, ADs, Assistant ADs. If you only want one contact to evaluate and not the rest, go to the Assigning tab and the Teams sub-tab. Click the number of contacts and only have the box for Evals checked for the contacts that should complete the evaluation.
Decide who can view the responses.
- Officials: If you gave a number under the Publish Responses # section, you’ll need to have this checked to give officials permission to view responses that were made for them.
- Sub-Admins: Allows sub-admins to see responses. This goes according to Scope as well.
If you would like to receive an email after an evaluation is completed, under the section of “Send Emails when Evaluations Completed”, click on the green plus sign. Enter the email that would like to receive these emails. Click on the green plus sign again to save the email.
Inside the Scoring Description box, describe what you want each individual rating to count as. This will explain to the evaluator what would be expected of the official to receive the intended score. Click “Save” or “Build Criteria” to save the report.
Categories and Building Criteria
If you’re using Sub-categories, you’ll need to add them first under the “Evaluation Report Categories” by hitting the plus sign, typing in the name of the category, and clicking on blue floppy disk icon. Next, build your criteria. These criteria are the points of officiating that you want the officials to be evaluated on.
- Click on the green plus sign. This will pull up its fields.
- Enter the name
- Enter the standard. This is what the evaluator should be looking for to rank the official.
- Enter the weight. The weight determines how much each criterion will affect the overall score. The total weight of your criteria will need to add up to 100%.
- Select the category that best fits the criteria.
After all is set and done, hit save at the top or bottom of the page. Congratulations! You have set up an evaluation!
Admins have the ability to edit an evaluation that was created previously.
- Go to Assigning and Users.
- Click on Evaluations located at the left-hand side of the screen.
- Click on Edit under the name of the evaluation you want to edit.
- Make any necessary changes.
- Click Save at the top or bottom of the page.
If the Assignor wants to submit an evaluation, the Arbiter allows this after the game has happened.
- Go to Assigning and find the game that you are looking to evaluate.
- Click on the number of slots. When they open, you’ll see a red A+ at the right side of the slot.
- Select the form that will be filled out and click on Go.
- Fill out the form in a honest way.
- Click on Save.
To monitor responses to any evaluation, go to the Users tab and click on Evaluations at the left-hand side of the screen. Under the name of the evaluation, click on Monitor. Click on the envelope of the desired evaluation report. This will display a listing of officials and their average scores from every submitted response. To change the sort order, click on the column headers. To view the average score for each criterion click on the number under “overall score."
To view an individual's reports click on the numbers in the "# Responses" column. This will display all of the reports submitted for the official and the evaluator who submitted the report. To view an individual report click on the score on the right of the page. If there is an issue the assigner may unsubmit the report, which sends it back to the person who submitted the report for revision and resubmission. The official then will have the opportunity to either completely redo the report or just modify it. The assigner can change the scores or comments from the report view page.
You can also print out a report of all the responses that have been made for a certain evaluation. Simply click the Responses link on the left side under the "Reports" section. Choose your export format, and then check which Report Type you want. It must be either "Individual Responses" or "Individual Summary." To view the average ratings of an evaluator change the pull-down button on the left side of the screen from officials to evaluators. This shows the average score that this person has rated on others. By clicking on the number under "#Responses" the assigner can see all the reports that a person has submitted and the score associated with each. An individual report can be viewed by clicking on the score on the right of the page.
If you no longer need an evaluation, you may delete it.
- Go to Users and click on Evaluations
- Under the evaluation you would like to delete, click on the Delete option.
- Click on OK to confirm the deletion.