When does a Strike happen?
When an annotator obtains an accuracy score of less than 90%, an investigation is automatically triggered.
The Quality Assurance team will identify the mistakes made by the annotator and determine the severity of the mistakes made. Generally, severity of mistakes are as below:
| Mistake | Severity Level | Likely Action |
| Misdrawn / Poor boundaries | 3 : Critically High | Strike |
| Mislabel | 2 : Medium | Further investigation |
| Extra or Missing Annotation | 1 : Low | Further investigation |
The investigation will take into consideration the ratio of mistakes made. The team will also review the project instructions and the work of the annotator in question. Comparison with other annotators will also be made. In cases of ambiguity, a user interview may be scheduled to ensure the team has sufficient data to make a firm decision.
Based on the data gathered, the team will decide if a Strike is warranted. If it is, the individual will be informed, and a Warning Letter will be issued along with personalized feedback in hopes the individual will learn.