Since we are trying to keep from letting the elite only run the system. I think it may be okay to not have a concrete rating scheme from 1 to 5 with absolute criteria for each value. Instead we could have a STANDARD course and then rate relatively to that course.
So for example, we deem course X to be a 5 in the category for teaching ability, this is our standard. Say then a few guys go to course Y and it's just not as good as X in terms of that category. So then course X gets a 4 in that category. Then we have a few guys that go to course Z and Z is better than X. So therefore we rate Z as 6.
I expect that most that will be using this will have the sense to identify that higher scores means better. Then we don't really have to get swamped with written criteria and standards because it's all relative to each other.
I highly disagree with that.
If we had one course that is the golden standard then why recommend anything else?
My golden standard and needs might be different than a CCW, or deploying army Snipers needs. So everyone will have a different golden standard.
Just like stated earlier if we have criteria and we can all discuss the requirements of the criteria then I think we will have a good baseline of what is required from a professional teaching within his lane. I think the 1-5 is a good system with an average based on categories. Again I bring this back to how a lot of police FTO programs work. You have one rookie that is being taught/evaluated. You have several graders who all have different backgrounds and personality but we try to keep the grading standardized. No it's not perfect. However this is better. We don't have just four FTOs. We have lots of good dudes bringing a lot of information so the more info the more better.