Instructors endorsement and rating scheme

OpDSource

'Merica!
Moderator
Vendor
#1
With a select few individuals attempting to corner the market on creating an accreditation for instructors current and future, PS can be a resource for potential students. Offering a rating scheme that an electoral college of industry professionals can build out on each instructor (or company) to give students a no bullshit idea of the relevant quality of their offerings.
What needs to happen now is building what that rating scheme looks like. In the following post, I will throw some ideas up to start the conversation.
 

OpDSource

'Merica!
Moderator
Vendor
#2
Should be broken down then an average score extracted from all sections combined.
Instructors also need to be broken down into the different categories they teach -
Tactical // Marksmanship // Medical // Combatives

So a rating scale for a tactical instructor should go something like this.
Capability - 4
Real World Experience - 3
Teaching Ability - 5
Time Management - 4
Cost - 3
Total Score - 3.8 out of 5
Safety should be a separate Go / No-GO quality. I can't be 3.8 out of 5 safe. Someone is getting shot.
Overall rating out of 5, anything below a 2.5 does not receive a PS recommendation. Anything above 4.5 receives PS Top Choice. 5 specific aspects for each type of instructor.
 
Last edited:

Tore Haugli

Moderator
Moderator
#3
As you posted in the thread on FB, I think a set of criteria that are scored in accordance with a scale based on how well or how poor that criteria was received.

Here is my suggestion, which is based on an academic grading system. These are already in use, so no need to reinvent the wheel, but could probably use some refinement. You can use letters or numbers to rate, though numbers make it easier to calculate an average score.


A/6 Excellent
An excellent performance, clearly outstanding. The candidate demonstrates excellent judgement and a high degree of independent thinking.

B/5 Very good
A very good performance. The candidate demonstrates sound judgement and a very good degree of independent thinking.

C/4 Good
A good performance in most areas. The candidate demonstrates a reasonable degree of judgement and independent thinking in the most important areas.

D/3 Satisfactory
A satisfactory performance, but with significant shortcomings. The candidate demonstrates a limited degree of judgement and independent thinking.

E/2 Sufficient
A performance that meets the minimum criteria, but no more. The candidate demonstrates a very limited degree of judgement and independent thinking.

F/1 Fail
A performance that does not meet the minimum academic criteria. The candidate demonstrates an absence of both judgement and independent thinking.
 

OpDSource

'Merica!
Moderator
Vendor
#4
As you posted in the thread on FB, I think a set of criteria that are scored in accordance with a scale based on how well or how poor that criteria was received.

Here is my suggestion, which is based on an academic grading system. These are already in use, so no need to reinvent the wheel, but could probably use some refinement. You can use letters or numbers to rate, though numbers make it easier to calculate an average score.


A/6 Excellent
An excellent performance, clearly outstanding. The candidate demonstrates excellent judgement and a high degree of independent thinking.

B/5 Very good
A very good performance. The candidate demonstrates sound judgement and a very good degree of independent thinking.

C/4 Good
A good performance in most areas. The candidate demonstrates a reasonable degree of judgement and independent thinking in the most important areas.

D/3 Satisfactory
A satisfactory performance, but with significant shortcomings. The candidate demonstrates a limited degree of judgement and independent thinking.

E/2 Sufficient
A performance that meets the minimum criteria, but no more. The candidate demonstrates a very limited degree of judgement and independent thinking.

F/1 Fail
A performance that does not meet the minimum academic criteria. The candidate demonstrates an absence of both judgement and independent thinking.
I would keep it to 5. The mean of the rating system will end up falling in between somewhere. 5 specific rating points keep the value high for each one, so if one part falls dramatically low it will affect the mean score quite a bit more. So I would just get rid of the E/2 portion of your post.
 
#6
Should be broken down then an average score extracted from all sections combined.
Instructors also need to be broken down into the different categories they teach -
Tactical // Marksmanship // Medical // Combatives

So a rating scale for a tactical instructor should go something like this.
Capability - 4
Real World Experience - 3
Teaching Ability - 5
Time Management - 4
Cost - 3
Total Score - 3.8 out of 5
Saftey should be a separate Go / No-GO quality. I can't be 3.8 out of 5 safe. Someone is getting shot.
Overall rating out of 5, anything below a 2.5 does not receive a PS recommendation. Anything above 4.5 receives PS Top Choice. 5 specific aspects for each type of instructor.

I second this method. I like the subcategories being averaged into an over all rating. the only thing I would add is that we clearly define what is being looked for in each category and examples of proper scores.

It reminds me of how we evaluate rookies while in FTO. every category is scored and all FTO's are on the same page as what is expected and how we grade. (or as much as we can be all having different personalities and pet peeves.)

it also helps having the reviewer background posted that way you know where he is coming from. I will look at different topics in a different light and it will be nice to get perspective from different personalities.
 

OpDSource

'Merica!
Moderator
Vendor
#7
Eric Graves made a very good point on FB when he wrote that he doesn't care what Joe Bob thinks of Jimbos training because that is the most exposure Joe Bob has ever had. The reviews and ratings need to be given by Industry professionals that have a bunch of exposure so an objective rating and review can be given. This needs to be done totally in the professional realm. An AAR from a typical student is encouraged, but this is not that.
 

Matt Landfair

Matt Six Actual
Administrator
#8
Eric Graves made a very good point on FB when he wrote that he doesn't care what Joe Bob thinks of Jimbos training because that is the most exposure Joe Bob has ever had. The reviews and ratings need to be given by Industry professionals that have a bunch of exposure so an objective rating and review can be given. This needs to be done totally in the professional realm. An AAR from a typical student is encouraged, but this is not that.
I have set up an area for people to provide their frames of reference, experience, and training.

I'm afraid keeping it to industry pros puts us near the same issue of why we are doing this.

How do we define industry professional?
 

OpDSource

'Merica!
Moderator
Vendor
#9
I have set up an area for people to provide their frames of reference, experience, and training.

I'm afraid keeping it to industry pros puts us near the same issue of why we are doing this.

How do we define industry professional?
That's why in the OP I referred to such a group as an electoral college of professionals. There needs to be a vote held for the members of such a group so the community feels well represented. If it is done any other way then I don't honestly believe it will be objective and therefore will fail.
This is a position of responsibility and if this will be taken seriously. I think PS needs to invest to get members of the reviewing group out to training to conduct the review research. This isn't an easy endeavour to do, but if its not done to this depth then we may as well ask for $375 a year and give instructors a made up cred that they can get a patch made of and mark their website.
 
#11
its not an AAR. It's more of a go/no validation of a company or an instructor. like stated above they would receive a score based off of knowledge/ experience in the subject being taught, cost/ value, time usage, safety, etc.

At least that's how I'm understanding it.
 
#12
If you guys want a mini AAR template everyone can do I can bang one out. I have a long format template I use for my reviews on my site but a mini one should be easy to do.
In any case a short AAR template would be sweet. I've gotten off in the weeds writing stuff up, having an outline to keep on track could be a benefit to readers.
 
#13
That's why in the OP I referred to such a group as an electoral college of professionals. There needs to be a vote held for the members of such a group so the community feels well represented. If it is done any other way then I don't honestly believe it will be objective and therefore will fail.
This is a position of responsibility and if this will be taken seriously. I think PS needs to invest to get members of the reviewing group out to training to conduct the review research. This isn't an easy endeavour to do, but if its not done to this depth then we may as well ask for $375 a year and give instructors a made up cred that they can get a patch made of and mark their website.

I don't think anyone would turn down free training. I'm sure there are a lot of dudes here that go to a lot of training though. I try to get atleast 6 classes a year in. I've invested a lot of money into my training. Was able to get advanced and top shooter at Rogers shooting school my first go so I guess some paid off.

I've also wasted some money in training. So it would be really cool to get this spun up. Something like this could have saved me thousands and a lot of wasted time. Atleast it would have for me with what I want to acclomplish with my training.

I would hope that within this group there are some dudes that know right from wrong most of the time. Then within that group there is a large portion that pay their own way or have paid their own way in training before. So assuming we have that it shouldn't be hard to lay a foundation with some good to go and stay away from companies.

#my2cents
 
#14
Since we are trying to keep from letting the elite only run the system. I think it may be okay to not have a concrete rating scheme from 1 to 5 with absolute criteria for each value. Instead we could have a STANDARD course and then rate relatively to that course.

So for example, we deem course X to be a 5 in the category for teaching ability, this is our standard. Say then a few guys go to course Y and it's just not as good as X in terms of that category. So then course X gets a 4 in that category. Then we have a few guys that go to course Z and Z is better than X. So therefore we rate Z as 6.

I expect that most that will be using this will have the sense to identify that higher scores means better. Then we don't really have to get swamped with written criteria and standards because it's all relative to each other.
 
#15
Since we are trying to keep from letting the elite only run the system. I think it may be okay to not have a concrete rating scheme from 1 to 5 with absolute criteria for each value. Instead we could have a STANDARD course and then rate relatively to that course.

So for example, we deem course X to be a 5 in the category for teaching ability, this is our standard. Say then a few guys go to course Y and it's just not as good as X in terms of that category. So then course X gets a 4 in that category. Then we have a few guys that go to course Z and Z is better than X. So therefore we rate Z as 6.

I expect that most that will be using this will have the sense to identify that higher scores means better. Then we don't really have to get swamped with written criteria and standards because it's all relative to each other.
I highly disagree with that.

If we had one course that is the golden standard then why recommend anything else?

My golden standard and needs might be different than a CCW, or deploying army Snipers needs. So everyone will have a different golden standard.

Just like stated earlier if we have criteria and we can all discuss the requirements of the criteria then I think we will have a good baseline of what is required from a professional teaching within his lane. I think the 1-5 is a good system with an average based on categories. Again I bring this back to how a lot of police FTO programs work. You have one rookie that is being taught/evaluated. You have several graders who all have different backgrounds and personality but we try to keep the grading standardized. No it's not perfect. However this is better. We don't have just four FTOs. We have lots of good dudes bringing a lot of information so the more info the more better.
 
#16
Since we are trying to keep from letting the elite only run the system. I think it may be okay to not have a concrete rating scheme from 1 to 5 with absolute criteria for each value. Instead we could have a STANDARD course and then rate relatively to that course.

So for example, we deem course X to be a 5 in the category for teaching ability, this is our standard. Say then a few guys go to course Y and it's just not as good as X in terms of that category. So then course X gets a 4 in that category. Then we have a few guys that go to course Z and Z is better than X. So therefore we rate Z as 6.

I expect that most that will be using this will have the sense to identify that higher scores means better. Then we don't really have to get swamped with written criteria and standards because it's all relative to each other.
I highly disagree with that.

If we had one course that is the golden standard then why recommend anything else?

My golden standard and needs might be different than a CCW, or deploying army Snipers needs. So everyone will have a different golden standard.

Just like stated earlier if we have criteria and we can all discuss the requirements of the criteria then I think we will have a good baseline of what is required from a professional teaching within his lane. I think the 1-5 is a good system with an average based on categories. Again I bring this back to how a lot of police FTO programs work. You have one rookie that is being taught/evaluated. You have several graders who all have different backgrounds and personality but we try to keep the grading standardized. No it's not perfect. However this is better. We don't have just four FTOs. We have lots of good dudes bringing a lot of information so the more info the more better.
I don't mean to get away from the concept of categories. I think that is absolutely critical. The reason i say to pick a golden standard is its only a golden standard in that particular category. So if Z is a 6 in teaching ability because we started with the standard of X (teaching ability). Im not advocating for deciding arbitrarily on a course that we deem perfect. I'm advocating on saying that X course is the standard of a given category and rating other courses relative to that in that category only.

Sent from my SM-G930V using Tapatalk
 
#17
This is more difficult to explain than it is to do in practice. So I'll explain the whole concept.

You have your 5 categories, then you decide which courses hold a 5 in which category.

Hypothetically we'll say:

Mr Roger's class takes the 5 in capability.
Mr Jones is 5 in RWE.
Mr Tom in TA
Mr Al in time management
And Mr Konner in Cost
(all other categories could be whatever you like)

A class by Mr Rich comes along and a few guys go to it. They say it was better in capability than Mr Roger's, but not as good in TA as Mr Tom, significantly worse than Mr Jones in RWE, and it was more expensive than Mr Konner's. Time management seemed just as good as Mr Al at times management.

So at this point Mr. Rich's class gets the following :

Capability 6
RWE 3
TA 4
TM 5
Cost 4
For an average of 4.4

I'm fairly certain I could write up a spreadsheet that would automatically change values of all instructors based on input values.



Sent from my SM-G930V using Tapatalk
 
#18
Instead of a spreadsheet maybe having a sub-forum for instructor aar's and in the sub each instructor has their own sub, it'll get populated as time goes on and people post up. That way people have a sub to go to for each instructor and they can see all the aar's/scores for each instructor?

Having a spreadsheet sounds tedious and without someone constantly working with it the concept may be fail down the road. This way the users can just post to a sub and if none exists they post to the main instructor sub and a mod/admin can start a sub for them.

I'll work on a mini AAR template - how mini are we talking about though? 5 categories? With a rating system at the bottom? Rating system integrated into the categories?
 
#20
Instead of a spreadsheet maybe having a sub-forum for instructor aar's and in the sub each instructor has their own sub, it'll get populated as time goes on and people post up. That way people have a sub to go to for each instructor and they can see all the aar's/scores for each instructor?

Having a spreadsheet sounds tedious and without someone constantly working with it the concept may be fail down the road. This way the users can just post to a sub and if none exists they post to the main instructor sub and a mod/admin can start a sub for them.

I'll work on a mini AAR template - how mini are we talking about though? 5 categories? With a rating system at the bottom? Rating system integrated into the categories?
Yeah, the spreadsheet would require a level of maintenance by just class reviews but that's really it, but it wouldn't be tedious at all. I'd automate it and run it as long as I can and teach others to run it so it can function in my absence. It's actually very simple that just about anyone who notes how to press buttons on a keyboard could run it flawlessly with almost zero training. If ya'll would like, I could just write up a demo. I could also integrate the standardized options stated above.

Sent from my SM-G930V using Tapatalk