Instructors endorsement and rating scheme

#21
I might be missing something but ain't there already a class AAR section?

In my mind this is strictly an instructor/company rating. Much like a BBB rating or anything else.

I like what y'all are saying I just think the application does not apply to this as much as it would a section that already exists.

This is not an instructor AAR. It's a rating. I'm not saying the rating is concrete. I believe that over time a rating could go up or down. however, this needs to be consistent. I know when I put my name to something I want it to be right. So if I say hey this dude is a X Rating it should mean something based on a procedure we establish. My background is what it is so obviously on some training I would have to reference someone else. I stay in my lane.
 
#22
So here's what I'll do. I'll write up my spreadsheet at some point in the next week. I'll include voodoo's method in my spreadsheet as well as my own. Unless I'm not tracking an existing thread, I think it will be inherently necessary to start another thread for getting reviews from people who have already been to whichever courses.

Sent from my SM-G930V using Tapatalk
 

ggammell

Established
Network Support I
#23
I might be missing something but ain't there already a class AAR section?

In my mind this is strictly an instructor/company rating. Much like a BBB rating or anything else.

I like what y'all are saying I just think the application does not apply to this as much as it would a section that already exists.

This is not an instructor AAR. It's a rating. I'm not saying the rating is concrete. I believe that over time a rating could go up or down. however, this needs to be consistent. I know when I put my name to something I want it to be right. So if I say hey this dude is a X Rating it should mean something based on a procedure we establish. My background is what it is so obviously on some training I would have to reference someone else. I stay in my lane.
I couldn’t help but notice that FTA, despite having no information on their website, already has an A+
BBB rating...
 
#24
As I said yesterday on FB, the scoring areas are not necessarily evenly weighted. In particular, if an instructor can deliver solid information and tactics to me, do I care how many doors he or she has kicked in? Consider for a moment someone who trained directly under and has been blessed by an instructor that we deem to have the requisite RWE, but have limited or no experience of that training. If they can deliver the exact same content to me as effectively as the door kicker, why are they less valuable?

Take two otherwise identical ratings:

Capability - 4
Real World Experience - 3
Teaching Ability - 5
Time Management - 4
Cost - 3
3.8 Average

Capability - 4
Real World Experience - 0
Teaching Ability - 5
Time Management - 4
Cost - 3
3.2 Average

Obviously, there could be larger or smaller disparities.

I believe the scores should be weighted, capability and teaching ability having the most weight, followed by RWE, cost, and time management. The cost/time aspect is interesting because the two come together to provide a value statement based on the other three, but when looked at individually, they don't necessarily provide an accurate picture.

An instructor here on the east coast could have extremely low operating costs and be compared to someone on the west coast with extremely high operating costs. Now, if you have the time, money, and inclination to fly or drive all over the U.S., then maybe saving a few bucks on your course makes sense, but if you are trying to take a two-day and have it within driving distance, then you are stuck choosing between instructors with artificially low ratings. The same two instructors above could have a spread of more than one point, depending on cost and experience, despite being able to deliver the same content.

Time management becomes an interesting evaluation criteria as well. Do we score it separately or as an aspect of the cost? It's an intersection of the cost of the course and the value of the trainee's time. If a course costs $100/day but takes five days to deliver the same content as a $400/day course can deliver in two, which one is more valuable?
 
#25
As I said yesterday on FB, the scoring areas are not necessarily evenly weighted. In particular, if an instructor can deliver solid information and tactics to me, do I care how many doors he or she has kicked in? Consider for a moment someone who trained directly under and has been blessed by an instructor that we deem to have the requisite RWE, but have limited or no experience of that training. If they can deliver the exact same content to me as effectively as the door kicker, why are they less valuable?

Take two otherwise identical ratings:

Capability - 4
Real World Experience - 3
Teaching Ability - 5
Time Management - 4
Cost - 3
3.8 Average

Capability - 4
Real World Experience - 0
Teaching Ability - 5
Time Management - 4
Cost - 3
3.2 Average

Obviously, there could be larger or smaller disparities.

I believe the scores should be weighted, capability and teaching ability having the most weight, followed by RWE, cost, and time management. The cost/time aspect is interesting because the two come together to provide a value statement based on the other three, but when looked at individually, they don't necessarily provide an accurate picture.

An instructor here on the east coast could have extremely low operating costs and be compared to someone on the west coast with extremely high operating costs. Now, if you have the time, money, and inclination to fly or drive all over the U.S., then maybe saving a few bucks on your course makes sense, but if you are trying to take a two-day and have it within driving distance, then you are stuck choosing between instructors with artificially low ratings. The same two instructors above could have a spread of more than one point, depending on cost and experience, despite being able to deliver the same content.

Time management becomes an interesting evaluation criteria as well. Do we score it separately or as an aspect of the cost? It's an intersection of the cost of the course and the value of the trainee's time. If a course costs $100/day but takes five days to deliver the same content as a $400/day course can deliver in two, which one is more valuable?
I don't think that the idea of averaging the scores is to dictate so it's necessarily better but more who MAY provide more bang for your buck. I don't think that the categories need to be weighted, mainly because anyone who is actually searching intelligently for instruction will decide personally the which of the categories mean more to them than the others.

I totally agree that teaching ability essentially trumps rwe. Instruction requires instruction.

Sent from my SM-G930V using Tapatalk
 
#26
Sorry for the double post. I haven't been on forums in a while and it seems I can't edit previous posts.

I think it is critical to define the criteria for each category. We have each of these categories and I think we all generally understand what they mean, but can somebody absolutely define them so that we can all be on the same page.

Sent from my SM-G930V using Tapatalk
 
#27
Isn't time management a kinda, sorta a subcomponent of teaching ability? A teacher could be great at conveying a point or making things click, but if he's disorganized, ineffecient and spends half his time telling war stories or looking at his phone, then it would be reflected in teaching ability.

Just a thought.
-------------

This whole thing will never get off the ground (nor will the FTA.)

1) If a person cares enough to discover a rating body exists and can look up the instructor rating, he's probably already online and reading reviews and doesn't need to be told that Voda is a joke and Yeager is questionable.

2) Who's paying for the electoral college to fly around the country and review classes? (Secretly, I presume, restaurant-critic stye as that's the only way to get a true feel for the class.)

3) Why will the electoral college be any better than the "5 Founders"? They will naturally have their own biases, regardless of how they are selected. P&S has it's one little clique. If you make this a P&S thing, forget about it. YOu'll have to get a wide, wide cross-section of the firearms training community to reduce the bias and

Most importantly...

4) Where's the demand? The public is not clamoring for a rating or reviewing body. The only people I ever see talking about the need for a rating system or instructor accrediting body are those that are already educated about instructors but are annoyed that some people choose to train with 'inferior' instructors. So the people that want this are the people that don't need this. The people need it are those that don't know that they need it and don't care.
 
#28
I second this method. I like the subcategories being averaged into an over all rating. the only thing I would add is that we clearly define what is being looked for in each category and examples of proper scores.

It reminds me of how we evaluate rookies while in FTO. every category is scored and all FTO's are on the same page as what is expected and how we grade. (or as much as we can be all having different personalities and pet peeves.)

it also helps having the reviewer background posted that way you know where he is coming from. I will look at different topics in a different light and it will be nice to get perspective from different personalities.
I woudn't average them. Let each category stand on it's own. As Savage noted, it has to do with real world experience. (I assume that relates to military/LEO experience.)

If I take a Rob Leatham class, I'm there to learn to shoot fast and accurately. So he doesn't have "real world experience" as you define it. It doesn't matter. So why average the categories and bring his score down since no one goes to Rob to learn tactics? Just note he doesn't have a tactical background and mark him zero for "RWE" and leave it at that.
 
#29
Isn't time management a kinda, sorta a subcomponent of teaching ability? A teacher could be great at conveying a point or making things click, but if he's disorganized, ineffecient and spends half his time telling war stories or looking at his phone, then it would be reflected in teaching ability.

Just a thought.
-------------

This whole thing will never get off the ground (nor will the FTA.)

1) If a person cares enough to discover a rating body exists and can look up the instructor rating, he's probably already online and reading reviews and doesn't need to be told that Voda is a joke and Yeager is questionable.

2) Who's paying for the electoral college to fly around the country and review classes? (Secretly, I presume, restaurant-critic stye as that's the only way to get a true feel for the class.)

3) Why will the electoral college be any better than the "5 Founders"? They will naturally have their own biases, regardless of how they are selected. P&S has it's one little clique. If you make this a P&S thing, forget about it. YOu'll have to get a wide, wide cross-section of the firearms training community to reduce the bias and

Most importantly...

4) Where's the demand? The public is not clamoring for a rating or reviewing body. The only people I ever see talking about the need for a rating system or instructor accrediting body are those that are already educated about instructors but are annoyed that some people choose to train with 'inferior' instructors. So the people that want this are the people that don't need this. The people need it are those that don't know that they need it and don't care.
I'm new to p&s, I agree with your points, but to add an anecdote: I'm here and want to make this work because I'm the guy that will be using it. There are a lot of guys like me that desperately want to go to courses but because of all the tactical derp fog out there it's difficult to commit because we're worried about wasting money or worse: absorbing bullshit. I'm not finished with the military either so I'm also trying to stay honing skills.

I have the frame of reference from working as a weapons instructor while in the marine corps so I did a lot of study and analysis on how to instruct well, so I can usually identify bullshit, but many can't.

Sent from my SM-G930V using Tapatalk
 
#30
I woudn't average them. Let each category stand on it's own. As Savage noted, it has to do with real world experience. (I assume that relates to military/LEO experience.)

If I take a Rob Leatham class, I'm there to learn to shoot fast and accurately. So he doesn't have "real world experience" as you define it. It doesn't matter. So why average the categories and bring his score down since no one goes to Rob to learn tactics? Just note he doesn't have a tactical background and mark him zero for "RWE" and leave it at that.
Maybe I didn't convey it well. It's a real world experience for what is being taught. Rob absolutely has the RWE for a shooting class. So that would be high marks.

However if you wanna go learn high risk vehicle take downs, patrol counter ambush, etc.... Then I would go to a guy that has some RWE performing those tasks.

I'm not saying hey this guy ain't a super cool looking big city swat cop so don't go. Or hey he's not CAG so don't go.

That one particular grading area to me is just how said instructor experiences apply to what is being taught.
 
#31
I don't think that the idea of averaging the scores is to dictate so it's necessarily better but more who MAY provide more bang for your buck. I don't think that the categories need to be weighted, mainly because anyone who is actually searching intelligently for instruction will decide personally the which of the categories mean more to them than the others.

I totally agree that teaching ability essentially trumps rwe. Instruction requires instruction.

Sent from my SM-G930V using Tapatalk
The problem I have with it is that we are also talking about an overall score which gets someone recommended or not recommended. So, if one guy has a 2.6 he's barely recommended and maybe has lower actual ability than a gal who has a 2.4 because she wasn't running a PSD for 10 years prior, but she's actually better at delivering material.

I do agree that certain tasks are better learned from someone who has some real world experience in them. I don't want to take a breaching class from someone who has only ever breached the shoot house. This brings up another interesting question, which is if there's an overall rating, how does one weight RWE at a school that teaches multiple skills? Or any of the other aspects, for that matter. Do we have a rating for each class category?
 
#32
I'm not entirely sold over the overall score concept either for this very reason. I can include it, plus your weighted catagorization, plus my rating system all separately in the spreadsheet so that we can compare.

Sent from my SM-G930V using Tapatalk
 
#33
Maybe I didn't convey it well. It's a real world experience for what is being taught. Rob absolutely has the RWE for a shooting class. So that would be high marks.

However if you wanna go learn high risk vehicle take downs, patrol counter ambush, etc.... Then I would go to a guy that has some RWE performing those tasks.

I'm not saying hey this guy ain't a super cool looking big city swat cop so don't go. Or hey he's not CAG so don't go.

That one particular grading area to me is just how said instructor experiences apply to what is being taught.
I will admit that I was reading this as someone who would have RWE as a cool guy, cop, etc. So, that was a bit of confusion on my part. I still think, however, that the ability to teach the concepts and get the student prepared is more important to me that RWE.

I also think it's a lot harder to vet. Some instructors we know have BTDT, others use their prior service as a shield and without a robust mechanism for checking the veracity it gets a little murky.
 
#34
I disagree that teaching ability is more important than rwe. I think that you can compensate for lack of rwe with teaching ability and somewhat do the same conversely. So they are very close to identical in importance to me. I don't really take well too some guy teaching stuff he hasn't practiced with good effect in the field. I've done different martial arts for a number of years and I see people who are good teachers get stumped on the concepts of techniques because of their lack of experience with fighting.

Like stated earlier. If I'm learning XB at a cqb course. I don't want to be taught where to stand from a guy who's never blown a door or entered buildings in the real world. I'm a very inquisitive person and I like to ask a ton of questions. If I ask a particularly digging scenario question about a ttp, I really don't respond well to

"I see what you're worry is, conceptually, if you do x and y you should be fine because of z reasons."

I respond way better to things like:

"I see what you're worry is, we conducted x in Iraq numerous times and built upon it, y became sop because of z reasons. You should be fine."

Sent from my SM-G930V using Tapatalk
 
#35
I disagree that teaching ability is more important than rwe. I think that you can compensate for lack of rwe with teaching ability and somewhat do the same conversely. So they are very close to identical in importance to me. I don't really take well too some guy teaching stuff he hasn't practiced with good effect in the field. I've done different martial arts for a number of years and I see people who are good teachers get stumped on the concepts of techniques because of their lack of experience with fighting.

Like stated earlier. If I'm learning XB at a cqb course. I don't want to be taught where to stand from a guy who's never blown a door or entered buildings in the real world. I'm a very inquisitive person and I like to ask a ton of questions. If I ask a particularly digging scenario question about a ttp, I really don't respond well to

"I see what you're worry is, conceptually, if you do x and y you should be fine because of z reasons."

I respond way better to things like:

"I see what you're worry is, we conducted x in Iraq numerous times and built upon it, y became sop because of z reasons. You should be fine."

Sent from my SM-G930V using Tapatalk
I think the martial arts analogy is a very good one, because we see in martial arts and in sports many times where coaches are not top players, but produce top players. Being able to produce an elite athlete is not the same skill as becoming one. Fud is fud, there are "martial artists" who don't have any understanding of application. There are also martial artists who have an internal understanding but cannot externalize it.
 
#36
I disagree that teaching ability is more important than rwe. I think that you can compensate for lack of rwe with teaching ability and somewhat do the same conversely. So they are very close to identical in importance to me. I don't really take well too some guy teaching stuff he hasn't practiced with good effect in the field. I've done different martial arts for a number of years and I see people who are good teachers get stumped on the concepts of techniques because of their lack of experience with fighting.

Like stated earlier. If I'm learning XB at a cqb course. I don't want to be taught where to stand from a guy who's never blown a door or entered buildings in the real world. I'm a very inquisitive person and I like to ask a ton of questions. If I ask a particularly digging scenario question about a ttp, I really don't respond well to

"I see what you're worry is, conceptually, if you do x and y you should be fine because of z reasons."

I respond way better to things like:

"I see what you're worry is, we conducted x in Iraq numerous times and built upon it, y became sop because of z reasons. You should be fine."

Sent from my SM-G930V using Tapatalk
I think the martial arts analogy is a very good one, because we see in martial arts and in sports many times where coaches are not top players, but produce top players. Being able to produce an elite athlete is not the same skill as becoming one. Fud is fud, there are "martial artists" who don't have any understanding of application. There are also martial artists who have an internal understanding but cannot externalize it.
This is why I feel that teaching is not greater weight than rwe and vice versa.

I would like it if your presented your weight ratios regarding the categories so I can include them in the spread sheet.

Sent from my SM-G930V using Tapatalk
 
#37
This is why I feel that teaching is not greater weight than rwe and vice versa.

I would like it if your presented your weight ratios regarding the categories so I can include them in the spread sheet.

Sent from my SM-G930V using Tapatalk
Dan, I'm thinking this through and may back off if we are rating instruction and "value" separately. When I think of Capability, Teaching Ability and RWE averaged in one rating, the Capability and Teaching Ability combined already outweigh the RWE. As do any two of the three, so it's a nice balance. I do think the value should be separated though, as it will change regionally and with goals.
 
#38
Sounds good. We should definitely define what each category actuality means though. I'll define to the best of my understanding each.

Capability: an instructor's resources such as location and training equipment.

Real World Experience: instructor's experience applying the techniques and skills he is teaching in real world situations.

Teaching ability: instructors ability to positively convey the techniques and skills he is intending to teach.

Time management: instructor's ability to convey the entirety of the course, conduct practical application, and properly conduct student critique in the time allotted by the course.

Are we all on the same page?

Sent from my SM-G930V using Tapatalk
 
#39
My understanding on Capability was more of the instructor's ability to actually perform the tasks. IE an instructor who shoots their own course to show you how it's done, instead of someone who teaches classroom skills without ever demonstrating their own ability to perform. I think your definition is more a facilities ranking, although we know a lot of instructors travel and rely on whatever facilities are available.
 
#40
My understanding on Capability was more of the instructor's ability to actually perform the tasks. IE an instructor who shoots their own course to show you how it's done, instead of someone who teaches classroom skills without ever demonstrating their own ability to perform. I think your definition is more a facilities ranking, although we know a lot of instructors travel and rely on whatever facilities are available.
Okay cool, yeah. This is why I wanted to define this is because I want completely tracking on that particular category and figured there may be confusion elsewhere for others.

Capability: instructor's ability to apply the skills and the techniques they are teaching effectively during the class.


I think we may want to have a "training aids" category for this system as well. I'm imagining a TCCC style course without having wound packing simulation equipment or an analog for needle decompression.

Sent from my SM-G930V using Tapatalk