- Joined
- Nov 29, 2013
- Messages
- 149
- Reaction score
- 23
From: https://apetest.com/edu/casper-faqs-regarding-coaching-and-advanced-preparation/
What is the benefit of coaching (advanced preparation) for the CASPer®, Situational Judgment Test (SJT)?
The current accepted research literature suggests that coaching effects are positively large, with an increase of 0.50 standard deviation after coaching and an increase of 0.32 standard deviation for reapplicants that retook their SJT test in the following admission cycle. This recent study might be deemed more relevant as the cohort studied were applying for medical school: although the sample was not randomized, applicants who were coached showed a clear improvement in their scores.
By comparison, the Scholastic Aptitude Test (SAT), and Medical College Admissions Test (MCAT), have a smaller a coaching effect benefit of 0.25 standard deviation.
The larger the standard deviation increase from coaching, the most likely an applicant is to benefit from advanced preparation. Therefore applicants are likely to benefit significantly from coaching (advanced preparation) for CASPer® .
http://onlinelibrary.wiley.com/doi/10.1111/j.1468-2389.2012.00599.x/abstract
http://onlinelibrary.wiley.com/doi/10.1111/j.1744-6570.2005.00713.x/abstract
Standardized Tests Predict Graduate Students' Success
Why does Altus Assessments state that there seems to be no evidence of any practice effects with CASPer®?
Altus Assessments is the company that administers CASPer®. They reported that they “see absolutely no change in CASPer® scores as students progress through the test”. They conclude that there seems to be no evidence of any practice effects with CASPer®. Their findings are based entirely on data collected from the same interval (ie. during the same sitting of the test). The general literature shows that practice effects take time to develop and CASPer® is not immune to these practice effects.
Why does Altus Assessments state a 6-8, month prep time for the MCAT and a 0 hour (no prep time) for CASPer®?
Applicants benefit more from CASPer preparation than MCAT preparation based on the current research. Altus acknowledges the cost of an MCAT prep course at $2,299 and that the test requires up to 8 months of advanced preparation time. Ironically, the research shows that applicants are likely to benefit half as much from coaching (advanced preparation) for tests like the MCAT, compared with coaching (advanced preparation) for tests like CASPer®.
In one study on near peer-assisted learning to improve confidence for medical students' situational judgment tests, forty-four students completed the feedback. There were significant post-course improvements in familiarity with structure, scoring system, knowledge and content, and overall SJT confidence (P < 0.05) with advanced preparation. Delayed feedback showed sustained improvement in familiarity with knowledge and content (P < 0.05). Qualitative analysis revealed themes of improved confidence, approachable tutors, and identifying question strategies.
https://altusassessments.com/2017/10/price-admission-medical-school/
Near peer-assisted learning to improve confidence for medical students' situational judgment tests Gostelow N, Soothill G, Vawda S, Annan D - Educ Health
Do I have to pay for coaching to get all the benefits of advanced preparation for CASPer®?
Concerns about potential unfairness of coaching can be countered through Altus Assessments making effective coaching available to all examinees, in the form of transparent, organizational endorsed coaching. A test design which is not amenable to formal coaching has the appeal of ensuring ‘fairness’ between candidates, especially as the cost of training schemes may distort access to such schemes and medical schools are likely to differ in the level of support they provide. Despite this, the existing evidence suggests that SJTs are well amenable to coaching.
https://onlinelibrary.wiley.com/doi/full/10.1111/medu.12070
I've heard that CASPer® is considered a more complex SJT. Is coaching (advanced preparation) still effective?
The current literature suggests that individuals have explored variations in SJTs designed to be less susceptible to coaching by tailoring their content, complexity and the response formats used and instructions given. Complexity built into scenarios such that the candidate must engage with the question rather than employing a simple response strategy has only been able to reduce, not eliminate coaching effects.
https://onlinelibrary.wiley.com/doi/full/10.1111/medu.12100
Disclaimer: CASPer SIM is a trademark of Advisor Prep Education Services. CASPer® is a registered trademark of Altus Assessments Inc. The CASPer® test is delivered by Altus Assessments at takecasper.com. This website is operated by Advisor Prep Education Services, a provider of test preparation services. Advisor Prep is an independent service provider which has no role in creating or administering the CASPer® test. Altus Assessments and Advisor Prep have no relationship whatsoever, whether by way of affiliation, endorsement, or otherwise.
What is the benefit of coaching (advanced preparation) for the CASPer®, Situational Judgment Test (SJT)?
The current accepted research literature suggests that coaching effects are positively large, with an increase of 0.50 standard deviation after coaching and an increase of 0.32 standard deviation for reapplicants that retook their SJT test in the following admission cycle. This recent study might be deemed more relevant as the cohort studied were applying for medical school: although the sample was not randomized, applicants who were coached showed a clear improvement in their scores.
By comparison, the Scholastic Aptitude Test (SAT), and Medical College Admissions Test (MCAT), have a smaller a coaching effect benefit of 0.25 standard deviation.
The larger the standard deviation increase from coaching, the most likely an applicant is to benefit from advanced preparation. Therefore applicants are likely to benefit significantly from coaching (advanced preparation) for CASPer® .
http://onlinelibrary.wiley.com/doi/10.1111/j.1468-2389.2012.00599.x/abstract
http://onlinelibrary.wiley.com/doi/10.1111/j.1744-6570.2005.00713.x/abstract
Standardized Tests Predict Graduate Students' Success
Why does Altus Assessments state that there seems to be no evidence of any practice effects with CASPer®?
Altus Assessments is the company that administers CASPer®. They reported that they “see absolutely no change in CASPer® scores as students progress through the test”. They conclude that there seems to be no evidence of any practice effects with CASPer®. Their findings are based entirely on data collected from the same interval (ie. during the same sitting of the test). The general literature shows that practice effects take time to develop and CASPer® is not immune to these practice effects.
Why does Altus Assessments state a 6-8, month prep time for the MCAT and a 0 hour (no prep time) for CASPer®?
Applicants benefit more from CASPer preparation than MCAT preparation based on the current research. Altus acknowledges the cost of an MCAT prep course at $2,299 and that the test requires up to 8 months of advanced preparation time. Ironically, the research shows that applicants are likely to benefit half as much from coaching (advanced preparation) for tests like the MCAT, compared with coaching (advanced preparation) for tests like CASPer®.
In one study on near peer-assisted learning to improve confidence for medical students' situational judgment tests, forty-four students completed the feedback. There were significant post-course improvements in familiarity with structure, scoring system, knowledge and content, and overall SJT confidence (P < 0.05) with advanced preparation. Delayed feedback showed sustained improvement in familiarity with knowledge and content (P < 0.05). Qualitative analysis revealed themes of improved confidence, approachable tutors, and identifying question strategies.
https://altusassessments.com/2017/10/price-admission-medical-school/
Near peer-assisted learning to improve confidence for medical students' situational judgment tests Gostelow N, Soothill G, Vawda S, Annan D - Educ Health
Do I have to pay for coaching to get all the benefits of advanced preparation for CASPer®?
Concerns about potential unfairness of coaching can be countered through Altus Assessments making effective coaching available to all examinees, in the form of transparent, organizational endorsed coaching. A test design which is not amenable to formal coaching has the appeal of ensuring ‘fairness’ between candidates, especially as the cost of training schemes may distort access to such schemes and medical schools are likely to differ in the level of support they provide. Despite this, the existing evidence suggests that SJTs are well amenable to coaching.
https://onlinelibrary.wiley.com/doi/full/10.1111/medu.12070
I've heard that CASPer® is considered a more complex SJT. Is coaching (advanced preparation) still effective?
The current literature suggests that individuals have explored variations in SJTs designed to be less susceptible to coaching by tailoring their content, complexity and the response formats used and instructions given. Complexity built into scenarios such that the candidate must engage with the question rather than employing a simple response strategy has only been able to reduce, not eliminate coaching effects.
https://onlinelibrary.wiley.com/doi/full/10.1111/medu.12100
Disclaimer: CASPer SIM is a trademark of Advisor Prep Education Services. CASPer® is a registered trademark of Altus Assessments Inc. The CASPer® test is delivered by Altus Assessments at takecasper.com. This website is operated by Advisor Prep Education Services, a provider of test preparation services. Advisor Prep is an independent service provider which has no role in creating or administering the CASPer® test. Altus Assessments and Advisor Prep have no relationship whatsoever, whether by way of affiliation, endorsement, or otherwise.
