Share on Pinterest

College Success Awards

Methodology for establishing award-winning schools

The College Success Awards recognize and celebrate schools that excel in preparing students for postsecondary success. In 2018, we awarded 814 schools across 9 states and published a report with our findings. In 2019, we awarded 1,722 high schools across 25 states. In 2020, we awarded 2,158 high schools across 29 states with College Success Awards. Here we share the methodology used to determine 2020 award-winning schools.

2020 methodology2019 methodology2018 methodology

2020 College Success Awards Methodology

Background

The objective of the College Success Awards is to recognize and celebrate public high schools that are successfully preparing students to succeed in college. Award winners are determined by a methodology that evaluates school-level data on college preparation, college enrollment, and college performance.

States included

After a national data collection effort, GreatSchools.org collected sufficient data to calculate the award in 29 states: Alabama, Arkansas, Colorado, Connecticut, Florida, Georgia, Hawaii, Idaho, Indiana, Iowa, Kentucky, Louisiana, Massachusetts, Michigan, Minnesota, Mississippi, Missouri, Montana, Nebraska, New Jersey, North Carolina, North Dakota, Ohio, Oklahoma, Tennessee, Texas, Vermont, Virginia, and Washington.

Fourteen of the states where schools are eligible for a 2020 College Success Award provided additional data on measurements of college success for low-income students, which allowed GreatSchools.org to review the schools’ college success efforts with a greater lens on equity. The states providing this additional data for students receiving free and reduced-price lunch are Connecticut, Georgia, Indiana, Iowa, Kentucky, Louisiana, Massachusetts, Michigan, Mississippi, Nebraska, New Jersey, Texas, Virginia, and Washington.

Data included

Award winners are determined by calculating a school-level score and applying a threshold to delineate which schools receive an award and which do not. This school-level score is comprised of three components:

  1. The College Preparation, which includes the percentage of students from a 4-year cohort who graduate from high school as well as measures of College Entrance Exam (ACT or SAT) performance.
  2. The College Enrollment, which includes school-level metrics that vary by state. Some states report this data as “Percent Enrolled In Any Postsecondary Institution Within 12 Months After Graduation,” others as “Percent Enrolled In Any Institution Of Higher Learning In The Last 0-16 Months”
  3. The College Performance, which includes remediation and persistence metrics, which also vary in availability by state. The remediation metrics capture the percentage of students needing remediation for college either for 2-year or 4-year institutions. Measures of college persistence are for the percentage of students returning for a second year at both 2-year and 4-year institutions.

Data limitations

There are three principal limitations to the data:

  1. College enrollment and persistence data do not take into account institution quality, thus college preparation data is included as a proxy for quality.
  2. In some states, data are not disaggregated by student group, so results may not be equally distributed across groups. When disaggregated student group data is provided by states, we calculate college success metrics for students receiving free and reduced-price lunches (FRL student group) and incorporate this data for low-income students into the methodology.
  3. Analysis inputs are “point in time” data, not longitudinal data, so results will not necessarily isolate the value added by the school from any out-of-school factors that may be influencing performance.

Award calculation methodology

Initial scoring

First, we standardize each of the inputs available for a particular school. To do this, we calculate the school’s position in the statewide distribution as a percentile for each metric.

Some states mandate that a specific college entrance exam is taken by all high school graduates. In these states, if data for multiple college entrance exams is obtained, only the data for the mandated exam is used. In states that do not require a specific exam to be taken, and where data for more than one college entrance exam is available, the data for the exam with the higher participation rate statewide is used. If participation rates are not available for either test then we include the data for both in the analysis.

Within each component, we calculate an average of the percentiles of the available metrics, resulting in a score for each of the three categories (College Preparation, College Enrollment, College Performance) for each school. This approach ensures that we give equal weight to the three types of college success metrics in the final school-level score. Eligibility for the award is based on schools having recent data in each of the components.

If a school is missing metrics in one category then that school is not included in the analysis and is not eligible for an award that year.

We calculate a single school-level score by averaging the three component scores and then identify the highest-scoring 30% of schools in each state. Our 2020 College Success Awards methodology incorporates the same equity-focused approach that we introduced in 2019. To ensure that GreatSchools.org’s College Success Awards recognize success through different lenses, including equity, we identify schools that are “beating the odds” by better serving low-income students who might otherwise not be prepared for or attending college. We estimated how well a school would do by exploring the correlation between average school-level scores and student income levels and awarded schools credit for doing better than predicted. For schools outperforming expectations, we added their residuals to their initial school-level score, to generate an adjusted school-level score.

Adjusting qualifications

After calculating school-level scores, we apply several rules to ensure that only schools that are performing well for all students move forward in the award process. First, schools that perform below the weighted average in a metric in their state will not receive the award. Second, schools that perform at or below the 50th percentile in a metric in their state are also disqualified. Third, in the 14 states where low-income student data is made available, schools in the top 25 percent for low-income students statewide (whether or not the school previously qualified) are automatically qualified; schools in the bottom 50 percent for low-income students statewide are disqualified. This step does not apply to schools in states where low-income student performance data are not available. Finally, schools that are performing below expected levels, i.e. those with negative residuals based on the correlation between average scores and low-income student enrollment, are disqualified regardless of their scores or previous qualification.

Final awards

Schools with adjusted scores in the top 30 percent for their state that also meet the other criteria listed above and schools in which low-income students perform better than 75 percent of the low-income students at other schools in the state (where low-income data are made available), are the final College Success Award winners for 2020!

A visual of the 2020 methodology:

A visual of the 2020 methodology


2019 College Success Awards Methodology

Background

The objective of the College Success Awards is to recognize and celebrate public high schools that are successfully preparing students to succeed in college. Award winners are determined by a methodology that evaluates school-level data on college preparation, college enrollment, and college performance.

States included

After a national data collection effort, GreatSchools.org collected sufficient data to calculate the award in 25 states: Arkansas, Colorado, Connecticut, Florida, Georgia, Hawaii, Idaho, Indiana, Kentucky, Louisiana, Massachusetts, Michigan, Minnesota, Mississippi, Missouri, Montana, Nebraska, New Jersey, North Carolina, North Dakota, Ohio, Oklahoma, Texas, Vermont, and Wyoming.

Eleven of the states where schools are eligible for a 2019 College Success Award provided additional data on measurements of college success for low-income students, which allowed GreatSchools.org to review the schools’ college-success efforts with a greater lens on equity. The states providing this additional subgroup data for students receiving free and reduced-price lunch are Connecticut, Georgia, Indiana, Kentucky, Louisiana, Massachusetts, Michigan, Mississippi, Nebraska, New Jersey, and North Dakota. For more information about state-by-state eligibility rates, see Appendix A.

Data included

Award winners are determined by calculating a school-level score and applying a threshold to delineate which schools receive an award and which do not. This school-level score is comprised of three components:

  1. The College Preparation component includes the percent of students from a 4-year cohort who graduate from high school, the SAT or ACT participation rate, and the average performance on the SAT or ACT.
  2. The College Enrollment component component includes school-level metrics that vary by state. Some states report this data as “Percent Enrolled in College Immediately Following High School,” others as “Percent enrolled in any institution of higher learning in the last 0-16 months,” or “Graduating seniors pursuing either a 2-year or 4-year college/university.”
  3. The College Performance component component includes remediation and persistence metrics, which also vary in availability by state. The remediation metric is “Percent of students needing remediation for college” and is sometimes disaggregated by subject. The persistence metric is “Percent enrolled in college and returned for a second year.”

Data limitations

There are three principal limitations to the data:

  1. College enrollment and persistence data do not take into account institution quality, thus college preparation data is included as a proxy for quality.
  2. In some states, data are not disaggregated by subgroup, so results may not be equally distributed across groups. When disaggregated subgroup data is provided by states, we calculate college success metrics for students receiving free and reduced-price lunches (FRL subgroup) and incorporate this data for low-income students into the methodology.
  3. Data are “point in time” data, not longitudinal data, so results will not necessarily isolate the value added by the school from any out-of-school factors that may be influencing performance.

Award calculation methodology

Initial score calculation methods

First, each of the inputs available for a particular school is standardized. To do this, the school’s position in the statewide distribution for each metric is calculated as a percentile.

Some states mandate that a specific college entrance exam is taken by all high school graduates. In these states, if data for multiple college entrance exams is obtained, only the data for the mandated exam is used. In states that do not require a specific exam to be taken, and there is data for more than one college entrance exam available, the data for the exam with the higher participation rate statewide is used. If participation rates are not available, the exam on which the school’s students performed better is used.

Within each component, we calculate an average of the percentiles of the available metrics, resulting in a score for each of the three categories (College Preparation, College Enrollment, College Performance) for each school. This approach ensures that we give equal weight to all three components in the final school-level score. For schools with no available data in one of the metrics, the average across the other metrics is taken. For example, if a school does not have graduation data, then the College Preparation component’s average percentile is based only on College Entrance Exam Performance and Participation data.

The last step in calculating the single school-level score is to calculate the average of the subscores of the three components. To limit the advantage of missing data, schools that do not have data in all three categories are considered ineligible for the award. Once these schools are removed, the eligible schools in each state receive a single school-level score calculated from the three components. We adjust the single school-level score to account for student income levels during the subsequent equity-focused score adjustments.

Equity-focused score adjustments

To ensure that GreatSchools.org’s 2019 College Success Awards recognize success through different lenses, including equity, we revised the 2019 College Success Award methodology to identify schools that are “beating the odds” by better serving low-income students who might otherwise not be prepared for or attending college. Our equity-focused methodology includes: 1) adjusting a school’s single school-level score based on how the school performs relative to expected levels based on their low-income student enrollment to create a final College Success Award score for each school; 2) assigning awards to the top 20 percent of schools based on those scores; and 3) calculating the school-level scores for low-income students in the 11 states where low-income student subgroup data is made available. Schools in the top 25 percent for low-income students statewide (whether or not the school was previously awarded) are included; schools in the bottom 50 percent for low-income students statewide (even if the school was previously awarded) are disqualified.

We adjusted the average College Success Award scores to boost the rankings of schools that are better serving low-income students. We estimated how well a school would do by exploring the relationship between average school-level scores and student income levels and awarded schools credit for doing better than predicted. After calculating the adjusted scores for each school, we awarded the top 20 percent of schools based on this final College Success Award score.

Finally, for states providing low-income student subgroup data, we applied thresholds based on the performance of low-income students at each school. Using the same methods as for initial overall College Success Award scores, we calculated College Success Award scores for low-income students (defined as those enrolled in Free and Reduced-price Lunch programs) in the 11 states that provided subgroup performance data. We adjusted the award assignments in these states based on low-income performance thresholds of 50 percent for low scores and 75 percent for high scores. For schools earning initial awards based on adjusted scores, schools retained the award if low-income subgroup performance is in the top half of scores for low-income students in that state. If the low-income subgroup performs in the bottom half, then the award is removed. For schools not earning awards in the initial steps, if low-income students at a school perform better than 75 percent of the low-income students at other schools in the state, the school earns the final College Success Award. Award assignments are not altered by these methods for schools in states where low-income student subgroup performance data are not available. All schools earning awards after the equity-focused methodology is applied are the final College Success Award winners.

A visual of the methodology

A visual of the 2019 methodology


2018 College Success Awards Methodology

Background

The objective of the College Success Award is to recognize and celebrate high schools that are doing the best at preparing students to succeed in college. Award winners are determined by a methodology that evaluates school-level data on college preparation, college enrollment, and college performance.

States included

After a national data collection effort, GreatSchools collected sufficient data to calculate the award in nine states: Colorado, Florida, Kentucky, Indiana, Georgia, Ohio, Michigan, Arkansas, and Missouri.

Data included

Award winners are determined by calculating a school-level metric and applying a threshold to delineate schools receiving the award and those that are not. This metric is comprised of three components:

  1. The College Preparation component includes the percent of students from a 4 year cohort who graduate high school and the average performance and participation for the SAT or ACT.
  2. The College Enrollment component includes school-level metrics that vary by state. Some states report this data as ‘Percent Enrolled in College Immediately Following High School,’ others as ‘Percent enrolled in any institution of higher learning in the last 0-16 months,’ and ‘Graduating seniors pursuing either a 2-year or 4-year college/university.’
  3. The College Performance component is comprised of remediation and persistence metrics, which also vary in availability by state. The remediation metric is ‘Percent of students needing remediation for college’ and is sometimes disaggregated by subject. Persistence metric is ‘Percent enrolled in college and returned for second year.’

Data limitations

There are three principal limitations with the data:

  1. College enrollment/persistence data do not take into account institution quality, thus college preparation data is included as a proxy for quality.
  2. Data are not disaggregated by subgroup, so results may not be equally distributed across groups.
  3. Data are “point in time” data, not longitudinal data, so results will not necessarily isolate the value-add of the school from out-of-school factors influencing performance.

Methodology

First, each of the inputs available for a particular school is standardized. To do this, the school’s position in the state-wide distribution for each metric is calculated as a percentile.

Some states mandate that a specific College Entrance exam be taken by all high school graduates. In these states, if data for multiple College Entrance exams is obtained, only the data for the mandated exam is used. In states that do not require a specific exam to be taken, and there is data for more than one College Entrance exam available, the data for the exam with the higher participation rates statewide is used. If participation rates are not available, the exam in which the school performed better is used.

Within each component, an average of the percentiles of the available metrics is calculated, resulting in a score in all three categories (College Preparation, College Enrollment, College Performance) for each school. This approach ensures that equal weight is given to all three components in the final for calculating a school’s level metrics. For schools with no available data in one of the metrics, the average across the other metrics is taken. For example, if a school does not have graduation data, then the College Preparation component’s average percentile is based only on College Entrance Exam Performance and Participation data.

The last step in calculating the single school level metric is to average the scores in the three buckets. To limit the advantage of missing data, schools that do not have data in all three categories are considered ineligible for the award. To ensure that only schools that are performing well are identified, schools that perform below the 50th percentile and below the weighted average in a metric in their state are also ineligible. Once these ineligible schools are removed, the top 30% of all schools from each state that are left are the College Success Award winners.

A visual of the methodology

A visual of the 2018 methodology


1 The availability of College Enrollment metrics varies by state.

Appendix A

Appendix A