SES – Random Critical Analysis

No, US school funding is actually somewhat progressive.

June 22, 2017November 17, 2018by rcafdm10 Comments

Note: I have already touched on these issues in a much longer, much broader post on education and on twitter at some length, however I thought it’d be useful to zoom in on this issue and marshal the evidence in one place for future reference.

It is still commonly supposed by much of the public that school funding is terribly unequal due to reliance on local funding mechanisms (especially property tax). Although there were once modest inequalities associated with local income levels (several decades earlier), this information is generally wildly out of date today. Within the vast majority of states districts with less advantaged students (read: higher poverty, lower income, fewer parents with college degrees, minority, etc) actually spend at least as much money per pupil (often more), both overall and in the narrower instructional expenditure category, and where there are inequalities these differences are usually quite modest and fleeting.

Though school funding is still significantly a local affair in most states there is substantial progressive redistribution of state and federal funds that effectively offset these potential inequalities (and then some). Some districts may choose to spend more controlling for income/wealth (tax effort) and there is some variance (mostly poorly explained by any SES measure), so that malcontents can always find isolated examples to complain about, but various formulas employed at the state and federal level sets a floor and effectively acts to prevent there being substantial positive correlations between school spending and district median income (or low poverty rates, percent minority, school free lunch percentage, and so on …. this holds across multiple measures).

No, the SAT doesn’t just “measure income”

November 25, 2015November 17, 2018by rcafdm8 Comments

I started this post to refute some specific arguments, but I changed my mind midstream and decided to add a lot more material than I initially envisioned. This is best viewed as being akin to a FAQ (Frequently Refuted Objections – FRO?) relating to the standardized tests and their use in higher education.

table of contents

1) SAT and income are not perfectly correlated
2) Income correlates well with lots of tests
3) mean GPA, academic rigor, and the significance HSGPA varies with SES
4) Even controlling for SES, different racial groups see different average scores
5) SES & B-W cognitive gaps start very young and keep growing
6) The B-W test score gaps are poorly explained by schools
7) The differences in “school quality” are generally overstated
8) The SAT’s association with family income is almost entirely mediated by family education
8) SES does not significantly mediate the predictive power of the SAT
9) Some have misconstrued the Univ. of Calif. data to argue otherwise
10) Nor are these tests biased against minorities
11) The predictive power of the SAT is vastly under-estimated
12) The strength of this prediction does not weaken past freshman year
13) Holding HSGPA constant the SAT offers significant incremental validity
14) The SAT is well correlated with IQ tests
15) Lumosity’s cognitive tests show strong correlations with SAT across universities
16) SAT test prep has little effect
17) Test prep rates do not vary all that much
18) SATs predict graduation rates between schools too
19) When schools systematically discount standardized tests the effects are very obvious
20) Heritability explains a great deal of these systematic SES relationships

**SAT and income are not perfectly correlated**

The SAT is certainly modestly correlated with parental income, but it is simply not true that the SAT is nothing more than a measure of family income.

I will briefly plot the 2011 SAT reading scores by income level to illustrate that the r**2 is considerably less than one.

There is significant overlap across the entire income distribution:

Box and whisker plot of simulated test scores

Simulated distribution by actual test taker counts

On sex differences within California public schools

June 17, 2015November 17, 2018by rcafdmComments

A year or two ago I read an article which demonstrated that sex differences in math and reading achievement are inversely related within and across countries on the PISA tests. The smaller the male-female math gap, the larger the verbal gap and vice versa. This tends to support the view that there are innate underlying differences in average abilities and interests between the sexes that strongly influence these patterns.

On educational attainment rates and income as a causative factor

June 13, 2015June 14, 2015by rcafdm3 Comments

As I mentioned to Robert VerBruggen in his latest piece on educational attainment and income, I do not believe that economic concerns are a major cause of differences in education attainment rates by income.

I previously analyzed this and related issues using the ELS:2002 data, but I decided to extend my analysis with NLSY97 and clarify my views, now that I have marshaled a fair amount of data to support my arguments.

First

Although the IQ (ASVAB) is an excellent predictor and generally mediates these differences fairly well, there are other systematic differences that are not fully accounted for when you control for IQ. High SES people, whether measured by income or educational attainment, typically have higher GPAs even with the same test scores.

(these differences would likely to be larger still if I did this as a composite SES index using education, income, occupational prestige, etc)

IQ, test scores, GPA, income, and related correlations from NLSY97

June 11, 2015June 13, 2015by rcafdm1 Comment

Diving into the NLSY97 dataset a bit right now and I thought I’d share some plots pertaining to IQ/achievement tests, income, GPA, educational attainment, and more. Nothing here is particularly novel, if you’ve looked for this sort of data before, but sometimes it’s nice to have additional independent analysis or alternative presentations of the data.

High school GPA-test score relationships

HS GPA by ASVAB in percentiles (IQ test)

HS GPA by PIAT (Peabody Individual Achievement Test, IQ test admin. in early childhood)

Understanding socioeconomic mobility

June 7, 2015June 11, 2015by rcafdm4 Comments

Although it may not seem like it at first blush, given the apparently modest correlations, the socioeconomic figures that I blogged about earlier largely agree with the published data on economic mobility.

They are measuring income (or earnings) whereas I am measuring with the ELS SES index (which is an equally weighted average of the respondents own earnings as of 2011, educational attainment, and occupational prestige), but the systematic income differences between classes (however measured) are almost certainly virtually fully mediated by this more comprehensive SES measure.

ELS data, students 2011 SES by 2002 SES of parents

The correlation between parent SES and child SES is 0.35. This may not sound like much, but if you bin child SES and parent SES into quintiles the mobility estimates are very similar to the widely publicized economic mobility estimates.

The average person born at the top of the SES distribution has little chance of ending up at the bottom and vice versa, but there is clearly a great deal of mobility that happens amongst less extreme groupings.

[Note: I didn’t correct these figures for oversampling, so I won’t claim they’re a perfect representation of reality, but they are generally pretty close in practice and they still are good for illustrative purposes.]

Mobility delta (child SES – parent SES) by parent SES

On average, as compared to their parents, high SES people are downward mobile and low SES people are upward mobile. This may seem counter-intuitive to some, but this makes sense because r < 1, i.e., there is non-trivial mobility, and this must be true for there to be meaningful relative mobility. Of course, just because there is mobility doesn’t imply that all people have an equal chance of ending up in any place in the SES distribution.

As I mentioned in my last post, there are other differences between groups besides just propensity to end up in particular SES bins and many of these differences are highly predictive of mobility — see test scores, HS GPA, etc. Indeed, they almost fully mediate outcomes, so talking in terms of “mobility” here can be a bit misleading because relative starting position (parent SES) tells you relatively little about what is likely once you have better information (e.g., test scores, HS GPA, etc).

Predicting economic mobility from 10th grade test scores

June 5, 2015June 6, 2015by rcafdm3 Comments

In my last post I briefly touched upon economic mobility vis-a-vis the link between test scores and subsequent adult incomes. Because these individuals were still pretty young, just a few years out of college (if they graduated), the earnings correlations were weaker than one might have expected. Since then I discovered an interesting continuous SES variable (F3SES) in the ELS:2002 data set that is probably a better measure of future earnings or mobility.

F3SES is the average of 3 inputs (2011 earnings from employment, the prestige score associated with the respondents current/most recent job, and educational attainment), each of which is standardized to a mean of 0 and a standard deviation of 1 prior to averaging.

Data users should note that, as of the third follow-up, socioeconomic status may be less-than fully stable for some third follow-up respondents, e.g., respondents with graduate-level education who are just beginning or have yet to begin their careers. Users should also note that F3SES does not account for the income, occupation, or education of the respondents spouse/partner, and therefore may not be fully indicative of household socioeconomic status as of the third follow-up.

NOTE: While the two versions of the BY family SES composites (BYSES1 and BYSES2) were created by differential assignment of prestige scores based on the 16-category BY occupation variables, F3SES is created by assigning prestige scores based on the 2-digit ONET code associated with the respondents current/most recent job as of the third follow-up.

While I am sure I could derive my own formula to produce a similar composite score, I’ll just use theirs for the time being.

There is no statistically significance difference between blacks and whites here.

Asian SES is higher than white SES for most of the distribution, but that’s not statistically significant either.

Exploring ELS 2002 data

June 4, 2015June 6, 2015by rcafdm1 Comment

The NYTimes turned me onto a new data source in a recent article on college graduation rates by SES. They implied that college graduation rates are better predicted by “wealth” than by the students test scores (10th grade ELS scores taken in 2002).

Being both curious about the underlying data and somewhat skeptical of the particular claims (or, at least, its interpretation) I decided to investigate it for myself. Having done so now, I can tell you that it’s a pretty rich data set. Unfortunately, a few key data points (e.g., SAT scores, ~~HS GPA~~, etc) are censored or rounded/binned to protect anonymity, but there are still a lot of interesting data there to analyze.

Update (6/6/15):

As in my follow-up post on economic mobility, I realized that they actually provided 9th-12th grade high school GPA as a non-continuous variable in the publicly accessible file. I have updated my post to reflect this new information in a few places!

First point

The parent’s educational attainment is a much better predictor of both test scores and subsequent child educational attainment than economic measures…..

(Bachelor degree) Attainment rate by test scores, grouped by parent income levels

Some additional plots of within school achievement gaps

May 30, 2015May 30, 2015by rcafdm6 Comments

In a prior post I compared the performance of various ethnic groups directly against each other within the same schools. That method has the advantage of reducing the effect of racial/ethnic composition from influencing the apparent relationship between each groups performance within school environments. However, there aren’t always enough of both groups to clear the minimum reporting requirements for student privacy and the school average is a better indicator of “school quality” in some peoples’ minds. Below I simply took the reported average for all students on the x-axis so that I could compare multiple groups at once.

I do not believe that this is a particularly good way to view the data, since it’s confounded by racial/ethnic composition and sorting by education levels (despite the obvious correlation), but if you do happen to think that the average score is a particularly good measure of “school quality” this approach might be eye opening.

I also plotted this same data for poor and non-poor blacks and whites and (all race/ethnic) scores by parental education level below. You’ll need to scroll down a ways to see it though.

English

English Grade 2

English Grade 4

English Grade 6

Small update to my prior post on concentrated poverty

May 19, 2015May 20, 2015by rcafdm4 Comments

This is just quick update to my prior post on concentrated poverty. I re-ran the California test score data at a school-level to compare within school black-white differences in test scores and converted all of the scores data to standard deviation units relative to (above) the non-hispanic mean by school, weighted by the number of test takers. The pattern can be observed as early as 2nd grade and it is quite consistent for all major/mandatory tests.

Black vs White within school comparison

Grade 2 English

Grade 2 Math