Research Methods has always been one of my favorite parts of psychology; whether you want to be a researcher or not, knowing how research works can be the difference between really understanding your field, or believing every headline that shows up in your newsfeed. I'd like to take a moment to discuss some research issues that, in my opinion, drive one of the biggest challenges in autism research today. Unfortunately, to make my point, I'll need to review some basic research concepts, so if you're already familiar then feel free to scroll on down (I recommend jumping back in around the bold "Let me give an example").
Not too long ago, I saw a talk by a researcher who wanted to know whether autistic individuals predict/interpret the perspectives of others differently than neurotypicals. They gave autistic and neurotypical children and adolescents very limited information about a fictional character (something like, "This is Sally. Her favorite color is purple.") and asked them how much they thought the fictional character liked certain food groups ("On a scale from 1 to 10, how much do you think Sally likes bananas?"). Then they asked how much the individual answering the questions liked that same food group ("On a scale from 1 to 10, how much do you like bananas?"). Then they used statistics to see whether autistic children were more likely than the neurotypical children to predict that the fictional characters were similar to themselves (or, were autistic children more likely to say that Sally liked bananas the same amount that they did?). This turned out to be the case, and they concluded that autistic children approach "theory of mind" (a construct related to empathy) by thinking about how they would feel in the other person's shoes, while neurotypical children are more likely to use what they know of other people to guide their guesses.
A Review of Research Methods 101
Empirical Research is research that is based on data; a researcher collects information, making efforts to maximize both precision and accuracy in their measurement, and then looks for relationships in the information they’ve collected. Generally, a researcher goes in with two or more variables (constructs that can occur in different ways, or vary, from individual/situation to individual/situation) in mind: an independent variable (the thing they think is driving a change, or the “cause”), and the dependent variable (the thing they think is changing, or the “effect”).
In most cases, "cause" and "effect" are theoretical only; there are specific criteria for actually being able to draw conclusions about causality. In order to determine causality, the researcher must demonstrate covariance (that the two variables vary together -- as one changes, the other also changes), temporal precedence (that the causal variable changes before the effect occurs), and internal validity (that there isn't another variable driving the effect). Only a true experiment can address all three factors sufficiently to infer causality: in an experiment, the researcher recruits a sample of participants that are believed to be from the same population (the group of people the researcher wants to draw conclusions about), randomly assigns them to different "levels" of the independent variable (for example, assigning half of participants to receive a new medication, while the other half receive a placebo; or assigning half of participants to exercise for 45 minutes 4 times a week, while the other half of participants does not exercise and serves as a comparison group), exposes them to their respective levels of the independent variable, then measures the dependent variable.
Because all participants are believed to have been comparable before being assigned to different levels of the independent variable, the researcher believes that the only difference between the groups is their experience of the independent variable. Therefore, if the two variables do covary, then temporal precedence has been established: the groups were the same prior to the researcher introducing the independent variable, and the groups are different after introduction of the independent variable. Some researchers will also use a pre-test and post-test to ensure the two groups were really comparable at the start of the experiment, but it is largely accepted to assume that because the participants were randomly assigned to their groups, the “error” (individual differences within the population) will be equally distributed across groups. (To put it another way, if you "randomly assign" people to group A or group B, you expect groups A and B to have similar average heights; you don't expect all the tall people to end up in group A and the short people to end up in group B.) This assumption is safe at large enough sample sizes because error, if it is random, will balance itself out if there is sufficient data (if you have 4 people, the average height may be pretty different between groups, but if you have 1,000 people, the heights should balance out to be about the same). And, because the researcher manipulated the independent variable but otherwise left the two groups the same, experiments are believed to have high internal validity, because no other variables were introduced differently for one group than for another (or at least, that's the goal).
Here's the problem: you can't randomly assign some participants to be autistic. If you want to draw conclusions about autistic individuals compared to neurotypical individuals, you're going to have a problem because being autistic changes so much about your life experiences, perception, communication, and... well, pretty much everything. Without random assignment you can't do a true experiment; and lack of control over the independent variable eliminates internal validity for your study.
This means that you cannot conduct a true experiment to compare autistics to non-autistics. You can look for correlations, and you can even conduct a quasi-experimental study (a study that mimics an experiment but lacks random assignment -- so you still manipulate some variable, but who is in which group is determined by other factors, like neurotype), but by definition you're going to lose some internal validity, because you don't know whether autism is really what's driving any differences you do find.
Let me give an example:
Not too long ago, I saw a talk by a researcher who wanted to know whether autistic individuals predict/interpret the perspectives of others differently than neurotypicals. They gave autistic and neurotypical children and adolescents very limited information about a fictional character (something like, "This is Sally. Her favorite color is purple.") and asked them how much they thought the fictional character liked certain food groups ("On a scale from 1 to 10, how much do you think Sally likes bananas?"). Then they asked how much the individual answering the questions liked that same food group ("On a scale from 1 to 10, how much do you like bananas?"). Then they used statistics to see whether autistic children were more likely than the neurotypical children to predict that the fictional characters were similar to themselves (or, were autistic children more likely to say that Sally liked bananas the same amount that they did?). This turned out to be the case, and they concluded that autistic children approach "theory of mind" (a construct related to empathy) by thinking about how they would feel in the other person's shoes, while neurotypical children are more likely to use what they know of other people to guide their guesses.
This might sound really interesting -- and it could be! If it's true, then that could help develop interventions for autistic children with social anxiety or social difficulties, by allowing therapists to understand how the autistic child is interpreting their social environment! But here's the thing: there were other variables that the study didn't account for, that could be driving the relationship:
For one thing, autistic individuals tend to think more literally about questions; when I asked how the researchers had accounted for frustration on the part of the autistic participants, they had no response except to acknowledge that the autistic children had voiced uncertainty about how the researcher wanted them to draw these conclusions (is Sally's favorite color supposed to tell me something about whether she likes bananas? How am I supposed to know how much she likes bananas if you haven't told me??). Personally, if I don't know how to answer a question, I give up and hope I can answer the next one; and in a case like this, I might simply default to just saying how much I like bananas because I don't have any way to actually know what Sally likes.
And that's before factoring in demand characteristics: if people think they know what you're looking for, they'll (often subconsciously) adjust their responses to give you what they think you should see. So for an autistic child who has been told they're bad at empathy, they might deliberately give more similar answers for both Sally and themselves because they want you to see that they can relate to Sally! They might think you're testing whether or not they can empathize with the fictional character, and they want you to know they do feel empathy, so they deliberately give responses similar to their own preferences in order to show that they know other people also have feelings and can have things in common with them.
Even more than that though, the study hadn't accounted for whether the autistic children even knew what their peers liked. The whole idea behind this paradigm was that some children would respond that Sally likes what their friends like, while other children would have difficulty realizing other children might like different things than they do, and thus "empathy" would be reflected by guessing that Sally is more similar to the average child rather than more similar to themselves. But what if the autistic children have had less opportunity to learn what it is that the average child likes?
For one thing, autistic individuals tend to think more literally about questions; when I asked how the researchers had accounted for frustration on the part of the autistic participants, they had no response except to acknowledge that the autistic children had voiced uncertainty about how the researcher wanted them to draw these conclusions (is Sally's favorite color supposed to tell me something about whether she likes bananas? How am I supposed to know how much she likes bananas if you haven't told me??). Personally, if I don't know how to answer a question, I give up and hope I can answer the next one; and in a case like this, I might simply default to just saying how much I like bananas because I don't have any way to actually know what Sally likes.
And that's before factoring in demand characteristics: if people think they know what you're looking for, they'll (often subconsciously) adjust their responses to give you what they think you should see. So for an autistic child who has been told they're bad at empathy, they might deliberately give more similar answers for both Sally and themselves because they want you to see that they can relate to Sally! They might think you're testing whether or not they can empathize with the fictional character, and they want you to know they do feel empathy, so they deliberately give responses similar to their own preferences in order to show that they know other people also have feelings and can have things in common with them.
Even more than that though, the study hadn't accounted for whether the autistic children even knew what their peers liked. The whole idea behind this paradigm was that some children would respond that Sally likes what their friends like, while other children would have difficulty realizing other children might like different things than they do, and thus "empathy" would be reflected by guessing that Sally is more similar to the average child rather than more similar to themselves. But what if the autistic children have had less opportunity to learn what it is that the average child likes?
Autistic children are more likely to be excluded, more likely to experience separate learning environments, and if they are with a large group of peers, they're more likely to be overstimulated, reducing their ability to process the information from their peers. Too often, if you place an autistic child in a school cafeteria, they'll be completely excluded by their peers; and even if their peers try to include them, they'll be more overstimulated, and (I would argue) less likely to notice what their classmates like and dislike eating.
So while it's possible that autistic children center themselves in trying to understand what others are feeling, it's also possible that they just haven't had as much exposure to the preferences of their peers. And with the information that we have, we just can't know.
That's poor internal validity. It's uncertainty about whether the dependent variable (self-reference or norm-reference when trying to predict information about a stranger) is actually related to the independent variable (neurotype), or actually a result of some other variable (like social experiences, demand characteristics, or frustration with a confusing question).
So what do we do? How do we draw meaningful conclusions?? And how do we account for all the other factors that could be at play???
When a simple experiment is not an option, the best way to maximize internal validity is to account for other likely variables and account for them in your data analysis. The study in my example could easily have included additional questions to evaluate the autistic children's knowledge of peer preferences; it could have included some measure of frustration, or asked about how the children reached their guesses about Sally and the other fictional characters. It could have used a totally different design to avoid introducing frustration for the autistic children -- perhaps showing the children a selection of foods that can be packed for lunch, telling them which of those foods are most popular, and then asking the child to help them pack a lunch for themselves and for another child, to see whether the child chose more of the "popular" foods for the other child and more of their own favorite foods for themselves. These are just a few ideas. None of them would have made the study quite as good as a true experiment -- without random assignment, you can't really be sure which variables you haven't accounted for -- but it would have been a big improvement.
This is why we need more neurodivergent individuals conducting research on neurodivergent populations. It will be infinitely easier for an autistic individual to anticipate what factors might affect responses from other autistics than for a neurotypical individual to make the same predictions. Including us in the early stages of research design gives us the opportunity to identify design flaws before the data has been collected, when the study can still be adjusted and questions added. We also bring a different perspective to interpreting findings; because neurotypical folks have a hard time understanding how we think about the world, they're going to be less good at analyzing why they got the results they did.
Neurodivergent researchers have lived experience with these confounds. We know whether question wording will make sense to us. We know how our experiences may have shaped how we tackle problems, how we interpret scenarios, and how we think about questions.
Including autistic individuals in autism research is a fantastic way to reduce the number of unaccounted for variables in a research project. And because the best way to account for confounds if you can't control them is to measure them and include them in your data analysis, this all means that we really need to identify these other variables before data collection, so that if we need to collect additional data we can.
Neurodivergent researchers can help studies to be more ethical.
All research with living participants -- and especially human participants -- must meet certain ethical criteria; a study should never cause more discomfort than is necessary for the research question, and the amount of discomfort you can ethically introduce depends on how much the participant could benefit from the research (for example, you can test a risky new drug if a person is likely die a painful death without it and they consent, but you can't give a risky new drug to someone who's perfectly healthy just to see what happens). Neurodivergent researchers bring a unique perspective regarding what could cause discomfort; while neurotypicals might look at a space as bright and cheerful, we may see it as overstimulating; while neurotypicals may consider a question fairly harmless, it may cause an autistic individual stress and confusion completely avoidable with better word choice. And of course, the most ethical research is research whose findings are aligned with reality, improve understanding, and do not contribute to misconceptions or stigma.
And finally, we can bring research to our own community.
In a day when so many of the most effectively-publicized autism organizations support the eugenics of the autistic population, it is hard to trust researchers. Including the autistic community in the design of research makes it infinitely easier for us to trust the research itself, increasing participation and enabling the research to actually impact and benefit our community in positive ways. And -- let's be honest -- the language used in Academia isn't always clear to non-academics; constructs get labeled with terms that we associate with much broader concepts (for example, see my post about Empathy). Having autistic adults involved in the research doesn't just engage that one autistic researcher; it engages countless other autistic individuals who are more likely to participate in, engage with, and understand the research thanks to that representation.
Autism research has suffered for too long from a lack of diversity and a lack of perspective. From neglecting key variables, to drawing questionable conclusions from the data, autism research without autistic representatives is unhelpful on average and harmful at worst.
But this isn't how autism research has to be.
The most efficient way to fix these problems is by including autistic individuals in the research process, from beginning to end. With so many of us so eager to volunteer our time, energy, and insights, there's really no excuse not to.
Comments
Post a Comment