SIFTing Out the CRAAP: Warrants, Dragons, Togas, and "Alaska"
In a world of fake stuff,
perhaps we should spend some time figuring out how to figure things out. Here goes.
We tend to teach a concept called warrants to help students understand how arguments build assumptions from evidence. I have a whole blog series that explains how this works. It starts here.
That generally helps with arguments made in some degree of good faith. They even help us with paranoid and outlandish claims. But we need a bit more to help us figure out how to critically read in a world where social media exposes us to a lot of manipulative trickery based on evidence selection.
This is hard, which is why we all tend to fail at it. I can take your evidence at face value and then explore the nature of your claims and how you might be using your evidence inappropriately. Even given what's been happening recently, I still think that this is far more common than simply making up evidence, and it gives us some defense against so called fake news.
Think of that this way, If I made up a damaging argument about Person X: "Person X is, in fact, A SECRET AGENT OF THE NEVER-COLLAPSED ROMAN EMPIRE, OPERATING IN SECRET FOR MORE THAN A THOUSAND YEARS TO DESTABILIZE SOCIETY AND BRING US BACK TO NERO'S RULE!"
Okay. Yeah.
Evidence A, that Person X, say, wears togas at home, only gets us to the claim that they are a Neroian agent if we have the warrant that links the evidence to the claim that in this case has to be "Only Neroian agents wear togas, ever." Well, that person has never been to a toga party or Halloween or who knows what else. Perhaps togas are terribly comfortable, as the Dread Pirate Roberts suggests in this clip from The Princess Bride:
Toga!
We tend to teach a concept called warrants to help students understand how arguments build assumptions from evidence. I have a whole blog series that explains how this works. It starts here.
That generally helps with arguments made in some degree of good faith. They even help us with paranoid and outlandish claims. But we need a bit more to help us figure out how to critically read in a world where social media exposes us to a lot of manipulative trickery based on evidence selection.
This is hard, which is why we all tend to fail at it. I can take your evidence at face value and then explore the nature of your claims and how you might be using your evidence inappropriately. Even given what's been happening recently, I still think that this is far more common than simply making up evidence, and it gives us some defense against so called fake news.
Think of that this way, If I made up a damaging argument about Person X: "Person X is, in fact, A SECRET AGENT OF THE NEVER-COLLAPSED ROMAN EMPIRE, OPERATING IN SECRET FOR MORE THAN A THOUSAND YEARS TO DESTABILIZE SOCIETY AND BRING US BACK TO NERO'S RULE!"
Okay. Yeah.
Evidence A, that Person X, say, wears togas at home, only gets us to the claim that they are a Neroian agent if we have the warrant that links the evidence to the claim that in this case has to be "Only Neroian agents wear togas, ever." Well, that person has never been to a toga party or Halloween or who knows what else. Perhaps togas are terribly comfortable, as the Dread Pirate Roberts suggests in this clip from The Princess Bride:
So the warrant analysis shows us that the evidence, whether true or not, doesn't justify the claim. That shields us from some kinds of bad/fake argument.
The hard part is that we may already be committed, because of our political party or religion or pop culture fandoms to the notion that togas are super bad. We won't see, as easily, the flaws of a warrant we have been trained to believe since we were kids.
Evidence B would have to be something that helps us believe that the Roman Spy Network still exists. I don't know what that could be. Latin on our coins. Roman style pillars on government buildings around the world? Olive oil on EVERY. SINGLE. SALAD!!!!
Again, warrant analysis should help.
But it doesn't do the whole job here. Warrants are still best at analyzing arguments the rhetor is making in good faith. They may be wrong or bizarre or mendacious, but the rhetor's stance of belief helps us to figure out what they are all about and whether or not we can believe them.
What do we do about lies?
Well, some lies are clearly for a purpose, like the political fake news circulating in the 2016 US Presidential Election. Warrants can help us there.
But what about lies designed for no clear purpose other than the get you to believe?
Dragons!
I think about this because I used to teach a class called "Ghosts, Aliens, & Monsters" about how we should decide what counts as evidence for things we might believe but have ambiguous or controversial evidence for that we often believe in (God, love, "Alaska" <-- have you been there? what evidence do you have for it? maybe it's all a ruse!!) and that we usually don't (UFOs, cryptids like bigfoot, ghosts). Do we treat evidences for these things the same or not? Should we? How would we know if we were making a mistake?
Anyway, one day in class I
showed this fake documentary thing where Patrick Stewart (Captain Picard,
Professor X) took us through the evidence for dragons. Here's 5 minutes of
excerpts:
It had some sequences where it takes you through a fictional researcher finding a a dragon and then explores questions of belief from there, the same approach I had for the class. A student came late to the screening and sat down and watched. She got wide-eyed and then stepped over to ask me in a whisper when the show had the "researcher" finding a "dragon carcass", "Dr. Vrooman, when did they find that?" I tried to break it to her that it was fake gently, but half the class heard her.
It had some sequences where it takes you through a fictional researcher finding a a dragon and then explores questions of belief from there, the same approach I had for the class. A student came late to the screening and sat down and watched. She got wide-eyed and then stepped over to ask me in a whisper when the show had the "researcher" finding a "dragon carcass", "Dr. Vrooman, when did they find that?" I tried to break it to her that it was fake gently, but half the class heard her.
Think of it. She was IN a class
ALL ABOUT what to do with fake reports, and just a few minutes of video and she
totally bought it.
Aha! Those of you who actually read the warrant section are on me here. How do you know she bought it (the claim) based on the evidence (her question)? Maybe she thought the whole thing was crap and didn't quite know how to bring that up to a professor who was showing this fake science to his class? Could be.
Aha! Those of you who actually read the warrant section are on me here. How do you know she bought it (the claim) based on the evidence (her question)? Maybe she thought the whole thing was crap and didn't quite know how to bring that up to a professor who was showing this fake science to his class? Could be.
So what about us?
Most of you likely learned the
CRAAP test in high school: https://libguides.cmich.edu/web_research/craap.
Mike Caulfield,
though, notes that the test fails on this example: https://zapatopi.net/treeoctopus/.
His solution is the SIFT
test: https://hapgood.us/2019/06/19/sift-the-four-moves/.
Check it all out and see if the
SIFT test helps with the tree octopus.
Do you really want that, though?
But I'm not going to do that because this blog post is designed for my students, and they are going to have to figure that out as part of their homework. I will update this post in a few weeks with an analysis of their collected answers to the problem of the SIFT test and the tree octopus.
Until then....
Insufficient Conclusion
Look, I know you want me to pull this together and tell you what to think about the SIFT test.Do you really want that, though?
But I'm not going to do that because this blog post is designed for my students, and they are going to have to figure that out as part of their homework. I will update this post in a few weeks with an analysis of their collected answers to the problem of the SIFT test and the tree octopus.
Until then....
