Steven Lubet, a trial lawyer, has given some useful advice for improving validation techniques in Interpretivism; and he has given some awful, self-destructive advice, which, if followed, would ruin the practice.
The Good Advice
In a blog post, [1] as well as a book,[2] Lubet has offered his counsel on ethnography as a social science practice. Generally, he is concerned that not enough is known in the social sciences about the principles for testing truth claims as used in the Law of Evidence. (43) I agree. The Common Law, as practiced in Great Britain and the US, has a long and honorable history of formulating principles by which to settle disputes between litigants and to test the veracity of their allegations and testimony.
Law is a substitute for settling disputes through violence. It is the collective product of some of the most practical minded, educated, fair, and intelligent men and women in history. As a set of such principles, it is an amazing, invaluable contribution to Civilization.
The “Hearsay Rule,” for example, allows, as admissible evidence, testimony as to what a witness has heard another person say. For instance, Mary heard John in another room scream “I hate you, Julie!” Then Mary heard a shot. Later, the police found Julie in that room, dead of a gunshot wound. Ergo; Mary’s testimony tends to show that John had the intent to kill Julie. Other evidence might be required to prove that he did it. But, if Mary had seen John shoot Julie after screaming at her, that would be Mary’s direct evidence, and could be conclusive.
Lubet advises ethnographers to be mindful of direct and indirect, or hearsay, evidence. (I.e., “What I saw” vs “What I was told.”) Referring to his book, Lubet says, “Thus, although I have never proposed that ethnographers adhere to strict legal rules, I did suggest that they should be more wary of vouching for their sources’ stories.”
Also, verbal testimony, or storytelling, can sometimes be checked against existing documentation, like court records, or security camera films. He says that ethnography needs more of this. And, look for “circumstantial evidence” to lend support to what informants say, or to debunk it.
While all this may appear to be good advice, actually it is misdirected. Lubet wants to put the Truth Verification burden on the ethnographer. But he appears not to understand that the relationship between the ethnographer and the subject is different from that between the ethnographer and her or his professional peers.
The Bad Counsel
First, contrary to Lubet’s advice, ethnographers should NEVER try to imitate “Attorneys and other fact-seeking professionals.” (43)
Lawyers want to win cases for their clients. They really don’t give a hoot for the ethnographer’s ideal of striving to understand the experiences and cultures of other people – to just understand their thoughts, feelings, and practices.
In the conduct of a trial, telling lies to pass as “truth” is a regular practice. Trial lawyers are experts in this art, and they expect it from one another's witnesses. That is most defiantly NOT the model ethnographers ought to be following.
The mental orientation of ethnographers is nearly the exact opposite of trial lawyers. Ethnography requires empathic listening. A good ethnographer can learn to hear something in an account, or story, that even the speaker may not understand.
Listening to people as an ethnographer is a completely different process than listening to witness testimony in a trial. The lawyer is a critical listener, ready to pounce upon those falsehoods and distortions that will help win his case. The ethnographer wants to encourage trust and free expression without judgment. So what if it is false, it is what gets expressed that matters. Interpretation asks, ‘what was the meaning to the speaker underlying the literal words?’ The subject is not under cross examination.
Lubet tells the story of a Muslim woman in court due to a dispute over a car rental bill. She was wearing a face veil. The judge told her to take it off so he could assess her truthfulness by seeing her face. She refused. She lost. (45-46)
Lubet says that seeing a face only has a 50-50 chance of detecting untruth; therefore, the judge was factually wrong. But Lubet’s interpretation exposes his literalism. An ethnographer might have seen that, for example, the judge just gave an excuse for his deeper feelings that it was his duty to protect American Law and values from foreigner disrespect. Whether the judge was right or wrong, Lubet’s lawyerly interpretation – that the judge acted on a factual mistake – may completely fail to understand the deeper meanings of the moment; namely, understanding what the situation meant to the judge.
Ethnography requires that informants feel comfortable and accepted for who they are without judgment. However, the skepticism that Lubet urges would diminish the very attitude that a talented ethnographer depends upon for her or his studies. The techniques used by a lawyer cross examining a witness would make the ethnographer appear as a police agent, not a confidant.
Ethnography requires naiveté and sensitivity, not skepticism. Lubet fails to understand this, and gives ethnographers advice that would be self-destructive for their practice. Thinking like a lawyer, he urges ethnographers in the field to maintain “distance between researcher and source,” to avoid “undermining the much-needed critical evaluation of a subject’s story.” (44) What he just doesn’t get is that A) ethnography requires emersion, not standoffish distancing, and B) participant observation is not the same as a trial lawyer's cross examination.
Also, contra Lubet, journalism is not ethnography. Journalists do not have the ethnographer’s responsibility for understanding the meanings with which people act. A journalist can interrogate his or her witnesses with skepticism and “distance,” because the careful building of trust is not required. Lubet confuses ethnography with reportage, a completely different practice for acquiring and conveying knowledge. Apples and Oranges.
Lubet's skepticism is appropriate for peer review. Alice Goffman’s account of ghetto urban life in On the Run may have factual errors.[3] Maybe “A” wasn’t really “B’s” girlfriend, like he claimed. But the Big Picture she provides is nonetheless true and important: the “get tough on crime” policy of that city caused disruption in the lives of folks in the Black community. The implication for local government was true and salient, no matter if some details were off. Big Picture: the policy was inhumane, and ought to be rescinded.
A critic of Goffman, Lubet declares boldly, “I reject this paradigm.” (46) He thinks the little facts should have priority over the “big picture.”
But he “rejects” a paradigm he fails to understand!
Lubet would like to turn his own lawyerly insensitivity into a methodological technique when he proclaims, “There is no contradiction between providing a ‘feel for the culture’ and obtaining accurate data.” (46) In his insistence on factual correctness over deeper meaning, Lubet makes of himself, like the philosopher says of the rich man, "he knows the price of everything, but the value of nothing."
The Onus of Responsibility for Validation
Lubet recounts the many instances in which social scientists said the same thing to him that I have said here. His response is tone deaf and stubborn. Nevertheless, his counsel is not entirely “value free,” so to speak.
In my view, the problem for interpretive social and political science is not to train skeptics to interrogate witness while in the field. The problem is to train peer reviewers in these techniques of scrutinizing evidence. In other words, the final responsibility for fact checking is with the profession, not the immersed ethnographer. Lubet’s advice is helpful for peer reviewers sitting at their computers in their offices, but misapplied when targeted towards ethnographers in the field.
Perhaps more importantly, is the problem of “meaning checking.” Lubet does not address this, in part, because he rejects big pictures. Testing facts for pinpoint accuracy is one approach to criticizing ethnographic reports. But the unique challenge for interpretivists is to formulate rules and practices for criticizing interpretations. Does this study mean what the author says it means? Does On the Run expose the unintended consequences of the city’s crime policies?
The profession, as an institution, not the ethnographer,
needs to arrange itself to take final responsibility for what it allows out
into public view, as scientifically validated.
Once “adversary testing” becomes institutionalized as a later phase in the interpretive social science process, ethnographers will be conscious of the reputations they are building among their peers. Those who may have been lazy, or too imaginative, in the past will then try to report, or write up, their interpretations more responsibly.
Science is a collective enterprise, based on a division of labor. The ethnographer’s talents are not those of the lawyer, but of a key worker in the knowledge producing process. Empathic observing and listening are required. However, an awareness of the Rules of Evidence when formulating principles to be applied in the peer review validation process is a reasonable proposal.
Finally, if Lubet has anything to contribute to
interpretive political science, it is the challenge that if lawyers have Rules
of Evidence for their profession, why can’t we have Rules of Validation for our
profession?
William J. Kelleher, Ph.D.
@InterpretivePo1
1st Post
Script:
Burawoy’s Reply
The sociologist, Michael Burawoy replied to Lubet in the
same publication. [4]
Burawoy agrees with the observations often made to Lubet that factual errors do not ipso facto invalidate an ethnographic study. They must be weighed against the bigger picture of the study’s contribution to knowledge, “Here the crucial criterion is not verisimilitude but the growth of knowledge.” (50)
Burawoy notes, like I said above, that, “The ethnographer is not a heroic individual but a member of a scientific community engaged in the collective enterprise of advancing knowledge.” (50) “As social scientists, our final goal is not to learn about the case, which is what the participant wants, but to learn from the case in order to expand scientific knowledge.” (52)
Burawoy, a sociologist, refers often to a distinction he sets up between “empiricist ethnography” and “theory-driven ethnography.” (48f) For political science, David Easton has clarified the role of “theory” for his profession. That is, the “political system” is the interpretive framework that authentic political science works within, and takes research guidance from.[5] Easton would agree with Burawoy’s comments that, “Without a lens the world is a blur,” and “Theory tells us what to look for and sensitizes us to things out of place. A good theory makes predictions and fosters surprises.” (50-51)
2d Post Script:
Example of a Dispute over a Fact in Peer Review
Lubet denounces as impossible Goffman’s claim that the Philadelphia police “wait outside hospitals serving poor Black communities and run the IDs of the men walking inside.” Apparently, in his view, police don’t play dirty tricks.
I guess he has never heard of ICE. They not only wait outside of hospitals, but court buildings, and the schools where undocumented parents can be nabbed while dropping off their children. He ought to watch the ethnographic film, “Immigrant Nation.” Perhaps his experience of white privilege obscures his ability to understand.
Lubet asks, surely factiously, if ethnographers can be excused for committing crimes.[6] I doubt any judge would hesitate to knock down that defense.
Finally, Lubet casts doubt on the depth of his own self-awareness when he pleads innocently that, “In contrast, my own research was open-ended. I would have reported finding any corroboration for Goffman’s story …” Oh? Not having considered ICE was convenient.[7]
Notes and References
[1] Steven Lubet (2019) Accuracy in Ethnography: Narratives, Documents, and Circumstances 18 Contexts 42-46. https://doi.org/10.1177/1536504219830676 For quotes I use the pdf at, https://journals.sagepub.com/doi/full/10.1177/1536504219830676
[2] Interrogating Ethnography: Why Evidence Matters (2017) Steven Lubet
[3] See my discussion of Goffman in this blog at, Replication Indoctrination – An Aim of Teaching in Political Science https://interpretat.blogspot.com/2020/08/replication-indoctrination-aim-of.html
[4] Michael Burawoy (2019) Empiricism and Its Fallacies. 18 Contexts (1) 48-53. https://doi.org/10.1177/1536504219830677 I use the pdf at, https://journals.sagepub.com/doi/pdf/10.1177/1536504219830677
[5] For more on this see: William J. Kelleher (2017) Letting Easton Be Easton—An Interpretivist. 15 QMMR (2) https://osf.io/preprints/socarxiv/cmnvx/ DOI 10.31235/osf.io/cmnvx
For Easton’s plan on the organization of political science see, Back to the Future: How Understanding David Easton Can Give Guidance to the Caucus for a New Political Science
[6] Lubet replies to Burawoy: “ethnographers in cars with guns.” Contexts Blog Post by Steven Lubet | March 22, 2019 https://contexts.org/blog/ethnographers-in-cars-with-guns/
[7] An interesting challenge to Lubet is made by Mikaila Mariel Lemonik Arthur, based on interpretations of legal history and evidentiary principles at, https://scatter.wordpress.com/2019/04/23/ethnographers-are-not-lawyers-ethnographies-are-not-trials-standards-of-evidence-hearsay-and-the-making-of-good-analogies/
Lubet’s response at, https://scatter.wordpress.com/2019/04/30/reply-ethnographers-are-not-lawyers-and-nobody-ever-said-they-should-be/
Lubet sums up their discussion in his statement, “Ethnography ‘isn’t law, it isn’t journalism, and it isn’t a laboratory experiment,’ says Arthur. Well sure, but neither should it resemble oral history, memoir, or fiction.”
Comments
Post a Comment