Research methodology for case law analysis: An appeal for openness

By Saïla Ouald Chaib

The enrolment as a PhD student does not come with a handbook. Consequently, you are somewhat forced from the start to reflect not only about the research subject, but also on the methodology you will use. In my case indeed, I spent some time not only doing research on the substantive part of my dissertation topic, but also doing research on doing research. My dissertation focused on the right to freedom of religion in the jurisprudence of the Strasbourg Court and case law analysis occupied therefore a central position in my work. Very soon I was confronted with a lot of questions. Do I first dive into the literature on the subject or do I first analyse the case law? Which cases should I read and how many? How should I approach the case law? This blogpost does not intend to draw a roadmap of how to conduct case law analysis. In fact, there is no such thing as one case-law analysis method. Instead, I want to share one of the ways I analysed the case-law and how methodologies from outside the legal sciences inspired me in the process. Besides defining the research questions and the theoretical framework within which the research will take place, two elements are essential when case-law needs to be analysed. The first one is the case law selection. In my PhD dissertation, I decided to examine all judgments and inadmissibility decisions (including partially inadmissible) ruled under article 9 since the reform of the Strasbourg Court in 1998. In the literature on freedom of religion, I noticed that often the same cases were referred to and the reasons why this occurred were not clear to me. This partially explains my decision to examine all cases. I wanted to have a broad overview of the article 9 case law in order to come to a deep and meaningful understanding of it. The second step consisted of determining how I would approach the case law and the methods and tools I would use. I first turned to literature and examined how other authors conduct their case-law analysis. I noticed however that very few explain their methodology. The inspiration for my own case-law analysis came from a rather unexpected corner, namely from social science methodology. In the remainder of this post, I will elaborate more on this aspect of my research.

What I learned from social science research method and tools

A few years ago I was part of a research team conducting an empirical research on women wearing a face veil in Belgium. We interviewed 27 women on their experiences concerning the face veil and the face veil ban. (More information on this research can be found in earlier posts here and here) For the analysis of these interviews, I was introduced to a social science qualitative research method called Grounded Theory.

Central to this method is the inductive approach, involving a bottom-up mindset where the theory emerges from the data, instead of starting from a hypothesis or a central argument and then deductively turning to the data in search of illustrations or confirmations of this hypothesis.[1] This requires an open-minded approach towards the data to let the data ‘speak’. Additionally, I also discovered the world of qualitative data analysis software programs (in my case Nvivo, but there are others) which allowed to do this analysis in a structured way. Applying this analysis on interviews made me reflect on the potential of both the technique and the program for legal researchers, in particular for the analysis of case-law. So instead of approaching the case law in a way that I would limit myself to searching illustrations relevant for supporting the theory I apply in my dissertation, I decided to undertake my case-law analysis in two stages. In a first stage, inspired by the grounded theory method, I approached the judgments and decisions with rather broad and open questions such as “How is the Court approaching the religious aspect of the case?” and “How does the Court approach applicant believers?” I was curious to see what would come out of this research. Additionally, I also decided to experiment with Nvivo for doing case-law analysis. This analysis is shaped through the process of coding. This is a deconstructing process in which segments of the data (in my case the Court’s judgments and decisions) are categorized under several themes and subthemes that are formed and refined during the analysis. For example, under the general theme of ‘reasoning’, one of my codes was ‘Alternative’ and this code comprised several sub-codes such as ‘Applicant had an alternative’ and ‘Applicant could have found an alternative’. I didn’t define these codes before the start of the analysis, but I created them during the coding process where I observed how the Court used the concept of ‘alternative’ in diverse ways in its reasoning. So every time I came across reasoning in the cases which uses the concept of alternatives, I would classify it under this code. This categorization of the case law already consists of analysis in itself, but the analysis also requires constant reflection and comparison during the process of coding. I first applied this method of coding on a diverse sample of 10 cases. As a result I built a first list of codes which I would refine, correct and develop during the chronological analysis of the remainder of hundreds of cases to code.

Only in a second stage of the analysis, I examined the codes from the perspective of the theoretical framework within which my research was conducted, namely the criteria of procedural justice. (See for more information on this concept here and here)

This approach allowed me not only to have a better overview of the Court’s reasoning in article 9 case-law, but it also enabled a deeper analysis. I could easily have found examples of reasoning illustrating procedural justice flaws in the Court’s case law, however, the results would have been less developed than the results I obtained from this way of working. Indeed, as a result of this two-stage approach, I discerned three levels in the Court’s case law where procedural justice flaws occurred. Some of the findings could only be observed because of the possibility to compare the different codes with each other. Also, most of the analytical work takes place during the process of coding itself. Not only the codes are important, but also the reflections obtained during the process, which researchers are encouraged to write down. (In grounded theory terms this is called memoing)

As mentioned earlier, I conducted this analysis in a qualitative data analysis program called Nvivo which helped me to process the large amount of cases in a structured way. Important to mention however is that a qualitative research software program, such as Nvivo (or other programs), does not do the work for the researcher, but it is a tool that helps to organize the data and it helps to bring structure in the analysis. If I wouldn’t have used this program I would probably have ended up with a long word document with all the selected parts of the case law or a pile of notebooks with my notes. Instead, everything was centred in one program: the cases, the codes and the notes. Moreover, I also classified the cases according to several criteria, such as gender and religion of the applicant, which was particularly interesting since this information is not available in the Hudoc database. I think there is potential in these kind of programs for lawyers which is worth further exploration, but this depends also on the preferences of every researcher.

Concluding reflections

What is the best method to conduct case-law analysis? I still don’t have an answer to that question, but I can only conclude that there is no such thing as THE best method. Finding the best method of research is a very individual choice. It depends on your research aim, your research questions, the nature and amount of your data, the available time and also of the researcher. The most important thing I would advice is to reflect on methodology from the beginning and to keep notes of that process as well. (Believe me; this will facilitate the writing of the methodology section in your dissertation!) Although there is not one clear roadmap for conducting a good case law analysis, I personally find it important (as a reader) that it is clear what your roadmap is. It is interesting to know why you chose a certain method and not the other and how you selected your case-law.

This is why I would like to conclude this post with a call for openness. An open mind towards other disciplines that put a lot of emphasis on methodology might offer some inspiration for legal research. Also an open mind towards the data can help to gain new insights. Finally, I would also encourage more openness towards the reader. Indeed, legal researchers, among whom the Strasbourg Observers who examine the case law of the Court expect courts to be transparent in their reasoning and to sufficiently motivate their decisions. It is a matter of good judicial practice, it ensures qualitative decision-making, it is a sign of respect towards the applicants in the case and it allows researchers like us to examine and to understand the thinking process leading to the resulting decision. Why not expect the same openness in our own work? It might not offer the best recipe for other researches, but it might inspire others in their own work to develop a methodology which is best for them.

[1] For literature on grounded theory see e.g. Jonny Saldana, The Coding Manual for Qualitative Researchers, (Sage Publications, 2012); Kathy Charmaz, Constructing Grounded Theory: A Practical Guide through Qualitative Analysis (Sage Publications, 2006) and in Dutch: Dimitri Mortelmans, Kwalitatieve analyse met Nvivo, (Acco, 2011).