English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

In this article we analyse the effects of different data collection strategies in the study of local participatory experiences in a region of Spain (Andalusia). We examine the divergences and similarities between the data collected using different methods, as well as the implications for the reliability of the data. We have collected participatory experiences through two parallel processes: a survey of municipalities and web content mining. The survey of municipalities used two complementary strategies: an online questionnaire and a CATI follow-up for those municipalities that had not answered our first online contact attempt. Both processes (survey and data mining) were applied to the same sample of municipalities, but provided significantly different images of the characteristics of Andalusia’s participatory landscape. The goal of this work is to discuss the different types of biases introduced by each data collection procedure and their implications for substantive analyses. KeyworDs Citizen participation; Data collection procedures; Internet data mining; Local participation; Participatory experiences; Survey administration mode. resumen En este artículo analizamos los efectos de diferentes estrategias para la recolección de datos en el estudio de las experiencias participativas andaluzas. Examinamos para ello las diferencias y similitudes entre los datos recogidos mediante diferentes métodos, así como las implicaciones para la fiabilidad de los datos. Para ello, hemos utilizado dos procedimientos paralelos. En primer lugar, una encuesta a municipios y la minería de datos en Internet. La encuesta se realizó utilizando dos modos de administración diferentes, un cuestionario online y un cuestionario telefónico de seguimiento a los municipios que no respondieron al primer intento de contacto vía correo electrónico. Tanto la encuesta como la minería de datos fueron aplicados a la misma muestra de municipios, aunque arrojaron diferencias significativas en cuanto a las características del panorama participativo en Andalucía. El objetivo de este trabajo es discutir los diferentes tipos de sesgos introducidos por cada procedimiento de recogida de datos y sus implicaciones para posteriores análisis sustantivos. Palabras Clave Experiencias participativas; Minería de datos online; Modo de administración de encuestas; Participación ciudadana; Participación local; Procedimientos de recogida de datos. revista internaCional De soCioloGía (ris) Special issue on methodological challenges in participation research Vol. 70, extra 2, 65-87, diciembre 2012 ISSN: 0034-9712; eISSN: 1988-429X DOI:10.3989/ris.2012.04.04 66 • CAROLINA GALAIS, JOAN FONT, PAU ALARCÓN and DOLORES SESMA RIS, VOL. 70. EXTRA 2, 65-87, DICIEMBRE 2012. ISSN: 0034-9712. DOI: 10.3989/ris.2012.04.04 introDuCtion1 Most previous attempts at providing a general picture of local participation activity have used self-administered surveys sent to municipalities (DETR 1998; Birch 2002; FEMP 2002; Ajángiz and Blas 2008). Is this a reliable strategy that can provide a good overall picture of reality? Are there other alternatives that could provide better information? In this paper we try to answer these questions through an analysis of the data collection process regarding participatory experiences completed at a local level in Andalusia, a region of Spain. To do so, we examine the divergences and similarities that arise from the comparison of two different methods (one of them with two modes of administration, making three different data sources) of collecting and coding information regarding a few hundred participatory experiences. The first main goal of this paper is to discuss the virtues and limitations of two contrasting strategies of data collection. The first strategy used was the more traditional one, a survey of municipalities. The second was a data mining strategy using the Internet. With this aim, we conducted two parallel data collection processes that tried to capture the same reality. In addition, the survey faced a common problem related to this methodology: dealing with refusals and with the resulting moderate response rates. To address this, the first mode of administration (Computer Assisted Self Interview, CASI) was complemented with a Computer Assisted Telephone Interview (CATI) survey. This allowed us to address two subsequent research questions. First, we asked whether the differences between our surveys were a product of comparing two different sets of municipalities (the larger and more engaged with the research topic, which answered our online survey in the first place versus the remaining ones that answered the telephone survey) or whether some of the differences were the result of using two different modes of administration (CATI vs. CASI). Second, once we aggregated these two sources of data, we were able to compare them with the results from our data mining approach to learn more about the biases each of them produced on the pictures of the reality obtained. The structure of the paper is as follows. In the next section we justify why making these comparisons is important and present the research design and data collection procedures we have used. Section 3 makes the first comparison between the two stages of the survey (CASI vs. CATI). In this first comparison, our two universes were different and, as a result, we also expect to find important differences in the characteristics of the experiences collected. We discuss whether all differences were compositional (i.e., caused by the fact that we are measuring two different parts of our final universe). The complementary explanation is that some of these differences may be the result of the 1 A previous version of this paper was presented at the Conference “Methodological challenges in participation research”, IESA (CSIC), Córdoba, November 4-5, 2011. We thank the session discussant, the participants and Donatella della Porta for helpful comments. METHODOLOGICAL CHALLENGES FOR THE LARGE N STUDY OF LOCAL PARTICIPATORY EXPERIENCES • 67 RIS, VOL. 70. EXTRA 2, 65-87, DICIEMBRE 2012. ISSN: 0034-9712. DOI: 10.3989/ris.2012.04.04 two modes of administration used. Section 4 moves to the comparison of the final results of both data collection procedures (survey vs. Internet-collected information). We follow the same logic as in section 3, showing the differences in a few important variables and analysing to what extent they are due to the data collection mode. Section 5 briefly presents three potential future research strategies to continue exploring the causes of the remaining differences. theoretiCal FrameworK anD researCh DesiGn This paper stems from the decisions and challenges faced when gathering information on participatory experiences at the local level for the MECPALO project2. One of the main goals of the project is to build several regional databases of participatory experiences developed at the sub-regional level3. This is aimed at making a description of the characteristics of these experiences, as well as answering a series of questions related to the origins, democratic qualities and attitudinal consequences of those experiences. We have argued elsewhere about the need to build close to local participation realities that go beyond the prevalent case study strategy (Font et al. 2011; Font and Galais 2011)4. However, drawing such a picture is not an easy task. Three different approaches are found in previous research. First, the selection of a limited subset of experiences that share some common organisational or territorial characteristics (“focused mappings” e.g., Schattan 2006; Sintomer et al. 2008). Second, the gathering of several varied experiences that try to capture the maximum diversity regarding those processes (Subirats et al. 2001; Della Porta and Reiter 2009). Third, the development of a survey of municipalities to obtain a list of the municipalities’ responses (DETR 1998; Birch 2002; FEMP 2002). Since the first approach allows building a more reliable but also more incomplete picture of reality, we wanted to assess the advantages and problems of the two remaining strategies. It should be noted that we lack a census of experiences. That is, there is no sampling frame with which to start. Bearing this in mind, we started by designing a representative sample of Andalusian municipalities. Andalusia has 770 municipalities, from which we selected a sample of 400. These 400 municipalities are representative of the municipa2 MECPALO is the Spanish acronym for the project Local participation processes in Southern Europe: causes and consequences. The project’s principal investigator (PI) is Joan Font and the research team includes researchers from three Spanish institutions, as well as a French team (PI: Yves Sintomer) and an Italian team (PI: Donatella della Porta). 3 The universe of analysis is formed by any participatory process (from a 2 hour consultation to a stable and periodical mechanism) whose aim is to discuss local policies or issues and which has either been promoted or has gained recognition from local authorities. 4 A similar argument has also been developed by other authors (e.g., Baiocchi et al. 2011). 68 • CAROLINA GALAIS, JOAN FONT, PAU ALARCÓN and DOLORES SESMA RIS, VOL. 70. EXTRA 2, 65-87, DICIEMBRE 2012. ISSN: 0034-9712. DOI: 10.3989/ris.2012.04.04 lities with more than 1000 inhabitants5. The sample was stratified by province and city size (Font et al. 2011). Our main unit of observation comprises experiences and not municipalities. Sampling Andalusian municipalities (those that develop participatory experiences as well as those that do not) allows us to answer additional research questions (e.g., why some councils conducted few or no participatory practices while others undertook quite a few), but this is not the aim of our particular research. In addition, our sampling strategy guarantees acceptable variability among the contextual explanatory factors. We then

Methodological challenges for the large N study of local participatory experiences. Combining methods and databases