© S. Quiédeville et al., Published by EDP Sciences 2018

1 Introduction

This paper discusses the effect of performing a Social Network Analysis (SNA) for an ex-post evaluation of the impacts of a research and innovation program in agriculture. Our study is based on the methodology proposed by Quiédeville et al. (2017), which is based on the Participatory Impact Pathway Analysis (PIPA) and other methods including SNA. The organization of stakeholder workshops is a key element in the analysis, and the approach can be described as participative. This can enhance the process of learning by the actors and increase the uptake of research results.

Our analysis focuses on the transition to organic farming that represents a paradigm shift, requiring the creation of substantial knowledge and techniques (Lamine et al., 2009). In 2000, INRA (French National Institute of Agronomic Research), CIRAD (Agricultural Research Center for International Development) and CFR (French Centre of Rice) launched a research program to develop organic crop production systems in the Camargue region in France. The research program contains six projects, focusing mainly on weed management (Tab. 1).

The Camargue territory is situated in the southeast of France over an area of some 145 000 ha where rice is the main crop grown. The transition to organic farming, with a shift from 600 ha in 2003 to 1400 ha in 2014, has implied several innovations, e.g. the development of crop rotation systems and false seedbed techniques.

This paper endeavors to assess:

• the effectiveness of using SNA to explore stakeholders’ statements on actors’ relationships;

• the limitations of SNA in evaluating the impacts of the research.

We investigate stakeholders’ statements using SNA since participatory approaches are often criticized for their lack of scientific rigor.

In the next section, we present the background to this case study and explain the method. Then, results are outlined along with conclusions on the main lessons learned.

2 Background

Changes in relationships between actors and their effects are very important in driving innovation processes. This causal link can be explained through the concept of social capital, which can be defined as a set of diverse entities where actors’ actions are facilitated inside a given social structure (Coleman, 1988). Social capital is constructed through three main underlying mechanisms (Aldrich, 2012): bonding (horizontal links within a group), linking (vertical links among actors), and bridging (horizontal links between groups). The role of knowledge brokers is seen as crucial in enabling innovation as they bring together stakeholders and facilitate their interaction in the wider innovation system (Klerkx et al., 2009). Furthermore, Krugman (1991) stressed the positive role played by clusters of actors on innovation and economic development via, in particular, increased information flow and knowledge spillovers. Gnyawali and Srivastava (2013) also demonstrated the existence of a positive link between network quality and innovation development through, inter alia, social interactions and competition intensity.

Different methods can contribute to evaluating how innovations have developed. Some of these focus on adoption decisions of individuals while others, including SNA, take a systems perspective and place particular emphasis on complex actors’ interactions.

SNA aims to explore the content, type, structural pattern, and intensity of social relationships within an organization, a group, or other structures (Casieri et al., 2008). Specifically, SNA can help to understand the position of the actors in the network as well as how and which resources and information are transferred or exchanged among actors (Wasserman and Faust, 1994).

The indicators of betweenness, clustering coefficient, and degrees (Haythornthwaite, 1996) can be used to analyze impacts of the research. The identification of actors with a high betweenness, representing a high degree of intermediation in the network, is of particular interest. The clustering coefficient of an actor can help gauge his/her access to relevant information and resources by calculating the level of connectivity between actors in the neighborhood (Scott, 2000). The measurement of degrees accounts for the number of relationships among actors. The higher the level of degrees, the more the likelihoods of getting experiences and skills are (Freeman, 1978).

Several evaluations of research or development programs used SNA in the past. It was for instance used for the assessment of impacts of regional development programs in Japan, on the structure, pattern and innovativeness of R&D networks (Yokura et al., 2013).

3 Materials and approach

The approach proposed by Quiédeville et al. (2017), to which the SNA described in this paper contributes, is made up of 5 steps. In this paper, we use only 4 steps, which are specifically relevant to SNA.

3.1 Initial screening, identification of main impacts, and SNA data collection and preliminary analysis

At the end of 2014, 19 stakeholders were interviewed in face-to-face: 1 key researcher from INRA, 1 responsible from the CFR, the natural park of Camargue and the Tour du Valat, as well as 1 manager from 4 different rice traders, and 11 farmers (7 partially-organic and 4 organic out of 35). The aim was to identify research outputs (first and tangible results), the general factors that eased or hampered the innovation pathway, the most important impacts and to collect SNA data. Regarding SNA, stakeholders were firstly asked to identify their useful relationships with others working on organic rice, for six periods of the program (corresponding to important changes) as well as to evaluate the intensity of those relationships (from 0 to 3) on information flows, financial and collaboration links. More information is provided in Table 2.

3.2 Building the impact pathway

A stakeholder workshop was organized in 2015 in order to reconstruct the theory of change of the research program. The stakeholders involved are 2 researchers from INRA, 3 researchers from the CFR, 3 participants in total from 2 organic rice traders and 7 organic and partially-organic farmers. Stakeholders were asked to draw the impact pathway of the research by linking the different components (e.g. the output x with the outcome y or activity z). The outcomes include changes in actions undertaken, behaviors and actors’ relationships.

3.3 Refining the pathway and collecting indicators

The evaluators populated the impact pathway that stakeholders drew in the second step (b) with relevant elements in relation to actors’ relationships, gathered in the first step (a). The elements taken from the first step are the statements on relationship issues, which were quoted by at least half of the stakeholders.

3.4 Process tracing and evaluation

This step is composed of two relevant stages for SNA.

3.4.1 Exploring alternative explanations

The first stage is to evaluate whether the first and second events (activity, output or outcome) of each pathway link actually happened. Equally important is the examination of the underlying mechanisms of these pathway links, which were challenged with possible alternative explanations derived from logical reasoning made by the SNA analyst.

3.4.2 Elimination phase and further network analysis

The Hoop test, as part of the approach of Process tracing (Mahoney, 2012), is applied with a view to validating the pathway links. It focuses on the necessary conditions to make the different links happen. The test is successful if the necessary conditions are fulfilled. A necessary condition (e.g. increased relationships between two institutions) for an event Y (e.g. development of a specific field trial) is a condition that must be fulfilled for Y to be reached. Further SNA calculations (with the same indicators but testing different configurations) were performed to better understand the first results obtained and check the validity of possible alternative explanations.

The SNA indicators used in the first step (a) and further in the fourth step (d) are calculated (the relationships are undirected) as follows (Wasserman and Faust, 1994):

• Betweenness centrality: $C_b\left(n_i\right)=\frac{\frac{\sum g_{jk}\left(n_i\right)}{g_{jk}}}{\frac{\left(g-1\right)\left(g-2\right)}{2}}\text{,}$

where g _jk is the number of geodesics connecting jk, and g _jk (n_i) the number of geodesics that actor i is on.

• Clustering coefficient: $C_i=\frac{2n_i}{k_i\left(k_i-1\right)}\text{,}$

• Average clustering coefficient: $\left(C\right)=\frac{1}{n}\sum C_i\text{.}$

• Degree centrality: D_i = Number of connections of actor i.

• Average Degree centrality: $\left(D\right)=\frac{1}{n}\sum D_i\text{.}$

4 Results

4.1 Identified pathway links

Table 3 presents 12 pathway links derived from discussions in the stakeholder workshop. These results were then turned into 4 different groups of pathway links (PL). These PL are as follows:

PL1: Four research activities, i.e. the CEBIOCA (Organic Cereals in the Camargue) project, diverse experimentations and the ORPESA (Organic Rice Production in Environmentally Sensitive Areas) project, contributed to the growing influence of INRA in the actor network.

PUM (Possible Underlying Mechanism): Increased exchanges between INRA and farmers.

PL2: The CEBIOCA project as well as the experimentations undertaken (by INRA and CIRAD) have contributed to an increasing influence of CIRAD in the actor network.

PUM: Increase of exchanges between CIRAD and farmers.

PL3: Both the increasing influence of INRA and CIRAD in the actor network has substantially developed relationships in the actor network, in a way that affects positively innovation development and the transition to organic farming.

PUM: INRA and CIRAD have become an important and moderately important knowledge broker, respectively.

PL4: The high selling price and demand growth for organic rice, as well as the adoption of organic farming, have contributed to an important and growing influence of Biosud (associating a cereal cooperative and two trading companies) in the actor network.

PUM: Opportunity for Biosud to increase incomes.

4.2 SNA tests

SNA results, corresponding to the testing of the 4 groups of PL, are presented in this section. Table 4 summarizes the indicators characterizing the actor network over time.

4.2.1 Increasing influence of INRA in the actor network

The purpose of the first test is to verify whether four particular research activities, which were mainly conducted by INRA, contributed to the growing influence of INRA in the network (Fig. 1).

The analysis allows to confirm this trend. The betweenness score (degree of intermediation) of INRA increased by 46% from 1999 to 2014 whereas the betweenness score of other actors remained stable or even decreased, apart from CIRAD and the private trader Biocamargue. The trend in average degrees also confirmed the increasing centrality of INRA. The average degrees of INRA increased by about 84% compared to an increase in average degrees of 29% for the entire network. Furthermore, we can validate the PUM of the pathway link: an increase of around 80% occurred in degrees between INRA and farmers. These bilateral relationships started to increase in the year 2005, which means that the CEBIOCA project (2000–2004) that attempted to review agronomic conditions and problems in the Camargue, did not play a direct and significant role in developing relationships between farmers and INRA in the Camargue.

The first possible alternative explanation for the increased centrality of INRA in the transition to organic agriculture was an increase in relationships between the adjacent actors of INRA, which could explain also the higher influence of INRA in the network. This was validated, as we observed a growth of 60% in the clustering coefficient of INRA. The two other possible alternative explanations for INRA’s increasing centrality have been rejected. The first explanation was that INRA had significant stronger relationships with other institutes, but from 1999 to 2014, the SNA data did not confirm this trend. Secondly, the SNA results show that relationships among actors in the network would not be lower if INRA was excluded from the network; and the average degrees, in that scenario, would even be higher (+ 18%) in 2014 as compared to 1999 (Fig. 2).

This first SNA test shows that results are not fully in line with what the stakeholders claimed when constructing the impact pathway during the workshop. The increasing role of INRA was slightly influenced by the growing cooperation between its adjacent actors, as shown by the clustering coefficient of INRA. Relationships between CIRAD and FranceAgriMer (French public agency for agriculture) increased over time and particularly since 2010 (projects partly financed by FranceAgriMer).

Fig. 1 Map of actors in 1999 (initial situation). NB: “Agri” means “farmer”. This map of actors corresponds to the initial situation of the network, before the research program was launched. We can observe a relative scattering of the actors in the network with no dominating institutions or persons, and with relatively little links. Carte des acteurs en 1999 (situation initiale). NB: “Agri” signifie “agriculteur”. Cette carte des acteurs correspond à la situation initiale du réseau, avant que le programme de recherche ait été lancé. Nous pouvons observer une relative dispersion des acteurs dans le réseau, sans institutions ou personnes dominantes et avec relativement peu de liens.

Fig. 2 Map of actors in 2014. This map of actors shows the strong importance of INRA, Biosud and CIRAD, which are all situated in the core of the actor network and that are all well or very well connected together and to farmers. That said, the CFR has become slightly less important. Carte des acteurs en 2014. Cette carte des acteurs montre l’importance de l’INRA, de Biosud et du CIRAD, qui sont tous situés au cœur du réseau d’acteurs et qui sont tous bien ou très bien connectés ensemble et avec les agriculteurs. Cela dit, le CFR a légèrement perdu en importance.

4.2.2 Increasing importance of CIRAD in the actor network

The SNA confirms the growing role of CIRAD in the network. During the time span of the research program, the betweenness of CIRAD increased by about 34% and the average degrees by around 61%, whereas the average degrees only increased by about 29% in the overall network. Furthermore, the PUM was confirmed: growing relationships between CIRAD and farmers with an increase in the bilateral degrees of about 45% from 1999 to 2014.

The first possible alternative explanation to the PUM has been validated: we observed growing relationships (bilateral degrees) between CIRAD and the cooperative SudCéréales associated in Biosud (development of a joint rice breeding program) as well as between CIRAD and INRA. The second possible alternative explanation was also confirmed; the SNA demonstrated an increase in the relationships between the adjacent actors of CIRAD: growth of 60% in the CIRAD’s clustering coefficient. This was mainly due to growing relationships (bilateral degrees) between INRA and both the French Centre of Rice (CFR) and FranceAgriMer. This can also explain the increasing importance of CIRAD in the network, but indirectly. However, the third possible alternative explanation was rejected: it was not confirmed that relationships in the network (average degrees), when excluding CIRAD from the analysis, were decreasing over time. In other words, CIRAD has not become more important in the network because of relationships between the other actors decreasing during the time span of the research program.

This SNA test deepened our understanding on the network since it was identified, in particular, an influence of FranceAgriMer and CFR on the growing role played by CIRAD alongside with the development of innovations focalized on the transition to organic farming.

4.2.3 Role of INRA and CIRAD on the actor and organic network development

The third test seeks to validate that the increasing influence of INRA and CIRAD across the actor network substantially affected its establishment and development. The positive development of the network is demonstrated by the increase of both the overall clustering coefficient (44%) and average degrees (29%) during the implementation of the research program (from 2000 to 2014). Moreover, the PUM, i.e. the fact that INRA and CIRAD have become knowledge brokers, was validated as already said in the previous two sections. A possible alternative explanation to the PUM was the emergence of other knowledge brokers in the network; however, this possibility was not confirmed by SNA data.

4.2.4 Influence of Biosud on the actor and organic network development

The SNA partially confirms the significant and rising role played by Biosud in two ways. The first is the rise of about 18% (in 2005 compared with 2003) in the overall network clustering coefficient after Biosud was created. The second aspect is the development of exchanges between Biosud and farmers since the year 2003, illustrated by an increase of 67% in the bilateral degrees from 2003 to 2014.

However, this SNA test could not determine the impacts of the creation of Biosud on the actor network. Surprisingly, the betweenness of Biosud declined by 27% from its creation (in 2003) to 2014, contrary to its clustering coefficient that considerably increased by 129%. Biosud was not positioned as an obligatory crossing point and so did not facilitate much communication between actors in the network (Tab. 4).

5 Discussion

5.1 Application of SNA

By conducting an SNA we were able to confirm that:

• the influence of Biosud has steadily increased since its creation in 2003;

• INRA has become central due to closer relationships with farmers (exchange of information) and played a key role in the adoption of organic farming;

• the last two elements have significantly affected the establishment and development of the network of actors.

The SNA indicators of betweenness, clustering coefficient, and degrees were relevant but also complementary in their capacity to elucidate the different pathway links that revealed actors’ relationships. The clustering coefficient was particularly useful to help understand and interpret the betweenness score of Biosud.

SNA was valuable in confirming or identifying the importance and role of the different actors in the network and their determinants. Additionally, the reconstruction of the actor network at 5 different periods, corresponding to significant changes within the actor network (scaling-up of the value chain, cooperation building or ceasing), allowed a significant deepening of the analysis.

5.2 Limitations and generalization of SNA

SNA proved to be an excellent research tool for confirming stakeholders’ statements on the impact pathway. It did, however, prove to have weaknesses in its ability to describe the effects on actors of the information they receive from the network, whether and to what extent their behaviors changed, why and how. Because of these weaknesses, we had to assume that closer links between actors were positively linked to innovation and to organic farming development. This assumption remains to be further tested in future research.

Furthermore, the conduct of an SNA requires substantial resources in terms of time and finances. At the same time, if evaluation costs are surpassed by the payback of higher positive impacts from future research, larger budgets to the purpose of evaluations would in the long run save public resources.

When the evaluator utilizes participatory instruments to assess the impacts of research, stakeholders’ statements on relationship issues can be further analyzed by SNA in order to verify their accuracy. SNA can potentially answer the same objectives as those enounced in this paper to evaluate the impacts of research in the frame of approaches like ASIRPA (Evaluating Impact of Public Agricultural Research) (Joly et al., 2015) and IMPRESS (Impact of Research in the South) (Temple et al., 2016).

6 Conclusion

The SNA method successfully contributed to the evaluation of the impacts of research. It permitted us to:

• investigate stakeholders’ statements on relationship issues;

• analyze the accuracy of alternative explanations to stakeholders’ views;

• deepen our understanding as to the role and importance of different institutional actors.

The development of relationships in the network was assumed to reflect the development and maintenance of organic agriculture over time. It would be of interest to complete this analysis by calculating indicators of resilience so that the capacity of the network to react, face disturbances and adapt could be further examined. This may allow a better understanding of the long-term effects of the research intervention. Finally, we recommend the use of SNA to help evaluate the impacts of research and innovation programs, in particular when participatory procedures are used.

Disclaimer

The content of this publication is the sole responsibility of the implementing partners of the IMPRESA project and can in no way be taken to reflect the views of the European Union.

References

All Tables

All Figures

Fig. 1 Map of actors in 1999 (initial situation). NB: “Agri” means “farmer”. This map of actors corresponds to the initial situation of the network, before the research program was launched. We can observe a relative scattering of the actors in the network with no dominating institutions or persons, and with relatively little links. Carte des acteurs en 1999 (situation initiale). NB: “Agri” signifie “agriculteur”. Cette carte des acteurs correspond à la situation initiale du réseau, avant que le programme de recherche ait été lancé. Nous pouvons observer une relative dispersion des acteurs dans le réseau, sans institutions ou personnes dominantes et avec relativement peu de liens.

In the text

Fig. 2 Map of actors in 2014. This map of actors shows the strong importance of INRA, Biosud and CIRAD, which are all situated in the core of the actor network and that are all well or very well connected together and to farmers. That said, the CFR has become slightly less important. Carte des acteurs en 2014. Cette carte des acteurs montre l’importance de l’INRA, de Biosud et du CIRAD, qui sont tous situés au cœur du réseau d’acteurs et qui sont tous bien ou très bien connectés ensemble et avec les agriculteurs. Cela dit, le CFR a légèrement perdu en importance.

In the text