1. Speak to a diverse sample within the target client segment:
To gather a wider set of qualitative insights, the team looked to identify farmers representing varied profiles, which would include a balance in gender, geographic location, and type and stage of farming. (Providers may want to seek diversity within the bounds of a clearly defined target segment.) To achieve this, the team (i) worked with partner organizations on the ground to recruit farmers based on pre-established criteria and (ii) independently recruited farmers through local leaders and at local markets.
2. Utilize local experts:
We were aware that a team conducting interviews in a foreign country could encounter cultural barriers arising from differing social norms. To moderate this, the team (i) interviewed experts on the ground (e.g., field officers) to better understand local norms, and (ii) worked with translators to ensure that they had sufficient training to ask accurate questions and do so in a way that was sensitive to farmers.
3. Mitigate external influence on the participant’s narrative:
It was important to allow interviewees to articulate their views with honesty and transparency, in order for the participatory approach to be successful. The potential existed for community leaders or partner organizations to influence the process through making the farmer introduction (i.e., the farmer might have felt an implied expectation to respond in certain ways) or through their presence at meetings; as external evaluators we worked to temper this influence by: (i) communicating with farmers directly once given their contact information, (ii) conducting the interview without a local leader or partner organization staff present, (iii) spending the initial part of the conversation clarifying the purpose of the interview, and (iv) providing opportunities for the farmers to be in the “position of power” in interviewsby allowing them to lead us, show us, take us around, and ask us questions.
4. Preserve transparency of the respondent’s voice:
The team should pay attention to factors that may distract farmers from presenting their views accurately, including their past experience interviewing, to ensure responses best reflect the farmers’ true circumstances. In addition, it is important to ensure that the form of media capture chosen does not distract from the purpose above. For example, if video recording is necessary but puts farmers on edge, the team can use an approach that includes writing key insights from the farmers during the interview and have the farmer repeat these into the camera at the end of the interview.
5. Revise approaches on an ongoing basis:
Working in the field requires flexibility to adapt the interview process and content based on emerging findings. A field team should have daily check-ins to gauge the relevance and sensitivity of questions in the interview guide (in terms of both context and tone). In addition, it is important to have review sessions with translators to ensure that their approach to asking questions is appropriate. Finally, the team should be flexible enough to recruit a wide range of respondents and change its recruiting approach based on what is emerging as effective.
Making time and space for clients to influence our log frames can surface new ideas for how to serve them better – the aspirations of our interviewees have clear implications for which financial solutions might be most effective. Listening to clients can reveal which outcomes we should measure, and which would be a waste of effort. Just as importantly, it puts us in the position to hear a story that could change the way we think about our work.
Defining impact and developing measurement frameworks often take place in a conference room at a hotel in the capital city, where no clients are present. We seem to know the impact we are trying to achieve without asking clients what they care about. While spending hours talking to individual farmers is time-consuming and does not always reveal earth-shattering insights, can we fully understand our impact if we don’t know how our clients would define success?