Not Just a Phone Call Away

Not Just a Call Away

Reaching women through phone surveys in India and Nepal

Lessons from survey of

Gujarat, India

627 female respondents

Lessons from surveys of over

BACKGROUND

In May and June 2020, IFPRI interviewed 627 women in Gujarat, India, to assess the impact of COVID-19 on their livelihoods under a study funded by the German Federal Ministry for Economic Cooperation and Development (BMZ). The Self-Employed Women’s Association (SEWA) provided telephone numbers for 860 of its members in 10 districts of Gujarat. For various reasons, only 73% of the identified women could be reached.

Between May and August 2020, IFPRI conducted a two-round survey of 2,689 maize farmers in four municipalities of Dang, Nepal, to assess the impact of COVID-19 on their farming activities and pest awareness under the Cereal Systems Initiative for South Asia (CSISA). Earlier, in March, we had conducted a household listing exercise, during which we visited respondents at their doorsteps and 3,731 farmers provided their telephone numbers. For various reasons, we managed to reach only about 72% of the numbers collected. Of those we spoke to, 71% were women and 29% were men.

71% female

29% male

What lowered response rates?

Incorrect/invalid phone numbers: In Round 1, 14% of numbers either did not work or belonged to a different house, even though they came from a list of pre-confirmed/verified numbers provided by SEWA. In Round 2, only 3% of respondents were lost to this challenge. Refusal to participate: In Round 1, 6% of contacts were reluctant to participate, despite having been informed by SEWA to expect the call. This could be because women are sometimes hesitant to take part in phone surveys due to lack of familiarity with enumerators, fear of judgement, family pressure, or distrust, among other reasons. In Round 2, only 3% of respondents were lost for such reasons. No response: Attempts were made to reach every contact over a period of five to six days. Still, 3% of calls went unanswered or were rescheduled multiple times (N=16) or were prevented by network (N=5) or other issues (N=6). In the second round, 2% of respondents were lost for these reasons.

Incorrect/invalid phone numbers: In Round 1 of the survey, 8% of phone numbers either did not work or belonged to a different house, even though the intended respondents themselves had provided the numbers in a physical listing exercise. In Round 2, that percentage fell to just 1%. Refusal to participate: Over 3% of those contacted refused to participate in Round 1, and 2% refused in Round 2. No response: In both survey rounds, we made seven call attempts over a period of three to four days. Yet 17% of calls in Round 1 and 11% in Round 2 remained unanswered.

Major Considerations

Best Time to Call

In both survey rounds, more than 63% of successful calls were completed between 8 a.m. and 2 p.m. For farmers, the ideal time depends on the crop season. We moved our call times when paddy transplantation began, even surveying respondents at night when requested.

65% of successful calls were conducted during the afternoon and early evening (between 12 p.m. and 6 p.m.). In Round 2, this figure was 58%. The call time was linked to the respondent’s occupation.

Round 1

Round 2

Post-noon

Morning

Post-noon

Morning

Evening

Morning

Evening

Round 1

Round 2

0.15

0.10

0.05

Time during the day (00:00 hours)

0.15

0.10

0.05

Time during the day (00:00 hours)

Density

0.15

Density

0.10

Density

2,600 respondents

Dang, Nepal

As the health and economic impacts of COVID-19 continue to evolve, researchers have been using new and creative methods of data collection. How well do phone surveys perform in terms of response rates, response bias, and data quality—particularly for women respondents? IFPRI conducted two multi-round phone surveys in India and Nepal in 2020. In addition to the findings themselves, we gained a better understanding of the strengths and difficulties of this method and drew important lessons for future phone surveys.

Best Time to Call

Assessing Respondent's Privacy

Familiarity with Enumerators

Other Notable Issues

Training and Technical Requirements

Mobile Recharge: A Luxury

3% of identified respondents in Round 1 and 2% in Round 2 could not be reached because their phones were either turned off or disconnected due to insufficient mobile credit. 102 women (or 55% of the 184 who were asked this question) stated that they spent less on recharging their mobile phones to help offset income losses due to the pandemic. We asked this question in Round 2 only, based on our experience in Round 1. Many respondents provided the phone numbers of friends or neighbors, either because they did not own a phone or because they did not want to share their own number.

Mobile Recharging: A Luxury

Speakerphone use: 24% of respondents had their phone on speaker mode during the calls. 22% of men and 24% of women had their speaker mode on; a t-test revealed no significant difference between the two groups. The survey included a knowledge-based test. If the speakerphone was on at the time, it is possible that others present could have helped respondents complete the test. Given the privacy concerns associated with having phones on speaker mode, and the possibility that responses could be affected, it is important to find out if phones are on speaker.

Speakerphone use: 65% of respondents put their phones on speaker mode during Round 1. 53% of respondents did so in Round 2. To avoid discomfort and risk of harm to respondents, sensitive questions were skipped when speaker mode was on. Sensitive questions were moved to the end of the interview to avoid disrupting the rest of the survey. Given the privacy concerns associated with having phones on speaker mode, and the possibility that responses could be affected, it is important to find out if phones are on speaker.

Assessing Respondents' Privacy

Respondents were familiar with the survey from the listing exercise. This helped enumerators build rapport with respondents. It is advisable to have the same enumerator contact the same respondent across survey rounds to strengthen rapport and maintain response rates.

Female enumerators were hired to help put female respondents at ease. Enumerators were affiliated with SEWA, which helped them to build rapport with respondents. In this multi-round survey, the same enumerators contacted the same respondents across rounds to strengthen the relationships formed.

Familiarity with Enumerators

Hearing issues: If a respondent was hard of hearing, a family member sometimes had to answer certain questions or repeat the questions to the respondent. Language issues: Because of regional dialects and variances in spoken language, questions often had to be restated, increasing survey time. Frequent call drops: Technical glitches made it difficult to maintain survey continuity.

Hearing issues: If the respondent was hard of hearing, a family member sometimes had to answer certain questions or repeat the questions to the respondent. Language issues: Because of regional dialects and variances in spoken language, questions often had to be restated, increasing survey time. Frequent call drops: Technical glitches and poor network connectivity made it difficult to maintain survey continuity for some locations.

Other Issues

Computer-assisted telephone interviewing (CATI) procedures and tools require a solid understanding of call-management practices. To ensure enumerators’ understanding of survey methods, it is important to hold multiple training sessions for them. Enumerators should take photos of handwritten call and survey updates every evening if they do not have MS Excel. Tools like KoBo Toolbox that do not strictly require laptops are useful for this.

Data collection procedures and tools using computer-assisted telephone interviewing (CATI) require an understanding of call-management procedures. Given that all facets of conducting surveys—including training survey teams and holding discussions and follow-ups—take place virtually, enumerators should be tech-savvy. To ensure enumerators’ understanding of survey goals and methods, it is important to hold multiple training sessions. To overcome any technological barriers, manual notes should be kept in the first days of data collection and issues discussed with the team and supervisors to ensure that challenges are quickly identified and addressed. If laptops are not available, KoBo Toolbox or Whatsapp can be used for sharing and discussing challenges.

Training and Technical Requirements

14% respondents could not be reached in round 2, reducing the sample size to 2,305.

10% of respondents could not be reached in round 2, lowering the sample size to 567.

Attrition May Persist

Gujarat, India

Dang, Nepal

Gujarat, India

Dang, Nepal

Gujarat, India

Dang, Nepal

Gujarat, India

Dang, Nepal

Gujarat, India

Dang, Nepal

Round 2

Round 1

Attrition May Persist

Gujarat, India

Dang, Nepal

Best Time to Call

Major Considerations

Mobile Recharge: A Luxury

Assessing Respondents' Privacy

Familiarity with Enumerators

Training and Technical Requirements

Attrition May Persist

Other Notable Issues

As we became aware of the issues highlighted here, the attrition rate fell but still persisted. Attrition patterns are specific to the study population. Incentives to participate: To increase response rates, food mini-kits in Gujarat and a phone recharge (top-up) in Dang were provided as incentives.

Click the icons to learn more

Shweta Gupta and Smriti Saini, International Food Policy Research Institute

Not Just a Call Away

Shweta Gupta Research Analyst Smriti Saini Research Analyst

This work received financial support from the German Federal Ministry for Economic Cooperation and Development (BMZ) commissioned and administered through the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) Fund for International Agricultural Research (FIA), grant number: 81235251, with additional support from the CGIAR Research Program on Policies, Institutions, and Markets, the USAID supported Gender, Climate and Nutrition Integration Initiative (GCAN), the USAID supported Cereal Systems Initiative for South Asia (CSISA) and the CGIAR GENDER Platform.

0.05

0.15

0.10

0.05

Post-noon

Morning

Time during the day (00:00 hours)

0.15

0.10

Time during the day (00:00 hours)

Density