Designing dependable field research in complex operating environments

Field research can fail long before an analyst opens a dataset. A broad question, unrealistic sample, unclear translation, rushed training, unsafe interview setting, weak supervision, or excessive data collection can each undermine the evidence.

Complex operating environments make these weaknesses more consequential. Access may change quickly. Populations may move. Connectivity may be unreliable. Administrative lists may be incomplete. Language can alter meaning. Local power can shape who participates and what they feel safe to say.

Dependable research does not come from one quality-control checklist. It comes from a connected design in which the decision, question, ethics, sampling, operations, collection, assurance, analysis, and use reinforce each other.

Begin with the operational decision

The first task is to identify the decision the research must support. Who will use the finding? What choice will they make? When is the answer needed? What would change if the evidence points in different directions?

This discipline reduces unnecessary questions. It also helps distinguish information that is useful from information that is merely available to collect.

The UNHCR guidance on data and information management links information work to operational needs, gaps, risks, and response priorities. That connection should be explicit in a study design. A research team should be able to map each main question to an intended decision or learning use.

Build an evidence map before a questionnaire

An evidence map lists the study questions, concepts, indicators or themes, possible sources, required comparisons, and intended analysis. It exposes gaps that a questionnaire can hide.

For each question, ask:

• Is primary data necessary?
• Which people or records can answer credibly?
• Could asking create risk or burden?
• What comparison is required?
• What kind of claim will the evidence support?
• What limitation is already predictable?

Some questions are best answered through existing records. Some need key informants. Some require direct observation. Some require private individual interviews. Some should not be asked at all if there is no safe response pathway or clear use.

Design for access without letting convenience define the population

Field access affects who can be reached. It must not silently redefine who the study claims to represent.

If remote locations, mobile populations, insecurity, weather, permissions, or transport make parts of the intended population inaccessible, document the difference between the intended population and the reached sample. Do not describe an accessible subset as if it represented everyone.

A sampling plan should state:

• The population to which findings are intended to apply
• The available sampling frame and its weaknesses
• The selection stages and replacement rules
• Required subgroup comparisons
• Expected non-response and inaccessibility
• How weights or design effects will be handled, if relevant
• Which claims the final sample can and cannot support

Where probability sampling is not possible, explain the alternative and its inference limits. Precision language should never outrun the design.

Treat context as part of the method

Context is not a paragraph at the beginning of a report. It affects question meaning, participant safety, interviewer identity, timing, location, transport, device use, group composition, and interpretation.

Local review can identify issues that a central technical team may miss. These include terms that do not translate cleanly, topics that require privacy, locations where participation could be observed, social relationships that affect disclosure, seasonal routines, community entry expectations, and referral or escalation needs.

Contextual adaptation does not mean abandoning comparability. It means identifying which elements must remain consistent and which must change for questions to retain their intended meaning.

Put ethics inside the design

Ethics is not completed by adding a consent script. It shapes whether a question should be asked, who should ask it, where, how answers are stored, who can access them, and what happens if risk or distress is disclosed.

The UNICEF Innocenti approach to ethical evidence generation places ethical thinking across research, evaluation, and data work. The practical implication is that ethics should be revisited as the study changes, not treated as a single approval event.

Key decisions include:

• Whether participation is genuinely voluntary
• Whether the information provided to participants is understandable
• Whether power relationships could create pressure
• Whether privacy is possible in the collection setting
• Whether sensitive questions have a necessary and beneficial use
• Whether the team has a response pathway for disclosures or distress
• Whether incentives are appropriate and non-coercive
• Whether exclusion or accessibility barriers have been addressed

Do-no-harm also applies to findings. A table, quote, map, or small subgroup can expose people even when names are removed.

Collect less data, with clearer responsibility

The OCHA Data Responsibility Guidelines frame data responsibility around the safe, ethical, and effective management of data in humanitarian action. The ICRC Handbook on Data Protection in Humanitarian Action provides further guidance for handling personal data in humanitarian settings.

Together, they support a simple rule: collect only what has a defined purpose and can be protected through its life cycle.

Before collection, define:

• The purpose of each sensitive field
• The lawful and ethical basis for handling it
• Who owns or controls the data
• Who can access raw and processed files
• Where data will be stored and transferred
• How identifiers will be separated or removed
• How long each file will be retained
• How incidents will be escalated
• What will be shared, with whom, and at what level of detail

Avoid gathering names, precise locations, identity numbers, detailed protection information, or health information merely because a tool permits it. Data minimization reduces participant risk and operational burden.

Make tools usable under real field conditions

A technically correct tool can still perform poorly in practice. Questions may be too long, options may overlap, recall periods may be unrealistic, translations may alter concepts, or skip patterns may confuse users.

Tool review should test:

• Connection to the research question
• Plain-language comprehension
• Translation and back-checking of meaning
• Cultural and contextual appropriateness
• Sensitivity and interview privacy
• Response categories and units
• Recall demands
• Interview length and participant burden
• Skip logic and required fields
• Offline function, battery use, and device handling
• Safe handling of notes, audio, or location data

A pilot should reproduce likely field conditions. It should test the full workflow, including introductions, consent, selection, interview flow, synchronization, supervisor review, corrections, and escalation. A pilot that only checks whether the form opens is not enough.

Train for judgement, not memorization

Field teams need more than a presentation about the questionnaire. They need to understand the purpose of the study, what each question is trying to measure, how to remain neutral, how selection works, how to protect privacy, and when to stop or escalate.

Effective preparation uses demonstration, paired practice, observation, correction, and repeat practice. It includes difficult situations such as an unavailable selected respondent, an audience gathering, contradictory answers, a request for assistance, distress, device failure, or a supervisor asking for an undocumented shortcut.

Readiness should be observed. Attendance does not establish competence.

Supervisors also need a defined role. They should know what to observe, which checks to run, which problems they can correct, which require technical review, and how to document decisions without concealing errors.

Design quality assurance before deployment

Quality assurance should be proportionate to the study and active from the first day. Waiting until collection ends leaves too few options for correction.

A field quality plan may include:

• Direct observation of interviews or assessments
• Daily completeness and consistency review
• Duration and timestamp review used with care
• Location review only where justified and safe
• Callbacks or back-checks where appropriate
• Review of open-text responses and other signals of comprehension
• Sample progression and non-response monitoring
• Structured supervisor debriefs
• Version control for tools and guidance
• A documented issue and decision log

Automated flags are prompts for review, not proof of misconduct. A very short interview, repeated answer pattern, or unusual location can have several explanations. Investigation should be fair and documented.

Quality monitoring can also create risk. Excessive location tracking, audio capture, or staff surveillance should not be justified in the name of assurance without necessity, transparency, and safeguards.

Protect the chain from collection to analysis

Data cleaning is part of evidence production. Decisions to recode, exclude, impute, merge, or correct can materially affect findings.

Maintain a clear record of:

• Original received data
• Tool and variable versions
• Cleaning rules
• Material corrections and their reasons
• Derived variables
• Exclusions and missingness
• Weighting or adjustment procedures
• Qualitative coding decisions
• Links between tables, figures, and analytical files

Analysis should follow the evidence map and planned comparisons, while allowing careful investigation of unexpected patterns. Disaggregation should be meaningful and safe. Small groups may need suppression or aggregation to protect identity and prevent unstable claims.

Triangulation means more than finding two sources that agree. It asks why sources converge or differ, whether they measure the same concept, whose perspective each represents, and how timing or method affects the comparison.

Report limitations as decision information

Limitations are not a ritual paragraph. They tell users how much weight to place on a finding.

State clearly when access excluded locations, when a frame was incomplete, when a question had high non-response, when social desirability may affect answers, when administrative records use different definitions, or when timing limits comparison.

Then explain the implication. Does the limitation affect a subgroup, a single indicator, the estimated magnitude, or the direction of the conclusion? Can another source reduce the uncertainty? Does the decision require further inquiry?

Credibility grows when evidence boundaries are visible.

Plan use before the report

Dependable research is not complete when the dataset is clean. It must reach the people who need to interpret and act on it.

Before fieldwork, identify:

• Primary and secondary users
• Decision points and deadlines
• Required levels of detail
• Sensitive findings and safe communication
• Validation or interpretation needs
• Output formats for technical and non-technical users
• Responsibilities for response

This may lead to several outputs: a technical report, decision brief, verified dataset, GIS product, presentation, workshop, or action tracker. Each should preserve the evidence and its limits.

A dependable field-research test

Before deployment, a commissioner should be able to answer yes to the following:

• The intended decision and users are explicit
• Every major question has a defined use
• The sample and access plan support the intended claims
• Local context has shaped tools and operations
• Ethical and safeguarding responsibilities are assigned
• Sensitive data have a clear purpose and protection plan
• Tools have been piloted under realistic conditions
• Field-team readiness has been observed
• Supervisors have clear quality and escalation procedures
• Cleaning and analysis decisions will be traceable
• Limitations will be communicated with their implications
• Findings have a planned route into decisions

If several answers are no, adding more questions or a larger sample will not solve the underlying problem. The design needs attention first.

Further reading

• UNHCR: Data and Information Management
• OCHA: Data Responsibility Guidelines
• UNICEF Innocenti: Ethical Evidence Generation
• ICRC: Handbook on Data Protection in Humanitarian Action

Closing call to action

Heading: Planning research in a complex operating environment?

Body: ERC can help connect the decision, field design, quality controls, ethical responsibilities, and intended use before collection begins.

Action: Discuss a field research need