Triangulate for data reliability and validity. In order to enhance data reliability and validity designers should triangulate regarding discipline, investigators, data, theory / methodology, tools / techniques. CDD prescribes the use of multiple data sources, methods, tools and techniques, and prerequisites B, C and F ensure discipline, investigator and data triangulation. This prerequisite is added to stress the importance of these different types of triangulation.
Multidisciplinary team. Designers from multiple disciplines should be included in the design team: they should have different backgrounds, skills and knowledge. This leads to a balanced perspective and access to a range of participants. Thereby, when team members conduct activities - in pairs – at the same time, the process of user context exploration is sped up.
Establish local partnerships. Local partners are required in order to adjust quickly to the local circumstances, obtain information about the potential users and community structures, get advice on activities, help figuring out what to do, be properly introduced in the community, help provide access to an unbiased selection of participants, build trust in communities, and to be properly introduced to the local people. They can also aid in selecting participants and finding translators. A community partner should be someone who understands local things and is respected by the people.
Get the team, client and translator on board. It is important for all those involved to see the relevance of the CDD approach to ensure reliable, rigorous data collection.
Follow qualitative research and ethics training. In order to conduct sound, rigorous research that does not invade people’s private lives in an incompetent way, and which results in valuable data, designers should have a solid and broad understanding of doing good research in the field. The research should be executed in a systematic, sceptical, ethical and rigorous manner. Designers should not conduct extractive research, but ensure an interactive, participative process together with the potential end-users to their mutual benefit. Therefore designers need to follow a training in which they are taught the right attitude, behaviour and questioning skills, and during which they practice their learned skills and techniques. Designers should also continuously examine their attitude, behaviour and questioning when conducting user context research in order to improve upon them. Capability Driven Design contains a ‘training module’ that designers can use to learn about doing good research in the field. In addition, a card with the most important interviewing rules is added to the ODK toolkit. This module and card, however, do not replace practical training under guidance of an expert.
Learn the themes by heart. In order to obtain broad insight into all aspects that comprise a person’s life and context, the themes and topics are leading. The themes should therefore be learned by heart, in order to allow for quick changes in conversation topics and establishing a fluent dialogue in which participants truly open up. They also help to pay attention to a comprehensive set of aspects when observing potential users in their natural settings. It helps to study the themes and questions, to roleplay them and to pilot them in the field. The facilitator and / or note-taker can keep track of the themes and questions by using the question cards.
Plan for it. Conducting user context research takes time, especially in developing regions where ‘things do not always go as planned’, and often time is needed for travel, for establishing contacts to obtain access, and for acclimatisation to the local situation. It should not be a ‘side-activity’. Preparation takes time, conducting activities takes time, and data analysis and validation take time. Plan sufficient time to properly follow all the steps and to conduct rigorous user context research.
Select a variety of participants with different characteristics for a broad range of insights. Especially a variety in gender, ethnicity, social class, age, and religion are important to include. Do not only include potential users, but obtain a broader picture to learn more about task distributions and perceptions of the broader community. Be aware not to only select participants that are easy to access, as this results in bias. It is, however, not always possible to talk to an unbiased sample of participants, as some people are truly difficult or even impossible to reach. It often depends on the community partner what is possible.
Activities should be conducted in pairs and preferably be recorded. CDD already prescribes the use of multiple data sources, methods, tools and techniques, to improve data reliability and validity. To further improve outcomes, each activity should be conducted with multiple team members. By assigning one activity facilitator and one note taker, each of them can focus on their own specific task, while interpretations, experiences and perceptions can be compared afterwards, ensuring investigator triangulation and improved data reliability. A third person can be added to take photographs or shoot video (when consent is given), but more people can overwhelm participants. Activities should preferably be recorded, to enable the note-taker to focus on behaviour, body language and the environment. When it is not possible to conduct an activity with multiple team members present, for example when a situation with solely women needs to be created and there is only one female team member, the activity should be recorded to allow for the designer to focus on the activity and the participant and to enable other team members to listen back to the things being said. However, only when consent for recording is given by the participant.
Activities should be conducted in participants’ natural setting. Potential users should be directly observed and interacted with in their natural settings in order to improve learning and understanding by building a shared language, capturing detail, gather concrete data, develop empathy and reduce bias and rationalization, filtering and distortion of information. Preferably, the design team will be in the field throughout the design project, but if that is not possible, at least at the beginning, prior to problem definition, and during prototyping, in order to obtain feedback and make adjustments to the design.
Participatory, simple and fun activities. CDD stimulates the use of a variety of techniques and tools, which can be tweaked by the designers to better fit their purpose. For the ODK interview method, techniques and tools have been selected and defined, but can still be changed. When designers develop or adjust techniques and tools, it should be kept in mind that multiple techniques and tools should be used (prerequisite A), and that activities should be simple, engaging and interactive, in order to create an enabling atmosphere in which participants feel free to express themselves. It is advised to let participants perform tasks or to let them create things, to stimulate expression of latent and tacit needs and desires.
Use insights to inform the next activity. As newly obtained information leads to new understanding, research goals and methods should be changed accordingly to obtain additional information. The research outcomes should therefore be analysed by the team after each activity to adjust the activities based on new insights.
Discuss outcomes in a bigger group to improve their value. The information, knowledge and interpretations should be shared with participants to point out misunderstanding and to improve data validity. If participants agree, they should also be shared with the community and local partners to keep stakeholders involved, enhance transparency and openness and improve data reliability.
Critical reflection on limitations. The data obtained, the methods used, the researchers involved and the project executed all have limitations and the researchers should reflect on them and be open and honest about them. These limitations can depend on the following:
The facilitator’s quality, skills, behaviour, bias, subjectivity and terminology used;
The design team’s presence, biases, characteristics, agenda and perspective;
The participant’s character, motivation, interest, well-being, feelings, emotions, etiquette, availability of time, scepticism, distrust, suspicion, prior experiences, cultural background and values;
The setting of the interview, the audience present, gatekeepers present, disturbances and distractions from outside;
The translator’s presence, biases, skills, interest in and understanding of the project;
The amount of distortion due to translation;
- The presence of recording devices.