Data Readiness Toolkit

If your National Society or team has the ability and bandwidth to conduct a formal assessment, you can go use the Data Readiness Framework to identify your current strengths and gaps and the Data Readiness Theory of Change to identify a pathway toward your desired outcome.

How would this look?

1. Select your team

The first step is to identify the appropriate people to be involved in this diagnostic. Who should be included will depend on your goals, but generally, you’ll want to identify a team or department. For example, you can identify a team working on a specific program (such as cash or community health surveillance) or a team/department that regularly works with data (such as response or PMER).

2. Assess your team

With your team, go through the framework and rate yourselves from 1 (does not exist) to 4 (high performance). Use the descriptions to rate yourselves as objectively as possible. You may find you belong in two categories; if this happens, choose the level that represents your team most of the time. As you score yourselves, take notes on why you chose that level. This can help you decide which areas to prioritize and later on, shape your learning experience.

3. Identify which competencies to prioritize

Depending on your broader objectives, there may be critical data readiness competencies needed for your team to be successful. Review your results and objectives to identify which areas to prioritize. This stage takes a bit more thought and creativity — which competencies are key to the success of your program or department? Where is there the most room for improvement? In additional, you may want to consider your National Society’s strategic priorities, funding opportunities, and any time constraints.

How would you measure progress?

After conducting the initial assessment and developing your action plan in your priority areas, your team can decide the frequency upon which you would like to review your progress. We recommend that progress reviews be conducted at least quarterly to allow time for course correction if needed, and to maintain the momentum of the data readiness reviews. After each data readiness review meeting, your team should decide on actionable goals that you plan to work towards before the next meeting. Bottlenecks or blockers should also be discussed in each data readiness review meeting, and where possible, an action plan should be developed to overcome those blockers.

National Societies are often tasked with conducting various organizational assessments on top of their day-to-day work of preparing for and responding to disasters. This leads to “assessment fatigue” and lack of motivation and buy-in to conduct a rigorous and worthwhile assessment. We may be able to improve interest in the data readiness framework by starting the conversation at more convenient times: integrating or consolidating the data readiness maturity elements into other assessments that are being or are scheduled to be conducted within a National Society, such as the Cash and Voucher Assistance Preparedness (CVAP) assessment, Preparedness for Effective Response (PER) assessment, and the Digital Transformation Assessment.

If your National Society will be undertaking any of the below organizational assessments, click the links below to learn how to integrate a data readiness review into those assessments.

Cash and Voucher Assistance Preparedness (CVAP) Assessment

Preparedness for Effective Response

We may be able to apply the Data Readiness Theory of Change during the program planning stage, in order to help National Societies answer questions around data readiness and design more data-driven activities from the beginning. This method would encourage National Society teams to think about their unique role in the project’s overall data flow, and support their engagement with that data flow as the project moves forward. It would also help teams to identify their data challenges from the start of the project so they can work to address them or seek external support during the preparedness phase, instead of having data challenges pop up during operations or other activity implementation.

How would this look?

First, we would take the Data Readiness Theory of Change and reframe each box so that it is phrased as a planning question. This approach would give project design teams the right questions to think about as they are designing their project or activity. For example:

1.1 Data culture (current statement in the Data Readiness Theory of Change): National Societies recognize the value and importance of data in their work and actively promote data skills and use.

1.1 Data culture (rephrased as planning question): How will we promote a culture of data use in this project? (Hold monthly data review meetings, regular data skills training opportunities for any interested staff, post infographic posters with program data around the office, etc.)

The next step would be to go through the Data Readiness Framework with the project design team to identify two things: 1) at which tier do they feel they are currently placed at this beginning of this project/activity, and 2) what is the related tier that they want to reach by the end of the project? It is not expected that teams would find every data readiness element relevant to their project/activity, and that is okay! The important thing is that teams select a few data readiness elements that are within their reach, and think about how improving those data readiness elements would improve aspects of their project.

How would we measure progress?

In order to measure progress on their selected data readiness elements, we would convert those elements into data readiness indicators and incorporate them into the M&E plan that is developed at the start of the project. An example of how the data readiness indicator could be incorporated in an M&E plan is as follows:

3.3.35. Standardized format for reporting: “Currently we do not have any mechanisms for reporting (Tier 1). However, by the end of Y1Q3 we will have developed, iterated, and finalized a reporting template for quarterly reports to our donor/
other audience.”

The data readiness indicators should be then tracked along with the other project indicators.

Instead of administering the data readiness framework in the form of an organizational assessment, an abridged version of the framework could be administered to IM/IT/other relevant team members on a periodic basis. The assessment could be presented as a user-friendly, anonymous web form that would only take a few minutes to complete. Then, data could be monitored to look for areas of improvement or persistent challenges.

How would this look?

Develop a short survey that is focused on the data readiness priority areas that your organization has identified. Make sure to include a survey question about which team/department the user works on so that you can aggregate the results later by team/department. You could frame the survey question options to identify which tier the user thinks is most representative of their team or project. For example:

How does your team backup your data? Choose the option that is most accurate for your team or project:

a. Most of the data resides in its original form and there is not a backup version

b. There are copies made of important documents (e.g. copies on a USB drive or duplicate copies)

c. There are backup systems in place, or data is regularly copied and stored in multiple locations

d. All data is linked to a cloud account and is backed up live

Once you have received the results from the survey, you can aggregate the results by team/department to get a sense of the data readiness gaps and strengths in that team/department.

How would we measure progress?

Administer this survey quarterly with the same questions to identify progress made for each of the data readiness priority areas. Pay attention to data readiness priority areas where no progress is made over multiple quarters. This may be an area that should be escalated for support or prioritized in the team’s action plan. Because the survey is anonymous, you may find more insight into data readiness problems or challenges that people may be less willing to discuss in an open forum.

Instead of administering the data readiness framework in the form of an organizational assessment, participants from different units within a National Society could come together for a focus group discussion around a certain outcome area. Thoughtful facilitation of this discussion could produce some valuable, actionable insights into what your team members feel about the state of data readiness on your team or project.

How would this look?

In a focus group discussion around data culture, for example, participants could be asked questions that are framed around the outputs such as:

Resource allocation: How frequently does leadership request time or resources for data initiatives? How does leadership use data or IM products?

Leadership buy-in: How does leadership connect data to your National Society’s work? Can you think of any examples where leadership demonstrated that they see value in using data? How do you use data on your
project/team?

Staff engagement: How do you access technical support if you have questions about data? Are you a part of any working groups or informal chat groups outside of this National Society related to data? Who do you ask
for help for data-related questions?

Make sure to use best practices for guiding the focus group discussion, such as asking open-ended questions and prompting for more information when possible, avoiding leading questions, and creating a comfortable environment where participants are encouraged to speak freely. Also make sure to have a notetaker or recording device so you capture the group’s key insights! Data from the focus group discussions should be analyzed, visualized, and presented to decision makers at your organization, with actionable areas and learning and development suggestions highlighted.

How would we measure progress?

Following the focus group discussion, you should check in regularly with the focus group participants to get a sense of what action items and learning and development suggestions are moving forward. If activities are not moving forward, seek to engage decision-makers within the National Society to better understand hesitancies, resource gaps, or other barriers. Focus group discussions can be repeated and more targeted to those hesitancies, resource gaps, and other barriers, in order to try to brainstorm solutions with a diverse group of participants. The important thing is to keep the conversation moving forward, and qualitative evidence is a great way to understand needs from those data users with whom you might not normally work.

When used as an organizational assessment/framework, it may be overwhelming for a National Society to comprehend how improved data readiness can strengthen the organization as a whole. If we were to contextualize how improved data readiness can lead to improved response operations, using specific examples from a relevant technical area, the benefit of applying the data readiness framework may become more clear. The goal of this approach would be to demonstrate value and relevance of data readiness for operations, understanding data challenges, and gaining buy-in across a project team.

How would this look?

The first step would be to select a recently completed (or recovery phase) operation or activity that the National Society was involved in. Instead of the original data readiness framework, the National Society team would be referring to an adapted version of the data readiness framework that uses language, examples, and reflection questions relevant for the technical type of operation that they completed. The revised examples would be worded in a way that would encourage new thinking about how that element of data readiness would support/improve that operation. For example:

Data Readiness Framework 2.3.20: Sampling

For a full version of the Data Readiness Framework adapted for a cash program, click here.

Then the project team would be asked some reflection prompts about how improved data readiness may have strengthened the operation, for example:

* Using that sampling approach, was the data we obtained representative of the community?

* Using that sampling approach, did we make efficient use of our resources?

* What kind of sampling approach would we use next time?

* Do we have someone on our team who can develop a sampling approach?

Once this exercise is completed, teams will identify the “wishlist” of data readiness activities that they feel would improve their next operation. From there, data readiness activities or links to learning and development would be identified that meet the team’s specific operational improvement priorities.

How would we measure progress?

Progress would be measured by recording which data readiness activities or learning and development opportunities were conducted by the National Society and continuing to promote growth and advancement in the priority areas they identified through the reflection exercise. After the next operation, the National Society team would again complete the reflection exercise and see where improvements were made. Through a continuous process of reflection, learning, and adapting, the NS team will start to see improvements in the effectiveness and efficiency of their operations.

A data readiness simulation exercise can be conducted in conjunction with other disaster preparedness or CVA program simulations that your team is conducting. During the course of the simulation, pay attention to the data needs at each phase of the operation. Incorporate the data readiness framework into your simulation guides so that your team can understand their current data needs, gaps, and strengths. If a certain element of the data readiness framework is not present, or not satisfactory, during the simulation, it will not be present or satisfactory during a real-life operation! The simulation allows your team to identify data readiness gaps when there is still time to correct them.

How would this look?

There will be certain touch points within your simulated operation that you will have to collect, manage, analyze, visualize, or use data for decision making. Refer to the IFRC GO Platform Operational Toolbox, which links you with useful documents to support each stage of the operation. Start with the Assessment Phase. Clicking on the link next to Assessment in the Operational Toolkit will take you to a resource housing templates, tools, and guidance for conducting assessments. However, the data readiness framework can aid you in identifying which competencies may be missing from your team that you need in order to be able to conduct assessments. For example:

Data Readiness Framework 2.2.13 Secondary data sets

Review the different tiers associated with secondary data sets. Does your team have any secondary data sets collected that could support you with the type of assessment you want to conduct? If not, you may be at a Tier 0 level for the secondary data sets competency. What if you have some data sets, but they are not relevant to this type of assessment, or perhaps they are outdated. In this case you may be at a Tier 2. If you have some secondary data sets that could support the type of assessment you want to conduct, and they are current and accurate, you are probably at a Tier 2 or 3 for secondary data sets, and would be ready to use secondary data during an actual disaster operation.

As you are going through the simulation, keep track of the areas of the data readiness framework where your team does not feel data-ready. Then, incorporate those elements into an action plan, so that you will be data-ready by the time an actual disaster takes place. The data readiness action plan that you develop during the simulation should help you identify the highest priority areas to work on before an actual disaster takes place. If you are doing a CVA program simulation and you realize that your team does not have any data management tools in place that you can use to collect data from the affected population, that is a high priority area to work on before you do an actual CVA program!

How would we measure progress?

Your action plan should include a timeline by when you hope to achieve your desired tier, as well as a focal person who can monitor and report on your progress. Since a disaster can occur at any time, focus on your highest priority data readiness areas. In periods of “blue-skies”, you can expand your action plan to include your organization’s longer-term goals and priorities. You can also incorporate these data readiness action items into a larger organization readiness program, and monitor progress alongside your other readiness indicators.