fbpx
Categories
Impact Innovation

The Role of Predictive Analytics in Digital Advisory (Part 2)

By Osman Siddiqi

 

Our previous post on predictive analytics shared the incredible story of Zachary Ndegwa about how he used Arifu to multiply his income streams over the years. This story makes us ask every day how we can make Arifu a lifelong learning companion for people particularly to inform day-to-day decisions at scale. 

To achieve this vision, it becomes important to understand how to credibly and frequently know if an Arifu learner is learning, applying knowledge, and how to respond with information or service linkages across time, space, and needs in order to build the kind of agency, decision-making, and growth Zachary exhibited with a little help from Arifu. 

This is not straightforward work. But we have some serious momentum.

Arifu’s Theory of Change is complicated. It’s complicated because we’re sector and problem-agnostic. What this means is that our operations cut across agriculture, financial services, health, employment, business growth, across countries, learner segments, and their intersections. Even so, two aspects of our Theory of Change drive our strategy. First, is a deep focus on us being able to inform day to day decision-making and activities through accessible learning content. The second is about leveraging market forces from the private sector, government, and more to showcase our value-proposition to partners to drive their growth and spur further creation, iteration, and scaling of Arifu’s content. 

In 2020, however, we have laid strong foundations for honing in on these two big questions across multiple learner segments. This includes a recently concluded pilot study in partnership with AgriFin Accelerate and with researchers from the World Bank and Harvard Business School. This study showed linkages between our internal Learning Scores (our Knowledge Score and Skills Score) and with self-reported farm revenues and profits. It further showed linkages with learning retention.

Locations of all surveyed farmers for this study

Figure 1: Locations of all surveyed farmers for this study

 

The study categorized a random set of learners into four groups ranging from very low engagement with our content to very high engagement. Arifu’s backend Learning Scores were compared with findings from an in-person survey implemented with the World Bank. The survey asked learners about their demographics, farm-level outcomes, and replicated questions from within our content in person. The sample size in the pilot size was 153 learners. The learners had engaged with Arifu’s content at least three months before the survey was implemented. The study taught us two main things:

  1. Our Learning Scores are predictive of learning retention i.e. farmers’ responses to our quiz questions were highly correlated with their responses to questions in the content.
  2. Our Learning Scores were predictive of financial outcomes such as annual revenue and profits.

Importantly, the results showed strong associations between the Learning Scores and financial outcomes and resolved an open question on learning retention. Specifically, a one standard deviation increase in the Learning Score is predictive of an increase in annual revenue of approximately KSH 9,065 (USD 84) and in annual profits of around KSH 1865 (USD 17).

Figure 2: Arifu’s Learning Score associations with total revenue and profits. Given the low resolution of the image above, the metrics from top to bottom on the y-axis are Respondent’s earnings past 30 days, Total HH income past 30 days, Total revenue past year, Total profits past year.

 

The findings tell us the Arifu Learning Score has strong potential as a possible Credit Score. In addition, it is possible that the Learning Scores are good predictors of behavior change and adoption of practices, purchase of inputs, and more.

We have seen through several independent studies that providing actionable and relevant information can improve the quality of life of learners, show learners how to use products and services in beneficial and non-harmful ways, and for building resilience during lean periods. This includes a 60 Decibels implemented impact study on potato farmers which showed that between 47-71% of farmers saw their crop production improve and between 16-49% of farmers saw an improved ability to save attributable to Arifu [1]. 

So what’s next? We have launched and are part of over four large scale impact evaluations, including at least two randomized controlled trials, across three countries in Sub-Saharan Africa, covering merchants, farmers, and youth [2]. The projects will reach well over 600,000 learners to further inform the precision of our Learning Scores prediction strength towards behavior change, product and service utilization, generation of leads for partners, and understand whether outcomes may have improved for learners. 

We are continuously looking for new partnerships to refine our Learning Scores and to build research projects that can help inform and further our vision for how we can leverage predictive analytics, as one tool, to drive our impact and drive the goals of our partners. 

Footnotes
  1. A significant portion of farmers who responded to this survey had not seen improvements in their production since they had not harvested their crop yet. To unpack this we are conducting a follow-up survey one year since the original study.
  2. These projects are in partnership with Google.org, the World Bank, Government of Kenya, Mastercard’s Center for Inclusive Growth, and UC Berkeley among other renowned organizations.
Share on facebook
Facebook
Share on twitter
Twitter
Share on linkedin
LinkedIn

Useful Links

Subscribe

Join Our Mailing List and always be the first to know about what's going on with our organization.
© 2020 Arifu. All Rights Reserved
Categories
Impact Innovation

The role of predictive analytics in digital advisory

By Osman Siddiqi

It is difficult to speak about Arifu without bringing up Zachary Ndegwa. Zachary was once a farmer from a village called Luma in Meru county, Kenya. In 2015, when Zachary first interacted with Arifu on his feature phone, he had a quarter acre tomato farm, the yields from which were sole means for providing for his family. 

At first, Zachary thought Arifu’s free-to-use chatbot was a person. By chatting with Arifu, he began learning how to grow healthier tomatoes by choosing the right seeds, applying fertilizer correctly, and much more. After learning, Zachary decided to take a not so insignificant risk. He chose to trust and apply the advice he was receiving from these texts. Much to Zachary’s relief, his tomato production and his income increased. This realization continued to drive his engagement with the Arifu chatbot. 

In 2019, we revisited Zachary and found that he is no longer growing tomatoes himself. Instead, Zachary has trained other farmers using the practices he learned from Arifu and now buys tomatoes from those farmers to sell in the market. Both Zachary’s income and status in his community have grown and he attributes this change of fortune in his family’s life to his continuous interactions with Arifu. 

Enhancing skills at scale

Zachary’s story exemplifies the type of impact Arifu strives to have on the lives of all learners who interact with our platform. Arifu incorporates expert advice, insights from human-centered design research, and behavioral economics principles to create actionable educational content that builds learner capacity on a wide range of topics. In addition, through its interface, Arifu connects end-users to the ecosystem of products and services, such as financial services or inputs to agriculture. By delivering this content through multiple digital channels, including SMS, Arifu aims to bridge the digital divide by creating, curating, and delivering useful information for those without access. We aspire to make this information demand-driven, needs-based, timely and actionable in order to positively impact lives. 

Recent academic literature suggests that SMS-based extension services, particularly in the form of advice, if actionable and targeted well, can be quite effective in improving farm outcomes for the poor. In addition, insights from Arifu’s ongoing research suggests that learners who engage with content state they find the content useful and applying it has improved their quality of life. 

As of the end of 2019, Arifu has well over a million learners. As we continue to scale, it has become increasingly important to ensure that Arifu’s reach adds value to as many of our learners as possible. To drive impact, it is simply not sufficient to create content and deliver it en masse. Nor is it sufficient to have high engagement levels with the deployed content. Impact requires solving real challenges learners face by continuously measuring the right things and innovating in order to improve. To achieve this it is important to measure learning accurately and being able to understand whether learning is resulting in meaningful behavior change. One tool we have developed to tackle this challenge is an algorithm we have dubbed the Skills Score. 

The Skills Score measures learning and the ability of learners to apply the content they engaged with. The three key components which make up the Skills Score are knowledge graphs which map how concepts relate to one other, quizzes to assess and reinforce learning for learners, and increasingly complex questions within those quizzes that measure the ability of learners to use what they learned to, for example, evaluate situations and create solutions. We believe these questions should allow us to not just measure knowledge improvements, but to also predict how well learners are able to apply the content they engaged with. The Skills Score should add value in (at least) the following ways:

  1. Identify where learners have skill gaps allowing us to personalize trainings to help improve learning at the right level for the right content
  2. Identify how effective Arifu trainings are at building skills and how to improve them. 
  3. Enhance the quality of engagement with learners through the provision of increasingly timely and relevant content to enable them to make more informed decisions.
  4. Understand and drive predictive power of content engagement, Skills Scores, and meaningful behavior change.
  5. Communicate which skills have been learned by learners to our partner organizations so they can make more informed strategic decisions.

Ultimately the Skills Score can be used to enhance Arifu’s ability as a digital coach operating at massive scale. It promises to enhance the cost-efficiency, dynamism, and impact of delivering trainings on demanded and needed information to learners globally thus enabling Arifu to fulfill its mission in significant ways.

Our commitment to impact

The global digital divide is still a long way away in creating access to the treasure trove of publicly available information many of us take for granted. Even so, it is not easy to identify information that is immediately relevant for driving our own capabilities. This is the gap Arifu seeks to bridge. Our research strategy is predicated upon a theory of change aiming to bridge these gaps in knowledge and improve the chances of social mobility for households. We know there are many Zacharys among our learners. His story speaks to the need for tools and systems that are right fit solutions and enable people to drive change using newfound knowledge and utilizing immediate resources available in their ecosystem.

While theories, anecdotes, and visions are necessary, they alone are not sufficient. Learning, iteration, creativity, and credible proof points are needed. Arifu has committed to hold itself accountable to the goal of impacting lives in meaningful ways. We will pursue this by leveraging the suite of research tools that have been developed in the development and technology sectors. We believe the Skills Score is one tool which will help us serve people with what they need, when they need it, as we learn about what their capabilities are and what they could become. 

Share on facebook
SHARE
Share on twitter
SHARE
Share on linkedin
SHARE

Useful Links

Subscribe

Join Our Mailing List and always be the first to know about what's going on with our organization.
© 2020 Arifu. All Rights Reserved
Categories
Innovation Life At Arifu

Introduction to Arifu

When we first launched our digital learning platform, Arifu, in 2015, we still had many questions about how our product should be designed to address the deep information gaps that exist globally, preventing billions from accessing relevant, affordable training and information to improve their quality of life. 

One of those questions was, “will learners engage differently with our product  if Arifu is introduced as a person versus as a platform?”. We saw major tech companies around the world asking similar questions and investing in their conversational AI personalities — Google Assistant, Amazon Alexa, Apple’s Siri, and so on — to make access to information more natural, efficient, and friendly and usher in an entirely new era of digital services. Might the same approach yield results for mass-market audiences in emerging and frontier economies including the billions who have access to phones but not the internet? 

The answer, it turned out, was “yes”. Our research with CGAP, Vodacom, and TechnoServe delivering financial education to rural Tanzanian audiences over interactive SMS showed that the content introducing Arifu as a person had between 15% to 25% higher interactions than Arifu as a platform. Subsequent independent research showed upwards of 400-500% increase in contributions to savings among farmers in Tanzania. With that, the Arifu platform became an interactive chatbot with one mission: to place information and opportunity in reach of everyone, one chat at a time. 

 

As part of that mission, we are excited to launch the Arifu blog, a forum that allows us at Arifu to share what we are learning with a variety of audiences interested in the application of basic and advanced technology for social impact and shared prosperity. 

 

Our upcoming posts will be categorized around the four intersecting topics of Innovation, Impact, Partnerships, and Life at Arifu.  Each category speaks to essential aspects of Arifu’s engagement with our stakeholders from our learners, team members, corporate and government partnerships, and the wider ecosystem. The inaugural post on our blog is by Arifu’s Director of Research & Impact, Osman Siddiqi, on the promising role predictive analytics can play in digital advisory services for improving livelihoods. The post particularly speaks to Arifu’s Skills Score, a set of algorithms for dynamically measuring learning and predicting behavior change among learners.  

 

We hope you enjoy learning with us and invite you to share your thoughts on our blogs. Please reach out at info@arifu.com if you would like to collaborate and let’s keep learning together. We look forward to keeping the conversation going.

 

Marisa Conway & Craig Heintzman,

Chief Learning Officer, Chief Executive Officer, and Co-Founders of Arifu.

 

Useful Links

Subscribe

Join Our Mailing List and always be the first to know about what's going on with our organization.
© 2020 Arifu. All Rights Reserved