TalkingPoints offers embedded translation to allow parents to communicate with teachers in their home languages.
The Challenge
How might TalkingPoints increase parental engagement in learning at home?
How might TalkingPoints increase parental engagement in learning at home?
My Role
|
My Team
|
Key Questions
What are the mental models of the parents for their role in their children’s education?
How do we segment the parents?
What is the best alignment between parents needs and what TalkingPoints could offer?
How do we segment the parents?
What is the best alignment between parents needs and what TalkingPoints could offer?
Initial Research Plan
My focus at the beginning of the research project is to reach parents and build empathy. The company has done user research on teachers in the past but not with parents. I am starting the project with a carte blanche. The main goals for this phase are establishing an understanding of parents' mental models including expectations and concerns, around their role in their children's education. The findings would be used to identify target parent segment.
TalkingPoints has a strong hypothesis for a virtual assistant (VA) concept. Once the target parent segment has been identified, I will build an initial prototype and recruit parents on the existing platform to conduct remote testing on the VA concept. The learnings from this low-cost and low-friction testing round will inform the next phase of prototypes and in-person testing.
TalkingPoints has a strong hypothesis for a virtual assistant (VA) concept. Once the target parent segment has been identified, I will build an initial prototype and recruit parents on the existing platform to conduct remote testing on the VA concept. The learnings from this low-cost and low-friction testing round will inform the next phase of prototypes and in-person testing.
Process of Learning and Testing
Phase 1 Qualitative Research
I recruited 12 parents for 1-hour long phone interviews. I leveraged the user database and applied filters including Title I eligibility, K-3 grade, home language, geography, and whether the parent is on the app instead of SMS service. I did my best to find parents who speak English, Spanish, and Mandarin, the top home languages of the users. In addition, I looked for geographic diversity across different states. Focusing on low-income (Title I eligibility) and K-3 grade, I am aligning the project with the company's overarching mission of service those with the greatest needs.
For the interviews, I asked questions to understand core characteristics of parents:
- Demographics
- Expectations for children's education
- Current interactions with children
- Communication with teachers
- Priorities
- Needs
Based on what we learned about the various barriers preventing parents from engaging with their children, I perceived that certain constraints such as lack of access to wifi or racial discrimination are beyond TalkingPoints's power to address. Other constraints such as language barrier, relationship with the teacher, and knowledge about what goes on in school are issues that the current version of the app already addresses. The constraint that best segments parents for TalkingPoints 2.0's goals is around the parent's self-efficacy. Some parents believed they cannot play a significant role in their children's education because they lack the means/skills. Other parents believe they have the ability and the means to help their children excel. There is a group of parents sitting in the middle, who believe they can but do not have the know-how to help their children.
We decided this middle group is the target parent segment because it aligns well with TalkingPoints' core capability and mission. The parents on the left require much more hands-on support to change their mindset than a digital solution could offer. The parents on the right are in a good position and the value of one more service to them is limited.
In addition, we decided to focus on Spanish-speaking parents, because they represent by far the largest non-English speaking user group on the app.
In addition, we decided to focus on Spanish-speaking parents, because they represent by far the largest non-English speaking user group on the app.
Phase 2 Remote Testing of Virtual Assistant
For the remote testing, I recruited 30+ parents to test the virtual assistant (VA) concept. Drawing upon findings from the qualitative research, I used historical conversations between parents and teachers to try to gauge the parents' sense of self-efficacy. For example, are the parents confident enough to ask questions about their children's performance in specific subjects? Are parents expressing a desire to help their children at home?
I built a VA prototype based on needs that parents had expressed during the interviews: helping their children with math and reading, and building better relationship with the teachers and children. I was their personal "assistant," a human chatbot, who messaged them with tips and advice on how to meet the aforementioned needs.
The results were hit or miss. Parents on average responded to simple "Yes/No" or "Text 1 for Yes, 2 for No" prompts 20% of the time. The same results applied to parents I had interviewed whom I had good knowledge around needs and therefore sent them customized tips. The results raised a lot of questions around why the engagement is so low and where the parents thought the messages were coming from.
I built a VA prototype based on needs that parents had expressed during the interviews: helping their children with math and reading, and building better relationship with the teachers and children. I was their personal "assistant," a human chatbot, who messaged them with tips and advice on how to meet the aforementioned needs.
The results were hit or miss. Parents on average responded to simple "Yes/No" or "Text 1 for Yes, 2 for No" prompts 20% of the time. The same results applied to parents I had interviewed whom I had good knowledge around needs and therefore sent them customized tips. The results raised a lot of questions around why the engagement is so low and where the parents thought the messages were coming from.
Phase 3 In-Person Concept Testing
Before going on-site, we held a company-wide meeting to align everyone on the mission and collectively choose the top 3 ideas to prototype and test in person. The following 3 concepts were chosen:
- Virtual Assistant: a chatbot that asks parents about their needs and offers tips
- Parent Coach: a chatbot that guides parents through a goal-setting process
- Just-In-Time Tips: a chatbot that offers tips in the moment when a teacher and a parent discuss an improvement area for the child
"I don't like answering these questions. Who is going to see my messages? Why do they want to know this information?" - Mother of 7-year-old
In summary, we had 3 key learnings:
1) Distrust of technology: asking parents to answer questions within the app presented difficulties because parents are unsure about who is viewing their responses. It evokes “fear” in today’s political climate because of their precarious immigration status. In addition, parents have limited exposure to navigating apps and are confused by who is asking them questions.
2) Relevance is crucial. In this context, it means a tip needs to address a concern for the parent and suggest an activity within the parent's ability.
3) Community is a key aspect of the Hispanic culture and could be leveraged to drive engagement.
* Key learning 1 is in tension with key learning 2 because parent input is needed to increase the relevance of the content, but parents are uncomfortable to answer questions.
1) Distrust of technology: asking parents to answer questions within the app presented difficulties because parents are unsure about who is viewing their responses. It evokes “fear” in today’s political climate because of their precarious immigration status. In addition, parents have limited exposure to navigating apps and are confused by who is asking them questions.
2) Relevance is crucial. In this context, it means a tip needs to address a concern for the parent and suggest an activity within the parent's ability.
3) Community is a key aspect of the Hispanic culture and could be leveraged to drive engagement.
* Key learning 1 is in tension with key learning 2 because parent input is needed to increase the relevance of the content, but parents are uncomfortable to answer questions.
"I just want you to tell me what to do. If you tell me which books to get from the library, I'll go get them for my child." - Mother of 6-year-old
The learnings helped inform the next steps in the research project. The in-person testing underscored several factors that wouldn't work with the target parent population. My goal for the next round of testing is to peel back the layers of usability/top-layer barriers to answer the original research question of what is the best way to align what TalkingPoints offers and what the parents need. To prepare for the next in-person testing, I made several improvements:
- I hired a Spanish translator to translate all of the content to avoid misunderstanding due to poor translation quality (normally everything is automatically translated by Google Translate)
- I removed the questions from the content format because answering questions is uncomfortable and unfamiliar for the parents
Phase 4 In-Person Concept Testing Round 2
I created an activity finder prototype to focus parents attention on the content, not the interaction. The interaction is driven by parents tapping to unlock each tip. There are no questions posed to the parents.
In addition, I created a prototype that simulated a community challenge to appeal to parents' affinity for communities.
In addition, I created a prototype that simulated a community challenge to appeal to parents' affinity for communities.
Caption: Screenshots of the activity finder prototype I built using Sketch and Invision
Without translation quality issues and the discomfort of answering questions, the big elephant in the room finally revealed itself. Many parents could not read well enough to understand the tips! This is a big finding because I had known about parents limited education backgrounds and intentionally wrote the tips to align with grade 1-3 reading levels. Even so, I discovered literacy was a key constraint. The 300-character text-based tips were beyond their comfort zone.
Because of the literacy constraint, only 1 out of 12 parents I tested the community challenge with responded. I believe the community aspect is a big opportunity area but a more realistic simulation/a real challenge was outside the scope of the app's existing capabilities. It would require heavy lifting to build robust prototypes to fully explore this concept, which was outside the scope of this project.
On the bright side, the trust issue went away when the app is no longer pushing questions for them to answer.
For the parents to engage with the content, I realized that the presentation has to be much more visual, preferably like a menu with step-by-step picture depictions.
Because of the literacy constraint, only 1 out of 12 parents I tested the community challenge with responded. I believe the community aspect is a big opportunity area but a more realistic simulation/a real challenge was outside the scope of the app's existing capabilities. It would require heavy lifting to build robust prototypes to fully explore this concept, which was outside the scope of this project.
On the bright side, the trust issue went away when the app is no longer pushing questions for them to answer.
For the parents to engage with the content, I realized that the presentation has to be much more visual, preferably like a menu with step-by-step picture depictions.
Phase 5 Concept Testing of Visual Advice
I applied the learnings from Phase 4 to create a prototype, in the format of comic strips, that is much more accessible for parents with limited literacy skills. The result was much higher engagement from parents during remote testing. Instead of every 5 messages, parents responded on average every 2 messages.
Caption: Example of the comic strips I built using Canva
Overall Outcome and Impact
About the users:
- Established a framework for understanding parents’ mental models and narrowed the focus on parents who have a base level of self-efficacy but lack the know-how
- Developed a deep understanding of the low-income Spanish-speaking families, especially their key constraints: limited exposure to technology and low literacy levels
- Identified the content type and delivery format that would most likely engage the target parent segment: Tips that describe relatable activities for parents and their children (K-2nd grade), delivered in a visual and scenario-based format to maximize accessibility
- During the research project, my interactions with parents helped surface several key usability issues with the current app, especially for the onboarding process. These issues are important to address before parents could be introduced to new functionalities such as the feature I have been designing to offer them activity tips.
- Parents are not clear on why they need the app if they are already receiving messages from the teachers via text. This is a difficult problem because the current app doesn’t offer much value beyond messaging, but you need to be on the app in order to reap the future benefits of TalkingPoints 2.0.
- Parents are struggling with some core features of the app, including typing a response to a teacher’s message, using the option to listen to a message if reading ability is limited, and recognizing the name at the top of each thread as the sender
- Some parents don’t know about the app store and rely on their children to download the app, which begs the question of what’s in it for the children so that they won’t sabotage the set-up?
Caption: Introductory video I designed and created using Powtoons (A Spanish version is available too)