Constructed Research Plan; Lead user interviews & usability testings; Analyzed server log tickets and synthesized qualitative data; Presented to Stakeholders
This is my HCI Master's Capstone Project with Truist Financial. We helped assist their client, Digital Navigator (DN), to improve service flow through UX research. DN offer services such as signing up for affordable internet service, purchasing affordable device, and learning new digital skills etc. Through qualitative UX research such as server data analysis, interview, focus group and usability testings we hope to establish, deliver and expand access to the digital literacy world in the underserved communities of Charlotte, NC.
Problem
At the time of our project, Digital Navigator expressed interests in offering new services to their customers and concerns about accessibility to the service being offered.
Hypothesis
Through various brainstorming sessions, the team and I hypothesized the lack of access to digital literacy services are caused by the accessibility to to local resources to digital literacy and connectivity.
Solution
After completing a series of qualitative analysis of the server data, user interviews and usability testings. We have discovered 3 themes to the current issues at hand: Internet Access Obstacles, Basic Device/Connectivity and Affordable Devices.
Outcome
In the end of the project, we successfully developed 3 scalable design recommendations for Digital Navigators. The design recommendations were approved by our team at Truist Financial for further development and our client Digital Navigator was impressed with the UX Research Report. DN thought that 24/7 Chatbot Helpline and Physical Digital Literacy Toolkit in particular and currently in development for future implementation!
Process
01. Research Planning
Client Background: Digital Navigator (DN)
Digital Navigators will help community members navigate to and through:
Signing up for affordable home broadband internet service
Purchasing affordable technology
Basic device and connectivity issues
Learning new digital skills
How It Works:
Call 311 to get connected with a digital navigator
A Digital Navigator is assigned to respond
The community member and the digital navigator work together to accomplish the goal
Understanding the Current Service and Product Flow
With defined budget and timeline, I worked with a team of six - two researchers, an engineer, project manager, a service designer and an Executive Director to ensure that all research questions brought to the table. Our partner, Truist Financial, expressed interest in understanding the strength and weaknesses of their client, 311 & Digital Navigator. So they can better support them in the future through software development & device donations.
How did users heard about 311 or Digital Navigator service?
What issues come up during the Digital Navigator service from the operator side?
What issues come up during the Digital Navigator service from the user & customer side?
Besides software development & device donation, how can Truist Financial: The Home Page Project Further support DN?
Recruiting Users & Digital Navigator Operators
We recruited via email and internal referrals inside the Digital Navigator teams. We created a screener that filtered out the direct user population and stakeholders for DN. A total of 10 users were selected for in-depth reviews and unmoderated usability testings.
02. Collecting Data
Conducting Competitive Analysis
In addition to the User Interview and Usability Testings, our team felt the necessity to cross-reference the best and most popular digital literacy organizations in the country. By conducting a competitive analysis, we can further verify the design we later propose is valid for Digital Navigator's users and customers. Thus me and another UX Researcher set out to explore 3 different companies and non-profit organization in digital literacy service industry: EveryoneOn, Human IT and LFC (Loaves, Fishes & Computers). Since the world of Digital Literacy Industry and Underserved communities were foreign to me, I made understanding these organizations and their users/customers a priority.
After thorough analysis, although there are existing quality services, our client Digital Navigator lacked in areas like
Device Donation,
Live Tech Support (e.g., chatbot)
Offer Tech and Digital Literacy Training
Remote User Interview
The teams at Truist Financial stated there are current UX researchers that have-already conducted preliminary interviews to create several User Personas. But there were only 5 interviews done with limited user persona, which is not representative of the user population in the Charlotte Community. Thus we conducted additional interviews and Usability Testings with Cognitive Walk through to further understand of users & customers.
TLDR, these 5 were the main research questions I wanted to answer through these interview sessions:
Collecting Digital Navigator Server Log Data
Below are the types of data we collected from the Server Logs that records all tickets submitted either by users/customers through calling 311 or the online System. The Average Chat Wait Time is the time users or customer had to wait after requesting help via 311 or the online DN system.
Tickets submitted to the Digital Navigator are recordings of the phone conversation between the DN operator and community user & members.
Below are examples from the tickets submitted to the DN system. In the following Data Analyzing section, the research team and I each completed qualitative thematic analysis of the tickets.
Screenshots of the server data is not available due to the protection of our user privacy & NDA.
03. Analyzing Data
Thematic Analysis of Qualitative Data from the DN Tickets
Me and the other UX researchers synthesize the users & customer tickets data by conducting a bottom up Thematic Qualitative Analysis approach. We categorized the data in to the following themes.
Signing Up for Affordable Home Broadband Internet Service,
Basic Device and Connectivity Issues,
Learning New Digital Skills,
DN Volunteer/Service Applicant,
Purchasing Affordable Technology
Others (E.g., Collaborations and Sponsorships)
Analysis Result: Digital Navigator Tickets
Interview Result: Digital Navigator User Personas
After conducting the interviews, the team and I started establishing personas to decide what direction Digital Navigator should head to in order to reach their goal. To reach a consensus, our team worked with the Truist Design and Management Executive stakeholders to collectively make a business decision on the target consumer bucket. There was an alignment to focus on Fathers who are small business owner as our primary persona, and Mothers who are 1st Gen Immigrant field workers as our secondary persona as these were the most common types we’ve seen across our research.
04. Formulating Recommendations
Theme 1: Internet Access Obstacles
After qualitatively analyze the server ticket data we collected, the researchers and I have synthesized 4 themes of design recommendations. First one is Internet Access Obstacles. The largest subsection of the tickets we analyzed was access to internet. Common obstacles to this are affordability and quality of internet access.Digital Navigators does work to connect people with resources like EveryoneOn to find low-cost internet, but they don’t provide instant or 24/7 assistance.
Design Recommendation
Since Internet Access Obstacles is such a common request, our recommendation is to have a 24/7 physical service or product built-in to Digital Navigator to support communities without internet access. Examples of this kind of service include chatbots, flyers, publicly available devices, and 24/7 support lines.
Theme 2: Devices & Connectivity
Through our interviews with the community members & Digital Navigator Operators, Device access, troubleshoot and Internet connections was the 2nd most mentioned problems they needed help from Digital Navigators or from tech savvy family members.
Design Recommendation
This is our Theme 2 design recommendation for Devices & Connectivity.
Partnerships with device donation or refurbished device organizations
Similar to Loaves and Fishes in Monterey
Address access to devices or affordability concerns.
In addition we presented a digital billboard or robotic kiosk that provides internet sign up services on a 24/7 basis. if the digital billboard or robotic kiosk can be placed across community spaces like libraries, convenience stores or bus stops. Our design recommendation will able to solve theme 1 and 2 which is finding internet access and basic device connectivity. The picture on the right is an example of digital billboard deployed in Sweden by OOH that aims to direct homeless populations.
Theme 3: Digital Literacy
Digital literacy is the 3rd theme on the list. After analyzing the data, our users & customers mentioned they may be forced to learn a new technology for the first time.Thus they need help from their family or community members to get trained. In addition, finding job listings, submitting applications, and networking is another area that our community needed help with. Most of the career development takes place online so the users wanted to learn digital skills in order to improve their job prospects
In addition, through our interviews we discovered the older community members (35 -65 yrs old) have a Fear of Burdening Others Mindset. This refers to relying on others to deal with technology could inhibit people or straining the relationship.
Design Recommendation
Since Truist Financial already have an informational website called The Home Page Program, set up for Digital Navigator. We planned on revising the website to offer online digital literacy courses and a center to distribute off-line physical readings.
Utilize THPP as a hub of resources for quick self-help/learning.
More courses, videos on digital literacy topics Revised Info.
Architecture & NavBarClear contact for DN service & other relevant resources - Number, Live Chat Support
05. Conclusion & Limitations
Research Limitations
User Research Limitations:
Time: Our timeline was limited due to this being a corporate sponsored Master's capstone project, the overall length of the project was only 9 weeks so we could not conduct more in-depth research conducted such as Diary Studies.
Budget: We do not have the same amount of resources as an industry or funded academia UXR team to recruit research assistants to assist with the over all planning.
Lack of in-person access to local community/users due to our target community being located in the East Coast while our school is located on the west coast.
Limitations Impact on Future R & D
Incomplete understanding of direct end users: Due to the nature of large community populations, Other methods that would have provided the rich level of user feedback would have been field research of visiting homes personally to observe families who have trouble with internet access, but unfortunately, with limitation in both budget and time, this was not an option.
No in-person User Testing with Direct End users. The underserved community we studied was very protective of one another thus conducting research in-person vs. online could generate a different result.
Develop scalable, transferable Ideas: due to internal shift in management at Truist financial we could not proceed further into design but our client, Digital Navigator found our report to be impressive and would like to implement the recommendations in 2023!
Conclusion
One of the most valuable learning I gathered from this project is learning how to collaborate with a vulnerable population and the importance of incorporating stakeholders in as much of the research planning as possible. There is something really special about serving the community of Charlotte through Digital navigator and witness the changes being made to their lives. In addition, being able to witness user pain points first hand, instead of being just facts on paper, we can empathize with the user and digest the data more effectively.
With this information, I plan on continuing to collaborate with my stakeholders in as much of the research as possible and practice more ethnography & diary studies to get more in-depth experience inside the community I'm studying!
