Milestone 5: Glmpse User Testing

View our slide deck summarizing user testing results here

Introduction: Driving Questions and Hypotheses

Questions
We want to know whether the app is useful and in what way it is useful. When test users have this app in their hands, in their daily lives, how do they use it, what do they use it for, and what do they gain by using it?

Hypotheses
Test users will use the app to request, deliver, and receive short and playful-- but meaningful-- impressionistic videos. The majority of open glimpse requests will receive responses within hours. Different users will find very different ways to use the 4-second video medium to satisfy the request, including wide ranges of facial expressions, video pans of surroundings, and spoken words, but each glimpse video will give an insight into the user's life or disposition in that moment. Users will request glimpses from other friends testing the app and communication traffic between test users will occur without external prompting (though it's important to note this requesting behavior won't be wholly organic as they know this is a test).

Methods

  1. Recruit pairs or triplets of test users who know and like each other. (It's imperative that each test user knows more than one other test user so there's someone for them to communicate with on the app.)
  2. Get each user set up with the app (involving preparing the right iOS certificates and the like), and over the next day, have the user respond to our glimpse request and have them send us a glimpse request. This ensures the test user has gotten familiar with the app and knows how it works. Then make sure the test user knows that his friends A and B are also on the app and encourage the user to test the app with these friends over the next few days.
  3. Let him loose and see what happens. Now we're just observing how users actually use the app.
  4. Use exit interviews and usage statistics to begin to answer the questions listed above.

Results

We added our first tester on Monday evening, with the bulk installing the app at some point on Tuesday. As of Wednesday night, we have the following stats:



There was typically a constant number of unanswered requests from each user, likely due to some users possessing multiple accounts and only using one. These abandoned accounts accumulated unanswered requests.

Discussion

Experimental design issues What do users want from a glmpse? What is the value in a glmpse video? What barriers exist for using Glmpse?

Implications

The largest change we intend on implementing is the ability to attach a short text message with a request. This accomplishes two things:

Additionally, we are also considering adding a convenient way for users to send a request while replying to one. ie, send a "counter request" after recording a glmpse for someone. This may be as simple as double tapping an incoming request, similar to Snapchat.
This would hopefully increase the chance of glmpses starting a sort of conversation. As one user wrote, referring to her desire to respond to an enjoyable glmpse someone sent her,
"I could [respond] since he had also requested one from me, but if he hadn't I would have been frustrated I couldn't share my appreciation in-app and would have to text.

Appendix A: App usage data

To see a summary of app usage by test user, and to sample the kinds of glmpses shared, visit the glmpse webview.

Appendix B: Need-finding Facebook Survey

We thought glmpse would be a great medium for moms to communicate with their children. Glmpse is a way for moms to prompt their children to share, and it's a low-friction way for the children to satisfy the mom's request for sharing. We designed a survey to help us understand how mothers currently communicate with their remote children and what needs they have. We targeted moms on Facebook who are older than 40 and use mobile phones. Unfortunately, we did not get many results to make any definitive claims. We spent $80 for 160 click-throughs, but only 12 moms actually completed the survey. Based on the responses, our initial findings are that although moms do report having to initiate communication more frequently, the majority of moms are satisfied with their current communication with their children (which happens mainly over the phone and through text messages). The communication need is not as pressing as we thought.

Appendix C: How does M5 build on M4?

We decided to pursue our glmpse prototype and iterated on it in four ways--