View our slide deck summarizing user testing results here
Introduction: Driving Questions and Hypotheses
We want to know whether the app is useful and in what way it is useful. When test users have this app in their hands, in their daily lives, how do they use it, what do they use it for, and what do they gain by using it?
- What do users aim to accomplish by sending glmpse requests?
- What is the value in a glmpse video?
- What are the barriers to sending or responding to glmpse requests?
Test users will use the app to request, deliver, and receive short and playful-- but meaningful-- impressionistic videos. The majority of open glimpse requests will receive responses within hours. Different users will find very different ways to use the 4-second video medium to satisfy the request, including wide ranges of facial expressions, video pans of surroundings, and spoken words, but each glimpse video will give an insight into the user's life or disposition in that moment. Users will request glimpses from other friends testing the app and communication traffic between test users will occur without external prompting (though it's important to note this requesting behavior won't be wholly organic as they know this is a test).
- Recruit pairs or triplets of test users who know and like each other. (It's imperative that each test user knows more than one other test user so there's someone for them to communicate with on the app.)
- Get each user set up with the app (involving preparing the right iOS certificates and the like), and over the next day, have the user respond to our glimpse request and have them send us a glimpse request. This ensures the test user has gotten familiar with the app and knows how it works. Then make sure the test user knows that his friends A and B are also on the app and encourage the user to test the app with these friends over the next few days.
- Let him loose and see what happens. Now we're just observing how users actually use the app.
- Use exit interviews and usage statistics to begin to answer the questions listed above.
We added our first tester on Monday evening, with the bulk installing the app at some point on Tuesday. As of Wednesday night, we have the following stats:
- 15 unique users (including us)
- 149 glmpse requests (100 not including us)
- 118 glmpses shared (62 not including us) See appendix to view recorded glmpses by user
- Removing our participation in the testing entirely, there were 33 glmpses shared between testers.
- 8 people responded to a mostly qualitative exit survey in addition to interviews
There was typically a constant number of unanswered requests from each user, likely due to some users possessing multiple accounts and only using one. These abandoned accounts accumulated unanswered requests.
Experimental design issues
What do users want from a glmpse?
- Testers were recruited from our friends. This resulted in most communication centering around the recruiters. While not a problem per se, a request/response with a developer on one end is perhaps less than natural than otherwise.
- Users did not have a huge amount of time to use the app. The initial novelty of the app may affect behavior (eg, dramatically increased usage). Longer term use needs to be observed.
- Significant bugs still exist in the prototype. In particular, push notifications exhibit some mysterious behavior, often leaving users unaware that they received a request.
What is the value in a glmpse video?
When asked what kind of glmpses they liked to receive, no users reported preferring to see the just the face of the glmpse provider. They all chose either the other user's surroundings, or a combination of the other user's surrounds and face.
This suggests that users want to connect to others via their environment, seeing what they see, etc, as opposed to face/speech centered content.
Users want responses to be personal
“there could be a danger of glimpses feeling like those annoying snapchat videos sent to everyone--vaguely interesting but not meant for you” -- Alison, after receiving a particularly unsatisfying glmpse
Requests are often sent simply because the requesters wants to know where someone is, what they're doing, etc.
“I requested a glimpse from everyone at noon because I had some free time and wanted to see what others were up to” -- Roberto
What barriers exist for using Glmpse?
Users feel satisfaction in knowing that someone wants a video from them
“Getting a request for a glimpse was flattering.” - Roberto
“I liked knowing she [user Panda] wanted me to share something. It was like I knew I wouldn’t bother her with the video but I also didn’t feel any pressure to make it special somehow.” - Alex
The actual content of glmpse videos had a lot of variety, including a combination of:
Users convey personality through their videos
- Funny faces
- Short, spoken messages
- The surrounding environment (eg, a pan of the room they're in, or street corner they're waiting at)
- Footage of or interactions with nearby objects
- Something funny. Visual humor
- Accidental glmpses. Not everyone was prepared to record...
Some users didn't know how to respond to a request, and hesitated while thinking of something to record. Random nearby objects were not deemed "glmpse worthy" by all users
“I'm still figuring out what I should be doing and what the etiquette is” - ball
Other users felt less motivated to use the app because they didn't get much out of other people's glmpses. Received glmpses were thought to be uninteresting, or not what they were looking for.
Unfortunately limited by the experimental design/recruiting process, a large portion of users said they would have used the app more if they knew more of the other users. Most users only knew a couple of the others, and generally didn't like requesting/responding to strangers
The largest change we intend on implementing is the ability to attach a short text message with a request. This accomplishes two things:
- Text can serve as a prompt for users who otherwise don't know what to record. This gives a glmpse more explicit purpose for those who want it
- Text can also be a way for the requester to ensure he/she receives a satisfying glmpse. eg, if users want to see where another person is as opposed to their face, they can ask for it explicitly
Additionally, we are also considering adding a convenient way for users to send a request while replying to one. ie, send a "counter request" after recording a glmpse for someone. This may be as simple as double tapping an incoming request, similar to Snapchat.
This would hopefully increase the chance of glmpses starting a sort of conversation. As one user wrote, referring to her desire to respond to an enjoyable glmpse someone sent her,
"I could [respond] since he had also requested one from me, but if he hadn't I would have been frustrated I couldn't share my appreciation in-app and would have to text.
Appendix A: App usage data
To see a summary of app usage by test user, and to sample the kinds of glmpses shared, visit the glmpse webview.
Appendix B: Need-finding Facebook Survey
We thought glmpse would be a great medium for moms to communicate with their children. Glmpse is a way for moms to prompt their children to share, and it's a low-friction way for the children to satisfy the mom's request for sharing. We designed a survey to help us understand how mothers currently communicate with their remote children and what needs they have. We targeted moms on Facebook who are older than 40 and use mobile phones. Unfortunately, we did not get many results to make any definitive claims. We spent $80 for 160 click-throughs, but only 12 moms actually completed the survey. Based on the responses, our initial findings are that although moms do report having to initiate communication more frequently, the majority of moms are satisfied with their current communication with their children (which happens mainly over the phone and through text messages). The communication need is not as pressing as we thought.
Appendix C: How does M5 build on M4?
We decided to pursue our glmpse prototype and iterated on it in four ways--
- Made it a mobile app
- Added confirmation for the provider when a glimpse was seen
- Added audio
- Enabled video streaming before recording is triggered to ease framing