UX Research · Mixed Method · Case Study

Uncovering Friction in Payment Setup for a Global Ad Platform

Project Overview

  • Role

    LEAD UX RESEARCHER

  • Client

    GLOBAL SOCIAL MEDIA & ADS PLATFORM

  • Methodology

    MODERATED USABILITY TESTING

  • Tools

    LOOKBACK, TEAMS, MURAL, MS WORD, MS EXCEL ETC.

  • Team

    2UXRs, 1PM, 1RECRUITER

  • Timeline

    4WK (NOV~DEC 2024)

Background

A global social media and ads platform wanted to evaluate the mobile payment setup experience for ad campaign users.

The research was initiated in response to a noticeable rise in customer support requests related to payment failures. Our goal was to identify key usability issues in the payment flow from the user’s perspective—especially across platforms like iOS and Android.

Research Goals

  • 1

    Understand how users locate, interpret, and complete mobile payment setup

  • 2

    Identify pain points and drop-off moments during pre-pay and post-pay flows

  • 3

    Recommend product changes to increase task success and user trust

Research Scope

Moderated Usability Testing

We conducted moderated remote usability testing with 10 participants in the U.S., including small business owners and marketers for such businesses. This method was chosen to uncover deep usability insights through direct observation and probing. Sessions were conducted on both iOS and Android platforms to capture differences in experience.

Flow Evaluated

  • Mobile payment setup flow (iOS & Android)

  • Pre-pay (Add Funds) and Post-pay (Add Payment Method)

  • Adding and removing payment methods

Participants

  • 10 U.S.-based users

  • Mix of sole proprietors and social media marketers for small businesses

  • Even split across billing models (pre-pay / post-pay users)

Method

  • 1:1 moderated usability testing (Teams)

  • 45 to 60 min in-depth sessions

  • Task-based sessions using think-aloud protocol

My Roles

  • 1. Plan

    CREATING RESEARCH PLAN

    Defined research scope and participant criteria aligned with client goals

  • 2. Recruit

    PARTICIPANT RECRUITING

    Developed a screening survey for recruiters and reviewed candidate responses to assess fit

  • 3. Conduct

    CONDUCTING USABILITY TESTING

    Developed a moderator guide and facilitated or supported 10 moderated usability sessions

  • 4. Synthesize

    SYNTHESIZING QUALITATIVE DATA

    Analyzed user feedback through thematic coding and affinity mapping to identify patterns and sentiments per task

  • 5. Deliver

    DELIVERING REPORT

    Produced a comprehensive report detailing methodologies, findings, and actionable recommendations per task and user flow.

Process & Challenges

1.Scoping & Planning

At the start of the project, we faced several structural challenges that threatened to derail the research before it even began:

  • Scope misalignment: Recruitment began by PM before the research plan was finalized, risking mismatch between session goals and participant profiles.

  • Audience ambiguity: Too many potential personas with no clear priority—making meaningful insights from a small sample harder to achieve.

  • Compressed timeline: We had less than six weeks to go from planning to final report delivery—with only 10 sessions to generate impact.

💡Key Contribution

🧭 Reestablished Clarity

  • Proposed and led a formal kickoff meeting to reset alignment on goals, scope, roles, and deliverables.

  • Clarified unclear areas with PMs and stakeholders (e.g., deliverable format, usage criteria, platform focus).

👥 Refocused the Audience

  • Recommended holding off on recruitment until screener + criteria were finalized.

  • Created a clear, stakeholder-approved breakdown:
    → 5 sole proprietors
    → 5 small business owners/marketers

📊 Created Transparent Planning Tools

  • Broke down the 100-hour estimate into concrete workstreams (planning, setup, moderation, analysis, reporting).

  • Built and shared a visual timeline (Mural) to align all parties on key deadlines and dependencies.

✅ Impact

  • Avoided recruitment mismatches by halting premature outreach and aligning on screening needs.

  • Accelerated decision-making through structured communication and clear documentation.

  • Improved forecasting and enabled smoother resource allocation and billing prep.

  • Set a scalable research model for future sessions—client expressed interest in expanding based on this pilot.

2. Recruiting & Participant Alignment

As soon as we aligned on the new participant criteria, I took the lead in redesigning the screener survey to reflect our updated scope. Instead of simply asking about demographics or general experience, I focused the screener around qualifying factors that directly impacted our research goals—such as the participant’s role in the business, ad platform usage, payment method and frequency, and device type.

💡Key Contribution

  • Created a structured and focused screener that could serve both as a filter and quota-balancing tool

  • Used the screener to retroactively evaluate already-contacted candidates, minimizing waste

  • Introduced a real-time tracking system (Excel Sheet) that helped prioritize qualified participants and keep stakeholders aligned

  • Provided clear pass/fail notes and tagging system so decisions could be made quickly, even asynchronously

✅ Impact

  • 100% of participants matched final criteria

  • Achieved balance across platform, device, and payment types

  • Reduced turnaround time with real-time collaboration

  • Strengthened session quality through better-fit recruitment

3. Designing & Executing Moderated Sessions

I facilitated 5 moderated sessions and supported 5 others as a note-taker. Sessions were conducted remotely via Microsoft Teams, which allowed participants to join from both desktop and mobile devices—enabling mobile screen sharing while preserving face visibility for sentiment cues. The moderator guide, co-developed with another UX researcher, was task-based and focused on critical flows within the Facebook Ads payment system.

Each session was structured around hands-on tasks supported by open-ended questions to uncover pain points, behavioral patterns, and unmet needs.

🔍 Key Tasks
Task 1: Locate where to manage ads payment methods
Task 2: Add a payment method (Postpay) or Add funds (Pre-pay) based on participant’s usual payment method
Task 3: Remove a payment method

❓ Key Questions

How do users currently manage ad payments on Meta platforms?

What barriers or frustrations arise during these tasks?

How do different payment methods impact trust or usability?

What do users expect but not find in the current experience?

💡Key Contribution

  • Facilitated 5 sessions and supported 5 as note-taker

  • Proposed Teams for multi-device support and sentiment observation

  • Co-authored a task-based moderator guide

  • Managed session logistics end-to-end: MS Bookings, scheduling, participant tracking

  • Designed a theme-based Excel log for structured note-taking and faster synthesis

  • Tracked task completion rates and failure reasons in a standardized format to support quant synthesis in reporting

As a note-taker, I developed a theme-based Excel log with structured categories (user background, each task, overall sentiment), making later synthesis faster and more systematic. I also added a task-level table tracking completion status and failure reasons, giving our qualitative findings a layer of quantifiable structure we could easily reference in the report.

✅ Impact

  • Enabled clearer cross-team communication through shared trackers and pre-session logistics

  • Increased research rigor by standardizing how task performance and failure reasons were captured

  • Supported quantitative synthesis from qualitative sessions, enhancing clarity in the final report

  • Reduced time-to-synthesis by creating structured, ready-to-transfer data from sessions

4. Synthesizing

Worked with another researcher to synthesize insights from 10 usability sessions into a task-based report. Structured notes—organized by participant, task, and theme—were exported to Mural for affinity mapping, using color-coded sentiment tags to surface patterns efficiently.

To support consistency and speed, we used a reusable Mural synthesis template I had previously designed. This helped streamline collaboration and made insight clustering more scalable.

Qualitative findings were paired with a task completion tracker to highlight breakdowns and failure patterns across flows.

Synthesis Method

1. Exported notes by key questions and tagged sentiment using a color system

2. Used affinity mapping to identify common issues across flows

3. Summarized insights with clear severity, quotes, and opportunities

✅ Impact

  • Enabled fast pattern recognition across flows

  • Connected behavior with sentiment to uncover root issues

  • Combined quant/qual data for clearer storytelling

  • Built foundation for smooth transition into reporting phase

Client Impact & Research Expansion

Our task-based report surfaced key usability breakdowns in labeling, navigation, and confirmation flows, along with friction specific to iOS, payment method management, and information architecture. Each issue was paired with actionable UX recommendations, prioritized by frequency and severity.

📊 Quantitative Insights

  • Task Completion Rate
    • Post-pay: 90% (9 of 10 participants)
    • Pre-pay: 50% (5 of 10 participants)

  • Platform-Specific Findings
    • iOS-specific issues in 6 of 10 sessions
    • Most common issue: payment settings difficult to locate (low IA findability)

⚠️ Key Usability Issues

  • Auto-billing enabled by default without user awareness

  • No direct way to manage payment methods in iOS app → redirected to mobile web

  • Unexpected Apple service fees (up to 50%) discouraged top-ups

  • Unclear error messages made task recovery difficult

🧭 UX Recommendations

  • Improve visibility of payment settings through clearer IA and navigation cues

  • Provide user control over billing options and default settings

  • Increase transparency around third-party service fees

  • Clarify error messages to support recovery from failed tasks

  • Enable native payment management in iOS app to reduce redirect friction

💬 Client Feedback & Expansion

Positive stakeholder feedback led to a second research phase in India, Vietnam, Mexico, and Indonesia, aimed at testing localized flows and validating global consistency.

"Really comprehensive – thanks for sharing. Feel like we should produce this kind of output for local in-market testing in general, as it’s informative and actionable."

- Client-side Ads Payment Lead