Growing Square Savings User Base
Approach: Concept Testing
Scope: Retention Experiments
Impact: 2.79x Sign-up, 2.86x Folders Created, 2.9x Total Balance Retained
Challenge: Scope Ambiguity & Impact, External Team Pressure
New Tools: Cursor as a Prototyping Tool
Square Savings is a beloved product and enjoys the highest CSAT of all the products at Square, but has the lowest monthly active users (MAU) within its financial portfolio. The team aimed to understand why MAU was lower than other Banking products and needed to identify ways to grow its user base.
I led generative concept testing based on my prior research in order to inform any potential experiments that would produce sizable impact on the growth of Square Savings. My goal: to validate the hypothesis that the Job to Be Done “Budgeting” would innovate Square Savings product beyond a typical savings tool.
I faced a challenge in this work stream due to an assumption that pre-launch research was going to solve for the surmounting external pressure on my stakeholders. I had already done a lot of generative research in this space, so I was radically transparent - no more thinking, now is the time to act. While scoping, I successfully pushed the work to inform the parameters of an experiment since we would get significant value in A/B testing. This brought immediate relief to stakeholders.
Approach
Heuristics Analysis & Assumption Workshop: In partnership with design, we co-led a heuristics analysis to uncover pain points within the sign-up funnel that would prevent sign-up completion.
Concept Testing: 5 qualitative concept tests on net-new savings + budgeting tools to understand its importance and evaluate product fit that would encourage deposit retention.
Ideation Sessions: Invited from product, design, and marketing immediately after our concept tests wrapped up. This session included refining insights, creating opportunities through How Might We’s, and brainstorming via Crazy 8s to get to create criteria for the experiments. This sprint took about 2.5 weeks.
A/B Testing: I helped build out the requirements for the test with my product and data leads and tracked our defined metrics.
Findings
Maintaining Budgeting Discipline is Hard: In the face of income uncertainty and expense certainty, small business owners feel the dredges of staying consistently committed to budgeting and believe its a privilege for others.
Habits are Formed but Can be Adjusted: Budgeting habits are entrenched in years of experience and users are apprehensive to fully change their process if they can’t see immediate value. Users need easy ways to adapt their budgeting that are not disruptive to their current set-up.
Social Proof is Important: Small business owners have a hunger for more budgeting tips and support, regardless of how comfortable they are with budgeting and appreciate rules of thumb, best practices in the industry, and current financial insights of existing customers. They know what they don’t know and can always be better (even if they are doing great).
Experiment Recos
Automating for key organizational buckets based on a small business’ industry, business structure, and past spending behavior.
Clear CTAs of what a small business owner should be saving for.
Leveraging AI: Providing a couple of budgeting options based on defaults, industry best practices, and personalization.
Upfront Set-Up: Easy multi-folder creation at the upfront, rather than waiting several steps to set-up that makes the product more exciting for users to encourage sign-up.
Configurations: Allowing users to adjust percentages and see total contributions based on allocations.
Impact
After reviewing recommendations with product leads, we created an experiment with the Data and Eng team that solved for automation and our users educational needs.
The results were incredibly successful. Marrying a heuristics analysis with concept testing contributed to a 2.79x increase in sign-ups, a 2.86x increase in folders created, and a 2.9x increase in balances retained. From this experiment, the team had high confidence to bring to this to market to all new customers.
Reflections
This project was a great example of balancing scrappiness while not sacrificing quality. There was a lot of research already done in the space, so I pushed for a different method that would be more actionable, which meant bringing solutions in the form of design mock-ups and simple descriptions to users.
Key takeaways for me:
The importance of social proofing: Social proofing a product goes much farther than telling an SMB that its useful. Our users don’t work and think in silos, they frequently look for advice on Facebook, online forums, industry groups, friends, family, and 3rd parties like accountants. Successful products match this mental model.
The impact of combining research methods: This approach combined discovery questions and evaluative questions, greatly shortening timelines for the team. Half of the interview was understanding why budgeting was important and the pain points of setting aside funds, while the second part was facilitating solutions that would make solving for those pain points easier. Insights were thus more actionable.
The value of research partnership with experiments: I love partnering with Data and Eng teams to launch experiments based on research insights. I encourage experimenting in the process wherever applicable. Research can only get us so close to the target, but at some point we need to put something in the market and learn.