Informing Square Savings’ Growth Experiments
Concept Testing | Retention Experiments | Impact: 2.79x Sign-up, 2.86x Folders Created, 2.9x Total Balance Retained
Square Savings is a beloved product and enjoys the highest CSAT of all the products at Square, but has the lowest monthly active users (MAU) within its financial portfolio. The team aimed to understand why MAU was lower than other Banking products and needed to identify ways to grow.
I led generative concept testing based on prior research I led in order to inform any potential experiments that would produce sizable impact on the growth of Square Savings. My goal: to validate the hypothesis that Budgeting as an additional Job to Be Done would innovate Square Savings product beyond a typical savings tool.
Scrappy & Quick Approach to High Quality Insights
Background & Objective
The Savings team wanted to investigate pain points within the current sign-up flow and identify growth experiments with product. The Data team indicated notable drop-off in the funnel and revealed that small business owners only had one folder: General Savings, which is the default folder created at sign up. Besides evaluating and improving the acquisition funnel, the product team was keen on validating additional solutions around key areas of automation and defaults.
I had already conducted several research studies around Cash Flow Management, Expense Management, and Budgeting so the team was bought-in on solving for the job: “Help me budget better for my business expenses.” We had a hypothesis that onboarding was the most important touchpoint and indicative of future success with the Savings product. We focused on evaluating and generating solutions top-of-funnel that could increase balance retention within Savings, so the objectives were to provide rationale for said experiments.
Methodology
Heuristics Analysis: In partnership with design, we co-led a heuristics analysis to uncover pain points within the sign-up funnel that would prevent sign-up completion. This took about 2 hours, and we brought to product a list of low-hanging fruit that were organized by severity and risk to product acquisition. Top-level findings included: 26 steps to sign-up, multiple verification steps, and cognitive overload when setting up earnings contribution. Findings were consumed by leads, and engineering reduced steps to 12 - directly correlated to a 2.89x increase in sign-ups.
Concept Testing: I was the lead for this study. I interviewed target users in order to understand needs, motivations, and pain points around budgeting, and solicited concept feedback to help our team refine ideas and ensure our experiments aligned with our user’s needs.
Findings
I led 5 concept tests on net-new savings + budgeting tools to understand its importance and evaluate product fit that would encourage deposit retention. I made clear that this work was not to provide a clear business directive, but to provide inspiration that would inform quantitative experiments. Stakeholders were invited to each session to take notes and ask follow-up questions based on the nature of our discussion. Notes and interviews were aggregated into Dovetail, and findings synthesized with its AI tooling.
I hosted an ideation session with members from product, design, and marketing immediately after our concept tests wrapped up. This session included refining insights, creating opportunities through How Might We’s, and brainstorming via Crazy 8s to get to create criteria for the experiments. This sprint took about 2.5 weeks.
Key Findings:
Maintaining Budgeting Discipline is Hard: In the face of income uncertainty and expense certainty, small business owners feel the dredges of staying consistently committed to budgeting and believe its a privilege for others.
Habits are Formed but Can be Adjusted: Budgeting habits are entrenched in years of experience and users are apprehensive to fully change their process if they can’t see immediate value. Users need easy ways to adapt their budgeting that are not disruptive to their current set-up.
Social Proof is Important: Small business owners have a hunger for more budgeting tips and support, regardless of how comfortable they are with budgeting and appreciate rules of thumb, best practices in the industry, and current financial insights of existing customers. They know what they don’t know and can always be better (even if they are doing great).
Recommendations
My recommendations for the experiment focused on automating for key organizational buckets based on a small business’ industry, business structure, and past spending behavior:
Clear call-to-action of what a small business owner should be saving for.
Providing a couple of budgeting options based on defaults, industry best practices, and personalization (leveraging AI).
Easy multi-folder creation at the upfront, rather than waiting several steps to set-up that makes the product more exciting for users to encourage sign-up.
Allowing users to adjust percentages and see total contributions based on allocations.
Impact
After reviewing recommendations with product leads, we created an experiment with the Data and Eng team that solved for automation and our users educational needs.
The results were incredibly successful. Marrying a heuristics analysis with concept testing contributed to a 2.79x increase in sign-ups, a 2.86x increase in folders created, and a 2.9x increase in balances retained. From this experiment, the team had high confidence to bring to this to market to all new customers.
Reflections
This project was a great example of balancing scrappiness while not sacrificing quality. There was a lot of research already done in the space, so I pushed for a different method that would be more actionable, which meant bringing solutions in the form of design mock-ups and simple descriptions to users.
Key takeaways for me:
The importance of social proofing: Social proofing a product goes much farther than telling an SMB that its useful. Our users don’t work and think in silos, they frequently look for advice on Facebook, online forums, industry groups, friends, family, and 3rd parties like accountants. Successful products match this mental model.
The impact of combining research methods: This approach combined discovery questions and evaluative questions, greatly shortening timelines for the team. Half of the interview was understanding why budgeting was important and the pain points of setting aside funds, while the second part was facilitating solutions that would make solving for those pain points easier. Insights were thus more actionable.
The value of research partnership with experiments: I love partnering with Data and Eng teams to launch experiments based on research insights. I encourage experimenting in the process wherever applicable. Research can only get us so close to the target, but at some point we need to put something in the market and learn.