When Evergy approached our team, they recognized the need to enhance their app experience through comprehensive usability testing and thoughtful remediation. This case study outlines our research project, which included user interviews, an information architecture (IA) audit, tree testing, and targeted usability issue remediation. Let's dive in to explore the methodologies, challenges, and outcomes of this research ask to alleviate major pain points throughout the experience.
The Ask
Our primary objectives for this project were to:
Assess App Navigation: Evaluate the efficiency, user-friendliness, and effectiveness of Evergy’s primary navigation.
Analyze UI Design & Content: Determine the intuitiveness, clarity, and readability of the app’s interface and core pages.
Recommend Optimizations: Identify areas for enhancement, including potential updates to the menu structure, labeling, hierarchy, and interactions.
Deliver a Final Report: Summarize findings and provide actionable design improvements to guide Evergy’s next steps.
Following an alignment session with Evergy, we finalized a detailed research plan that outlined our goals, ensuring a clear, shared vision for the project.
Research Methodology
Our approach was grounded in a multi-phase research plan, combining heuristic evaluation, usability testing, and information architecture (IA) validation through tree testing.
Heuristic Evaluation We began with a heuristic evaluation based on Nielsen’s heuristics to identify baseline usability issues. This evaluation identified areas of friction in navigation, system feedback, and content consistency, providing us with initial insights into the app’s overall usability.
Navigation Structure: Users had difficulty locating key functions due to unclear labeling and organization.
System Feedback: Limited visual feedback left users uncertain about actions like payment confirmations and alert sign-ups.
Content Clarity: Dense text on some pages affected readability and task efficiency.
These insights shaped our usability testing focus, helping us prioritize the areas requiring user validation through usability testing and created the base for the formation of our usability test tasks.
Round 1 Usability Testing
Study Goals Our primary goal for Round 1 was to understand user expectations and identify potential friction points in navigating the homepage and primary features. These insights would lay a foundation for future evaluations, providing a clear focus for targeted improvements.
Sample Size We conducted the study with 7 participants:
Location: 4 from Kansas, 3 from Missouri
Audience Definitions:
Household Income: $40,000–$149,999
Age Spread: 31–79 (Average age: 44)
Gender: 4 Female, 3 Male
Requirements: All participants were Evergy customers and active mobile app users.
Key Tasks Participants were asked to complete the following tasks:
Evaluate the Homepage: Review the homepage layout, content, and overall intuitiveness.
Sign Up for Power Outage Alerts: Attempt to register for notifications on service restorations.
Add a Second Account: Locate the feature to add a new account within the app.
Update Primary Phone Number: Update the phone number linked to the Evergy account.
Key Findings The insights gathered in Round 1 highlighted areas for improvement that would be essential for refining user experience:
Ineffective Use of Whitespace: Participants pointed out the poor use of whitespace and disorganized content, especially on the outages page. Many expressed that all critical content should be accessible about the fold without the need to scroll.
Interest in Usage Data on Homepage: Users wanted quick access to their usage data and indicated it would be helpful to see this on the homepage.
Support Missing from Key Areas: Users noted that support functionality was difficult to access and expressed a desire for a support option directly on the homepage.
Lack of Contrast in Bottom Navigation: Some participants struggled with distinguishing tabs in the bottom navigation due to low contrast, suggesting that a darker or bolder color would improve visibility and accessibility
Note that while we were not on the hook for an entire accessibility audit, we did audit all of the pages we tested. Small changes to colors and typography were incorporated into the mockups.
Inconsistent Naming Conventions: Confusion arose when users tried to add a second account, as the labels in the "More" menu (e.g., "Manage Accounts" and "Add Account") were inconsistent and unclear.
Recommendations Based on Round 1 Findings:
To address these pain points, we outlined several recommendations to enhance the app's usability and align with user needs:
Group Content in Intentional Ways
Recommendation: Refine the use of whitespace on the homepage and outages page to better organize content. Pulling key information into view will reduce unnecessary scrolling and help prevent users from encountering “false bottoms” on smaller devices.
Incorporate Usage Data on the Homepage
Recommendation: Add a usage data component to the homepage, providing users with easy access to this frequently sought-after information.
Make Support Easily Accessible
Recommendation: Integrate a support or contact function on the homepage to ensure users can easily reach assistance when needed.
Ensure Contrast and Legibility of Bottom Navigation
Recommendation: Redesign the bottom navigation to improve contrast and align with accessibility standards, making the navigation tabs more visually distinct.
Align Naming Conventions
Recommendation: Standardize language across pages to reduce confusion, especially for features like "Manage Accounts" and "Add Account," which should use consistent terminology throughout the app.
These recommendations provided a clear direction for Round 2 usability testing, allowing us to validate these adjustments and further improve the app’s overall functionality and user satisfaction.
Additional Usability Testing: "Analyze My Usage" Page Evaluation
Study Goals At the client's request, an additional round of testing was added to evaluate the "Analyze My Usage" page, along with features related to rate plan selection and payment plan setup. The purpose of this round was to learn about users' expectations and uncover possible friction points within these specific areas. This was initially outside of our project scope, but we found a way to finesse the timeline to fit it in for them.
Sample Size The test was conducted with 5 participants:
Location: 4 from Kansas, 1 from Missouri
Audience Definitions:
Household Income: $40,000–$149,999
Age Spread: 36–52 (Average age: 44)
Gender: 2 Female, 3 Male
Requirements: All participants were Evergy customers and mobile app users.
Key Tasks Participants were asked to complete the following tasks:
Evaluate the "Analyze My Usage" Page: Review layout and data provided on the "Analyze My Usage" page.
Change Rate Plan: Locate and navigate to the rate plan section and select a new rate plan.
Set Up a Payment Plan: Find and complete the setup for a payment plan.
Key Findings The study surfaced several insights that revealed additional friction points and opportunities for refinement:
Difficulty Finding Payment Agreement Option: Users initially struggled to find the payment plan option, which was placed lower on the page. Despite ultimately locating it, the low placement caused confusion.
Uncommon Naming Conventions: Many users found the label "Payment Agreement" confusing, expecting terms more like "Payment Plans" that more clearly communicated the option’s purpose. Most users initially overlooked this label.
Inadequate Color Contrast in Key Areas: The color scheme used in sections like "Top Energy Costs" and the CTAs at the bottom of the page did not meet accessibility guidelines, affecting readability and visibility.
Low Placement of Top Energy Usage Days: Users expressed a desire for quicker access to information about their highest energy usage days, suggesting that this data would be better placed higher on the page for easy reference.
Recommendations
Based on the insights gathered from this additional test, we recommended several design adjustments to improve clarity, accessibility, and usability:
Restructure Page Hierarchy
Recommendation: Elevate the "Payment Agreement" option within the page hierarchy when relevant, potentially introducing dynamic error messaging to prompt its use when appropriate. Additionally, consider permanently raising the option within the Payment Options page.
Switch to Common Naming Conventions
Recommendation: Update labels to follow common naming conventions to better align with user expectations. Rename “Payment Agreement” to a more intuitive label such as “Payment Plans” or “Need Help Paying Your Bill.”
Ensure Colors Meet Accessibility Standards
Recommendation: Ensure that all colors used for key information and CTAs align with current accessibility guidelines, providing adequate contrast to improve readability and usability.
Elevate Key Information in Page Hierarchy
Recommendation: Position data about highest energy usage days higher on the "Analyze My Usage" page, allowing users to quickly access this important information.
This additional testing provided valuable insights and actionable recommendations that, although not implemented in Round 2, have been noted as priorities for future updates to further enhance the user experience.
How We Approached our Conceptual Mocks
We wanted to test how 2 different approaches resonated with users. Prior user feedback was mainly positive and many users noted that they enjoyed the simplicity and utility of the app. Knowing this, we wanted to approach the usability testing with 2 separate options to gauge their appetite for more fondational changes.
Option A This option focused on small optimizations to the existing homepage and navigation, without introducing complex functionality and changes. It builds upon what is already working, does not introduce big changes that restructure the information architecture and reduces risk by prioritizing small incremental improvements while paving the way for future enhancements.
Option B All the small optimizations included in Option A, along with usage data display on the homepage and a redesigned outage page. It introduces the concept of a centralized home ‘dashboard’ that allows users access to key information like payment information and usage in one place. Option B has the potential to.
Round 2 - Option A Usability Testing
Study goals Validate potential solutions to friction points observed in prior tests, learn about users' expectations and any possible new friction points by evaluating small optimizations to the existing homepage and navigation.
Key tasks:
Evaluate the homepage
Navigate to where you can sign up for outage alerts and sign up for alerts
Navigate to where you can add a new account to your Evergy app and add the account
Navigate to where can you locate information about a power outage in your area and evaluate the outage information provided
Sample Size The test was conducted with 5 participants:
Location: 3 from Kansas, 2 from Missouri
Audience Definitions:
Household Income: $80,000-$174,99
Age Spread: 26–52 (Average age: 36)
Gender: 3 Female, 2 Male
Requirements: All participants were Evergy customers and mobile app users
.
Round 2 - Option B Usability Testing
Study goals Validate potential solutions to friction points observed in prior tests, learn about users' expectations and any possible new friction points by evaluating new homepage dashboard features and the redesigned outage page.
Key tasks:
Evaluate the homepage
Navigate to where you can sign up for outage alerts and sign up for alerts
Navigate to where you can select a payment plan and sign up for a payment plan
Navigate to where can you locate information about a power outage in your area and evaluate the outage information provided
Insights Across Both Tests
Power status resonated with users: The power status component was well received by users. With some users noting that it exceeded their expectations.
Power status component was missed by some: Two users glossed over the outage details provided in the power status component entirely, going directly to the outage map.
Interest in a non-emergency phone number: Users called out a desire for a non-emergency number in addition to the emergency line, noting the possibility that only listing the emergency number could result in misuse.
Outage map caused confusion: Multiple users found the outage map confusing and weren’t clear on how to decipher the icons or legend.
Recommendations:
Explore introducing a power status to key pages: User feedback indicated that the introduction of a power status component would improve the overall experience.
Ensure the power status component has appropriate weight: Redesign the component to address its prominence on the page and the clarity of key details within.
Provide a non-emergency phone number: Consider adding a non-emergency phone number in addition to the emergency line, including clear direction on when to use each.
Introduce more clarity in the outage map: Open discussions with the current vendor to explore potential solutions for map functionality and clarity of design
Test Specific Insights & Recommendations
Option A Insights
Copy changes to Manage Account improved task completion: Changing the label in the Manage Account section from ‘Register Account’ to ‘Add Account’ led to a visible improvement. All users were able to complete the task more quickly than in R1 testing.
Recommendation:
Adjust label in Manage Account to Add Account: Switch ‘Register Account’ in the Manage Account section to ‘Add Account’ to align to user mental models and create parity with the subsequent page.
Option B Insights
Additional usage data features could be valuable: Users had varying expectations around how usage data was presented. Some expressed interest in additional functionality like projected usage, average daily usage, and a daily usage view.
Copy changes to Payment Agreement created clarity: Users had an easier time finding the correct path with language oriented to the core need (‘Need help paying your bill?’).
Treating Payment Agreement prominently was effective: Pairing the Payment Agreement content dynamically, and contextually with past due messaging, increased discoverability. However, some users still looked further down the page for this option initially.
Recommendations:
Iterate and test usage data component: Explore additional usage data features like projected usage, average daily usage, and a daily usage view. Further iteration and testing is recommended.
Adjust label in Payment Agreement: Edit the ‘Payment Agreement’ label to better align to user mental models, i.e., ‘Need help paying your bill’ or ‘Payment plans’.
Provide Payment Agreement option in key locations: When contextually relevant, use dynamic logic to elevate Payment Agreement higher and pair with past due messaging. Provide the option both dynamically and on-page to increase findability.
Tree Testing
Tree testing was ran to validate the current app's IA to see if any improvements need to be made. All task scores, completion rates, first click rates, and directness rates indicated that no changes were needed. Users had no problems finding the correct destinations. Our recommendation to the client was to run a tree test before the introduction of any major new categories to ensure that it follows user mental models and expectations.
Conclusion
Our research and usability testing for Evergy’s mobile app uncovered key areas for improvement, including navigation clarity, content organization, and accessibility. Recommendations such as refining the homepage layout, enhancing navigation contrast, and standardizing naming conventions aim to streamline the user experience. By addressing user feedback and incorporating features like usage data on the homepage, these insights provide a clear path for improving the app while strengthening our client relationship for future collaboration.
