[comment]: # (Set the theme:) [comment]: # (The list of themes is at https://revealjs.com/themes/) [comment]: # (The list of code themes is at https://highlightjs.org/) [comment]: # (Pass optional settings to reveal.js:) [comment]: # (markdown: { smartypants: true }) [comment]: # (Other settings are documented at https://revealjs.com/config/) ### Jayson Webb, PhD UX Research Manager | 16 June 2023 [comment]: # (A comment starting with three or more !!! marks a slide break.)

## Fun Facts - <span style="text-align:left"> I live in Boulder, CO with my wife Erika

## Fun Facts - <span style="text-align:left;opacity: 0.5">I live in Boulder, CO with my wife Erika </span> - <span style="text-align:left">I go by "Jay"

## Fun Facts - <span style="text-align:left;opacity: 0.5">I live in Boulder, CO with my wife Erika - <span style="text-align:left;opacity: 0.5">I go by "Jay" - <span style="text-align:left">We have 2 dogs and a cat

## Fun Facts - <span style="text-align:left;opacity: 0.5">I live in Boulder, CO with my wife Erika - <span style="text-align:left;opacity: 0.5">I go by "Jay" - <span style="text-align:left;opacity: 0.5">We have 2 dogs and a cat - <span style="text-align:left">We have 2 daughters, Maddie and Amelia

## Roy <img src="media/20230122_065318.jpg" height="100%" width="100%">

## Desi <img src="media/20230429_081357.jpg" height="30%" width="30%">

## Boo <img src="media/20230429_155412.jpg" height="30%" width="30%">

## The Fam <img src="media/20230527_102419_0_.jpg" height="70%" width="70%">

## Background

## Path to UX research - <span style="text-align:left;">Human Factors as an undergrad

## Path to UX research - <span style="text-align:left;opacity: 0.5">Human Factors as an undergrad</span> - <span style="text-align:left;">UX research internship at HP

## Path to UX research - <span style="text-align:left;opacity: 0.5">Human Factors as an undergrad</span> - <span style="text-align:left;opacity: 0.5">UX research internship at HP</span> - <span style="text-align:left;">UX research my whole career

## Career Overview - Large B2B Enterprises (Twilio, Oracle, HP)      

## Career Overview - <span style="opacity: 0.5">Large B2B Enterprises (Twilio, Oracle, HP)      </span> - Founder and principal in consulting companies

## Career Overview - <span style="opacity: 0.5">Large B2B Enterprises (Twilio, Oracle, HP)      </span> - <span style="opacity: 0.5">Founder and principal in consulting companies - Strong advocate for quantitative UX research

## Career Overview - <span style="opacity: 0.5">Large B2B Enterprises (Twilio, Oracle, HP)      </span> - <span style="opacity: 0.5">Founder and principal in consulting companies - <span style="opacity: 0.5">Strong advocate for quantitative UX research <p style="text-align:left;margin-left:70px;font-size: 0.7em"> Classical training in statistics (ANOVA, regression, factor analysis, cluster analysis) and probability during my graduate and undergraduate work and have worked to improve my quantitative skills throughout my career. </p>

## Roles - Built and led teams (both research and data science/analytics)

## Roles - <span style="opacity: 0.5">Built and led teams (both research and data science/analytics)</span> - Lead mixed-methods researcher

## Roles - <span style="opacity: 0.5"> Built and led teams (both research and data science/analytics) - <span style="opacity: 0.5">Lead mixed-methods researcher</span> - Trained and educated teams

## Roles - <span style="opacity: 0.5"> Built and led teams (both research and data science/analytics) - <span style="opacity: 0.5">Lead mixed-methods researcher</span> - <span style="opacity: 0.5"> Trained and educated teams - I have evangelized and advocated for UX research

## Roles - <span style="opacity: 0.5"> Built and led teams (both research and data science/analytics) - <span style="opacity: 0.5">Lead mixed-methods researcher</span> - <span style="opacity: 0.5"> Trained and educated teams - <span style="opacity: 0.5"> I have evangelized and advocated for UX research - I worked with research ops closely in my last position

### Why a UX research manager? - Whether I'm managing people or not, I love UX research

### Why a UX research manager? - <span style="opacity: 0.5"> Whether I'm managing people or not, I love UX research - I like managing and mentoring people

### Why a UX research manager? - <span style="opacity: 0.5"> Whether I'm managing people or not, I love UX research - <span style="opacity: 0.5"> I like managing and mentoring people - I like strategic thinking and working with cross-functional stakeholders

# Case Studies

## PaaS trial signup PaaS = Platform as a Service (e.g. compute, storage)

## Situation - The PaaS organization wanted to improve the signup completion rate for getting a PaaS services trial account

## Situation - <span style="opacity: 0.5"> The PaaS organization wanted to improve the signup completion rate for getting a PaaS services trial account - About **3%** of potential trial customers who started the signup flow completed it

## Situation - <span style="opacity: 0.5"> The PaaS organization wanted to improve the signup completion rate for getting a PaaS services trial account - <span style="opacity: 0.5"> About **3%** of potential trial customers who started the signup flow completed it - PaaS UX Design director asked me what research we could do to help improve signup rate

### Approach, Challenges, Impact - I advocated for using analytics to understand dropoff in the signup flow

### Approach, Challenges, Impact - <span style="opacity: 0.5"> I advocated for using analytics to understand dropoff in the signup flow - PaaS wasn’t using Adobe Analytics data

### Approach, Challenges, Impact - <span style="opacity: 0.5"> I advocated for using analytics to understand dropoff in the signup flow - <span style="opacity: 0.5"> PaaS wasn’t using Adobe Analytics data - Data-informed design changes led to an increase in trial signup rate from 3% to 8.5%

## Challenges - Biggest challenge, just getting access to Adobe Analytics data - Next biggest challenge, mapping the analytics data to the UI

#### Not all elements instrumented <img src="media/Screenshot_from_2023-06-14_12-05-19.png" height="90%" width="90%">

### Problem Areas <img src="media/Screenshot_from_2023-06-14_12-17-18.png" height="90%" width="90%">

## Conclusion - The data convinced the org of the main areas to be improved

## Conclusion - <span style="opacity: 0.5"> The data convinced the org of the main areas to be improved - Qualitative research was then done by the designers to iterate and evaluate solutions

## Conclusion - <span style="opacity: 0.5"> The data convinced the org of the main areas to be improved - <span style="opacity: 0.5"> Qualitative research was then done by the designers to iterate and evaluate solutions - The redesigns were a success

## Conclusion - <span style="opacity: 0.5"> The data convinced the org of the main areas to be improved - <span style="opacity: 0.5"> Qualitative research was then done by the designers to iterate and evaluate solutions - <span style="opacity: 0.5"> The redesigns were a success - This was the beginning of the trajectory that led to me getting headcount for data scientists

#### Building a UX Research Data Science Program

## Situation - I had been the staff UX researcher for the Developer Experience business unit for about 10 months

## Situation - <span style="opacity: 0.5"> I had been the staff UX researcher for the Developer Experience business unit for about 10 months - I advocated for getting more quantitative data into our practice of understanding our users and improving their experience

## Situation - <span style="opacity: 0.5"> I had been the staff UX researcher for the Developer Experience business unit for about 10 months - <span style="opacity: 0.5"> I advocated for getting more quantitative data into our practice of understanding our users and improving their experience - I presented a vision for what a quantitative UX research program could look like

## Situation - <span style="opacity: 0.5"> I had been the staff UX researcher for the Developer Experience business unit for about 10 months - <span style="opacity: 0.5"> I advocated for getting more quantitative data into our practice of understanding our users and improving their experience - <span style="opacity: 0.5"> I presented a vision for what a quantitative UX program could look like - I got the green light to build that program, reporting to the Senior Director of UX

## Additional Context - Heady times at Twilio (July 2021). Hypergrowth mentality. Encouraged to think long term (3 year vision)

## Additional Context - <span style="opacity: 0.5"> Heady times at Twilio (July 2021). Hypergrowth mentality. Encouraged to think long term (3 year vision) - Fast forward to February 2023. Opposite mentality. Mass layoffs (as there were in the Fall of 2022), contraction of visions.

## The Team - A data engineer and a data scientist (hired in late Fall 2021) - A data analyst (hired in January 2022) - An additional data scientist (hired in March 2022)

## Vision - Get behavioral data across web properties (logged in and not) and CLI/API usage

## Vision - <span style="opacity: 0.5"> Get behavioral data across web properties (logged in and not) and CLI/API usage - Behavioral Personas. Segment users based on their cross-product behavior and account characteristics

## Vision - <span style="opacity: 0.5"> Get behavioral data across web properties (logged in and not) and CLI/API usage - <span style="opacity: 0.5"> Behavioral Personas. Segment users based on their cross-product behavior and account characteristics - Find stumbling blocks in the user journey

## Vision - <span style="opacity: 0.5"> Get behavioral data across web properties (logged in and not) and CLI/API usage - <span style="opacity: 0.5"> Behavioral Personas. Segment users based on their cross-product behavior and account characteristics - <span style="opacity: 0.5"> Find stumbling blocks in the user journey - Enhance the work of researchers and research ops

## Biggest Challenges - Getting the right data to support the program. Implement Segment from the ground up

## Biggest Challenges - <span style="opacity: 0.5"> Getting the right data to support the program. Implement Segment from the ground up - Managing a team of data folks in a UX organization that was matrixed into a development organization

## Biggest Challenges - <span style="opacity: 0.5"> Getting the right data to support the program. Implement Segment from the ground up - <span style="opacity: 0.5"> Managing a team of data folks in a UX organization that was matrixed into a development organization - Management stakeholders who greenlighted the creation of the group had all left by about the half-way point in our journey

## Management Style - Inspiration and psychological safety - Belongingness and identity - Career growth - Accountability and performance management

## What did I do? - Manage the team - Translate the vision into tactical activities - Occasional ad hoc analyses - how many users do behavior x with product y? What documents are consumed during the first week of a user's experience? - Ran sprint planning and standups - Roadshows/communication

# Thank You A word about creating this presentation..