The 49th episode of Datacast is my conversation with Carl Gold— the Chief Data Scientist at Zuora. Give it a listen to hear about his electrical engineering background in Stanford, his Ph.D. work in computational neuroscience at CalTech, his move from academic to working as a quant analyst for a Wall-Street finance firm, his transition into data science, the Subscription Economy Index, “Fighting Churn with Data,” and much more.
Listen to the show on (1) Spotify, (2) Apple Podcasts, (3) Google Podcasts, (4) Stitcher, (5) iHeart Radio, (6) Radio Public, (7) Breaker, and (8) TuneIn
Key Takeaways
Below are highlights from my conversation with Carl:
On Studying Electrical Engineering at Stanford
My college experience at Stanford was great. In the technology world, there was a lot more optimism back when the Internet was first invented. There were also fewer negative consequences that have come out of technology. Today, Palo Alto is the most affluent, expensive, gentrified town you’ll find. But back in the 90s, Palo Alto wasn’t even gentrified with diverse communities.
I was attracted to electrical engineering because I wanted a fundamental understanding of computers. But the truth is I gave up electrical engineering as soon as I graduated. I didn’t remember much about the courses that I took, but such knowledge turned out to be useful for my Ph.D. work later on.
On Pursuing a Ph.D. in Computational Neuroscience at CalTech
The computational and neural systems program at CalTech brings together neuroscience and AI researchers, hoping that the two fields can reinforce each other to understand both the brains and machine learning better. As someone who had learned both non-neural machine learning and neural-based learning, I was attracted to the theory of neural networks.
I have to say that I had a big disillusionment in that program. When I went to it, I thought that the neural networks research was closely related to the brain. The program would take in people who didn’t have any neuroscience background, gave us core training in neuroscience, and assigned us to work in different labs. Once I actually learned about neuroscience, I quickly realized that neural networks don’t have anything to do with how the brain works. Many people in the machine learning and data science community do not appreciate that the models they use don’t map well to the human brains. There is a lot of debate over this.
My thesis ended up examining the brain from the first principles. The research is called “Biophysics of Extracellular Action Potentials”:
An action potential is an electric spike in a neuron, which propagates down the nerve fiber like a pulse.
Neurons also broadcast electric fields around them extracellularly. Such an extracellular field has many different shapes, meaning that the spike's polarity could be positive or negative. This was unusual because there could only be one polarity inside the cell.
There was not much understanding of how the spikes inside the cell led to the electric field outside the cell.
I worked on a biophysical model based on physics and electricity. That was where my electrical engineering background came in handy. I used this model to compare with real-world data from extracellular recordings to figure out what explained the different characteristics of the extracellular spikes.
On Transitioning Into Quantitative Finance
Right towards the end of my Ph.D., I was getting married and expected our first child. If I continued the academic path, I would do post-docs in biology, usually 8 years long with a low salary. The option of leaving academia and going to the industry made sense.
In 2007, there was a different perception of the finance world in a strong economy. There was a general belief that quantitative finance genuinely was doing good for the world and making a lot of money for those involved.
Until data science became a thing, if you did a Ph.D. in science and did not want to stay in academia, your best option was Wall Street.
On Valuable Lessons Learned at Wall Street
I saw the power of statistics and explainable models:
In the world of fixed incomes, I worked within data-scarce environments — considering that bonds are illiquid and don’t get traded very often.
The intuition and domain-knowledge of the traders became key. Any models which did not contribute to the traders’ knowledge and understanding would not be accepted.
Machine learning would certainly not be accepted back then and probably still isn’t now. The slow pace of trading means you don’t have many observations to train your model on.
I also saw the importance of not having correlations in input features:
Because finance mostly uses statistically-based regression models, the uncorrelated features were essential.
Considering the interpretability requirement, we wouldn’t use techniques like PCA to make things un-correlated. We would have to understand the problem and engineer features that are powerful yet un-correlated.
On Transitioning Into Data Science
After the financial crash in 2008, the image of finance took a beating. The work environment and compensation in finance wasn’t so good anymore. During this period, data science was exploding, and machine learning/statistics was at the heart of it. Given my academic background, I decided to make the transition.
I ended up taking my first data science job thanks to a connection with a friend. His startup, Sparked, has a data science-based product for recommending people and projects. I got involved with their churn analysis projects based on their customer requests.
On Subscription Business Model
In a subscription business model, your customers sign up and commit to using your product. It is now the main model for software delivery.
Freemium is a kind of subscription service with a free option. The free tier offers a basic level of service.
Direct-to-consumer and in-app purchases are other popular business models.
Churn analysis remains critical for any business models with recurring revenue / repeated sales.
On Key Trends In The Subscription Economy
We observed a drop in the growth rate of subscription companies’ sales. Last year, the average growth rate for them was 20%. This year, given the current crisis, that number became 12%. However, for the non-subscription-based companies, sales contracted by 11%.
There are many new and exciting subscription-based products/services that we rely on to deal with this crisis.
On The Difficulties of Fighting Churn
Churn is hard to predict because you never have complete information about your customers.
Churn is harder to prevent because you have to make sure that your customers get more value from your product.
Churn is a multi-team effort because you need to make a better product (product and engineering), design a better customer experience (marketing), or create a more attractive pricing plan (sales and finance).
On Creating Great Customer Metrics
The basic behavioral metrics are not effective because many customer behaviors are correlated with others.
The ratio metrics are an interpretable way to unpack such correlation effects (ad view per post, dollars per call, etc.). These metrics are also great features to improve your machine learning models for relevant prediction/forecasting tasks.
On Writing “Fighting Churn With Data”
The book's final version uses a simulation dataset to illustrate many concepts because I didn’t have a true churn dataset to share (customer data is very sensitive).
I had to update earlier chapters to accommodate the direction I took in later chapters.
On The Benefits of An Academic Background
In graduate school, I got a lot of practice with scientific methods to master a subject via reading papers and conducting research.
Figuring out what to learn and chasing references is also a crucial skill that I picked up. All science is built on past work.
Computer scientists tend to reinvent things. In a computer science education, you focus on creating your own algorithms and not getting any credit from copying others. But for data science, you must do the background research and understand state-of-the-art before building your own.
Lastly, hypothesis testing is a valuable mindset to have when dealing with uncertainty. There are many possible hypotheses to test, so you have to use your domain knowledge and intuition to come up with the right ones to test.
Show Notes
(1:57) Carl recalled his undergraduate experience studying Electrical Engineering at Stanford back in the early 90s.
(3:58) Carl recalled his graduate experience pursuing Master's degrees in Computer Science at NYU and King’s College in the late 90s. For his Master’s Thesis, he investigated Support Vector Machines with a Bayesian algorithm programmed in C.
(6:45) Carl walked over his Ph.D. work in Computation and Neural Systems at CalTech, where he did a thesis on Biophysics of Extracellular Action Potentials.
(13:11) Carl provided brief thoughts about his experience working as a business analyst and consultant for HBO during his Ph.D. period.
(14:55) Carl went over his rationale behind his decision to move from academic neuroscience to quantitative finance.
(19:19) Carl discussed his proudest accomplishments and valuable lessons learned from spending seven years at Morgan Stanley Capital International and rising to a leadership role as Vice President of Risk Modeling.
(23:17) Carl uncovered his move to San Francisco to work as a lead data scientist at Sparked back in 2014, which builds a customer success SaaS solution.
(27:10) Adding to his move to Zuora in 2015, Carl explained how the subscription business model works in layman terms.
(31:44) Carl unpacked the common patterns that he saw from analyzing subscriber churn for companies across industries due to his work on Zuora Analytics.
(33:30) Carl shared the process of creating the Subscription Economy Index, Zuora’s landmark index tracking the rapid ascent of the Subscription Economy, and distilled the key trends of the 2020 edition.
(39:59) Carl unpacked the three reasons that make churn hard to fight: (1) Churn is hard to predict, (2) Churn is harder to prevent, and (3) Churn requires a multi-team effort (Watch his talks at the 2019 Data Council San Francisco and the 2020 Subscribed Online Conference).
(44:46) Carl shared advice for data scientists who want to collaborate more effectively with other functional departments.
(46:30) Carl emphasized the importance of creating great customer metrics, which are ratios of basic behavioral metrics to fight churn effectively.
(53:49) Carl went over the challenges of writing “Fighting Churn With Data,” which provides a clear overview of churn concepts, along with hands-on tricks and tips developed through years of experience analyzing customer behavior.
(55:53) Carl reflected on how his academic background in computational neuroscience contributes to his success as a quant analyst and a data scientist.
(59:37) Carl compared his experience living and working across Los Angeles, New York, and San Francisco.
(01:02:04) Closing segment.
His Contact Info
His Recommended Resources
“Fighting Churn With Data” by Carl Gold
Konrad Kording (Professor of Computational Neuroscience at the University of Pennsylvania)
Kate Crawford (Distinguished Research Professor in Tech, Culture, and Society at New York University)
Cassie Kozyrkov (Chief Decision Scientist at Google)
“Freakonomics” by Stephen Dubner and Stephen Levitt
Here are the discount codes that you can use to purchase “Fighting Churn with Data” with 40% off:
fcddcr-6D84
fcddcr-4AE7
fcddcr-6D9C
fcddcr-30BF
fcddcr-9705
About the show
Datacast features long-form conversations with practitioners and researchers in the data community to walk through their professional journey and unpack the lessons learned along the way. I invite guests coming from a wide range of career paths - from scientists and analysts to founders and investors — to analyze the case for using data in the real world and extract their mental models (“the WHY”) behind their pursuits. Hopefully, these conversations can serve as valuable tools for early-stage data professionals as they navigate their own careers in the exciting data universe.
Datacast is produced and edited by James Le. Get in touch with feedback or guest suggestions by emailing khanhle.1013@gmail.com.
Subscribe by searching for Datacast wherever you get podcasts or click one of the links below:
If you're new, see the podcast homepage for the most recent episodes to listen to, or browse the full guest list.