The 44th episode of Datacast is my conversation with Shreya Shankar— a Machine Learning Engineer at Viaduct. Give it a listen to hear about her Computer Science education at Stanford, her research in Machine Learning Security at Google AI, the challenges of building Machine Learning system in production, the difficulty of recruiting female engineers, and much more.
Listen to the show on (1) Spotify, (2) Apple Podcasts, (3) Google Podcasts, (4) Stitcher, (5) Overcast, (6) Breaker, and (7) TuneIn!
Key Takeaways
Below are highlights from my conversation with Shreya:
On Studying Computer Science at Stanford
When I first showed up at Stanford, I initially wanted to major in Economics. But in the fall quarter of my freshman year, I also took an introduction to CS course because everyone in my down was taking one, and I didn’t want to miss out. The course was so well-taught, while I never showed up for any of my Econ courses. I realized that I should probably major in the field that I actually went to class for and enjoyed.
There was also a vibrant community around teaching people how to code. I went to it solely because of the community like Women in CS and the potential that coding was advertised.
My all-time favorite CS course at Stanford is Advanced Topics in Operating Systems, where we read papers every week about system design. Getting that bird-eye view on how to be a good engineer, in general, is such a valuable skill that a lot of CS courses did not teach.
On Being a Teaching Assistant at Stanford
The biggest thing that I learned was the ability to communicate technical concepts to people who are not technical. In the real world, you are not in a position of authority to explain anything and need to have a humble perspective.
When I started as a TA in my freshman year, I had seniors and graduate students in my group. I learned how to distill technical concepts in a way that got them excited about tech and wanted to work in the same team as mine.
On Co-Directing SHE++
There are tons of hurdles that keep women away from considering technology as a career path. The thing that is most striking to me is that there doesn’t seem to have much women representation in technical roles. When going to college, it didn’t appear that I could be an engineer, because you couldn’t be what you couldn’t see.
The biggest initiative that I helped run at SHE++ is Pao Include. We paired high school students with college mentors to develop computer science education initiatives in their hometowns, like hackathons or summer camps. We also flew 25–30 high school students to the Bay Area every year and took them around tech companies. I think just being able to see somebody in a position you can potentially be in is game-changing.
On Being a Software Engineer Intern at Facebook
I learned that I was not fond of software engineering much, as I didn’t have any unique skill set to contribute. Thankfully, my manager guided me towards a more multi-faceted role working with designers and PMs. This role culminated in a two-week data science sprint at the end of the summer to predict whether a FB page or profile belongs to a politician, based on the text in those pages or profiles.
The experience helped me think about the types of roles that I wanted to pursue after graduation. I enjoyed the collaboration with people, but not necessarily being a software engineer at a big company. It was helpful to learn that in three months, rather than signing up for that after graduation for three years.
On Researching ML Security at Google Brain
It’s almost complete luck. My former manager at Brain reached out to me and was interested in having an undergraduate researcher. I felt into ML Security by accident because it seems to be a field with a lot of interest during 2017, but there was not much novel work published.
What sustained my interest was the excitement of working with a toy problem in ML security that could have implications down the line. Over the past several years, it’s easy to construct adversarial attacks, but quite hard to construct robust defenses. I am currently interested in literature that uses adversarial training methods to make the classifier more robust to other situations.
I was exposed to colleagues who look at technical problems with curiosity. I was also impressed with people who are exceptionally good at math and good at explaining them. Wherever I work now, I always want to work with incredible people.
On Majoring in Computer Systems
The skillset that I wanted to develop in college is learning how to learn. Computer system is a good track to teach people how to learn new concepts that look daunting at first. System principles have been fundamental to computer science for a long time.
“Designing Data-Intensive Applications” by Martin Kleppmann is an incredible overview of computer systems from a data perspective.
Stanford’s CS110 course provides a great introduction to fundamental systems concepts and coding assignments.
On Debunking Misconceptions Between Research and Industry ML
ML in practice is a lot about the data than about the models. The biggest gains in system performance come from adding new data sources or rewiring the train and test split.
In applied ML, the engineering around building the systems — (1) to do ML and (2) allow multiple data scientists to collaborate/iterate on their models without losing their results — is way more challenging than coding the algorithms themselves.
The silo between data scientists and ML engineers need to be bridged in some way. Data scientists are fundamentally different from engineers — as they care about building insightful models to deliver value, not on specific tooling at a tactical level.
On Recruiting Female Engineers
There are many women in CEO, COO, and CMO positions at tech companies, but not many women individual contributors.
In terms of hiring, I wish there was more representation of women who code day-to-day, do system design, and contribute to open-source projects.
Women in AI and Women in DS are incredible groups that you can follow.
On Advocating For Mental Health
I am trying to develop intrinsic motivation to do work, even when I feel depressed. It’s tough for me because of bipolar disorder.
Working with a therapist has been helpful, but I need to accept that sometimes I can only perform at 50% capacity. I felt so much better to have a therapist whom I can trust as a mental outlet. I am glad that mental illness has been de-stigmatized to some extent.
Show Notes
(2:02) Shreya discussed her initial exposure to Computer Science and her favorite CS course on Advanced Topics in Operating Systems at Stanford.
(4:07) Shreya emphasized the importance of distilling technical concepts to a non-technical audience, thanks to her experience as a section leader and teaching assistant for CS198.
(6:26) Shreya shared the lack of representation in technical roles that keep women away from considering technology as a career path, and the initiative she was involved with at SHE++.
(9:40) Shreya reflected on her software engineering internship experience at Facebook, working on Civic Engagement tools to help representatives connect with their constituents.
(12:33) Shreya went over the anecdote of how she worked on Machine Learning Security research at Google Brain.
(15:36) Shreya unpacked the paper “Adversarial Examples That Fool Both Computer Vision and Time-Limited Humans,” — where her team constructs adversarial examples that transfer computer vision models to the human visual system.
(20:08) Shreya reflected on the lessons learned from her experience working with seasoned researchers at Google Brain.
(23:31) Shreya gave her advice for engineers who are interested in multiple specializations.
(25:34) Shreya provided resources on the fundamentals of computer systems.
(27:15) Shreya explained her reason to work at an early-stage startup right after college (check out the blog post on her decision-making process).
(28:41) Shreya was the first ML Engineer at Viaduct, a startup that develops end-to-end machine learning and data analytics platform to empower OEMs to manage, analyze, and utilize their connected vehicle data.
(32:27) Shreya discussed two common misconceptions people have about the differences between machine learning in research and practice (read her reflection on one-year of making ML actually useful).
(35:24) Shreya expanded on the organizational silo challenge that hinders collaboration between data scientists and software engineers while designing a machine learning product.
(40:48) Shreya has been quite open about the challenge of recruiting female engineers, explaining that it is hard to sell women candidates when their alternatives are “conventionally sexy.”
(47:24) Shreya and a few others have developed and open-sourced GPT-3 Sandbox, a library that helps users get started with the GPT-3 API.
(51:52) Shreya explained her prediction on why OpenAI can be the AWS of modeling.
(54:24) Shreya shared the benefits of going to therapy to cope with mental illness challenges.
(58:36) Closing segment.
Shreya’s Contact Info
Shreya’s Recommended Resources
Martin Kleppmann’s “Designing Data-Intensive Applications”
Stanford’s CS110 — “Principles of Computer Systems”
Steve Krug’s “Don’t Make Me Think”