James Le

View Original

2020 Annual Review: The Year of Resilience

James Le

Every December since 2015, I have taken some time to write my annual review, a practice that was originally inspired by James Clear (see my 2019, 2018, 2017, 2016, 2015, and 2014 versions). The review's goal is to reflect on the previous year and identify what went well, what could have gone better, and what I’m working toward. This practice enables me to celebrate the efforts and milestones I have made over the past 12 months while examining the bottlenecks and limitations I could have improved upon. The review is a deeply personal report, letting me see myself for who I am and think about the type of person I want to become. This year’s format is inspired by David Perell, which consists of five parts. I’ll start with the highlights, reflect on 2020 goals, set 2021 goals, celebrate milestones, review areas of improvement, and conclude with a list of open questions.

Highlights Of The Year

1 — Reflecting on 2020 Goals

1.1 — Finishing Graduate School

My biggest achievement is definitely completing my M.S. program in Computer Science at RIT in early December. These past 2.5 years have equipped me with a rigorous level of academic pursuit, research expertise, and programming sharpness. I still recalled my first semester at RIT two years ago, which was probably the most intense period of my education due to the challenging workload. That semester I took “Deep Learning For Vision” — a highly challenging course that formally introduced me to machine learning research. This past fall, I became a Teaching Assistant for that course (thanks to Professor Kanan): helping students with programming assignments, research projects, and course lectures. It’s amazing how much I learned throughout these years.

My Master’s thesis lies at the intersection of Meta-Learning and Recommendation Systems. More specifically, I applied an algorithm called Model-Agnostic Meta-Learning to improve collaborative filtering systems' accuracy performance. Throughout this process, I also wrote a series of blog posts about recommendation systems and meta-learning, which have gathered some public attention. Thanks to my advisor Alex Ororbia and other members at the Neural Adaptive Computing Lab, who have supported me in the final weeks of completing the thesis write-up. I hope to publish this work at a top-tier academic conference next year, such as RecSys, KDD, SIGIR, WSDM, to name a few.

The Meta-Rec Framework in my thesis

1.2 — Writing

I wrote 39 blog posts in 2020 (all cross-posted on my Medium account). Besides the ones which are podcast recaps (discussed below), here are the posts that I’m most proud of:

  • Data management and model deployment practices: these are lecture summaries from the Full-Stack Deep Learning Bootcamp I attended in November 2019, which led to fruitful collaboration later in the year (discussed later).

  • “Meta-Learning Is All You Need” is the inaugural piece on my meta-learning series. I think this line of research is groundbreaking as real-world data is limited and long-tail. I plan to continue this series next year, diving into topics such as continual learning and multi-task learning.

  • Recaps of virtual events, including Spark AI and Toronto ML summits: I have had a habit of writing up my experience attending events since last year. It’s a fantastic way to synthesize knowledge from the talks and connect with the speakers afterward. An upcoming event that I’m excited to recap is the REWORK Deep Learning Summit at the end of January.

1.3 — Podcasting

I released 25 podcast episodes in 2020 (with a few already recorded but yet to be released). The number of downloads increases by 41% as a result. Here are my top five favorite ones:

Growing a podcast sustainably requires patience and experiment. Since August, I have diligently written key takeaways from these conversations in both short-form post and Tweet thread. These practices seem to drive engagement incrementally.

Datacast downloads in 2020

1.4 — Side-Hustling

I got involved with the Full-Stack Deep Learning team around the summer — putting together the boot camp's online version and helping with the bi-weekly virtual meetup about production ML (check out our Twitter to know what our industry speakers share at these meetups). We will provide an updated version of the course next spring, so sign up if you’re interested!

I also started working part-time for Snorkel AI under the Marketing team around the fall — putting together its solutions page and crafting a blog post that recaps our CEO’s talk at the MLSys seminar. Snorkel provides a great solution called programmatic data labeling to tackle the challenges of getting good training data for machine learning pipelines, so follow our content for more information!

2 — Goals for 2021

2.1 — Diving Deeper Into MLOps

I’m excited to start a new role soon with Superb AI, an enterprise-grade training data platform. Training data is still the biggest bottleneck of the ML development process, especially since the importance of proper data operations within an ML organization continued to increase. The Superb AI team is building a product that could help facilitate awareness and education around building better AI, starting with proper data management principles. Given my increasing level of interest in production-level machine learning in the past year, this is a great opportunity to be in the trenches and navigate the ML tooling landscape.

If readers want to learn more about MLOps, I suggest joining this Slack community.

2.2 — Exploring AI Safety Research

Advances in AI within the past decade have brought excitement about its positive potential to transform science, medicine, transportation, along with concerns about the privacy, fairness, security, economic, and longer-term implications of powerful AI. It is worth giving serious thought to such potential challenges and risks. Many experts believe that there is a significant chance we’ll create artificially intelligent machines with abilities surpassing those of humans (superintelligence) sometime during this century. Researchers working on this problem aim to maximize the chance of a positive outcome while reducing the chance of catastrophe.

AI safety is a research line that addresses the problem of accidents in machine learning systems. Accidents here are defined as unintended and harmful behavior that may emerge from machine learning systems when we specify the wrong objective function, are not careful about the learning process, or commit other machine learning-related implementation errors. Under this broad theme, I am mostly interested in the value alignment problem. In an “aligned” AI system, the actions it pursues move the world towards states that humans want and away from states that humans don’t want. Next year, my goal is to explore literature that overcomes the conceptual or engineering issues need to create aligned AI more comprehensively.

If readers are interested in these ideas, I suggest looking at these AI Safety resources and this AI Alignment research overview.

2.3 — Getting More Involved With Effective Altruism

I have been a passive consumer of effective altruism content in the past three years, mostly via the EA newsletter and the 80,000-hour podcast. For those who are not familiar, effective altruism is a field that uses high-quality evidence and careful reasoning to work out how to help others as much as possible. It is also a community of people taking these answers seriously by focusing on the most promising solutions to the world’s most pressing problems.

This past November, I attended EA’s virtual student summit, a fantastic event that covered various problem profiles ranging from existential risk to animal welfare. I networked with tons of interesting young adults who are motivated to tackle these problems from that event. Since then, I have been participating in two reading groups on forecasting and global priorities research, where we read relevant content on these topics and discuss practical applications. Next year, my goal is to continue being a part of this community by attending more conferences/reading groups if time allows.

If readers are interested in being involved with this community, feel free to reach out, and I’ll add you to the Slack/Discord groups that I’m a part of!

The forecasting reading group I’m involved with

3 — Things To Improve

3.1 — Early To Bed, Early To Rise

Based on my Sleep Cycle app, my average sleep quality in 2020 is 70%. My average time in bed is 7 hours, with sleep time around midnight and waking-up time around 7 AM. This is obviously not ideal since I used to go to bed around 10 and woke up around 5 regularly just the year before. I suppose that the pandemic really made us anxious (around March-April, I watched COVID 19 videos religiously while in bed). Going into the new year, I aim to adjust this sleep window back to my 2019 routine.

3.2 — Saying No

I recently finished reading Greg McKeown’s “Essentialism” — which explains a disciplined, systematic approach for determining where our highest point of contribution lies, then making execution of those things almost effortless. If I took away only one thing from the book, it is about deliberately distinguishing the vital few from the trivial many, eliminating the non-essentials, and removing any obstacles so the essential things have a clear, smooth passage. It’s been a common thread since college that I said yes to many non-essential activities, leading to plenty of unnecessary stress along the way. Previously, I can reason that it’s important to take time exploring so I don’t end up stuck on a local maxima (read “Climbing the Wrong Hill” for the metaphor). But now, as I know I’m sort of on the right career path, it’s important to commit to specific things instead of just collecting options (read “The Trouble with Optionality” for a sophisticated take on this).

To say no more effectively, I need to identify those essential activities. That’s why I commit to spending January to figure them out (read the Career Planning section at the end of the review).

Greg McKeown’s “Essentialism”

3.3 — Fitness

Given that I spent most of the time indoor throughout 2020, it was challenging to find a consistent fitness rhythm. For the first three quarters of the year, I probably exercised three times a week, mostly doing HIIT and yoga. In the fall, when the campus gym re-opened, I got back to lift heavy weights and swimming, which was a major boost in mood and productivity. If there is a highlight in this realm, it would definitely be the YouTube channel “Yoga with Adriene.” Her yoga practices are mindful and compassionate if I have to describe them. Check out this New York Times article to learn more and support her work.

Next year, I certainly will take fitness more seriously. If readers have any advice on incorporating a fitness routine into your daily schedule, I’d love your suggestions!

3.4 — Cooking

Another aspect that I hope to improve is cooking. When moving to a new apartment in the fall, I got excited about this — reading “The 4-Hour Chef” and watching Gordon Ramsey’s MasterClass. However, as my workload piles up, I found it hard to spend proper time making complicated dishes from these resources. I ended up retreating to the easy meals that I could make.

Next year, I plan to make a genuine attempt to improve my cooking skill. If readers have any valuable resources, feel free to share them in the comment!

4 — Things To Celebrate

4.1 — Daily Journaling/Planning/Time-Blocking

I didn’t miss a single day of morning journaling. It’s a simple practice where I write down three things that I’m grateful for, three tasks that I want to accomplish, and one manifestation of who I want to be.

Later in the year, I bought the Productivity Planner from Intelligent Change. It’s a beautifully put together planner that integrates time-proven productivity hacks used by some of the most successful and productive people in the world. Using the planner has been critical to ensuring that I work on the most important task of the day first before tackling secondary tasks.

Finally, I time-block my calendar every night to make sure that I can commit to “deep work” for at least 50% of my next workday. I’m very proud of that.

My typical Google Calendar

4.2 — Virtual Connecting

I’ll remember 2020 as the year that I fell in love with virtual connections, as the working world embraces Zoom. Besides doing countless work and academic meetings, I also joined many virtual reading groups, attended many virtual meetups, and conducted many virtual interviews. One of my favorite experiences is a month-long essay club over the summer organized by Patricia Mou, who has a fantastic newsletter about wellness. We discussed a variety of essays ranging from the philosophy of reliance to the trouble with career climbing.

4.3 — Reading

In 2020, I read 27 books (only counting hard copies, no online books/white-papers/reports). Here are my top five:

  • Nassim Taleb’s “Anti-fragile”: After reading “The Black Swan” and “Skin In The Game” last year, I slowly became a devoted Taleb fan. The thesis of the book is that some systems thrive from shocks, volatility, and uncertainty. By adopting more anti-fragile traits myself, I can thrive in an uncertain and chaotic world. There are so many interesting concepts from the book relevant to our 2020: via negative, the Lindy effect, the Turkey problem, etc.

  • Ryan Holiday’s “Stillness Is The Key”: After reading “Ego Is The Enemy” and “Obstacle Is The Way,” I decided to complete Holiday’s trilogy. This is probably his best book! One-sentence summary: Stillness is the key to modern success — being steady while the world spins around, acting without frenzy, hearing only what needs to be heard, possessing quietude on command, etc. It makes room for gratitude, happiness, and inspiration.

  • Clayton Christensen’s “How Will You Measure Your Life”: I picked up this book after Clayton’s passing earlier this year. This HBR article does a great job of summarizing core ideas from the book.

  • Nadia Eghbal’s “Working In Public”: I chose this book due to my interest in open-source development and the passion economy. Eghbal offers a new taxonomy of communities — including newer phenomena such as “stadiums” of open source developers, other creators, and really, influencers who perform their work in massive spaces where the work is public (and not necessarily participatory). Also, listen to her a16z podcast episode for the key points.

  • Maria Konnikova’s “The Biggest Bluff”: As a previous fan of Annie Duke’s “Thinking In Bets,” I was not disappointed with this New York Times best-selling piece. The book is a whirlwind tour into human psychology and poker aficionados.

5 — Open Questions

5.1 — Growing A Newsletter

2020 is the year in which I consumed many newsletters, with the rise of a platform like Substack. Given my commitment to writing and podcasting, it’s challenging to find time to build a newsletter. But if my long-term goal is to identify my 100 true fans, then it’s critical to build an audience via email. There’s a lot of newsletters that round up links around the Internet. Some newsletters basically serve the purpose of a blog. I think combining these two formats would be ideal.

Then there’s the part about choosing what type of content to put in the newsletter. I believe the more niche the content is, the better subscriber engagement will be. Given my interest, looking at recommendation systems from both the technical and societal perspectives would be something that I like to explore.

5.2 — Community Building

Given my involvement with multiple communities, I have been more and more intrigued about the art of community building. For example, I have been listening to the Get-Together podcast, poking around community forums, and will join a reading group to read “The Art of Gathering” soon.

5.3 — Ph.D.

I applied to multiple Computer Science Ph.D. programs this past fall. A Computer Science Ph.D. offers the chance to become a leading researcher in a vital field with the potential for transformational research. While I remain open to exploration, I believe much of the research needed to enable recommendation systems better align with user values has not been done. For example, here are a couple of questions that I would like to consider during a Ph.D.:

  • How to construct optimization metrics that lead to objectives which are substantially well-aligned with human values? Recent work has provided theoretical frameworks for objective functions that capture the desired notion of “value.” I want to make those frameworks practical by building models that can learn complex human values and measure their response to the recommendations — borrowing ideas from the technical AI safety agenda.

  • What are the side effects of recommendation systems? How to effectively detect the undesired side effects on the user? Avoiding side effects has been studied during the design of reward functions for reinforcement learning. I aspire to apply that idea here and develop tools that can introduce guardrails against specific and quantifiable undesired side effects of the recommendations.

  • How to design systems that require minimal supervision from the user side? This would likely be an extension of my Master’s thesis work. More specifically, I am fascinated by the potential of other meta-learning frameworks and related domains (active learning, continual learning, multitask learning) to learn from limited user data.

There are both pros and cons of going this route. Here are the pros:

  • Potential for the large impact of my research.

  • Opportunity to become an expert in AI.

  • Freedom to pursue research topics that most interest me.

  • Brilliant colleagues.

  • Helps me enter technical jobs in the industry, providing a backup to academia.

And here are the cons:

  • Academia is a zero-sum game.

  • Only a small percentage of Ph.D. graduates end up with tenure-track jobs.

  • Takes a long time (5 to 7 years) with relatively low pay.

  • Doing highly open-ended research provides little feedback, which can be de-motivating.

  • About half of those who enter the industry afterward don’t end up with research positions.

I think for my situation, given the opportunity and financial costs, I’ll only pursue this route if accepted into a prestigious program with an exceptional research environment and well-funded fellowship.

5.4 — Career Planning

Lastly, thanks to my involvement with Effective Altruism later in the year, I want to get more serious about deliberate career planning. The best resource that I found on this topic is this updated guide from the 80,000 hours team. It presents a series of concrete steps and questions to work through, starting with longer-term goals and working towards actionable next steps. Here I wrote down the first few steps that I have been working through:

Step 1 is to define a fulfilling, high-impact career. Based on the guide, it is useful to divide ultimate career aims into (1) positive impact and (2) personal fulfillment. These goals are mutually reinforcing.

  • The expected impact of a problem = pressing-ness of the problem (how large is it, how neglected is it, how solvable is it) x effectiveness of the opportunity (how effective is the intervention, how many resources can be leveraged) x personal fit (how likely am I to excel, how well does it match my strengths, will I be motivated in the long run)

  • The main components of a satisfying job are: a sense of meaning or helping others, a sense of achievement, engaging work with autonomy, supportive colleagues, and sufficient ‘basic conditions’ such as fair pay and non-crazy working hours.

Step 2 is to identify the most pressing global problems. I found 80,000 hours’ problem profiles to be really informative. Here are the 3 problems that I am most interested in addressing (due to their importance, neglected-ness, and tractability):

  • Recommendation systems at top tech firms: The technology involved in recommender systems — such as those used by Facebook or Google — may turn out to be important for positively shaping progress in AI safety, as argued here. Improving recommender systems may also help provide people with more accurate information and improve the quality of political discourse.

  • Improving individual reasoning or cognition: Better reasoning and cognitive capacities usually make for better outcomes, especially when problems are subtle or complex. Work on improving individual decision-making is likely to be helpful no matter what challenges the future throws up.

  • Positively shaping AI development: The problem of how one might design a highly intelligent machine to pursue realistic human goals safely is very poorly understood. If AI research continues to advance without enough work going into the research problem of controlling such machines, catastrophic accidents are much more likely to occur.

Step 3 is to generate ideas for long-term paths. The challenge of longer-term planning is to find narrow enough goals to provide direction and motivation but broad enough that they’re sufficiently stable over time and let me stay open to opportunity. Anecdotally, while people often seem overambitious about what they can achieve in a year, they often underestimate what they can achieve over several decades as their skills build up. Here are the four ways to generate options for long-term roles:

  • Aim at top problems: This entails generating ideas for roles that tackle the pressing problems from step 2 (research scientist, data scientist, ML engineer, tech journalist, AI product manager, etc.).

  • Develop transferable career capital: This entails generating ideas for roles that build ‘transferable career capital’ — skills, connections, or other resources that I will be able to use to tackle a variety of pressing problems (any roles in early-stage startups).

  • Capitalize on my strength: This entails clarifying my strengths, skills, and other career capital and use them to generate more options (written and oral communication, research, software development, etc.).

  • Coordinate with others in a community: This entails considering roles to help communities I might join to have more impact (machine learning, effective altruism, stoicism, etc.).

Step 4 is to clarify my strategic focus, which is what I’m currently working on. The idea is to look for unusually good opportunities that happen to be right in front of me and work backward from potentially great longer-term paths. The image below summarizes the important factors to look for in my next career steps:

80,000 Hours Career Guide (https://80000hours.org/career-planning/article/)

  • Immediate impact potential — How much will this next step allow me to contribute to solving a pressing problem (in expectation)?

  • Specialist career capital potential — How much will this step advance me towards my best-guess longer-term paths?

  • Transferable career capital potential — What opportunities will this step give me to gain transferable skills, impressive credentials, useful connections, or otherwise invest in my self-development, which will put me in a generally better position to contribute to pressing problems?

  • Personal fit — What are my chances of outsized success in this path? Relatedly, will I be sustainably motivated and happy, and does it match my strengths?

  • Information value — How much will I learn about myself, the world, and my longer-term options, to help me uncover even better longer-term paths than my current top options (or further narrow them down)?

  • Fit with other personal priorities — Will this next step fit with the priorities that I might have?

The Bottom Line

Sitting down to do an annual review takes time, patience, and some tolerance for discomfort as we ask ourselves the hard questions. When holiday hectic-ness takes hold and we are longing for a break, the end of the year hardly feels like a time to dig through our year in search of answers. But an annual review is an opportunity to discover what may have been hidden and search for insights we can use in the year to come. If you want to conduct your own review for the upcoming year, I recommend this guide from Todoist.

That’s it for my 2020 version of the annual review. See you in 2021!