Credit: OpenAI Dall-E2
What progress did I make?
Welcome to another chapter of the machine learning scholar adventure. It’s been a while since the last chapter update I posted but I’m so glad to be back on this exhilarating path! One thing I’ve come to realise is that progress is not linear.
Usually, when learning, there are periods of consistent progress and also periods with setbacks and also times when you’re in a plateau. This is all part of the journey so the most important thing is to keep going (and remember to take breaks). Since I last wrote, I have experienced consistency…but also setbacks and plateaus. However, I refuse to give up on becoming a world class AI practitioner. My machine learning scholar adventure must go on! So, here’s what I’ve been up to recently:
Machine learning
Datacamp: Data scientist with python
I started the data scientist with python career track because even though I was previously making progress in the fast.ai course, I was keenly aware that there were significant gaps in my understanding of machine learning fundamentals and lack of basic fluency with popular tools such as pandas and scikit-learn. I’m committed to providing value when I get my first role so it’s important that I have capability in these frameworks. At the moment, I’m 52% of the way through the track and am enjoying the structured way in which my knowledge is growing. Whilst the coverage is fantastic, I’m going to need to apply what I learn to novel scenarios via projects and kaggle to really improve my skills. The certificates for the courses I’ve done so far are linked below:
- Introduction to python
- Intermediate python
- Data Manipulation with pandas
- Joining Data with pandas
- Introduction to data visualisation with matplotlib
- Introduction to data visualisation with seaborn
- Python data science toolbox (Part 1)
- Python data science toolbox (Part 2)
- Intermediate data visualisation with seaborn
- Introduction to importing data in python
- Intermediate importing data in pythoh
- Cleaning data in python
Meta Learning: How To Learn Deep Learning And Thrive In The Digital World
When reflecting on my endeavour to learn machine learning, I realised that my approach was not effectively ramping up my ability to actually do machine learning. I was being too passive! From reflecting on my previous post and Radek’s Meta learning book, I’m focusing more on practice. So instead of theory → practice, I am now starting with practice → theory → practice. This can be seen with my greater emphasis on kaggle and open source below (so I apply what learn by building more and learn things just in time). Some other great tips from the book that have proven useful in the past week are using stack overflow more often, navigating developer documentation and making myself more productive with pycharm, my IDE of choice.
Fastai
Previously, I was working through the book and online course simultaneously which was not great as my focus was too split. Hence, I’m now just doing the online course. My learning loop that’s inspired by Radek’s experience in the course is:
- Run the full lecture jupyter notebook to get an overview of what will be covered in this section
- Watch the corresponding fastai video lectures at 1.5x speed to get a more comprehensive gist (move to step 3 when adequate content amount is covered)
- Work through the streamlined jupyter notebook to replicate results covered in lecture so far
- Go back to step 2, switching between 2–3 until entire jupyter notebook has been run
- Recreate training in completely new jupyter notebook to consolidate understanding (can refer to lecture or notebook when stuck)
- Find a similar dataset and apply techniques covered to gain applicable skills
- Answer questions in the course by using book as reference only
Kaggle
To help me apply my knowledge from the above courses, learn to iterate on ideas quicker and connect with other machine learning practitioners, I’m investing more time in the kaggle platform.
Titanic
I’d done a basic submission for the Titanic — Machine learning from from disaster competition a while back but in the last week, I’ve been trying to improve my score. It was previously 0.77511 and is currently 0.78468 which corresponds to position 1912/14640 on the leaderboard. This means I’m in the top 15% in the rankings! Using ideas from this helpful beginner walk-through as well as this datacamp kaggle tutorial, I made the following changes to my original notebook:
- Added Age, Fare and Embarked as features
- Replaced NaN values with suitable statistics
- Scaled numerical features
- Explored model baselines (previously only random forest classifier was used)
- Started doing some model tuning
Phrase matching
Entered the phrase to phrase matching competition and am exploring Jeremy’s starter notebook to get me started in natural language processing. Also made my first contribution to a discussion and got an upvote!
Inspiration
Graphcore
I had the pleasure of attending two talks by Tim from Graphcore recently: Inspiration: Accelerating data-intensive ML applications with the Graphcore IPU and From Zero to SOTA: How to fine-tune BERT using Hugging Face for world-class training performance. It was fantastic seeing how the novel IPU architecture and associated software is helping innovators find the next breakthrough. While CPUs are great for normal apps and web whose computation model is scalar, GPUs for matrix based apps whose computation model is vector, IPUs are especially designed for graph based computations which are the bread and butter of cutting edge deep learning approaches. In the BERT demo, I thought it was super cool how the model could be partitioned across multiple IPUs to maximise hardware usage.
Hugging face
Whilst I’d seen hugging face mentioned in a random tweet before, I never really grokked exactly what they did until earlier this week. This hugging face crash course video offered a comprehensive survey of their ecosystem whilst this BERT 101 blog post helped me to understand how this popular and versatile language model works at a high level. I especially like how easy it is to embed intelligence into projects using the transformer library and I’m looking forward to contributing models, datasets and making spaces during my machine learning journey.
Two Minute Papers
My favourite research development recently has been this mind blowing demonstration of OpenAI’s capabilities. It’s beyond incredible how we now have models that can generate media that has never existed before just from a text description. A new era of human-machine creativity is definitely upon us! I highly recommend checking out OpenAI’s related instagram account for more AI generated artworks.
Miscellaneous
Below are resources I found useful during the week for refreshing and extending my knowledge:
Articles
Twitter threads
Videos
- Dummy variables in pandas
- Cross validation and k-fold cross validation
- False negatives and false positives
- Machine learning lifecycle
- Zero-to-1: How to start a career in ML
Open source
Programming consensus is that reading code and writing code are the best way of improving programming ability. Consequently, I’ve started contributing to open source. At the moment, my machine learning understanding still needs to improve more before I can contribute to projects like fastasi and pytorch so in the meantime, I’m going after more low-hanging fruit. This week, I stumbled upon Naomi’s projects and contributed the pull requests below which both got merged 💪🏾 I also became part of the corresponding community, Naomi’s Nook.
What am I exploring now?
- Datacamp’s Data scientist with python career track
- FastAI’s Practical deep learning for coders course
- Radek’s Meta learning book
What did I learn from the challenges I’ve conquered?
- Prioritise just-in-time learning to maintain momentum
- Deliberate practice and building make understanding theory much easier
- Progress is not linear so keep going (and remember to take breaks)
What are my next steps?
Kaggle
- Refine titanic submission further
- Make progress in the Phrase to phrase matching competition
Graphcore
- Work through PopTorch MNIST tutorial
- Convert my GAN code to work on IPU
Open source
- Submit pull request for another issue
Thanks for reading my latest update 😄 Want to connect? Or see more frequent updates of my journey? Then feel free to reach out and follow me on twitter 🤖
Originally published at https://www.machineplay.io on May 2, 2022.
