(1) Bython, a basic Python interpreter written in C. I just want to have a taste of what it's like programming out a language so I won't go too far. It is dynamically typed and has automatic memory management. https://github.com/remykarem/bython. PRs welcome!
(2) minishell. This is a simple shell with only 2 commands: either you (i) hit enter to view 10 files of a folder at a time, or you (ii) enter a filename and view nbytes. I built this because a co-worker wanted to view a folder with 20+GB of files but couldn't do it with an `ls`. With this, I hope that we can casually explore a folder without having to print everything. https://github.com/remykarem/minishell.
You need a way to organise and prioritise these. Use a Trello board.
Because you're self-studying, you're sort of going to a self-study university taking different modules from different faculties. This Trello board spells out your self-study university curriculum and you're in charge of it.
Here is a 5-step process to build this curriculum.
Firstly, create say 3 lists on the Trello board: ML, CS and Math. Each list represents a 'faculty'.
Then, for every list, create Trello cards where each card is a 'module'. For example, you would create 'Data structures & algorithms' in the CS list and 'Decision trees' under ML.
The next step is to figure out for each module if it's something you either (i) wish to know or (ii) must know. You can use Trello labels or even use the Trello separators for this.
This following step requires a bit of work and it's the fun part, only because it's self-study. For each module, list down (you can list things in a card) the resources you have for that module. For this there are various resources you can get from the comments, search engines, and your peers. Consider the different modes of instruction: books, e-books, videos, lecture notes, slides, articles, blog posts, online learning platforms and so on. Choose what's best for you. If you can't decide just pick something first and find another time to source for another material.
Lastly, prioritise the modules. This can be done by easily dragging the modules which you want to do first on top of the list (having considered what you wish to know and what you must know). Set, say, top 3 modules for each list then you'd like to do for the next 2 weeks.
This is the high-level curriculum planning. If you plan on a micro-level planning like what modules to do for this week or for today, that I leave it to you.
Say no more to copying code line-by-line to Jupyter notebook. This conversion just needs your code to be paragraphed, like how codes should be. Comments are converted to markdowns. Codes are converted to cell codes. It's as easy as running pip install p2j and run p2j your_code.py
Explore how (stochastic) gradient descent works on a simple linear regression. I built this demo to help me better understand backpropagation, by keeping an eye on the values of the weights as they get updated.
I've also added step-by-step remarks and graph plots to the values of the weights and the loss function.
Things you can play around with: optimiser, learning rate, variable initialiser, loss function, batch size, no. of epochs
At the end of this post you will get a cheat sheet of the 10 common gradient descent optimisation algorithms.
Using more readable notations, I will walk you through how the vanilla stochastic gradient descent slowly evolved into the popular Adam optimiser and others. I also came out with an ‘evolutionary map’ of the optimisers to visualise this.
The motivation for writing this post is that there is a lack of simple-to-read equations for parameter update and a compiled list of these optimisers.
(1) Bython, a basic Python interpreter written in C. I just want to have a taste of what it's like programming out a language so I won't go too far. It is dynamically typed and has automatic memory management. https://github.com/remykarem/bython. PRs welcome!
(2) minishell. This is a simple shell with only 2 commands: either you (i) hit enter to view 10 files of a folder at a time, or you (ii) enter a filename and view nbytes. I built this because a co-worker wanted to view a folder with 20+GB of files but couldn't do it with an `ls`. With this, I hope that we can casually explore a folder without having to print everything. https://github.com/remykarem/minishell.
(3) Scrollable Python documentation, a hack from the scrollable interface found in https://allennlp.org/tutorials. Use case is for people who are explaining Python code. https://github.com/remykarem/scrollable-python-documentation.
My command of C isn't that great so if you're interested to collaborate, I'm happy to be your apprentice :)