Life lessons from Machine Learning 01: What’s your Loss Function?
Personally, the most thrilling part of learning is that it always carries a big deal of irony. Nothing really turns out expectedly as you delve into something. When I went to China to learn how to speak in this new alien language, instead I’ve learned how to listen(如果这不是，还有什么是外星语呢?). Mostly because, for the first time ever, I had to hold myself back simply due to my own inability to articulate my thoughts in Mandarin.
I came to graduate school to learn about machine learning， and instead I’m learning how humans learn.
MACHINE vs HUMAN
As an average person who didn’t have anything to do with tech, I used to think humans and machines were a rather antagonistic pair. Machines meant ‘lacking something humane’. In fact, the more I learn about machines, I feel less of the juxtaposition and see more similarities. There are multiple interesting concepts in machine learning that made me reflect on the way humans learn.
- Loss function: Stop taking actions that lead to negative consequences.
One of the things machines are better at than humans is that they are the masters of iteration. Machines can replicate the same function over and over without getting bored(computers do get tired tho, I feel ya, buddy). And in machine learning, they don’t merely duplicate exactly the same thing they did in the former round. As machines learn, they systemically measure how wrong or right they are and update how they learn in the next round.
As machines learn, they systemically measure how wrong or right they are and update how they learn in the next round.
So, how do they calculate the ‘wrongness’ of them? ML algorithms use something called ‘Loss function’(objective function). Sometimes they measure how far they are from the correct answer(usually by Euclidean distance), or take the average of their performances to measure the distance. Sometimes they ignore the direction but just grasp how the current features are working, and sometimes they can precisely readjust the next step to the opposite direction of the last trial.
There I got to ask myself, what’s my loss function?
It’s struck me so hard that I have never thought about observing and correcting my way of learning. Harvard Business School professor Gerald Zaltman says that 95 percent of our purchase decision making takes place in the subconscious mind. We use the same mind when it comes to learning, as well. So we unintentionally end up with a very inefficient way of learning(Imagine an ML algorithm without a loss function! It’s almost impossible to expect them to return a meaningful output.)
By adopting the loss function as the machines do, we can finally ‘work smarter, not harder’. Or by updating our behaviors little by little until we reach the optimized routine, at least we can avoid the mockery from Einstein: “Insanity is doing the same thing over and over again, but expecting different results.”
So, what’s a humane way to adopt this so-called loss function that measures our wrongness in achieving the desired outcome? Instead of asking, ‘what am I doing wrong now?’, ask yourself, ‘what do I have to know to evaluate my current performance?’
‘what do I have to know to evaluate my current performance?’
I’d imagine something like this: Loss = what’s lacking from (objective evaluation +subjective satisfaction +progress from last week) I want to know the objective evaluation level of my current performance(from a review/feedback/quiz…) as well as my personal satisfaction and progress levels.
Let’s not confuse this with finding an equation to achieve the goal, like this one: Achieving goal = doing A + thinking B + not doing C + a
This is what the learning algorithm itself is set to find and optimize, not the loss function. The goal here is to think about how to assess the wrongness of my current output.
As it’s always debatable which loss function to use for each dataset in ML, The beauty of loss function in life as well is that it’s not about having the perfect formula — which is thought to be impossible with the decision makings that we’re usually dealth with as it’d involve the entire universe — but it’s about having one.
Having a loss function of your own helps you to be mindful of your wrongdoings and sets a redirection mechanism for yourself every step.
Well it happens to be January now at the time of writing and we’re about to begin the next iteration of a ‘year’. whoa I’m doing this for the 27th time! And this year I am going to ask myself a different question:
What can we do differently this time? What should we really STOP doing?
