This is a crosspost from a Linkedin article. ML algorithms are pervasive and impact consumers in a big way! Here is a commentary on what’s working, what could be better, and the opportunities ahead.
ML Algorithms have a big influence on us.
Whether we know it or not, ML Algorithms are a big part of many digital experiences and applications consumers use every day. From voice assistants like Alexa and Siri to translating a street sign from any foreign language to English, and to predicting traffic and the estimated time of arrival at the destination, ML Algorithms are pervasive.
ML Algorithms also have a big influence on our everyday lives. They influence our news feeds, what we buy, what we watch on TV, ads presented, answers to questions we might be interested in, and in some cases, even the questions we should be asking. In a nutshell, ML Algorithms either make decisions on our behalf or present us with options to choose from to make a decision.
This is an irreversible change, and it’s becoming the new normal as technology becomes more intelligent and people become more dependent on technology.
ML Algorithms get a ‘B’ for performance and ‘D’ for societal impact.
Over the last few years, ML Algorithms have attempted to perform a small set of human tasks at scale. ML Models and Algorithms do this by consuming gobs of raw content and metadata, how humans interact with it in the digital world, learning from those interactions, and fine-tuning the ML model until the expected outcome is achieved.
For example, speech recognition, natural language understanding, categorizing or classifying content in text, images, and videos, translating text from one language to another all work the same way. This has helped reimagine and build innovative consumer experiences and has an overall positive impact on user experience.
Another area where ML Algorithms have made an impact is curation and recommendation of digital content and information that reaches consumers – e.g., search, news, social, video, commerce, and ads. While ML Algorithms increased user engagement, not all of that increase has been for the right reason!
The biggest issue with ML Algorithms for content curation and recommendations has been about encoding historical human biases in data and amplifying them at scale to drive user engagement. Examples of this include:
- Presenting information that validates user’s own assumptions, or putting more emphasis on information that they already know, while ignoring new information that might be contrary to what they believe in.
- Presenting a user with information (such as content or products) based on what the majority of other users like, and not necessarily what that user needs.
- Not presenting diversity in content/information on topics or questions the user might not have any foreknowledge about. For example, if somebody was searching for information about whether or not their child should go to a public or private school, the first few results presented might heavily influence their decision about which school to choose.
While it might seem like a relatively small issue, the societal cost of ML Algorithms for content curation and recommendation amplifying inherent human bias has been huge and should be addressed.
Take the example of news feeds that are presented to a user. The top candidate stories are selected based on the cohort that a user belongs to. A cohort could be defined by the location the user lives, their affiliations, demographics, behaviors and several other factors. First of all, cohort-based candidate identification of new stories promotes group-think and decreases diversity!
These candidate stories are then ranked based on the likelihood of the user clicking on the content and engaging with it. Users are more likely to read the news that is aligned with their own views and opinions than something that is contrary. Engaging with the new story further strengths the user’s views and opinions. The societal cost for this is the lack of openness to new opinions, tolerance and losing the ability to respect contrarian viewpoints.
Practitioner’s dilemma – should humans influence ML Algorithms?
The issues of the current crop of ML Algorithms mentioned above can be traced back to a few primary reasons:
- ML Algorithms were trained to find patterns of average user behavior by using data only as input.
- ML models were lacking in true domain knowledge and understanding about a topic/domain.
- ML algorithms could not filter their responses based on the situational context. For example, boosting the distribution of a funny cartoon isn’t the same as boosting the distribution of a potentially horrific accident or incident.
- ML algorithms could not comprehend the impact of the decision made for the users or the choices presented.
Some of these issues can be resolved with human input to make ML Algorithms more intelligent, responsible, and effective.
The ‘practitioner’s dilemma’ is: Should human input be allowed to correct historical biases, and will it help or hurt in the long run? For example, differences in lines of credit available for men versus women, or one social group versus another, implicit gender bias when screening for job applicants are just some examples of the nuanced nature of the problem. While it is possible that human input might solve the current set of issues where legal, it might also open up the gate for humans to potentially abuse it.
Solving that human abuse of ML algorithms requires processes to maintain quality and integrity, as well as governance and clearly defined roles and responsibilities for the entire ML Algorithm pipeline. Additionally, the ‘black box’ approach to ML Algorithms would need to be replaced with ‘explainability’ and ‘reproducibility’ of the output. Easily said, but a long list of to-dos for an emerging capability area.
So the answer to the ‘practitioner’s dilemma’ might be: It depends on the use case where ML Algorithms are applied to.
The first phase for ML Algorithms and the initial hype is almost over. The real work, such as opportunities and challenges that come with them, starts now. As discussed before, the future belongs to intelligent user experiences, and ML Algorithms will likely play a big role in deciding the winner in the ‘intelligent era’ of applications. It is going to be an exciting decade ahead for ML (Machine Learning) researchers, practitioners, engineers, and enthusiasts.
In short, a few key takeaways to conclude this article are:
- A tech-only approach to ML Algorithms doesn’t work.
- With big user data comes even bigger responsibility.
- Understanding the problem to solve, the domain, and the context in which ML Algorithms will be useful are critical requirements.
- There are challenges ahead to make ML algorithms more robust. However the promise and potential of intelligent applications are well worth it.