Not Every Data Problem Requires Artificial Intelligence.
Let me share an insight with you. In the industry, you don’t have to use AI to solve all the problems. Not all problems require Artificial Intelligence, anyway.
There are problems that can be solved with traditional programming, regular expression, SQL and there are problems that can be solved with data analytics and data visualisation as well and there are many other solutions to problems that that you might face.
You don’t actually have to have a very large team of machine learning engineers because you might have heard from someone or you might feel that the problems you are facing can only be solved with Artificial Intelligence.
This is a misconception going around that all difficult problems can only be solved with Artificial Intelligence. Because some problems are so difficult that artificial intelligence algorithms can just learn how to solve them. Humans cannot devise non-AI algorithms for them. That’s not true at all. A lot of problems in fact, majority of the problems can be solved without the use of AI and majority of the problems are solved without artificial intelligence algorithms.
Only some problems require AI and it depends on what the business problems are and what you’re trying to answer and only from that perspective, you can know whether the problem you’re trying to solve requires AI and if it does, to what extent it requires AI.
There are AI APIs out there for example Google cloud vision APIs, Google Translate API and voice API and not only that, there are AWS APIs as well and there are Azure APIs and there are many other APIs available that do inferences where you send an image and it will send you the labels back or send the converted file back depending on what you send it.
Now most of the times you can simply integrate those services into your products. So, you don’t even have to develop your own algorithms from scratch.
There are very few companies out there that actually need to develop their own algorithms. There are even fewer companies that have the capacity to develop the AI algorithms to the required standard.
Let me share an amazing story here from an Amazon employee. He writes:
At Amazon, I was a part of the fraud detection team. Obviously, Machine Learning algorithms were a big part of the job. Most of the people on my team were recent grads and extremely pumped up to use cool new tech to solve problems.
There was once a specific classification problem that we were trying to solve that involved complex domain names. We spent three months building and training a model to do this, many days tweaking the parameters and finally ended up with a model that had an accuracy of ~70%.
Enter new Business Analyst.
He looks at our model, understands the problem that we’re trying to solve and then sits at his computer for a couple of hours and comes back and tells us it’s done. We’re not sure what he means by done so we go over to look and see a complex Regex Statement and it worked with a >99% accuracy.
We felt like such idiots that day but I learned a very important lesson.
The smartest people are those who spend more time understanding the problem and then come up with the most simple and elegant solutions to it.
Cool and complex tech is not always the best route and ML is definitely not the answer to everything under the sun.
What matters is not using AI to solve a problem. What matters is whether you are trying to solve your business problem using the most effective way possible and maximising on the savings and profits.
If you are not sure whether you need to use AI in your projects, what type of AI you need, how to know if your data team is right in their reasonings that they need more AI, drop me a message. I can give you clarity on your approach towards doing data science. I can devise a strategic plan for you based on your business goals, aspirations and what you would like to achieve with data.
About the Author:
Zain Daniyal is a Data Strategy Consultant