02_MachineLearningTermonology
Machine learning (ML) is a branch of artificial intelligence (AI) focused on creating systems that can learn from data and improve over time without being explicitly programmed. The core idea behind machine learning is to enable computers to learn from experience and automatically adapt to new data, rather than relying on explicit programming instructions
Category
🤖
TechTranscript
00:00Let's understand as to what is the key difference between machine learning from traditional
00:06programming in traditional programming. If you're from the programming background, you'd realize
00:11that when you when you are given the assignment of writing a traditional program, you have been
00:19given a set of rules to encode them in your program, right? If you're writing a Python program,
00:26or if you're writing a traditional Java program, etc, etc. For developing an application you have,
00:32you have to be given I mean, the normally the business or the or the business analysts, etc.
00:38They will give you a set of rules, a set of business rules. And those business rules have
00:44to be encoded. And those business rules are fixed fixed rules. I mean, they are known
00:48from from before, okay, this is what we expect our programs to do. This is what we expect from
00:55the data. And this is how the these are the set of rules that have to be applied on the data
01:00to produce this kind of output. So those whole thing whole set of logics are known,
01:08and the format of the data and the nature of the data are also expected from before. Okay,
01:15so there it's a deterministic deterministic situation, we encode the deterministic rule
01:21within our conventional or traditional programs. However, machine learning is completely different.
01:29The way it is different is that we are not writing programs per se. Okay, we are actually not
01:34writing programs per se. Our programs at the end, like we write traditional programs, and those those
01:42are the programs, the programs we write are the programs that will run in live in the production
01:47environment ultimately. Now in machine learning, we do not code any program that runs in in the
01:55production scenario or production environment or the live environment. It's models that run
02:04in the production environment and how how the models get created. The models, what we throw
02:10in the mix is an is a learning algorithm, a machine learning algorithm. By learning algorithm,
02:19it actually indicates that it learns it learns like a human being.
02:25It learns from data, more more specifically, almost always it learns from historical data.
02:34Now, how does it learn, it learns by using a lot of mathematical computations. And there are
02:43various different types of scenarios for which various different types of algorithms that are
02:49available. And in subsequent lessons, we are going to look at some of those different types of
02:55learning algorithms. But all those algorithms have some mathematical formula or mathematical
03:01computations behind it. And those mathematical computations when applied on the historical
03:07data that is input to the learning algorithm, it learns the patterns within the data.
03:15Alright, so when it learns the patterns include in inside the data, those patterns
03:20automatically encoded in the model. And when the patterns are get going to get encoded in the model,
03:27this model when applied on new data, are able to make decisions or predictions for yourself,
03:36which normally are not possible by writing traditional programming because we do not
03:43know what rules or patterns have to be encoded in our final program. Okay, so this is the key
03:51difference between machine learning and traditional programming. As we go on, we see more more and
03:56more scenarios and more and more different types of learn different data even more clear as to what
04:03we mean. Previous slide we have spoken about there are a number of related terms we have used up
04:12machine learning, we thought of artificial intelligence, we've heard of neural network,
04:17data science and big data. Now, this is quite confusing, we keep on hearing in various kind of
04:23lessons or various kind of articles on the net, or books, etc. This, these terms are related,
04:32they have a lot of overlap. And let's understand once and for all as to what each of them mean.
04:39So machine learning, I mean, in the in a couple of past slides, we were talking about machine
04:45learning. And there are use cases of machine learning. So for instance, the email filtering
04:51use case, or for instance, stock price prediction use case, there are learning algorithms or
04:58more specifically machine learning algorithms of different types that are existing,
05:03which can be applied on each of those kind of scenarios. And they produce results,
05:09they learn from the data and they produce results, either predictions or decisions,
05:13etc, etc. There are different types of machine learning algorithms, linear regression,
05:18logistic regression, random forest, decision tree, k nearest neighbor, k means clustering,
05:25etc, etc. So we will learn some of them in the subsequent lessons. But these are this is what
05:32machine learning is. Now deep learning is a particular, or rather, let's talk about neural
05:41network. neural network employs some of the machine learning algorithms in a slightly different
05:49architecture or rather vastly different architecture, which mimic the human brain.
05:56Now, how does it mimic the human brain, human brain is brain is comprised of neurons. And in
06:02a similar manner, the neural network employs something called nodes, which are actually
06:08similar to neurons, the neurons get activated in a neural network architecture, the neural networks
06:14have something called activation functions. And each of the neurons and nodes in a neural network
06:20architecture are connected to are organized in in terms of layers. And there are multiple layers
06:27within each neural network architecture. neural network architectures are used for solving very
06:35complex problems involving complex sets of data, etc, etc. And those nodes in neural network,
06:42as I said, organized in terms of layers and nodes in each layer are connected to
06:50the previous layers as well as the next layers. And each of those connections between layers of
06:57nodes are contain something called the weights which the neural network learn to solve the
07:07problem. Okay, so this is a special type of special type of architecture to solve a very complex set
07:15of problems. For instance, complex set of image processing problems, set of natural language
07:22processing problems are solved by neural network architectures. Right now, related to neural
07:29networks is the deep learning. Now, as I mentioned that in neural network, there are nodes which are
07:35organized in terms of layers. In deep learning, it's it's a neural network architecture. However,
07:44the number of layers in a deep, deep neural network are higher, there can be shallow neural
07:50network, and there can be deep neural network in shallow neural networks, there are small number of
07:56layers of nodes or neurons. And in deep neural network, there are many different number of
08:03layers. And when when there are deep neural network, they are most definitely are used for
08:08solving more complex problems. Now, the learning of learning from the data, I mean, as we mentioned
08:15in machine learning, there are algorithms that learn from the data. Similar in the case of neural
08:22network or deep neural network, it the architecture learns the algorithm and the and the architecture
08:29of a neural network learn from the data. So when you're using a deep neural network, it's also
08:35called the learning process is also called deep learning. Okay, so these two are related. And all
08:40this machine learning neural network, deep learning, all this combined together is comprising
08:46or creating the field of artificial intelligence, artificial intelligence is not something different.
08:51artificial intelligence is kind of a superset term, which encompass machine learning, deep
08:59learning, neural network, etc, etc. And also data science. Now, how is data science different from
09:07let's say machine learning, the difference key difference between machine and there's a lot of
09:11overlap between data science and machine learning. The kind of algorithms that machine learning uses
09:17are also the kind of algorithms data science uses. However, the difference between data science and
09:23machine learning are subtle, very subtle. The goal of machine learning is most always
09:29solving, solving a problem like spam filtering, or, for instance, creating a recommendation engine,
09:36etc. Now, the goal of data science is slightly different. The goal of data science almost always
09:43is finding patterns and insights and presenting them to stakeholders from the data. And in a lot
09:51of times, also predict the future also help create a decision. And in those cases, it uses classification
10:02algorithm, regression algorithms, just like machine learning, as well. So the importance of
10:08algorithms is more on machine learning, the importance of algorithms, predictive algorithms,
10:16like regression, and the classification, etc, are a little bit less on data in data science.
10:22However, the importance of data analysis, which are commonly called exploratory data analysis,
10:29visualizations using statistical techniques, etc, to find out insights and patterns of data,
10:36and presenting them to stakeholders for making business decision is commonly the goal of
10:42data science. So that's a subtle difference between machine learning and data science. As I
10:47mentioned earlier, there is still a lot of overlap. There are exploratory data analysis that you also
10:52need to perform for machine learning projects. That is also needed on data science, probably
11:00more, the goals of data science are a little different. The algorithms like linear regression,
11:05logistic regression are still the same algorithms for prediction, etc, that are used for data
11:10science. Alright, hopefully, it's getting a little bit clearer in your mind. Big data in the on the
11:19other hand is a little different beast. Now, big data is a science or set of approaches to our set
11:29of technologies to manage very large sets of data, which cannot be managed in traditional data
11:37management or data processing technologies. Say for instance, Python programs or a common Java
11:44program may not be or the normal Rdm is that we are very familiar with may not be suitable for
11:49managing such large sets of data. Okay, so thereby, for managing such large sets of data, there are
12:00data warehouses or data lakes, which are commonly called there are data processing large data
12:06processing systems like Hadoop, Apache, Spark, etc, which are used which are big data systems.
12:12For data storage, there are NoSQL databases, like for instance, for both structured and
12:19unstructured data sets, which are like MongoDB, Cassandra, DynamoDB, etc. Those are NoSQL
12:26database databases that are commonly used for managing big data. Now, how big data is related
12:33to all of this machine learning and artificial intelligence and deep learning neural network,
12:37etc. They are related because after the management of big data, these big data are normally fed into
12:47various data science endeavors or deep learning endeavors or machine learning endeavors to further
12:54produce results in terms of data analysis results or predictive results and etc, etc.
13:02So, that is the reason big data are commonly mentioned in connection with machine learnings
13:09and artificial intelligence, etc. So, this is what these terms mean for each of them. So,
13:15there is a lot of overlap. These terms are related