Machine learning is solving such a large number of sophisticated problems today that it seems like magic. But there isn’t any magic in machine learning rather it has a strong mathematical and statistical foundation.

如今，机器学习正在解决如此众多的复杂问题，似乎就像魔术一样。 但是机器学习没有任何魔力，它具有强大的数学和统计基础。

While trying to understand the important and somewhat difficult concepts of machine learning, we sometimes do not even think about some of the trivial concepts. Maybe you think of those, but I know that I ignore a lot of simple things many a times. The reason is the amazing machine learning and deep learning libraries that have functions and methods to quickly do this for us. ????

在尝试理解机器学习的重要且有些困难的概念时，我们有时甚至没有考虑一些琐碎的概念。 也许您会想到这些，但是我知道我很多次都忽略了许多简单的事情。 原因是令人惊叹的机器学习和深度学习库，它们具有可为我们快速完成此操作的功能和方法。 ????

One such trivial problem is to find the total number of trainable parameters in a feed-forward neural network by hand. A question that I encountered in one of my exams and confused me with the options provided. This question has also been asked in many different forums by many machine learnimg practitioners. ????????

一个这样的琐碎问题是用手找到前馈神经网络中可训练参数的总数。 我在一项考试中遇到的一个问题，使我对所提供的选项感到困惑。 许多机器学习实践者也在许多不同的论坛中提出了这个问题。 ????????

The problem discussed in this post is:

这篇文章中讨论的问题是：

如何找到前馈神经网络中可训练参数的总数？ (How to find the total number of trainable parameters in a feed-forward neural network?)

You must be wondering why is that even an important one to discuss. It is indeed! The time taken to train a model is dependent on the number of parameters to train so this knowledge can really help us at times.

您一定想知道为什么这甚至是一个重要的话题。 这的确是！ 训练模型所需的时间取决于训练参数的数量，因此这些知识有时确实可以为我们提供帮助。

By looking at a simple network, you can easily count and tell the number of parameters. In the worst case, you can draw the diagram and tell the number of parameters. But what happens when you encounter a question of a neural network with 7 layers and a different number of neurons in each layer, say 8, 10, 12, 15, 15, 12, 6. How would you tell how many parameters are there in all?

通过查看一个简单的网络，您可以轻松地计算并告诉参数数量。 在最坏的情况下，您可以绘制图表并告诉参数数量。 但是，当您遇到一个神经网络的问题，该问题具有7层并且每层中有不同数量的神经元时，比如说8、10、12、15、15、12、6，该怎么办？所有？

Let us together find a mathematical formula to get the count. But before moving to the calculation, let us first understand what a feed-forward neural network is and what characteristics it possesses. This will help us in finding the total number of parameters.

让我们一起找到一个数学公式来获取计数。 但是在进行计算之前，让我们首先了解什么是前馈神经网络以及它具有的特征。 这将帮助我们找到参数的总数。

A feed-forward neural network is the simplest type of artificial neural network where the connections between the perceptrons do not form a cycle.

前馈神经网络是最简单的人工神经网络，其中感知器之间的连接不形成循环。

Despite being the simplest neural network, they are of extreme importance to the machine learning practitioners as they form the basis of many important and advanced applications used today. ????

尽管它们是最简单的神经网络，但是它们对于机器学习从业者来说却极为重要，因为它们构成了当今使用的许多重要和高级应用程序的基础。 ????

Characteristics of a feed-forward neural network:

前馈神经网络的特征：

1. Perceptrons are arranged in layers. The first layer takes in the input and the last layer gives the output. The middle layers are termed as hidden layers as they remain hidden from the external world.
   感知器分层排列。 第一层接受输入，最后一层提供输出。 中间层被称为隐藏层，因为它们对外部世界保持隐藏。
2. Each perceptron in a layer is connected to each and every perceptron of the next layer. This is the reason for information flowing constantly from a layer to the next layer and hence the name feed-forward neural network.
   一层中的每个感知器都连接到下一层的每个感知器。 这就是信息不断从一层流到下一层的原因，因此也就称为前馈神经网络。
3. There is no connection between the perceptrons of the same layer.
   同一层的感知器之间没有连接。
4. There is no backward connection (called a feedback connection) from the current layer to the previous layer.
   从当前层到上一层没有反向连接(称为反馈连接)。

Note: A perceptron is the fundamental unit of a neural network that calculates the weighted sum of the input values.

注意：感知器是计算输入值的加权和的神经网络的基本单位。

Mathematically, a feed-forward neural network defines a mapping y = f(x; θ) and learns the value of the parameters θ that helps in finding the best function approximation.

在数学上，前馈神经网络定义了一个映射y = f(x;θ)并学习了有助于寻找最佳函数逼近的参数θ的值。

Note: There is also a bias unit in a feed-forward neural network in all the layers except the output layer. Biases are extremely helpful in successful learning by shifting the activation function to the left or to the right. Confused? ???? In simple words, bias is similar to an intercept (constant) in a linear equation of the line, y = mx + c, which helps to fit the prediction line to the data better instead of a line always passing through the origin (0,0) (in case of y = mx).

注意：前馈神经网络中除输出层以外的所有层中还存在一个偏置单元。 偏向通过将**功能向左或向右移动，对于成功学习非常有帮助。 困惑？ simple简单来说，偏差类似于直线的线性方程y = mx + c中的截距(常数)，这有助于使预测线更适合数据，而不是始终穿过原点的线(0 ，0)(如果y = mx)。

Let us now use this knowledge to find the number of parameters.

现在让我们使用此知识来查找参数的数量。

Scenario 1: A feed-forward neural network with just one hidden layer. Number of units in the input, hidden and output layers are respectively 3, 4 and 2.

方案1：仅具有一个隐藏层的前馈神经网络。 输入，隐藏和输出层中的单位数分别为3、4和2。

Assumptions:

假设：

i = number of neurons in input layer

i =输入层中神经元的数量

h = number of neurons in hidden layer

h =隐藏层中神经元的数量

o = number of neurons in output layer

o =输出层中神经元的数量

From the diagram, we have i = 3, h = 4 and o = 2. Note that the red colored neuron is the bias for that layer. Each bias of a layer is connected to all the neurons in the next layer except the bias of the next layer.

从图中，我们有i = 3， h = 4和o =2。请注意，红色神经元是该层的偏差。 一层的每个偏置都连接到下一层的所有神经元，除了下一层的偏置。

Mathematically:

数学上：

1. Number of connections between the first and second layer: 3 × 4 = 12, which is nothing but the product of i and h.

   第一和第二层之间的连接数：3×4 = 12，这不过是i和h的乘积。

2. Number of connections between the second and third layer: 4 × 2 = 8, which is nothing but the product of h and o.

   第二层和第三层之间的连接数：4×2 = 8，这不过是h和o的乘积。

3. There are connections between layers via bias as well. Number of connections between the bias of the first layer and the neurons of the second layer (except bias of the second layer): 1 × 4, which is nothing but h.

   层之间也通过偏置连接。 第一层的偏置与第二层的神经元之间的连接数(第二层的偏置除外)：1×4，仅是h 。

4. Number of connections between the bias of the second layer and the neurons of the third layer: 1 × 2, which is nothing but o.

   第二层的偏置与第三层的神经元之间的连接数：1×2，不过是o 。

Summing up all:

总结所有：

3 × 4 + 4 × 2 + 1 × 4 + 1 × 2

3×4 + 4×2 +1×4 +1×2

= 12 + 8 + 4 + 2

= 12 + 8 + 4 + 2

= 26

= 26

Thus, this feed-forward neural network has 26 connections in all and thus will have 26 trainable parameters.

因此，该前馈神经网络总共具有26个连接，因此将具有26个可训练的参数。

Let us try to generalize using this equation and find a formula.

让我们尝试使用该方程式进行概括并找到一个公式。

3 × 4 + 4 × 2 + 1 × 4 + 1 × 2

3×4 + 4×2 +1×4 +1×2

= 3 × 4 + 4 × 2 + 4 + 2

= 3×4 + 4×2 + 4 + 2

= i × h + h × o + h + o

= i × h + h × o + h + o

Thus, the total number of parameters in a feed-forward neural network with one hidden layer is given by:

因此，具有一个隐藏层的前馈神经网络中的参数总数由下式给出：

(i × h + h × o) + h + o

(i×h + h×o)+ h + o

Since this network is a small network it was also possible to count the connections in the diagram to find the total number. But, what if the number of layers is more? Let us work on one more scenario and see if this formula works or we need an extension to this.

由于该网络是小型网络，因此还可以对图中的连接进行计数以找到总数。 但是，如果层数更多怎么办？ 让我们在另一个场景中工作，看看这个公式是否有效，或者我们需要对此进行扩展。

Scenario 1: A feed-forward neural network with three hidden layers. Number of units in the input, first hidden, second hidden, third hidden and output layers are respectively 3, 5, 6, 4 and 2.

方案1：具有三个隐藏层的前馈神经网络。 输入，第一隐藏，第二隐藏，第三隐藏和输出层中的单元数分别为3、5、6、4和2。

Assumptions:

假设：

i = number of neurons in input layer

i =输入层中神经元的数量

h1 = number of neurons in first hidden layer

h1 =第一隐藏层中的神经元数量

h2 = number of neurons in second hidden layer

h2 =第二个隐藏层中的神经元数量

h3 = number of neurons in third hidden layer

h3 =第三隐藏层中的神经元数量

o = number of neurons in output layer

o =输出层中神经元的数量

1. Number of connections between the first and second layer: 3 × 5 = 15, which is nothing but the product of i and h1.

   第一层和第二层之间的连接数：3×5 = 15，这不过是i和h1的乘积。

2. Number of connections between the second and third layer: 5 × 6 = 30, which is nothing but the product of h1 and h2.

   第二层和第三层之间的连接数：5×6 = 30，仅是h1和h2的乘积。

3. Number of connections between the third and fourth layer: 6 × 4 = 24, which is nothing but the product of h2 and h3.

   第三层和第四层之间的连接数：6×4 = 24，这不过是h2和h3的乘积。

4. Number of connections between the fourth and fifth layer: 4 × 2= 8, which is nothing but the product of h3 and o.

   第四层与第五层之间的连接数：4×2 = 8，这不过是h3与o的乘积。

5. Number of connections between the bias of the first layer and the neurons of the second layer (except bias of the second layer): 1 × 5 = 5, which is nothing but h1.

   第一层的偏置与第二层的神经元之间的连接数(第二层的偏置除外)：1×5 = 5，仅是h1 。

6. Number of connections between the bias of the second layer and the neurons of the third layer: 1 × 6 = 6, which is nothing but h2.

   第二层的偏置与第三层的神经元之间的连接数：1×6 = 6，这不过是h2 。

7. Number of connections between the bias of the third layer and the neurons of the fourth layer: 1 × 4 = 4, which is nothing but h3.

   第三层的偏置与第四层的神经元之间的连接数：1×4 = 4，这不过是h3 。

8. Number of connections between the bias of the fourth layer and the neurons of the fifth layer: 1 × 2 = 2, which is nothing but o.

   第四层的偏置与第五层的神经元之间的连接数：1×2 = 2，这不过是o 。

Summing up all:

总结所有：

3 × 5 + 5 × 6 + 6 × 4 + 4 × 2 + 1 × 5 + 1 × 6 + 1 × 4 + 1 × 2

3×5 + 5×6 + 6×4 + 4×2 +1×5 +1×6 +1×4 +1×2

= 15 + 30 + 24 + 8 + 5 + 6 + 4 + 2

= 15 + 30 + 24 + 8 + 5 + 6 + 4 + 2

= 94

= 94

Thus, this feed-forward neural network has 94 connections in all and thus 94 trainable parameters.

因此，该前馈神经网络总共具有94个连接，因此具有94个可训练参数。

Let us try to generalize using this equation and find a formula.

让我们尝试使用该方程式进行概括并找到一个公式。

3 × 5 + 5 × 6 + 6 × 4 + 4 × 2 + 1 × 5 + 1 × 6 + 1 × 4 + 1 × 2

3×5 + 5×6 + 6×4 + 4×2 +1×5 +1×6 +1×4 +1×2

= 3 × 5 + 5 × 6 + 6 × 4 + 4 × 2 + 5 + 6 + 4 + 2

= 3×5 + 5×6 + 6×4 + 4×2 + 5 + 6 + 4 + 2

= i × h1 + h1 × h2 + h2 × h3+ h3 × o + h1 + h2 + h3+ o

= i × h1 + h1 × h2 + h2 × h3 + h3 × o + h1 + h2 + h3 + o

Thus, the total number of parameters in a feed-forward neural network with three hidden layers is given by:

因此，具有三个隐藏层的前馈神经网络中的参数总数由下式给出：

(i × h1 + h1 × h2 + h2 × h3 + h3 × o) + h1 + h2 + h3+ o

(i×h1 + h1×h2 + h2×h3 + h3×o)+ h1 + h2 + h3 + o

ŤHUS，公式找到可训练参数的总数在具有n个隐藏层的前馈神经网络由下式给出： (Thus, the formula to find the total number of trainable parameters in a feed-forward neural network with n hidden layers is given by:)

If this formula sounds a bit overwhelming ????, don’t worry, there is no need to memorize this formula ????. Just keep in mind that in order to find the total number of parameters we need to sum up the following:

如果这个公式听起来有点????，不要担心，没有必要记住这个公式????。 请记住，为了找到参数总数，我们需要总结以下内容：

1. product of the number of neurons in the input layer and first hidden layer
   输入层和第一个隐藏层中神经元数量的乘积
2. sum of products of the number of neurons between the two consecutive hidden layers
   两个连续的隐藏层之间的神经元数量的乘积之和
3. product of the number of neurons in the last hidden layer and output layer
   最后一个隐藏层和输出层中神经元数量的乘积
4. sum of the number of neurons in all the hidden layers and output layer
   所有隐藏层和输出层中神经元数量的总和

Now, I hope you can apply this method to find the total number of parameters in a feed-forward neural network with any number of hidden layers and neurons even without using the library. ????

现在，我希望您可以使用此方法来查找前馈神经网络中具有任意数量的隐藏层和神经元的参数总数，即使不使用该库也是如此。 ????

Reference:

参考：

https://cs.stanford.edu/people/eroberts/courses/soco/projects/neural-networks/Architecture/feedforward.html

https://cs.stanford.edu/people/eroberts/courses/soco/projects/neural-networks/Architecture/feedforward.html

Thank you! Do leave your comments if you have any feedback or suggestion regarding this post!

谢谢！ 如果您对此帖子有任何反馈或建议，请留下您的评论！

LinkedIn

领英