CNN参数计算

追风赶月的少年
计算机视觉卷积神经网络

发布于：2022年4月21日

字数：722字

时长：4分钟

在CNN的前向传播过程中，我们通常需要参数量，那么参数量的计算又当如何计算呢，不急，听我慢慢道来。

实例：

1.定义一个简单网络

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
# 该文件为test.py
import torch
import torch.nn as nn
import math


def conv_out_size_same(size, stride):
    return int(math.ceil(float(size) / float(stride)))


class discriminator(nn.Module):
    def __init__(self, d=128, input_shape=[64, 64]):
        super(discriminator, self).__init__()
        s_h, s_w = input_shape[0], input_shape[1]
        s_h2, s_w2 = conv_out_size_same(s_h, 2), conv_out_size_same(s_w, 2)
        s_h4, s_w4 = conv_out_size_same(s_h2, 2), conv_out_size_same(s_w2, 2)
        s_h8, s_w8 = conv_out_size_same(s_h4, 2), conv_out_size_same(s_w4, 2)
        self.s_h16, self.s_w16 = conv_out_size_same(
            s_h8, 2), conv_out_size_same(s_w8, 2)

        # 64,64,3 -> 32,32,128
        self.conv1 = nn.Conv2d(3, d, 4, 2, 1)

        # 32,32,128 -> 16,16,256
        self.conv2 = nn.Conv2d(d, d * 2, 4, 2, 1)
        self.conv2_bn = nn.BatchNorm2d(d * 2)

        # 16,16,256 -> 8,8,512
        self.conv3 = nn.Conv2d(d * 2, d * 4, 4, 2, 1)
        self.conv3_bn = nn.BatchNorm2d(d * 4)

        # 8,8,512 -> 4,4,1024
        self.conv4 = nn.Conv2d(d * 4, d * 8, 4, 2, 1)
        self.conv4_bn = nn.BatchNorm2d(d * 8)

        # 4,4,1024 -> 1
        self.linear = nn.Linear(self.s_h16 * self.s_w16 * d * 8, 1)

        self.leaky_relu = nn.LeakyReLU(negative_slope=0.2)
        self.sigmoid = nn.Sigmoid()

    def weight_init(self):
        for m in self.modules():
            if isinstance(m, nn.Conv2d):
                m.weight.data.normal_(0.0, 0.02)
            elif isinstance(m, nn.BatchNorm2d):
                m.weight.data.normal_(0.1, 0.02)
                m.bias.data.fill_(0)
            elif isinstance(m, nn.Linear):
                m.weight.data.normal_(0.0, 0.02)
                m.bias.data.fill_(0)

    def forward(self, x):
        bs, _, _, _ = x.size()
        # (3, 64, 64)->(128, 32, 32)
        x = self.leaky_relu(self.conv1(x))
        # (128, 32, 32)->(256, 16, 16)
        x = self.leaky_relu(self.conv2_bn(self.conv2(x)))
        # (256, 16, 16)->(512, 8, 8)
        x = self.leaky_relu(self.conv3_bn(self.conv3(x)))
        # (512, 8, 8)->(1024, 4, 4)
        x = self.leaky_relu(self.conv4_bn(self.conv4(x)))
        # (1024, 4, 4)->(bs, 16*1024)
        x = x.view([bs, -1])
        # (bs, 16*1024)->(bs, 1)
        x = self.sigmoid(self.linear(x))

        return x.squeeze()

2.调用summary()函数计算参数以及输出

1
2
3
4
5
6
7
from test import discriminator
from torchsummary import summary
import torch
if __name__ == '__main__':
    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
    m = discriminator(d=128, input_shape=[64, 64]).to(device)
    summary(m, input_size=(3, 64, 64))

结果如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
----------------------------------------------------------------
        Layer (type)               Output Shape         Param #
================================================================
            Conv2d-1          [-1, 128, 32, 32]           6,272
         LeakyReLU-2          [-1, 128, 32, 32]               0
            Conv2d-3          [-1, 256, 16, 16]         524,544
       BatchNorm2d-4          [-1, 256, 16, 16]             512
         LeakyReLU-5          [-1, 256, 16, 16]               0
            Conv2d-6            [-1, 512, 8, 8]       2,097,664
       BatchNorm2d-7            [-1, 512, 8, 8]           1,024
         LeakyReLU-8            [-1, 512, 8, 8]               0
            Conv2d-9           [-1, 1024, 4, 4]       8,389,632
      BatchNorm2d-10           [-1, 1024, 4, 4]           2,048
        LeakyReLU-11           [-1, 1024, 4, 4]               0
           Linear-12                    [-1, 1]          16,385
          Sigmoid-13                    [-1, 1]               0
================================================================
Total params: 11,038,081
Trainable params: 11,038,081
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.05
Forward/backward pass size (MB): 4.63
Params size (MB): 42.11
Estimated Total Size (MB): 46.78
----------------------------------------------------------------

Process finished with exit code 0

1.卷积（Conv)

以Conv2d-1为例：

变化：（3, 64, 64)->(128, 32, 32)

计算方法：

其中bias等于输出通道数

因此：params=128×(4×4×3)+128=6272

2.激活函数（Activation)

不产生参数

3.正则化（BN)

以BatchNorm2d-4为例：

（256，16， 16)->(256， 16， 16 )

计算方法：

因此：params=256×2=512

4.全连接（FC)

以Linear-12为例：

（1024， 4， 4）->(1)

计算方法：

其中：bias等于输出通道数

因此：params=1024×4×4×1+1=16385

5.池化层（pooling)

不产生参数

更新于：2022年11月3日

参数计算