二分类卷积核极限数量实验
摘要
通过分类mnist 0和2的实验证明神经网络的卷积核数量是有极大值的，超过极大值网络性能会下降。并且证实就是单核的网路的性能也比无核的网络性能要好。
实验过程
将mnist的图片变成9*9，先后分别用无卷积核网络和有1-12个卷积核的网络进行分类，停止收敛的标准是网络的输出函数与目标函数的差值小于δ，让δ等于1e-4到1e-6的20个值，每个δ统计199次，取平均值
得到数据
|
无核 |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
11 |
12 |
|
|
平均准确率p-ave |
平均准确率p-ave |
平均准确率p-ave |
平均准确率p-ave |
平均准确率p-ave |
平均准确率p-ave |
平均准确率p-ave |
平均准确率p-ave |
平均准确率p-ave |
平均准确率p-ave |
平均准确率p-ave |
平均准确率p-ave |
平均准确率p-ave |
|
|
1.00E-04 |
0.980931 |
0.9719622 |
0.9754188 |
0.9794774 |
0.9825319 |
0.9823871 |
0.9832263 |
0.9827917 |
0.9827617 |
0.9826443 |
0.9815729 |
0.9808411 |
0.9800618 |
|
9.00E-05 |
0.980409 |
0.9742924 |
0.9773045 |
0.9802542 |
0.9827318 |
0.982959 |
0.9828991 |
0.982462 |
0.9825045 |
0.9824645 |
0.9812732 |
0.9808536 |
0.9814555 |
|
8.00E-05 |
0.9805963 |
0.9734782 |
0.9782886 |
0.9811133 |
0.9827942 |
0.9827492 |
0.9822947 |
0.9828292 |
0.9817877 |
0.9818226 |
0.9807362 |
0.9808611 |
0.9805838 |
|
7.00E-05 |
0.9805189 |
0.9748269 |
0.9800194 |
0.9823421 |
0.9838731 |
0.9836184 |
0.9832512 |
0.9832762 |
0.98192 |
0.9825769 |
0.9813156 |
0.9803566 |
0.9805514 |
|
6.00E-05 |
0.9813631 |
0.9759159 |
0.9813256 |
0.9836334 |
0.9836509 |
0.9832887 |
0.983506 |
0.9827442 |
0.9826943 |
0.9818851 |
0.9809834 |
0.9814005 |
0.9821398 |
|
5.00E-05 |
0.9820649 |
0.9770622 |
0.9815554 |
0.9838731 |
0.9846124 |
0.9848622 |
0.9847923 |
0.9844576 |
0.9832487 |
0.9843552 |
0.9829865 |
0.9826793 |
0.9825569 |
|
4.00E-05 |
0.9827467 |
0.9787781 |
0.9825619 |
0.9839031 |
0.985022 |
0.9860685 |
0.9851694 |
0.9853767 |
0.984485 |
0.9842802 |
0.9844476 |
0.9833936 |
0.9833387 |
|
3.00E-05 |
0.9830564 |
0.9790029 |
0.9832737 |
0.9849821 |
0.986573 |
0.9864681 |
0.9861909 |
0.9859162 |
0.9856489 |
0.9854042 |
0.9852493 |
0.9853592 |
0.9847723 |
|
2.00E-05 |
0.983526 |
0.9790903 |
0.9848147 |
0.9860161 |
0.986628 |
0.9873223 |
0.9866704 |
0.9868528 |
0.9864057 |
0.9859361 |
0.9862134 |
0.9860086 |
0.9858188 |
|
1.00E-05 |
0.9821723 |
0.980966 |
0.9863482 |
0.9871974 |
0.988184 |
0.9875646 |
0.9877469 |
0.9880216 |
0.9876145 |
0.9869701 |
0.9874447 |
0.9866205 |
0.9870351 |
|
9.00E-06 |
0.9822897 |
0.9818076 |
0.9861859 |
0.9878218 |
0.9877769 |
0.9882389 |
0.988124 |
0.9884137 |
0.9875496 |
0.9869876 |
0.9871674 |
0.9869976 |
0.9870226 |
|
8.00E-06 |
0.9825819 |
0.9819875 |
0.986096 |
0.9873947 |
0.9883738 |
0.9886085 |
0.9884087 |
0.9877194 |
0.9878243 |
0.9875046 |
0.9879242 |
0.9867878 |
0.9869402 |
|
7.00E-06 |
0.9826743 |
0.9825819 |
0.9865256 |
0.9877794 |
0.9882589 |
0.9884712 |
0.9884937 |
0.9881964 |
0.9873673 |
0.9883038 |
0.9871425 |
0.987165 |
0.9872474 |
|
6.00E-06 |
0.9826368 |
0.9833062 |
0.9868428 |
0.9879966 |
0.9888408 |
0.9884187 |
0.9886235 |
0.9883838 |
0.9880766 |
0.9880366 |
0.9882289 |
0.9874447 |
0.9876195 |
|
5.00E-06 |
0.9826618 |
0.9830265 |
0.9868053 |
0.9883288 |
0.9883363 |
0.988144 |
0.9884462 |
0.9883188 |
0.9880041 |
0.987672 |
0.9875121 |
0.9874197 |
0.9877219 |
|
4.00E-06 |
0.9829016 |
0.9828242 |
0.98713 |
0.9884812 |
0.9894627 |
0.9890406 |
0.9889532 |
0.9882514 |
0.9885436 |
0.9883063 |
0.9882114 |
0.9879542 |
0.987587 |
|
3.00E-06 |
0.9834086 |
0.9839581 |
0.9874047 |
0.9888308 |
0.9893328 |
0.9890931 |
0.9892205 |
0.9887884 |
0.9885636 |
0.9882239 |
0.9882389 |
0.988124 |
0.9879692 |
|
2.00E-06 |
0.9839805 |
0.9847773 |
0.9881865 |
0.9889807 |
0.9895476 |
0.9900621 |
0.9892329 |
0.9895701 |
0.9892429 |
0.9891955 |
0.9887634 |
0.988094 |
0.988159 |
|
1.00E-06 |
0.9850945 |
0.9854841 |
0.9889957 |
0.9897649 |
0.9904767 |
0.9902844 |
0.9903394 |
0.9899123 |
0.9899897 |
0.9900821 |
0.9896076 |
0.9895801 |
0.9893054 |
这个数据很明显，这个网络的卷积核的极大值就是4个，超过4个以后网络的性能就是下降的。
与前面的实验相比改进了卷积核的写法后就是单卷积核的网络的性能也要好于无核的网络。
|
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
11 |
12 |
|
|
δ |
最大值p-max |
最大值p-max |
最大值p-max |
最大值p-max |
最大值p-max |
最大值p-max |
最大值p-max |
最大值p-max |
最大值p-max |
最大值p-max |
最大值p-max |
最大值p-max |
|
1.00E-04 |
0.9870775 |
0.9900596 |
0.9900596 |
0.9900596 |
0.9895626 |
0.9890656 |
0.9905567 |
0.9900596 |
0.9905567 |
0.9905567 |
0.9905567 |
0.9895626 |
|
9.00E-05 |
0.9885686 |
0.9900596 |
0.9900596 |
0.9900596 |
0.9890656 |
0.9895626 |
0.9900596 |
0.9900596 |
0.9905567 |
0.9900596 |
0.9895626 |
0.9905567 |
|
8.00E-05 |
0.9870775 |
0.9895626 |
0.9895626 |
0.9900596 |
0.9895626 |
0.9900596 |
0.9900596 |
0.9895626 |
0.9905567 |
0.9905567 |
0.9900596 |
0.9905567 |
|
7.00E-05 |
0.9865805 |
0.9920477 |
0.9920477 |
0.9895626 |
0.9905567 |
0.9905567 |
0.9910537 |
0.9905567 |
0.9910537 |
0.9900596 |
0.9905567 |
0.9910537 |
|
6.00E-05 |
0.9875746 |
0.9905567 |
0.9905567 |
0.9905567 |
0.9905567 |
0.9905567 |
0.9910537 |
0.9910537 |
0.9910537 |
0.9905567 |
0.9925447 |
0.9920477 |
|
5.00E-05 |
0.9890656 |
0.9900596 |
0.9900596 |
0.9910537 |
0.9915507 |
0.9905567 |
0.9905567 |
0.9905567 |
0.9915507 |
0.9920477 |
0.9910537 |
0.9915507 |
|
4.00E-05 |
0.9870775 |
0.9910537 |
0.9910537 |
0.9905567 |
0.9915507 |
0.9905567 |
0.9910537 |
0.9910537 |
0.9910537 |
0.9920477 |
0.9900596 |
0.9915507 |
|
3.00E-05 |
0.9890656 |
0.9905567 |
0.9905567 |
0.9920477 |
0.9915507 |
0.9925447 |
0.9920477 |
0.9920477 |
0.9920477 |
0.9915507 |
0.9925447 |
0.9910537 |
|
2.00E-05 |
0.9890656 |
0.9915507 |
0.9915507 |
0.9925447 |
0.9925447 |
0.9930418 |
0.9930418 |
0.9925447 |
0.9925447 |
0.9920477 |
0.9920477 |
0.9925447 |
|
1.00E-05 |
0.9890656 |
0.9925447 |
0.9925447 |
0.9935388 |
0.9935388 |
0.9945328 |
0.9930418 |
0.9930418 |
0.9935388 |
0.9930418 |
0.9930418 |
0.9925447 |
|
9.00E-06 |
0.9905567 |
0.9930418 |
0.9930418 |
0.9945328 |
0.9930418 |
0.9945328 |
0.9935388 |
0.9940358 |
0.9935388 |
0.9935388 |
0.9935388 |
0.9930418 |
|
8.00E-06 |
0.9895626 |
0.9945328 |
0.9945328 |
0.9935388 |
0.9940358 |
0.9945328 |
0.9935388 |
0.9940358 |
0.9935388 |
0.9940358 |
0.9935388 |
0.9935388 |
|
7.00E-06 |
0.9905567 |
0.9930418 |
0.9930418 |
0.9935388 |
0.9935388 |
0.9945328 |
0.9940358 |
0.9930418 |
0.9935388 |
0.9940358 |
0.9925447 |
0.9930418 |
|
6.00E-06 |
0.9895626 |
0.9940358 |
0.9940358 |
0.9945328 |
0.9940358 |
0.9935388 |
0.9940358 |
0.9940358 |
0.9940358 |
0.9950298 |
0.9940358 |
0.9935388 |
|
5.00E-06 |
0.9895626 |
0.9935388 |
0.9935388 |
0.9950298 |
0.9945328 |
0.9940358 |
0.9955268 |
0.9940358 |
0.9940358 |
0.9940358 |
0.9955268 |
0.9940358 |
|
4.00E-06 |
0.9895626 |
0.9935388 |
0.9935388 |
0.9945328 |
0.9945328 |
0.9950298 |
0.9945328 |
0.9950298 |
0.9945328 |
0.9945328 |
0.9940358 |
0.9945328 |
|
3.00E-06 |
0.9905567 |
0.9945328 |
0.9945328 |
0.9940358 |
0.9940358 |
0.9955268 |
0.9945328 |
0.9945328 |
0.9940358 |
0.9945328 |
0.9945328 |
0.9945328 |
|
2.00E-06 |
0.9920477 |
0.9950298 |
0.9950298 |
0.9950298 |
0.9950298 |
0.9960239 |
0.9950298 |
0.9950298 |
0.9955268 |
0.9950298 |
0.9940358 |
0.9945328 |
|
1.00E-06 |
0.9920477 |
0.9945328 |
0.9945328 |
0.9950298 |
0.9950298 |
0.9955268 |
0.9960239 |
0.9945328 |
0.9950298 |
0.9955268 |
0.9960239 |
0.9950298 |
虽然随着核数量的增加网络性能下降但是最大值差异不大，但与无核的网络相比至少要大0.002，这个对比很显著。
|
无核 |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
11 |
12 |
|
|
δ |
耗时 min/199 |
耗时 min/199 |
耗时 min/199 |
耗时 min/199 |
耗时 min/199 |
耗时 min/199 |
耗时 min/199 |
耗时 min/199 |
耗时 min/199 |
耗时 min/199 |
耗时 min/199 |
耗时 min/199 |
耗时 min/199 |
|
1.00E-04 |
0.2510167 |
1.2715 |
2.8217333 |
5.1089833 |
8.2244 |
11.63025 |
14.325867 |
18.464317 |
19.568783 |
24.883217 |
27.894917 |
30.5407 |
35.482333 |
|
9.00E-05 |
0.2691833 |
1.4081167 |
3.0315333 |
5.4479167 |
8.46165 |
11.841017 |
15.08195 |
18.796167 |
22.856683 |
26.815167 |
28.957083 |
32.827283 |
34.380633 |
|
8.00E-05 |
0.2690667 |
1.5663833 |
3.3068833 |
6.3013667 |
7.6072833 |
9.0804333 |
16.304467 |
20.220283 |
21.989017 |
25.436333 |
30.08465 |
32.092583 |
37.09115 |
|
7.00E-05 |
0.2797 |
1.6771667 |
3.6570833 |
6.3787167 |
10.332117 |
14.287483 |
17.20495 |
19.265317 |
24.540617 |
28.654333 |
31.4023 |
35.701783 |
38.670483 |
|
6.00E-05 |
0.29395 |
1.7820667 |
4.0756833 |
5.4802333 |
11.6074 |
15.76265 |
17.986717 |
21.974783 |
26.9867 |
29.22455 |
33.383133 |
37.477267 |
41.156033 |
|
5.00E-05 |
0.3057167 |
2.1413833 |
4.4190667 |
8.1032667 |
12.5821 |
16.450767 |
19.949167 |
22.1754 |
28.092133 |
32.563083 |
36.113567 |
38.653917 |
44.88355 |
|
4.00E-05 |
0.32555 |
2.39175 |
5.1381667 |
9.29685 |
13.4934 |
15.459617 |
22.3508 |
26.857917 |
31.82875 |
34.540633 |
38.003217 |
43.7331 |
46.465733 |
|
3.00E-05 |
0.3573333 |
2.8671 |
6.21925 |
10.574367 |
13.08 |
19.680767 |
25.4449 |
29.330183 |
34.697517 |
40.599 |
44.00425 |
48.46525 |
52.224633 |
|
2.00E-05 |
0.4487167 |
3.5389 |
7.6344833 |
12.621517 |
18.1334 |
23.220783 |
25.266033 |
33.831767 |
36.608567 |
42.10865 |
46.517267 |
50.2975 |
57.184533 |
|
1.00E-05 |
0.5657 |
5.16495 |
10.4814 |
16.757217 |
23.107117 |
29.435483 |
33.560983 |
39.054233 |
45.175417 |
54.56055 |
57.1836 |
64.459117 |
69.426083 |
|
9.00E-06 |
0.5779 |
5.4613667 |
11.08785 |
14.880367 |
23.517033 |
30.675617 |
38.538583 |
42.5816 |
49.113533 |
53.532183 |
59.51315 |
64.300083 |
69.5925 |
|
8.00E-06 |
0.6237333 |
5.8232 |
11.272 |
18.11145 |
23.764067 |
29.274283 |
35.182233 |
42.7501 |
48.519683 |
55.278533 |
60.619967 |
66.11955 |
73.005083 |
|
7.00E-06 |
0.6701667 |
6.1751 |
12.540717 |
19.626467 |
23.809333 |
33.355817 |
40.047483 |
44.642233 |
52.064817 |
57.1209 |
63.674367 |
68.816567 |
74.744667 |
|
6.00E-06 |
-1.043133 |
6.4720833 |
13.542133 |
20.0948 |
26.614533 |
31.45445 |
38.972967 |
46.176583 |
52.081467 |
60.708967 |
64.057983 |
72.303883 |
76.163333 |
|
5.00E-06 |
0.8278667 |
7.2051167 |
14.259883 |
21.24865 |
28.258033 |
34.109517 |
41.226333 |
48.910033 |
54.673833 |
62.315317 |
66.795883 |
75.911267 |
81.254883 |
|
4.00E-06 |
0.92955 |
7.6314833 |
15.4772 |
22.509267 |
30.298767 |
39.369883 |
46.338433 |
51.8897 |
59.972433 |
66.741217 |
73.211083 |
80.869683 |
85.531033 |
|
3.00E-06 |
1.0805333 |
6.5921667 |
15.777033 |
24.25435 |
33.881 |
41.083117 |
47.96175 |
56.240533 |
63.19005 |
69.45185 |
78.905017 |
85.909833 |
94.3877 |
|
2.00E-06 |
1.4429333 |
9.5656 |
15.067417 |
27.935483 |
30.0696 |
46.009817 |
53.679467 |
62.338233 |
70.84785 |
78.75425 |
86.743033 |
96.4067 |
100.94462 |
|
1.00E-06 |
2.15025 |
11.655817 |
22.0313 |
30.766883 |
43.671067 |
54.981717 |
63.107367 |
72.584117 |
82.2587 |
92.99845 |
100.5198 |
111.2489 |
120.38415 |
由于改变了核的写法，网络收敛的时间几乎就是线性的，12个核的收敛时间是4个核的收敛时间的2.79倍，但是性能只有4个核的99.8%。也就是浪费了2.79倍的时间和算力但是换来的却是性能下降了0.2%。
由此可见增加卷积核虽然不会导致网络性能大幅下降，但是确实可能造成大量的浪费，因此优化下卷积核的数量是很有必要的。
|
无核 |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
11 |
12 |
|
|
δ |
迭代次数n |
迭代次数n |
迭代次数n |
迭代次数n |
迭代次数n |
迭代次数n |
迭代次数n |
迭代次数n |
迭代次数n |
迭代次数n |
迭代次数n |
迭代次数n |
迭代次数n |
|
1.00E-04 |
3165.7085 |
6594.0653 |
7748.9648 |
9529.9246 |
11176.784 |
12379.889 |
13558.055 |
14096.704 |
15073.402 |
15596.523 |
15981.08 |
16323.141 |
16675.151 |
|
9.00E-05 |
3407.9095 |
7377.2462 |
8445.4271 |
10215.111 |
11433.447 |
12701.347 |
14308.377 |
14948.116 |
15857.749 |
16152.794 |
16621.407 |
16887.362 |
16965.899 |
|
8.00E-05 |
3486.7638 |
8280.9648 |
9263.9347 |
11330.613 |
12633.432 |
13926.367 |
15185.442 |
16021.583 |
16417.08 |
16968.116 |
17287.101 |
17275.548 |
17629.211 |
|
7.00E-05 |
3668.6834 |
8864.809 |
10286.09 |
11740.794 |
13858.935 |
15354.01 |
16078.221 |
16613.035 |
17118.744 |
17903.965 |
18037.789 |
18508.497 |
18403.266 |
|
6.00E-05 |
3962.5628 |
9461.8945 |
11461.08 |
12902.628 |
15540.714 |
17113.663 |
17103.623 |
17932.744 |
18677.93 |
18283.779 |
18910.442 |
19475.588 |
19564.789 |
|
5.00E-05 |
4167.6784 |
11471.121 |
12547.915 |
15065.08 |
16882.035 |
17916.347 |
18959.307 |
19536.06 |
20133.709 |
20395.01 |
20835.99 |
20913.201 |
20887.457 |
|
4.00E-05 |
4515.8291 |
12890.02 |
14653.377 |
17104.111 |
18521.171 |
19657.643 |
20620.779 |
21938.07 |
22008.899 |
22532.583 |
22735.92 |
22696.322 |
22777.518 |
|
3.00E-05 |
5059.8794 |
15370.442 |
17854.764 |
19685.779 |
21551.729 |
22471.181 |
23230.905 |
24183.859 |
24314.332 |
24922.613 |
24905.533 |
25087.668 |
24971.905 |
|
2.00E-05 |
6524.8744 |
19073.583 |
22028.106 |
23803.889 |
25042.492 |
26591.739 |
26606.236 |
26850.07 |
27237.663 |
27379.02 |
27449.276 |
27155.905 |
27362.648 |
|
1.00E-05 |
8527.8995 |
28067.92 |
30412.719 |
31097.427 |
32837.94 |
32943.824 |
33148.839 |
33615.513 |
33142.141 |
33984.995 |
33088.357 |
33560.442 |
33350.975 |
|
9.00E-06 |
8746.4221 |
29789.422 |
32222.844 |
32834.593 |
33713.864 |
34114.477 |
34996.824 |
34830.402 |
35206.653 |
34491.02 |
34649.281 |
34490.653 |
34142.261 |
|
8.00E-06 |
9474.0101 |
31697.523 |
32766.905 |
33801.07 |
34210.688 |
35002.668 |
36016.985 |
35084.824 |
35570.97 |
35643.171 |
35501.302 |
35080.271 |
35375.392 |
|
7.00E-06 |
10301.668 |
33712.196 |
35559.377 |
36198.317 |
36519.879 |
37369.322 |
36858.357 |
37080.543 |
37402.528 |
36598.528 |
36545.251 |
36462.261 |
36376.387 |
|
6.00E-06 |
11850.653 |
35417.709 |
37922.764 |
38082.186 |
38643.935 |
38723.136 |
38242.618 |
38353.578 |
38801.95 |
38738.503 |
38041.749 |
37771.367 |
37248.302 |
|
5.00E-06 |
13068.734 |
39449.126 |
39982.452 |
40338.633 |
41573.965 |
40671.688 |
40908.286 |
41188.02 |
40236.759 |
40241.07 |
39805.392 |
39794.266 |
39332.538 |
|
4.00E-06 |
14670.05 |
41822.709 |
43194.121 |
42763.352 |
41775.06 |
44011.492 |
43305.236 |
43111.573 |
43195.477 |
43185.925 |
42760.296 |
42320.879 |
41888.849 |
|
3.00E-06 |
17112.472 |
46296.513 |
46017.899 |
46075.472 |
45625.131 |
45659.955 |
46976.673 |
46795.719 |
46156.447 |
44958.789 |
46297.935 |
45931.382 |
45347.779 |
|
2.00E-06 |
23285.869 |
52644.849 |
51703.975 |
52201.01 |
52207.563 |
53319.307 |
51039.03 |
51098.784 |
50992.623 |
51048.387 |
50976.905 |
50405.407 |
49902.251 |
|
1.00E-06 |
35222.241 |
64157.402 |
63712.854 |
61751.06 |
62164.94 |
62810.206 |
61508.296 |
60616.573 |
60869.266 |
60364.543 |
59755.307 |
59271.693 |
58164.457 |
随着核的增加迭代次数是缓慢减少的。