Supplementary Material (PhD Thesis)
Open-Set Recognition for Different Classifiers

Set of results of neural networks in open-set scenarios

Pedro Ribeiro Mendes Júnior
Prof. Anderson Rocha (advisor)

Institute of Computing (IC)
University of Campinas (UNICAMP)

Information about this page can be found here.

Table. Information about training sets. All training sets include samples from MNIST dataset. Variations shown in this table indicate the portion of data included from CHARS74K additional dataset.
KU no training sample from CHARS74K.
KUn numbers for the KU classes.
KUl letters for the KU classes.
KUnl numbers and letters for the KU classes.
KUc numbers for the KN classes.
KUcn numbers for both KN and KU classes.
KUcl numbers for KN classes and letters for KU classes.
KUcnl numbers for both KN and KU classes and letters for KU classes.

Table. Information about testing sets.
KN numbers from the known (KN) classes of MNIST dataset.
KNc numbers from the known (KN) classes of CHARS74K dataset.
KU numbers from the known unknown (KU) classes of MNIST dataset.
KUl letters from the known unknown (KU) classes of CHARS74K dataset.
KUn numbers from the known unknown (KU) classes of CHARS74K dataset.
UU numbers from the unknown unknown (UU) classes of MNIST dataset.
UUl letters from the unknown unknown (UU) classes of CHARS74K dataset.
UUn numbers from the unknown unknown (UU) classes of CHARS74K dataset.
RN random data.
RNp test set with pixels randomly shuffled.

Table 1. OSNNet-TR-KU-normal-table_image.png
First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KU KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9935 0.4029 0.9939 0.3574 0.5069 0.1480 0.3624 0.2977 0.3296 0.2453
523 0.9941 0.2790 0.9948 0.4987 0.6579 0.3176 0.5038 0.4514 0.5400 0.5580
631 0.9916 0.2652 0.9947 0.6312 0.7418 0.4797 0.6261 0.6490 0.4997 0.5348
424 0.9941 0.3469 0.9944 0.5221 0.6866 0.3003 0.5265 0.4739 0.4466 0.3871
532 0.9932 0.2492 0.9953 0.7684 0.8281 0.5229 0.7442 0.7613 0.8585 0.6245
325 0.9951 0.2029 0.9968 0.7782 0.8931 0.4258 0.7677 0.7635 0.7802 0.7051
433 0.9922 0.2292 0.9974 0.7747 0.8689 0.5069 0.7635 0.7218 0.8563 0.5341
541 0.9918 0.1834 0.9955 0.8245 0.8748 0.5360 0.8154 0.8404 0.7573 0.6387
442 0.9910 0.1133 0.9954 0.9036 0.9524 0.6064 0.8805 0.9185 0.8562 0.8713
334 0.9943 0.2639 0.9970 0.7907 0.8765 0.5076 0.7960 0.7448 0.7992 0.7011
226 0.9941 0.4527 0.9984 0.6529 0.7592 0.4962 0.6445 0.6049 0.6726 0.7080
451 0.9891 0.1327 0.9966 0.9599 0.9781 0.6648 0.9563 0.9367 0.9639 0.9608
343 0.9924 0.1554 0.9969 0.9323 0.9469 0.6655 0.9332 0.9200 0.9568 0.7998
235 0.9940 0.2067 0.9983 0.9227 0.9826 0.6822 0.9202 0.9008 0.9517 0.9134
352 0.9907 0.1139 0.9970 0.9229 0.9588 0.8213 0.9256 0.9253 0.9555 0.8220
361 0.9910 0.0815 0.9969 0.9585 0.9881 0.8389 0.9511 0.9938 0.9886 0.8008
244 0.9929 0.2999 0.9982 0.8747 0.9018 0.7403 0.8622 0.8685 0.9145 0.8157
253 0.9921 0.0997 0.9978 0.9800 0.9944 0.8098 0.9744 0.9663 0.9999 0.8124
262 0.9879 0.1100 0.9982 0.9773 0.9895 0.8847 0.9815 0.9615 0.9957 0.8195
mean 0.9924 0.2204 0.9965 0.7911 0.8624 0.5766 0.7861 0.7737 0.7959 0.6975
Table 2. OSNNet-TR-KU-threshold-table_image.png
Results with threshold on softmax. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KU KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9805 0.1320 0.9988 0.9368 0.9577 0.6026 0.9067 0.9250 0.9514 0.9854
523 0.9830 0.1140 0.9983 0.9379 0.9600 0.6898 0.9155 0.9158 0.9872 0.9689
631 0.9746 0.0824 0.9990 0.9742 0.9881 0.7682 0.9596 0.9708 0.9467 0.9962
424 0.9852 0.1587 0.9987 0.8984 0.9586 0.6001 0.8691 0.8377 0.9176 0.9517
532 0.9795 0.0811 0.9989 0.9734 0.9859 0.8094 0.9634 0.9665 0.9983 0.9955
325 0.9880 0.1024 0.9990 0.9135 0.9354 0.6380 0.9084 0.9092 0.9223 0.9814
433 0.9802 0.0895 0.9993 0.9619 0.9783 0.7285 0.9517 0.9221 0.9930 0.9683
541 0.9752 0.0554 0.9989 0.9866 0.9949 0.8081 0.9831 0.9939 0.9983 0.9953
442 0.9744 0.0311 0.9988 0.9926 0.9983 0.8427 0.9837 0.9962 0.9717 0.9975
334 0.9861 0.1345 0.9987 0.9618 0.9917 0.7203 0.9508 0.9285 0.9935 0.9895
226 0.9879 0.2393 0.9994 0.9051 0.9532 0.6642 0.8945 0.8709 0.9613 0.9502
451 0.9707 0.0557 0.9991 0.9959 0.9970 0.8842 0.9934 0.9971 0.9999 0.9998
343 0.9800 0.0557 0.9990 0.9923 0.9942 0.8446 0.9898 0.9860 0.9991 0.9784
235 0.9854 0.1272 0.9994 0.9851 1.0000 0.8028 0.9811 0.9742 0.9983 0.9955
352 0.9760 0.0293 0.9991 0.9912 0.9948 0.9396 0.9878 0.9920 0.9995 0.9771
361 0.9761 0.0236 0.9992 0.9944 0.9994 0.9432 0.9899 1.0000 0.9999 0.9954
244 0.9843 0.1377 0.9996 0.9778 0.9781 0.8591 0.9648 0.9835 0.9981 0.9856
253 0.9807 0.0347 0.9994 0.9967 0.9984 0.9031 0.9947 0.9879 1.0000 0.9752
262 0.9721 0.0309 0.9995 0.9972 0.9983 0.9517 0.9974 0.9921 1.0000 0.9940
mean 0.9800 0.0903 0.9990 0.9670 0.9822 0.7895 0.9571 0.9552 0.9808 0.9832
Table 3. OSNNet-TR-KU-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KU KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.8183 0.0700 0.8853 0.9735 0.9740 0.6102 0.9620 0.9589 0.9994 0.9972
523 0.8453 0.0996 0.8759 0.9231 0.8790 0.6519 0.9118 0.9395 0.9622 0.9788
631 0.8603 0.0521 0.9144 0.9742 0.9674 0.5197 0.9674 0.9852 0.9863 0.9934
424 0.9020 0.1681 0.6660 0.8646 0.8858 0.4729 0.8355 0.8079 0.9409 0.9357
532 0.9012 0.0962 0.8076 0.9474 0.9556 0.6580 0.9278 0.9603 0.9310 0.9941
325 0.9460 0.1816 0.8616 0.7708 0.8463 0.4789 0.7573 0.7692 0.8014 0.8255
433 0.9385 0.1667 0.8098 0.9008 0.9287 0.5411 0.8945 0.8698 0.9843 0.9167
541 0.7335 0.1141 0.9263 0.9654 0.9608 0.6636 0.9651 0.9412 0.9756 0.9603
442 0.9384 0.1399 0.7157 0.9006 0.8817 0.5459 0.8887 0.8791 0.8595 0.9196
334 0.9424 0.1731 0.7675 0.8660 0.8788 0.5972 0.8593 0.8501 0.9282 0.8963
226 0.9447 0.2227 0.7044 0.7408 0.7489 0.5815 0.7255 0.7405 0.7808 0.8220
451 0.9862 0.1802 0.7917 0.8956 0.9053 0.5802 0.9097 0.8752 0.9555 0.9862
343 0.8685 0.0860 0.9380 0.9366 0.9435 0.7213 0.9329 0.9181 0.8891 0.8880
235 0.9605 0.2473 0.7753 0.7743 0.7915 0.7210 0.7739 0.8121 0.7957 0.7241
352 0.8651 0.1420 0.8603 0.8403 0.9038 0.7862 0.8571 0.8339 0.8772 0.7692
361 0.8230 0.1207 0.9221 0.9324 0.9301 0.7493 0.9164 0.9260 0.9659 0.8187
244 0.9167 0.3056 0.8367 0.6864 0.7124 0.5860 0.6758 0.6991 0.6643 0.6268
253 0.9667 0.1737 0.7561 0.8338 0.8294 0.7253 0.8082 0.8203 0.8247 0.6116
262 0.9604 0.0980 0.8814 0.9568 0.9566 0.7851 0.9407 0.9424 0.9540 0.7902
mean 0.9009 0.1494 0.8261 0.8781 0.8884 0.6303 0.8689 0.8699 0.8987 0.8660
Table 4. OSNNet-TR-KU-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KU KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9769 0.0636 0.8348 0.9792 0.9805 0.5537 0.9648 0.9579 0.9999 0.9996
523 0.9636 0.0925 0.6555 0.8912 0.8773 0.5411 0.8969 0.8918 0.9152 0.9654
631 0.9324 0.0701 0.8088 0.9455 0.9605 0.4880 0.9386 0.9670 0.9986 0.9804
424 0.9857 0.1781 0.7361 0.8330 0.8427 0.4310 0.8107 0.7749 0.8747 0.9249
532 0.9662 0.0797 0.7657 0.9595 0.9604 0.6260 0.9452 0.9638 0.9475 0.9981
325 0.9895 0.2207 0.8110 0.7383 0.8127 0.4372 0.7218 0.7229 0.7530 0.7967
433 0.9724 0.1700 0.6151 0.9185 0.9053 0.4427 0.9141 0.8715 0.9788 0.9562
541 0.9591 0.0863 0.7303 0.9565 0.9571 0.5707 0.9538 0.9577 0.9812 0.9761
442 0.9853 0.1279 0.7559 0.9107 0.9183 0.5611 0.8964 0.9149 0.8304 0.9446
334 0.9895 0.2072 0.7797 0.8520 0.8754 0.4752 0.8385 0.8033 0.9510 0.7962
226 0.9447 0.2227 0.7044 0.7408 0.7489 0.5815 0.7255 0.7405 0.7808 0.8220
451 0.9848 0.1694 0.8482 0.9100 0.9103 0.5890 0.9197 0.8458 0.9810 0.9900
343 0.9924 0.2520 0.7929 0.8065 0.7936 0.4584 0.7995 0.7951 0.7314 0.7958
235 0.9605 0.2473 0.7753 0.7743 0.7915 0.7210 0.7739 0.8121 0.7957 0.7241
352 0.9883 0.2115 0.7418 0.7785 0.8028 0.4730 0.7714 0.7649 0.8199 0.8141
361 0.9759 0.1182 0.7656 0.9173 0.9005 0.7052 0.9009 0.9167 0.9850 0.9267
244 0.9167 0.3056 0.8367 0.6864 0.7124 0.5860 0.6758 0.6991 0.6643 0.6268
253 0.9667 0.1737 0.7561 0.8338 0.8294 0.7253 0.8082 0.8203 0.8247 0.6116
262 0.9604 0.0980 0.8814 0.9568 0.9566 0.7851 0.9407 0.9424 0.9540 0.7902
mean 0.9690 0.1629 0.7682 0.8626 0.8703 0.5659 0.8524 0.8507 0.8825 0.8652
Table 5. OSNNet-TR-KU-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KU KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9718 0.1123 0.9854 0.9391 0.9510 0.7378 0.9236 0.9180 0.9408 0.9890
523 0.9836 0.1426 0.9843 0.8931 0.8793 0.6538 0.8717 0.8908 0.9607 0.9576
631 0.9723 0.0809 0.9884 0.9497 0.9711 0.7342 0.9373 0.9270 0.8880 0.9841
424 0.9878 0.1827 0.9868 0.8691 0.9050 0.5482 0.8379 0.8040 0.8848 0.9465
532 0.9813 0.1314 0.9802 0.9144 0.9218 0.7186 0.8963 0.9183 0.8210 0.9632
325 0.9932 0.1799 0.9890 0.8230 0.8864 0.5262 0.8283 0.8178 0.8366 0.9299
433 0.9873 0.1465 0.9873 0.9264 0.9324 0.6172 0.9179 0.8985 0.9637 0.9330
541 0.9812 0.1056 0.9801 0.9456 0.9564 0.7224 0.9365 0.9380 0.9550 0.9577
442 0.9862 0.1059 0.9789 0.9468 0.9547 0.6508 0.9237 0.9463 0.8675 0.9230
334 0.9925 0.2034 0.9860 0.9037 0.9416 0.5690 0.8934 0.8460 0.9706 0.9532
226 0.9954 0.3676 0.9889 0.7306 0.7782 0.4724 0.7058 0.6745 0.8415 0.8873
451 0.9863 0.1517 0.9810 0.9430 0.9562 0.7006 0.9418 0.8891 0.9420 0.9733
343 0.9930 0.1751 0.9803 0.9194 0.9146 0.6394 0.9161 0.9054 0.8967 0.9114
235 0.9955 0.2908 0.9852 0.8738 0.9262 0.5550 0.8703 0.8425 0.9233 0.8763
352 0.9932 0.1667 0.9725 0.9118 0.9365 0.7067 0.9042 0.8740 0.9043 0.8693
361 0.9935 0.1205 0.9680 0.9139 0.9156 0.7662 0.8964 0.9396 0.8950 0.9243
244 0.9967 0.3555 0.9876 0.8502 0.8852 0.6442 0.8247 0.8208 0.9461 0.7387
253 0.9969 0.2106 0.9570 0.8916 0.8956 0.6047 0.8955 0.8891 0.9781 0.7347
262 0.9963 0.2686 0.9743 0.8751 0.8969 0.6987 0.8939 0.8810 0.9244 0.7677
mean 0.9886 0.1841 0.9811 0.8958 0.9160 0.6456 0.8850 0.8748 0.9126 0.9063
Table 6. OSNNet-TR-KU-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KU KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9914 0.3118 0.6392 0.5125 0.5015 0.2803 0.4929 0.4809 0.4327 0.6237
523 0.9926 0.2678 0.7385 0.5491 0.5392 0.3203 0.5386 0.5650 0.5745 0.6498
631 0.9917 0.2697 0.6685 0.4971 0.5045 0.2527 0.4726 0.4801 0.3432 0.5703
424 0.9936 0.3065 0.8774 0.6568 0.6810 0.3215 0.6177 0.5750 0.6805 0.7360
532 0.9925 0.3054 0.7394 0.5658 0.6036 0.3793 0.5398 0.5800 0.5809 0.6614
325 0.9934 0.2040 0.9856 0.8007 0.8742 0.5118 0.8083 0.8027 0.8074 0.9117
433 0.9918 0.2653 0.7699 0.6674 0.6757 0.4038 0.6662 0.6456 0.7319 0.6899
541 0.9921 0.2307 0.7780 0.6565 0.6538 0.3954 0.6471 0.6906 0.6492 0.6817
442 0.9924 0.2122 0.8669 0.7879 0.7825 0.4383 0.7556 0.8083 0.6427 0.7277
334 0.9938 0.2498 0.9418 0.8293 0.8593 0.4880 0.8246 0.7762 0.9196 0.8723
226 0.9954 0.3676 0.9889 0.7306 0.7782 0.4724 0.7058 0.6745 0.8415 0.8873
451 0.9931 0.2991 0.8485 0.7448 0.7544 0.4091 0.7437 0.6445 0.8166 0.7861
343 0.9938 0.2134 0.9138 0.8549 0.8539 0.5786 0.8538 0.8391 0.8565 0.8538
235 0.9955 0.2908 0.9852 0.8738 0.9262 0.5550 0.8703 0.8425 0.9233 0.8763
352 0.9941 0.1980 0.9483 0.8370 0.8662 0.6359 0.8327 0.7514 0.8234 0.8216
361 0.9947 0.1659 0.9083 0.8676 0.8779 0.6777 0.8498 0.9027 0.8771 0.8604
244 0.9967 0.3555 0.9876 0.8502 0.8852 0.6442 0.8247 0.8208 0.9461 0.7387
253 0.9969 0.2106 0.9570 0.8916 0.8956 0.6047 0.8955 0.8891 0.9781 0.7347
262 0.9963 0.2686 0.9743 0.8751 0.8969 0.6987 0.8939 0.8810 0.9244 0.7677
mean 0.9938 0.2628 0.8693 0.7394 0.7584 0.4772 0.7281 0.7184 0.7552 0.7606

Table 7. OSNNet-TR-KUn-normal-table_image.png
First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUn KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9936 0.0538 0.9948 0.9768 0.9984 0.1553 0.9639 0.9825 1.0000 0.7266
523 0.9937 0.0512 0.9953 0.9749 1.0000 0.3404 0.9642 0.9908 1.0000 0.9370
631 0.9925 0.0382 0.9938 0.9863 0.9988 0.4670 0.9766 0.9818 1.0000 0.8778
424 0.9953 0.0422 0.9957 0.9835 0.9970 0.3362 0.9739 0.9828 1.0000 0.8202
532 0.9928 0.0330 0.9949 0.9907 0.9957 0.4718 0.9789 0.9868 1.0000 0.8899
325 0.9949 0.0319 0.9963 0.9811 0.9986 0.4173 0.9730 0.9850 1.0000 0.8067
433 0.9930 0.0389 0.9960 0.9917 0.9958 0.4815 0.9848 0.9776 1.0000 0.8570
541 0.9913 0.0212 0.9949 0.9935 0.9991 0.5162 0.9870 0.9958 1.0000 0.9280
442 0.9912 0.0157 0.9953 0.9936 0.9991 0.5992 0.9876 0.9948 1.0000 0.9772
334 0.9941 0.0368 0.9975 0.9918 0.9963 0.4969 0.9864 0.9940 1.0000 0.9009
226 0.9966 0.0878 0.9977 0.9777 0.9976 0.4401 0.9734 0.9696 0.9998 0.8147
451 0.9904 0.0149 0.9957 0.9959 0.9969 0.6717 0.9931 1.0000 1.0000 0.9885
343 0.9925 0.0204 0.9965 0.9955 0.9991 0.6660 0.9944 0.9962 1.0000 0.9199
235 0.9953 0.0476 0.9979 0.9922 0.9987 0.6420 0.9901 0.9929 1.0000 0.9498
352 0.9900 0.0031 0.9971 0.9982 1.0000 0.8523 0.9959 0.9986 1.0000 0.9485
361 0.9905 0.0078 0.9971 0.9954 1.0000 0.8651 0.9922 1.0000 1.0000 0.9392
244 0.9921 0.0289 0.9975 0.9941 1.0000 0.6975 0.9892 0.9935 1.0000 0.9320
253 0.9918 0.0096 0.9979 0.9979 0.9993 0.7984 0.9956 0.9988 1.0000 0.9682
262 0.9891 0.0207 0.9981 0.9987 0.9995 0.8793 0.9977 0.9921 1.0000 0.9295
mean 0.9927 0.0318 0.9963 0.9900 0.9984 0.5681 0.9841 0.9902 1.0000 0.9006
Table 8. OSNNet-TR-KUn-threshold-table_image.png
Results with threshold on softmax. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUn KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9802 0.0157 0.9988 0.9962 1.0000 0.6167 0.9894 0.9979 1.0000 0.9989
523 0.9828 0.0195 0.9985 0.9953 1.0000 0.6799 0.9898 0.9972 1.0000 0.9995
631 0.9762 0.0073 0.9987 0.9979 1.0000 0.7784 0.9944 1.0000 1.0000 0.9994
424 0.9867 0.0175 0.9988 0.9957 1.0000 0.6287 0.9917 0.9951 1.0000 0.9976
532 0.9790 0.0108 0.9987 0.9979 1.0000 0.7709 0.9950 1.0000 1.0000 0.9986
325 0.9882 0.0057 0.9988 0.9959 0.9986 0.6416 0.9921 0.9973 1.0000 0.9944
433 0.9803 0.0120 0.9990 0.9984 0.9989 0.7230 0.9953 0.9986 1.0000 0.9986
541 0.9739 0.0033 0.9986 0.9990 1.0000 0.8076 0.9973 0.9979 1.0000 0.9995
442 0.9747 0.0034 0.9985 0.9985 1.0000 0.8512 0.9962 0.9986 1.0000 0.9998
334 0.9856 0.0086 0.9990 0.9981 1.0000 0.7260 0.9951 0.9991 1.0000 0.9950
226 0.9907 0.0388 0.9992 0.9923 0.9976 0.6200 0.9879 0.9884 1.0000 0.9823
451 0.9729 0.0032 0.9990 0.9993 1.0000 0.8775 0.9983 1.0000 1.0000 0.9999
343 0.9809 0.0089 0.9989 0.9992 1.0000 0.8461 0.9984 1.0000 1.0000 0.9987
235 0.9879 0.0225 0.9992 0.9976 1.0000 0.7729 0.9965 0.9972 1.0000 0.9980
352 0.9744 0.0000 0.9991 0.9997 1.0000 0.9515 0.9986 1.0000 1.0000 0.9981
361 0.9751 0.0011 0.9993 0.9990 1.0000 0.9435 0.9975 1.0000 1.0000 0.9995
244 0.9822 0.0069 0.9993 0.9977 1.0000 0.8236 0.9958 0.9989 1.0000 0.9982
253 0.9811 0.0042 0.9992 0.9996 1.0000 0.8872 0.9984 1.0000 1.0000 0.9995
262 0.9726 0.0082 0.9995 0.9998 1.0000 0.9468 0.9992 1.0000 1.0000 0.9988
mean 0.9803 0.0104 0.9990 0.9977 0.9997 0.7838 0.9951 0.9982 1.0000 0.9976
Table 9. OSNNet-TR-KUn-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUn KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.7286 0.0399 0.9531 0.9703 0.9691 0.7278 0.9608 0.9779 0.9903 0.9997
523 0.8701 0.0597 0.7150 0.9381 0.9509 0.5574 0.9194 0.9380 0.9936 0.9848
631 0.7630 0.0399 0.9568 0.9882 0.9926 0.6537 0.9831 0.9740 0.9999 0.9947
424 0.9847 0.0656 0.8472 0.8985 0.8892 0.4528 0.9018 0.9115 0.9576 0.9819
532 0.8245 0.0475 0.7966 0.9578 0.9734 0.6430 0.9415 0.9447 0.9991 0.9972
325 0.9785 0.0797 0.8058 0.8992 0.9022 0.4363 0.8947 0.8896 0.9436 0.9308
433 0.9234 0.0351 0.8935 0.9833 0.9739 0.6178 0.9802 0.9763 0.9999 0.9810
541 0.8387 0.0683 0.7715 0.9212 0.9264 0.6220 0.9241 0.9391 0.9925 0.9810
442 0.8949 0.0474 0.8917 0.9829 0.9846 0.5865 0.9755 0.9912 1.0000 0.9355
334 0.9406 0.1070 0.9152 0.8580 0.8847 0.5835 0.8501 0.8834 0.8755 0.8652
226 0.8996 0.0540 0.9775 0.9793 0.9607 0.7407 0.9711 0.9812 0.9845 0.9482
451 0.8952 0.0636 0.8713 0.9826 0.9738 0.6124 0.9792 0.9268 1.0000 0.9570
343 0.9337 0.0646 0.8961 0.9110 0.9101 0.6976 0.9028 0.9155 0.9241 0.7969
235 0.9243 0.1725 0.7327 0.7257 0.7495 0.6097 0.7273 0.7692 0.7330 0.7927
352 0.9568 0.0371 0.8532 0.9742 0.9662 0.7531 0.9634 0.9523 0.9985 0.9121
361 0.9342 0.0499 0.8692 0.9467 0.9445 0.8666 0.9382 0.9563 0.9819 0.8450
244 0.9322 0.0571 0.9753 0.9664 0.9918 0.7977 0.9600 0.9907 0.9999 0.9551
253 0.9708 0.2973 0.6732 0.6502 0.6536 0.6826 0.6727 0.6969 0.7817 0.6864
262 0.9580 0.1126 0.8721 0.8556 0.8754 0.7590 0.8594 0.8507 0.8650 0.8225
mean 0.9027 0.0789 0.8562 0.9152 0.9196 0.6526 0.9108 0.9192 0.9485 0.9141
Table 10. OSNNet-TR-KUn-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUn KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9642 0.0426 0.8345 0.9576 0.9686 0.5532 0.9386 0.9641 0.9985 0.9989
523 0.9850 0.0611 0.7407 0.9209 0.9300 0.5973 0.9088 0.9335 0.9907 0.9962
631 0.9172 0.0401 0.8526 0.9668 0.9674 0.5320 0.9571 0.9699 0.9926 0.9964
424 0.9847 0.0838 0.8192 0.9143 0.8978 0.5205 0.9054 0.9128 0.9577 0.9529
532 0.9124 0.0697 0.7370 0.9341 0.9538 0.6689 0.9309 0.9296 0.9634 0.9956
325 0.9921 0.1476 0.7717 0.8202 0.8130 0.4005 0.8079 0.8152 0.8242 0.8948
433 0.9744 0.0828 0.7757 0.9649 0.9446 0.4586 0.9588 0.9459 0.9988 0.9580
541 0.9593 0.0860 0.6908 0.9089 0.9116 0.5165 0.9127 0.9450 0.9936 0.9976
442 0.9829 0.0731 0.7467 0.9148 0.9182 0.5236 0.9061 0.9315 0.9755 0.9875
334 0.9911 0.1251 0.8600 0.9149 0.9289 0.4530 0.8992 0.9225 0.9900 0.8606
226 0.8996 0.0540 0.9775 0.9793 0.9607 0.7407 0.9711 0.9812 0.9845 0.9482
451 0.9816 0.1095 0.7402 0.9178 0.9134 0.5458 0.9101 0.8259 0.9806 0.9697
343 0.9904 0.1177 0.7216 0.8343 0.8280 0.5459 0.8178 0.8285 0.7827 0.8455
235 0.9243 0.1725 0.7327 0.7257 0.7495 0.6097 0.7273 0.7692 0.7330 0.7927
352 0.9828 0.0953 0.7850 0.9148 0.8933 0.5964 0.9073 0.8927 0.9235 0.8893
361 0.9822 0.1272 0.6736 0.7612 0.7505 0.6724 0.7455 0.8354 0.7936 0.8074
244 0.9322 0.0571 0.9753 0.9664 0.9918 0.7977 0.9600 0.9907 0.9999 0.9551
253 0.9708 0.2973 0.6732 0.6502 0.6536 0.6826 0.6727 0.6969 0.7817 0.6864
262 0.9580 0.1126 0.8721 0.8556 0.8754 0.7590 0.8594 0.8507 0.8650 0.8225
mean 0.9624 0.1029 0.7884 0.8854 0.8869 0.5881 0.8788 0.8916 0.9226 0.9134
Table 11. OSNNet-TR-KUn-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUn KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9734 0.0349 0.9893 0.9888 0.9952 0.7129 0.9792 0.9866 1.0000 0.9879
523 0.9826 0.0589 0.9875 0.9866 0.9917 0.6819 0.9766 0.9847 1.0000 0.9944
631 0.9707 0.0342 0.9888 0.9937 0.9965 0.7342 0.9867 0.9895 1.0000 0.9641
424 0.9885 0.0442 0.9903 0.9876 0.9971 0.5995 0.9756 0.9777 1.0000 0.9875
532 0.9825 0.0556 0.9790 0.9846 0.9885 0.7274 0.9751 0.9874 0.9992 0.9914
325 0.9937 0.0557 0.9896 0.9798 0.9958 0.5205 0.9682 0.9717 0.9965 0.9464
433 0.9854 0.0549 0.9874 0.9927 0.9969 0.6240 0.9846 0.9738 1.0000 0.9801
541 0.9796 0.0367 0.9806 0.9913 0.9927 0.7198 0.9847 0.9947 0.9995 0.9926
442 0.9873 0.0355 0.9764 0.9885 0.9974 0.6562 0.9809 0.9962 1.0000 0.9822
334 0.9929 0.0771 0.9807 0.9851 0.9912 0.5464 0.9782 0.9799 1.0000 0.9711
226 0.9952 0.1157 0.9900 0.9766 0.9976 0.4599 0.9664 0.9611 0.9970 0.9300
451 0.9858 0.0734 0.9808 0.9902 0.9900 0.7036 0.9843 0.9971 1.0000 0.9649
343 0.9930 0.0667 0.9783 0.9882 0.9949 0.6500 0.9866 0.9747 0.9998 0.8888
235 0.9960 0.1199 0.9826 0.9753 0.9956 0.5536 0.9709 0.9715 0.9995 0.9580
352 0.9922 0.0542 0.9780 0.9901 0.9962 0.7251 0.9865 0.9806 1.0000 0.9226
361 0.9931 0.0564 0.9738 0.9863 0.9901 0.7654 0.9733 0.9880 1.0000 0.9380
244 0.9961 0.0848 0.9840 0.9833 0.9949 0.6242 0.9709 0.9841 1.0000 0.9283
253 0.9966 0.0957 0.9654 0.9724 0.9822 0.5866 0.9694 0.9729 0.9985 0.8234
262 0.9957 0.1114 0.9769 0.9862 0.9944 0.6913 0.9818 0.9745 0.9990 0.8746
mean 0.9884 0.0666 0.9821 0.9856 0.9936 0.6464 0.9779 0.9814 0.9994 0.9488
Table 12. OSNNet-TR-KUn-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUn KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9909 0.2137 0.6941 0.7442 0.7437 0.2803 0.7153 0.6894 0.8218 0.6214
523 0.9919 0.1521 0.8141 0.8425 0.8136 0.3681 0.8230 0.8331 0.8955 0.8015
631 0.9924 0.2667 0.5455 0.5950 0.5803 0.1995 0.5832 0.6111 0.5870 0.4582
424 0.9928 0.1071 0.9390 0.9294 0.9470 0.4436 0.9135 0.9054 0.9878 0.9069
532 0.9928 0.2472 0.5769 0.6894 0.6894 0.2918 0.6654 0.6685 0.7751 0.6155
325 0.9937 0.0557 0.9896 0.9798 0.9958 0.5205 0.9682 0.9717 0.9965 0.9464
433 0.9911 0.1462 0.8716 0.8968 0.9143 0.4144 0.8777 0.8604 0.9324 0.8242
541 0.9941 0.2549 0.5996 0.7002 0.7111 0.2464 0.6992 0.6912 0.6887 0.5968
442 0.9927 0.1137 0.8638 0.9259 0.9156 0.4490 0.9009 0.9300 0.9839 0.8607
334 0.9940 0.1338 0.8813 0.9285 0.9262 0.4364 0.9227 0.9129 0.9859 0.8927
226 0.9952 0.1157 0.9900 0.9766 0.9976 0.4599 0.9664 0.9611 0.9970 0.9300
451 0.9929 0.2215 0.7996 0.8762 0.8900 0.4249 0.8825 0.8292 0.9809 0.7077
343 0.9941 0.0988 0.9608 0.9635 0.9764 0.5861 0.9638 0.9529 0.9980 0.8418
235 0.9960 0.1199 0.9826 0.9753 0.9956 0.5536 0.9709 0.9715 0.9995 0.9580
352 0.9930 0.0616 0.9686 0.9854 0.9920 0.6914 0.9818 0.9729 1.0000 0.8821
361 0.9943 0.1076 0.8862 0.9235 0.9329 0.6571 0.9093 0.9140 0.9028 0.8291
244 0.9961 0.0848 0.9840 0.9833 0.9949 0.6242 0.9709 0.9841 1.0000 0.9283
253 0.9966 0.0957 0.9654 0.9724 0.9822 0.5866 0.9694 0.9729 0.9985 0.8234
262 0.9957 0.1114 0.9769 0.9862 0.9944 0.6913 0.9818 0.9745 0.9990 0.8746
mean 0.9937 0.1425 0.8573 0.8881 0.8944 0.4698 0.8772 0.8740 0.9226 0.8052

Table 13. OSNNet-TR-KUl-normal-table_image.png
First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9933 0.0000 0.9944 0.9989 1.0000 0.1809 0.9953 1.0000 1.0000 0.9288
523 0.9937 0.0007 0.9951 0.9993 1.0000 0.3441 0.9972 1.0000 1.0000 0.9883
631 0.9919 0.0000 0.9945 0.9995 1.0000 0.4191 0.9972 1.0000 1.0000 0.9712
424 0.9940 0.0000 0.9960 0.9991 1.0000 0.3480 0.9970 1.0000 1.0000 0.9780
532 0.9928 0.0014 0.9953 0.9992 1.0000 0.4882 0.9968 1.0000 1.0000 0.9321
325 0.9950 0.0000 0.9965 0.9995 1.0000 0.4226 0.9976 1.0000 1.0000 0.9885
433 0.9929 0.0036 0.9958 0.9994 1.0000 0.4666 0.9975 1.0000 1.0000 0.9730
541 0.9910 0.0021 0.9943 0.9997 0.9991 0.4988 0.9979 1.0000 1.0000 0.9849
442 0.9919 0.0000 0.9949 0.9996 1.0000 0.5813 0.9975 1.0000 1.0000 0.9978
334 0.9940 0.0042 0.9968 0.9995 0.9988 0.5350 0.9976 1.0000 1.0000 0.9871
226 0.9949 0.0053 0.9980 0.9993 1.0000 0.4888 0.9981 1.0000 1.0000 0.9944
451 0.9882 0.0033 0.9971 0.9998 1.0000 0.6911 0.9984 1.0000 1.0000 0.9980
343 0.9937 0.0040 0.9950 0.9995 1.0000 0.6231 0.9984 1.0000 1.0000 0.9869
235 0.9948 0.0038 0.9985 0.9998 1.0000 0.6846 0.9984 1.0000 1.0000 0.9845
352 0.9895 0.0000 0.9969 0.9997 1.0000 0.8404 0.9988 1.0000 1.0000 0.9834
361 0.9906 0.0033 0.9967 0.9997 1.0000 0.8734 0.9986 1.0000 1.0000 0.9905
244 0.9935 0.0028 0.9979 0.9998 1.0000 0.7148 0.9982 1.0000 1.0000 0.9794
253 0.9912 0.0019 0.9980 0.9999 1.0000 0.8132 0.9990 1.0000 1.0000 0.9989
262 0.9892 0.0129 0.9984 1.0000 1.0000 0.8766 0.9991 0.9988 1.0000 0.9961
mean 0.9924 0.0026 0.9963 0.9995 0.9999 0.5732 0.9978 0.9999 1.0000 0.9811
Table 14. OSNNet-TR-KUl-threshold-table_image.png
Results with threshold on softmax. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9799 0.0000 0.9989 0.9999 1.0000 0.6444 0.9988 1.0000 1.0000 0.9997
523 0.9826 0.0007 0.9987 0.9999 1.0000 0.6899 0.9990 1.0000 1.0000 1.0000
631 0.9755 0.0000 0.9990 0.9999 1.0000 0.7260 0.9988 1.0000 1.0000 0.9999
424 0.9845 0.0000 0.9989 0.9997 1.0000 0.6378 0.9986 1.0000 1.0000 0.9999
532 0.9790 0.0000 0.9988 0.9999 1.0000 0.7955 0.9990 1.0000 1.0000 0.9991
325 0.9885 0.0000 0.9989 0.9998 1.0000 0.6429 0.9992 1.0000 1.0000 0.9999
433 0.9808 0.0016 0.9986 0.9999 1.0000 0.7107 0.9990 1.0000 1.0000 0.9993
541 0.9738 0.0006 0.9988 0.9998 1.0000 0.7834 0.9992 1.0000 1.0000 1.0000
442 0.9763 0.0000 0.9988 0.9999 1.0000 0.8377 0.9992 1.0000 1.0000 1.0000
334 0.9858 0.0014 0.9987 0.9999 1.0000 0.7584 0.9991 1.0000 1.0000 0.9999
226 0.9880 0.0026 0.9993 0.9998 1.0000 0.6696 0.9992 1.0000 1.0000 0.9998
451 0.9680 0.0000 0.9993 1.0000 1.0000 0.8881 0.9995 1.0000 1.0000 1.0000
343 0.9828 0.0000 0.9986 0.9999 1.0000 0.8218 0.9996 1.0000 1.0000 1.0000
235 0.9874 0.0019 0.9995 0.9999 1.0000 0.8040 0.9995 1.0000 1.0000 0.9998
352 0.9749 0.0000 0.9992 1.0000 1.0000 0.9501 0.9997 1.0000 1.0000 0.9998
361 0.9765 0.0022 0.9991 1.0000 1.0000 0.9525 0.9995 1.0000 1.0000 1.0000
244 0.9856 0.0000 0.9993 0.9999 1.0000 0.8369 0.9992 1.0000 1.0000 0.9989
253 0.9787 0.0019 0.9993 0.9999 1.0000 0.8971 0.9995 1.0000 1.0000 1.0000
262 0.9735 0.0000 0.9995 1.0000 1.0000 0.9443 0.9999 1.0000 1.0000 1.0000
mean 0.9801 0.0007 0.9990 0.9999 1.0000 0.7890 0.9992 1.0000 1.0000 0.9998
Table 15. OSNNet-TR-KUl-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.7465 0.0120 0.9532 0.9773 0.9760 0.6252 0.9714 0.9952 0.9994 0.9987
523 0.7971 0.0632 0.7342 0.7457 0.7335 0.6244 0.7457 0.7569 0.7830 0.9964
631 0.6850 0.0283 0.9279 0.9755 0.9675 0.7648 0.9680 0.9917 0.9964 0.9997
424 0.9134 0.1022 0.7719 0.7278 0.7517 0.5071 0.7213 0.7455 0.7047 0.9765
532 0.8327 0.1004 0.7300 0.6864 0.6647 0.5823 0.6826 0.6546 0.6192 0.9968
325 0.8823 0.0499 0.9494 0.9409 0.9638 0.5532 0.9450 0.9383 0.9764 0.9736
433 0.8906 0.0673 0.8488 0.8979 0.8851 0.4978 0.8954 0.8773 0.9152 0.9243
541 0.8066 0.0583 0.9017 0.8990 0.9069 0.5945 0.9171 0.9352 0.9810 0.9938
442 0.9593 0.0459 0.8598 0.9054 0.8933 0.5698 0.9061 0.9063 0.9003 0.9428
334 0.9222 0.0494 0.8670 0.9141 0.9145 0.6285 0.9124 0.9235 0.9093 0.9596
226 0.9518 0.0640 0.9194 0.8968 0.8944 0.7184 0.8975 0.9081 0.9003 0.8998
451 0.8909 0.0645 0.9164 0.9137 0.9206 0.6950 0.9259 0.9062 0.9954 0.9646
343 0.9147 0.0444 0.8788 0.9178 0.9112 0.6603 0.8915 0.9066 0.8615 0.9604
235 0.9708 0.0995 0.7997 0.8497 0.8344 0.6788 0.8440 0.8684 0.8820 0.8078
352 0.9056 0.0266 0.9617 0.9922 0.9918 0.8136 0.9894 0.9986 1.0000 0.9148
361 0.9382 0.0696 0.8665 0.9304 0.9237 0.7220 0.9062 0.9090 0.8739 0.9100
244 0.9237 0.0651 0.7732 0.9113 0.9540 0.5614 0.9062 0.8940 0.9465 0.8859
253 0.9029 0.0762 0.9154 0.8182 0.8134 0.7719 0.8224 0.8127 0.8001 0.7867
262 0.9634 0.1098 0.9060 0.8519 0.8633 0.7641 0.8356 0.8382 0.8313 0.7267
mean 0.8841 0.0630 0.8674 0.8817 0.8823 0.6491 0.8781 0.8824 0.8882 0.9273
Table 16. OSNNet-TR-KUl-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9627 0.0250 0.8584 0.8866 0.8906 0.5635 0.8842 0.9041 0.9030 0.9983
523 0.9639 0.0713 0.7673 0.7893 0.7940 0.5813 0.7943 0.7717 0.8001 0.9964
631 0.9590 0.0375 0.8506 0.8021 0.8398 0.4885 0.7914 0.8258 0.7829 0.9983
424 0.9845 0.1510 0.7138 0.6511 0.6686 0.4384 0.6615 0.6587 0.6956 0.9775
532 0.9666 0.0814 0.7701 0.8890 0.8829 0.5621 0.8783 0.8688 0.9006 0.9712
325 0.9894 0.0796 0.7831 0.8978 0.8978 0.4648 0.8726 0.8695 0.9119 0.9359
433 0.9853 0.1040 0.6312 0.7780 0.7368 0.4764 0.7790 0.7215 0.7999 0.9593
541 0.9468 0.0813 0.8155 0.8542 0.8607 0.5185 0.8600 0.8935 0.8805 0.9987
442 0.9632 0.1217 0.7740 0.7692 0.7176 0.5971 0.7724 0.8113 0.7679 0.9354
334 0.9877 0.1125 0.7919 0.7436 0.7463 0.5251 0.7503 0.7463 0.7708 0.9437
226 0.9518 0.0640 0.9194 0.8968 0.8944 0.7184 0.8975 0.9081 0.9003 0.8998
451 0.9864 0.1632 0.7558 0.7366 0.7127 0.5122 0.7588 0.7463 0.9264 0.9638
343 0.9860 0.1441 0.8041 0.7053 0.7061 0.5093 0.7104 0.7184 0.6723 0.8475
235 0.9708 0.0995 0.7997 0.8497 0.8344 0.6788 0.8440 0.8684 0.8820 0.8078
352 0.9868 0.0766 0.8609 0.9569 0.9492 0.4870 0.9451 0.9325 0.9985 0.9523
361 0.9866 0.1223 0.7607 0.7341 0.7173 0.6282 0.7226 0.7511 0.7406 0.8394
244 0.9237 0.0651 0.7732 0.9113 0.9540 0.5614 0.9062 0.8940 0.9465 0.8859
253 0.9029 0.0762 0.9154 0.8182 0.8134 0.7719 0.8224 0.8127 0.8001 0.7867
262 0.9634 0.1098 0.9060 0.8519 0.8633 0.7641 0.8356 0.8382 0.8313 0.7267
mean 0.9667 0.0940 0.8027 0.8169 0.8147 0.5709 0.8151 0.8179 0.8374 0.9171
Table 17. OSNNet-TR-KUl-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9712 0.0030 0.9905 0.9992 0.9984 0.7406 0.9969 1.0000 1.0000 0.9963
523 0.9830 0.0061 0.9861 0.9991 1.0000 0.6798 0.9976 0.9992 1.0000 0.9954
631 0.9732 0.0026 0.9837 0.9987 0.9977 0.7115 0.9963 1.0000 1.0000 0.9940
424 0.9878 0.0032 0.9901 0.9992 0.9988 0.5780 0.9966 1.0000 1.0000 0.9958
532 0.9810 0.0110 0.9851 0.9991 0.9989 0.7397 0.9969 0.9970 1.0000 0.9914
325 0.9935 0.0021 0.9894 0.9988 1.0000 0.5467 0.9962 1.0000 1.0000 0.9821
433 0.9866 0.0135 0.9863 0.9994 0.9969 0.6387 0.9973 0.9980 1.0000 0.9901
541 0.9813 0.0153 0.9757 0.9974 0.9928 0.6591 0.9974 1.0000 1.0000 0.9889
442 0.9864 0.0119 0.9774 0.9986 0.9974 0.6610 0.9965 1.0000 1.0000 0.9973
334 0.9927 0.0103 0.9853 0.9982 0.9988 0.5785 0.9961 0.9961 1.0000 0.9779
226 0.9950 0.0219 0.9825 0.9978 1.0000 0.4566 0.9946 0.9988 1.0000 0.9901
451 0.9863 0.0253 0.9784 0.9983 0.9965 0.6844 0.9962 1.0000 1.0000 0.9676
343 0.9930 0.0211 0.9761 0.9986 0.9992 0.6471 0.9962 0.9944 1.0000 0.9674
235 0.9959 0.0223 0.9813 0.9984 0.9987 0.5727 0.9953 0.9977 1.0000 0.9713
352 0.9929 0.0070 0.9740 0.9982 0.9969 0.7226 0.9969 0.9985 1.0000 0.9529
361 0.9935 0.0213 0.9699 0.9973 0.9964 0.7690 0.9917 0.9979 1.0000 0.9735
244 0.9965 0.0454 0.9642 0.9959 0.9953 0.6042 0.9888 0.9926 1.0000 0.8930
253 0.9969 0.0234 0.9807 0.9975 0.9987 0.6063 0.9964 0.9946 1.0000 0.9517
262 0.9958 0.0434 0.9716 0.9978 0.9984 0.6849 0.9942 0.9906 1.0000 0.8950
mean 0.9886 0.0163 0.9804 0.9983 0.9979 0.6464 0.9957 0.9977 1.0000 0.9722
Table 18. OSNNet-TR-KUl-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9914 0.0922 0.7007 0.8633 0.8568 0.2652 0.8542 0.8387 0.8572 0.6416
523 0.9930 0.0733 0.7118 0.9236 0.8679 0.3082 0.9220 0.9032 0.9483 0.7528
631 0.9929 0.1574 0.4956 0.7625 0.7431 0.1680 0.7529 0.6962 0.7498 0.4772
424 0.9922 0.0209 0.9500 0.9938 0.9960 0.4025 0.9889 0.9915 1.0000 0.9448
532 0.9922 0.1230 0.6693 0.7894 0.8174 0.3422 0.7947 0.7470 0.8082 0.6403
325 0.9942 0.0032 0.9815 0.9983 1.0000 0.5184 0.9954 0.9994 1.0000 0.9693
433 0.9918 0.0726 0.8676 0.9492 0.9192 0.4480 0.9441 0.9388 0.9386 0.8465
541 0.9925 0.1365 0.6408 0.7502 0.7472 0.2652 0.7628 0.7630 0.7880 0.6349
442 0.9921 0.0616 0.8646 0.9690 0.9580 0.4339 0.9635 0.9588 0.9992 0.8965
334 0.9929 0.0177 0.9762 0.9966 0.9960 0.5517 0.9915 0.9870 1.0000 0.9510
226 0.9950 0.0219 0.9825 0.9978 1.0000 0.4566 0.9946 0.9988 1.0000 0.9901
451 0.9919 0.1056 0.8068 0.9106 0.9072 0.4386 0.9207 0.8609 0.9532 0.8032
343 0.9936 0.0378 0.9302 0.9936 0.9877 0.5967 0.9913 0.9761 1.0000 0.8700
235 0.9959 0.0223 0.9813 0.9984 0.9987 0.5727 0.9953 0.9977 1.0000 0.9713
352 0.9945 0.0558 0.8730 0.9780 0.9599 0.5878 0.9792 0.9671 1.0000 0.8428
361 0.9949 0.0471 0.9039 0.9652 0.9525 0.6838 0.9599 0.9779 0.9892 0.8736
244 0.9965 0.0454 0.9642 0.9959 0.9953 0.6042 0.9888 0.9926 1.0000 0.8930
253 0.9969 0.0234 0.9807 0.9975 0.9987 0.6063 0.9964 0.9946 1.0000 0.9517
262 0.9958 0.0434 0.9716 0.9978 0.9984 0.6849 0.9942 0.9906 1.0000 0.8950
mean 0.9937 0.0611 0.8554 0.9385 0.9316 0.4703 0.9363 0.9253 0.9490 0.8340

Table 19. OSNNet-TR-KUnl-normal-table_image.png
First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUnl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9928 0.0000 0.9948 0.9995 1.0000 0.1735 0.9962 1.0000 1.0000 0.9614
523 0.9935 0.0000 0.9944 0.9990 1.0000 0.3300 0.9973 1.0000 1.0000 0.9938
631 0.9913 0.0014 0.9952 0.9993 1.0000 0.4642 0.9971 1.0000 1.0000 0.9939
424 0.9935 0.0008 0.9960 0.9992 1.0000 0.3307 0.9967 1.0000 1.0000 0.9903
532 0.9927 0.0000 0.9955 0.9990 1.0000 0.5047 0.9968 1.0000 1.0000 0.9419
325 0.9951 0.0010 0.9958 0.9987 1.0000 0.4004 0.9976 0.9988 1.0000 0.9817
433 0.9922 0.0026 0.9966 0.9998 1.0000 0.5106 0.9981 1.0000 1.0000 0.9873
541 0.9921 0.0037 0.9951 0.9997 1.0000 0.5162 0.9984 1.0000 1.0000 0.9818
442 0.9920 0.0000 0.9945 0.9995 1.0000 0.6245 0.9976 0.9986 1.0000 0.9981
334 0.9943 0.0027 0.9970 0.9995 0.9976 0.5340 0.9976 0.9993 1.0000 0.9800
226 0.9958 0.0026 0.9977 0.9994 1.0000 0.4694 0.9983 1.0000 1.0000 0.9916
451 0.9896 0.0021 0.9965 0.9996 1.0000 0.7068 0.9983 1.0000 1.0000 0.9983
343 0.9926 0.0071 0.9961 0.9999 0.9992 0.6696 0.9988 1.0000 1.0000 0.9821
235 0.9942 0.0075 0.9978 0.9997 1.0000 0.6533 0.9984 1.0000 1.0000 0.9865
352 0.9892 0.0000 0.9973 0.9997 1.0000 0.8405 0.9990 1.0000 1.0000 0.9921
361 0.9905 0.0015 0.9967 0.9997 1.0000 0.8514 0.9983 1.0000 1.0000 0.9863
244 0.9923 0.0027 0.9979 0.9997 1.0000 0.7231 0.9981 1.0000 1.0000 0.9836
253 0.9905 0.0000 0.9980 0.9996 1.0000 0.8157 0.9983 1.0000 1.0000 0.9802
262 0.9894 0.0078 0.9977 0.9999 1.0000 0.8753 0.9993 1.0000 1.0000 0.9603
mean 0.9923 0.0023 0.9964 0.9995 0.9998 0.5786 0.9979 0.9998 1.0000 0.9827
Table 20. OSNNet-TR-KUnl-threshold-table_image.png
Results with threshold on softmax. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUnl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9788 0.0000 0.9987 0.9998 1.0000 0.6428 0.9990 1.0000 1.0000 0.9999
523 0.9828 0.0000 0.9985 0.9996 1.0000 0.6885 0.9988 1.0000 1.0000 1.0000
631 0.9742 0.0000 0.9988 0.9999 1.0000 0.7516 0.9991 1.0000 1.0000 1.0000
424 0.9830 0.0000 0.9989 0.9998 1.0000 0.6301 0.9986 1.0000 1.0000 0.9999
532 0.9784 0.0000 0.9986 1.0000 1.0000 0.8019 0.9990 1.0000 1.0000 0.9977
325 0.9882 0.0000 0.9988 0.9998 1.0000 0.6151 0.9992 1.0000 1.0000 0.9998
433 0.9793 0.0000 0.9988 1.0000 1.0000 0.7235 0.9994 1.0000 1.0000 1.0000
541 0.9742 0.0000 0.9987 0.9999 1.0000 0.8019 0.9993 1.0000 1.0000 1.0000
442 0.9763 0.0000 0.9987 0.9999 1.0000 0.8688 0.9991 1.0000 1.0000 1.0000
334 0.9857 0.0000 0.9989 1.0000 1.0000 0.7491 0.9992 1.0000 1.0000 0.9999
226 0.9891 0.0026 0.9994 0.9998 1.0000 0.6403 0.9991 1.0000 1.0000 0.9997
451 0.9716 0.0000 0.9994 0.9999 1.0000 0.9068 0.9995 1.0000 1.0000 1.0000
343 0.9802 0.0000 0.9988 0.9999 1.0000 0.8546 0.9997 1.0000 1.0000 1.0000
235 0.9874 0.0000 0.9994 0.9999 1.0000 0.7903 0.9992 1.0000 1.0000 0.9999
352 0.9741 0.0000 0.9992 0.9999 1.0000 0.9510 0.9995 1.0000 1.0000 0.9999
361 0.9756 0.0000 0.9993 0.9998 1.0000 0.9411 0.9995 1.0000 1.0000 1.0000
244 0.9819 0.0000 0.9993 0.9999 1.0000 0.8472 0.9995 1.0000 1.0000 0.9996
253 0.9773 0.0000 0.9992 0.9999 1.0000 0.9017 0.9994 1.0000 1.0000 0.9999
262 0.9744 0.0000 0.9994 0.9999 1.0000 0.9483 0.9998 1.0000 1.0000 0.9994
mean 0.9796 0.0001 0.9990 0.9999 1.0000 0.7924 0.9993 1.0000 1.0000 0.9998
Table 21. OSNNet-TR-KUnl-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUnl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.8354 0.0247 0.8246 0.8932 0.9147 0.6150 0.8686 0.9152 0.8844 0.9993
523 0.8274 0.0468 0.8080 0.8605 0.8247 0.6236 0.8578 0.8758 0.8575 0.9955
631 0.7517 0.0217 0.9045 0.9115 0.9072 0.6103 0.9112 0.9064 0.8999 0.9986
424 0.9363 0.1062 0.7907 0.8418 0.8627 0.5434 0.8409 0.8472 0.8915 0.9827
532 0.8749 0.0879 0.7253 0.6629 0.7227 0.6598 0.6457 0.6286 0.5845 0.9860
325 0.9265 0.0128 0.9348 0.9892 0.9950 0.5739 0.9850 0.9708 0.9984 0.9810
433 0.9290 0.0940 0.7619 0.8013 0.7370 0.5472 0.7770 0.7734 0.7683 0.9735
541 0.8898 0.0793 0.8177 0.7685 0.7843 0.6858 0.7749 0.8535 0.8483 0.9788
442 0.9789 0.0373 0.8457 0.8995 0.8799 0.5835 0.8907 0.9147 0.9037 0.9676
334 0.9060 0.0781 0.8481 0.8360 0.8471 0.6073 0.8449 0.8498 0.8956 0.9198
226 0.9783 0.1719 0.6452 0.6510 0.6378 0.4403 0.6429 0.6615 0.6484 0.6254
451 0.9039 0.0753 0.8699 0.8057 0.7806 0.6481 0.8184 0.7194 0.8339 0.9693
343 0.9320 0.0991 0.7781 0.7210 0.7361 0.6396 0.7137 0.7388 0.6965 0.9006
235 0.9536 0.1729 0.8536 0.7486 0.7453 0.7262 0.7586 0.7682 0.7287 0.7639
352 0.9411 0.0372 0.8651 0.9599 0.9479 0.7766 0.9447 0.9700 0.9572 0.9519
361 0.8996 0.0224 0.8921 0.9889 0.9764 0.8466 0.9810 0.9896 0.9858 0.8421
244 0.9765 0.1431 0.7223 0.7908 0.8095 0.5479 0.7849 0.7863 0.8058 0.7989
253 0.9689 0.0832 0.8497 0.8510 0.8760 0.8080 0.8446 0.8578 0.8421 0.7524
262 0.9514 0.1324 0.8402 0.8308 0.8339 0.7210 0.8317 0.8348 0.8128 0.8326
mean 0.9138 0.0803 0.8199 0.8322 0.8326 0.6423 0.8272 0.8348 0.8339 0.9063
Table 22. OSNNet-TR-KUnl-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUnl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9759 0.0515 0.8203 0.8697 0.8594 0.5488 0.8632 0.8788 0.9551 0.9999
523 0.9832 0.0472 0.8128 0.8971 0.8783 0.5597 0.8929 0.8816 0.8988 0.9976
631 0.9771 0.0316 0.8785 0.9383 0.9199 0.4674 0.9345 0.9668 0.9984 0.9993
424 0.9631 0.1296 0.8282 0.7562 0.7319 0.4622 0.7580 0.7568 0.7682 0.9849
532 0.9826 0.1069 0.6300 0.6062 0.6229 0.6219 0.5908 0.5968 0.4804 0.9953
325 0.9882 0.0402 0.8472 0.9068 0.9117 0.4492 0.8865 0.8779 0.9030 0.9209
433 0.9642 0.0610 0.8330 0.8663 0.8415 0.5135 0.8586 0.8579 0.8945 0.9871
541 0.9844 0.0995 0.6925 0.6999 0.6972 0.5035 0.7001 0.7191 0.7248 0.9788
442 0.9856 0.0618 0.7640 0.7945 0.7890 0.6388 0.7867 0.7980 0.6901 0.9637
334 0.9887 0.0988 0.6600 0.7666 0.7766 0.4785 0.7632 0.7683 0.7438 0.9113
226 0.9783 0.1719 0.6452 0.6510 0.6378 0.4403 0.6429 0.6615 0.6484 0.6254
451 0.9354 0.0863 0.8012 0.7989 0.7956 0.6211 0.8074 0.7852 0.8113 0.9891
343 0.9905 0.1185 0.7478 0.8399 0.8003 0.5380 0.8360 0.8057 0.8310 0.7929
235 0.9536 0.1729 0.8536 0.7486 0.7453 0.7262 0.7586 0.7682 0.7287 0.7639
352 0.9922 0.0941 0.8097 0.8863 0.8780 0.6221 0.8859 0.8775 0.9002 0.8129
361 0.9859 0.1209 0.5869 0.6785 0.6558 0.5224 0.6599 0.7153 0.6165 0.6980
244 0.9765 0.1431 0.7223 0.7908 0.8095 0.5479 0.7849 0.7863 0.8058 0.7989
253 0.9689 0.0832 0.8497 0.8510 0.8760 0.8080 0.8446 0.8578 0.8421 0.7524
262 0.9514 0.1324 0.8402 0.8308 0.8339 0.7210 0.8317 0.8348 0.8128 0.8326
mean 0.9750 0.0974 0.7696 0.7988 0.7927 0.5679 0.7940 0.7997 0.7923 0.8845
Table 23. OSNNet-TR-KUnl-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUnl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9712 0.0039 0.9882 0.9996 0.9957 0.7505 0.9973 1.0000 1.0000 0.9950
523 0.9823 0.0076 0.9841 0.9991 1.0000 0.6975 0.9974 0.9981 1.0000 0.9965
631 0.9689 0.0019 0.9895 0.9996 1.0000 0.7147 0.9976 1.0000 1.0000 0.9990
424 0.9880 0.0061 0.9906 0.9986 0.9974 0.5841 0.9947 0.9992 1.0000 0.9868
532 0.9815 0.0084 0.9826 0.9990 1.0000 0.7250 0.9965 0.9976 1.0000 0.9918
325 0.9939 0.0071 0.9871 0.9986 1.0000 0.5184 0.9962 0.9969 1.0000 0.9850
433 0.9864 0.0133 0.9881 0.9995 0.9969 0.6291 0.9969 0.9974 1.0000 0.9962
541 0.9818 0.0157 0.9758 0.9981 0.9890 0.6952 0.9973 1.0000 1.0000 0.9843
442 0.9871 0.0103 0.9783 0.9991 0.9991 0.6856 0.9957 1.0000 1.0000 0.9960
334 0.9927 0.0210 0.9838 0.9981 0.9976 0.5711 0.9960 1.0000 1.0000 0.9861
226 0.9954 0.0087 0.9814 0.9984 1.0000 0.4722 0.9964 0.9989 1.0000 0.9612
451 0.9848 0.0238 0.9838 0.9986 0.9982 0.7343 0.9965 0.9900 1.0000 0.9921
343 0.9931 0.0277 0.9795 0.9989 0.9992 0.6574 0.9973 0.9951 1.0000 0.9797
235 0.9961 0.0211 0.9878 0.9986 1.0000 0.5498 0.9957 0.9988 1.0000 0.9463
352 0.9926 0.0032 0.9766 0.9982 1.0000 0.7218 0.9966 1.0000 1.0000 0.9672
361 0.9934 0.0243 0.9633 0.9977 0.9967 0.7617 0.9927 0.9960 1.0000 0.9154
244 0.9963 0.0241 0.9870 0.9983 0.9992 0.6248 0.9940 0.9978 1.0000 0.9037
253 0.9964 0.0355 0.9739 0.9961 0.9980 0.6081 0.9913 0.9903 1.0000 0.8257
262 0.9959 0.0809 0.9188 0.9891 0.9836 0.6367 0.9827 0.9710 0.9975 0.8016
mean 0.9883 0.0181 0.9790 0.9981 0.9974 0.6494 0.9952 0.9962 0.9999 0.9584
Table 24. OSNNet-TR-KUnl-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUnl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9907 0.0842 0.7228 0.8099 0.7712 0.3217 0.7965 0.8146 0.8106 0.7128
523 0.9926 0.0833 0.5954 0.7338 0.6707 0.3050 0.7235 0.7135 0.7064 0.6611
631 0.9917 0.1340 0.5834 0.6949 0.6749 0.1945 0.6883 0.7008 0.6171 0.6461
424 0.9925 0.0429 0.9394 0.9829 0.9697 0.4034 0.9759 0.9729 0.9984 0.8808
532 0.9932 0.1433 0.5551 0.6589 0.6841 0.2529 0.6572 0.5556 0.7538 0.5295
325 0.9944 0.0071 0.9860 0.9983 1.0000 0.4986 0.9960 0.9969 1.0000 0.9776
433 0.9917 0.0896 0.7796 0.8857 0.8753 0.3871 0.8696 0.8215 0.8601 0.7614
541 0.9921 0.0997 0.7587 0.8847 0.8572 0.3763 0.8807 0.8708 0.9039 0.7631
442 0.9926 0.0595 0.8406 0.9808 0.9612 0.4498 0.9700 0.9767 0.9971 0.8871
334 0.9937 0.0311 0.9605 0.9956 0.9924 0.5127 0.9928 0.9974 1.0000 0.9609
226 0.9954 0.0087 0.9814 0.9984 1.0000 0.4722 0.9964 0.9989 1.0000 0.9612
451 0.9936 0.1999 0.6520 0.7757 0.7644 0.3372 0.7696 0.6966 0.7848 0.6405
343 0.9941 0.0505 0.9407 0.9932 0.9891 0.5844 0.9884 0.9847 1.0000 0.8477
235 0.9961 0.0211 0.9878 0.9986 1.0000 0.5498 0.9957 0.9988 1.0000 0.9463
352 0.9931 0.0214 0.9577 0.9921 0.9930 0.6688 0.9922 0.9830 1.0000 0.8950
361 0.9942 0.0493 0.9094 0.9906 0.9803 0.7100 0.9817 0.9960 0.9960 0.8716
244 0.9963 0.0241 0.9870 0.9983 0.9992 0.6248 0.9940 0.9978 1.0000 0.9037
253 0.9964 0.0355 0.9739 0.9961 0.9980 0.6081 0.9913 0.9903 1.0000 0.8257
262 0.9959 0.0809 0.9188 0.9891 0.9836 0.6367 0.9827 0.9710 0.9975 0.8016
mean 0.9937 0.0666 0.8437 0.9136 0.9034 0.4681 0.9075 0.8967 0.9171 0.8144

Table 25. OSNNet-TR-KUc-normal-table_image.png
First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUc KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9932 0.8402 0.9948 0.0253 0.1208 0.1708 0.0352 0.0000 0.0011 0.0551
523 0.9938 0.8311 0.9936 0.0137 0.1292 0.3030 0.0220 0.0367 0.0002 0.2562
631 0.9919 0.8383 0.9949 0.0237 0.1198 0.4353 0.0382 0.0534 0.0007 0.1697
424 0.9945 0.8813 0.9952 0.0272 0.1177 0.3206 0.0392 0.0248 0.0010 0.3235
532 0.9932 0.8526 0.9944 0.0369 0.1085 0.4695 0.0415 0.0275 0.0011 0.2818
325 0.9948 0.8865 0.9967 0.0261 0.2067 0.3966 0.0330 0.0491 0.0005 0.3105
433 0.9927 0.8367 0.9967 0.0494 0.1577 0.4863 0.0622 0.0692 0.0010 0.2335
541 0.9908 0.8195 0.9952 0.0455 0.1483 0.5347 0.0698 0.0866 0.0020 0.3124
442 0.9916 0.8523 0.9949 0.0438 0.1928 0.5970 0.0521 0.0864 0.0039 0.4767
334 0.9935 0.8945 0.9972 0.0527 0.1362 0.5305 0.0836 0.0487 0.0027 0.4126
226 0.9958 0.8927 0.9970 0.0455 0.1583 0.4552 0.0588 0.0480 0.0028 0.3610
451 0.9882 0.8349 0.9967 0.0652 0.1225 0.7300 0.0901 0.0767 0.0004 0.5658
343 0.9928 0.8574 0.9960 0.0617 0.1371 0.6586 0.0762 0.0900 0.0000 0.3695
235 0.9953 0.8544 0.9978 0.0534 0.1904 0.6374 0.0669 0.0407 0.0000 0.5080
352 0.9901 0.8559 0.9969 0.0543 0.1472 0.8426 0.0792 0.1194 0.0000 0.4118
361 0.9909 0.8768 0.9975 0.0718 0.1747 0.8592 0.0783 0.1282 0.0000 0.4786
244 0.9931 0.9273 0.9980 0.0776 0.1670 0.7216 0.0907 0.1214 0.0005 0.4299
253 0.9917 0.8848 0.9979 0.0653 0.1726 0.7951 0.0949 0.0952 0.0120 0.6134
262 0.9886 0.8791 0.9981 0.0887 0.1652 0.8564 0.1088 0.1366 0.0000 0.5057
mean 0.9924 0.8630 0.9963 0.0488 0.1512 0.5684 0.0642 0.0705 0.0016 0.3724
Table 26. OSNNet-TR-KUc-threshold-table_image.png
Results with threshold on softmax. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUc KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9799 0.5270 0.9988 0.8309 0.9060 0.6211 0.7850 0.8412 0.9791 0.9761
523 0.9835 0.5216 0.9979 0.7506 0.8388 0.6703 0.7330 0.8360 0.9529 0.9821
631 0.9752 0.4470 0.9992 0.8713 0.9161 0.7497 0.8283 0.8068 0.9870 0.9854
424 0.9862 0.6334 0.9985 0.6918 0.8047 0.6036 0.6716 0.7498 0.9263 0.9644
532 0.9810 0.5133 0.9987 0.7979 0.8661 0.7816 0.7564 0.8315 0.9744 0.9838
325 0.9883 0.6840 0.9992 0.6052 0.7713 0.6113 0.6265 0.6592 0.8151 0.9506
433 0.9799 0.5626 0.9990 0.7563 0.8304 0.7072 0.7519 0.7773 0.9130 0.9438
541 0.9742 0.4892 0.9989 0.8428 0.9003 0.8026 0.8200 0.8800 0.9508 0.9766
442 0.9760 0.5138 0.9987 0.7905 0.8966 0.8406 0.7767 0.8691 0.8900 0.9916
334 0.9852 0.6594 0.9989 0.6919 0.7676 0.7409 0.7001 0.7277 0.8551 0.8946
226 0.9902 0.7379 0.9989 0.4670 0.5664 0.6292 0.4870 0.4858 0.6301 0.8925
451 0.9694 0.5391 0.9991 0.8068 0.8712 0.8978 0.8071 0.8382 0.9111 0.9860
343 0.9815 0.5882 0.9987 0.7078 0.7985 0.8419 0.7378 0.7056 0.6983 0.9136
235 0.9871 0.6697 0.9991 0.5040 0.6498 0.7736 0.4999 0.5043 0.4757 0.8900
352 0.9740 0.5463 0.9992 0.7356 0.8174 0.9492 0.7305 0.7687 0.9068 0.9523
361 0.9749 0.5846 0.9992 0.7525 0.8349 0.9486 0.7312 0.6680 0.7644 0.9360
244 0.9843 0.8291 0.9993 0.5342 0.6638 0.8464 0.5161 0.6228 0.6845 0.9358
253 0.9807 0.6810 0.9993 0.5175 0.6409 0.8868 0.5699 0.5679 0.6842 0.8935
262 0.9744 0.7066 0.9997 0.5861 0.6446 0.9373 0.6194 0.6267 0.5423 0.9695
mean 0.9803 0.6018 0.9990 0.6969 0.7887 0.7810 0.6920 0.7245 0.8180 0.9483
Table 27. OSNNet-TR-KUc-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUc KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.8066 0.2145 0.9391 0.9601 0.9764 0.5843 0.9274 0.9495 0.9989 0.9915
523 0.8561 0.3026 0.8126 0.8699 0.8336 0.6173 0.8396 0.9010 0.9555 0.9742
631 0.7388 0.1529 0.9607 0.9671 0.9854 0.5963 0.9470 0.9843 1.0000 0.9976
424 0.9122 0.4688 0.7494 0.8064 0.8275 0.4663 0.7706 0.8425 0.9787 0.9748
532 0.8697 0.2795 0.8246 0.9187 0.9551 0.6651 0.8898 0.9237 0.9934 0.9951
325 0.9092 0.6155 0.7469 0.6280 0.6828 0.5041 0.6349 0.7170 0.8457 0.7831
433 0.8786 0.3849 0.7206 0.8495 0.8676 0.5506 0.8368 0.8707 0.9605 0.9662
541 0.8649 0.2927 0.8165 0.8940 0.9324 0.5910 0.8875 0.8783 0.9302 0.9693
442 0.8846 0.3844 0.7495 0.7754 0.7699 0.6363 0.7687 0.8198 0.9161 0.8341
334 0.9440 0.5840 0.9816 0.5897 0.6298 0.6103 0.6130 0.6011 0.5613 0.7091
226 0.8876 0.5450 0.9993 0.5696 0.5932 0.7431 0.5579 0.5818 0.5475 0.9052
451 0.8223 0.2808 0.8346 0.9232 0.9192 0.5932 0.9059 0.8629 0.9986 0.9744
343 0.8739 0.4632 0.8409 0.7308 0.7878 0.6856 0.7385 0.7883 0.7847 0.8566
235 0.9685 0.7218 0.8667 0.3396 0.4459 0.6850 0.3327 0.3465 0.2740 0.7254
352 0.9689 0.4580 0.7796 0.7115 0.7619 0.7042 0.7152 0.7509 0.9129 0.7796
361 0.8955 0.3971 0.8407 0.7502 0.8159 0.7724 0.7375 0.6150 0.8021 0.8479
244 0.9621 0.7159 0.7677 0.4417 0.5460 0.6642 0.4271 0.5080 0.5141 0.8340
253 0.9282 0.6162 0.7835 0.4330 0.4834 0.6830 0.4530 0.4233 0.4961 0.5221
262 0.9723 0.7770 0.7540 0.4114 0.4262 0.6593 0.4069 0.4533 0.4199 0.5877
mean 0.8918 0.4555 0.8299 0.7142 0.7495 0.6322 0.7047 0.7273 0.7837 0.8541
Table 28. OSNNet-TR-KUc-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUc KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9622 0.2313 0.7846 0.9486 0.9755 0.5445 0.9117 0.9588 0.9998 0.9971
523 0.9837 0.3002 0.7914 0.8989 0.8564 0.5648 0.8673 0.9451 0.9980 0.9382
631 0.9731 0.2045 0.7959 0.9586 0.9837 0.5187 0.9203 0.9770 1.0000 0.9943
424 0.9882 0.5098 0.7451 0.7692 0.8584 0.4349 0.7446 0.8282 0.9569 0.9302
532 0.9646 0.3265 0.7528 0.9158 0.9600 0.6285 0.8761 0.9314 0.9954 0.9677
325 0.9862 0.6339 0.7805 0.5586 0.6848 0.4036 0.5928 0.6046 0.7932 0.8527
433 0.9861 0.4313 0.6307 0.8467 0.8741 0.4756 0.8307 0.8556 0.9883 0.9728
541 0.9621 0.2671 0.7490 0.9336 0.9609 0.5065 0.9078 0.9058 0.9307 0.9842
442 0.9830 0.3480 0.6691 0.8493 0.8480 0.6167 0.8256 0.8808 0.9874 0.9381
334 0.9889 0.6503 0.7397 0.5552 0.6013 0.3878 0.5813 0.6128 0.7549 0.7493
226 0.8876 0.5450 0.9993 0.5696 0.5932 0.7431 0.5579 0.5818 0.5475 0.9052
451 0.9849 0.3888 0.7548 0.8505 0.8953 0.6484 0.8213 0.8567 0.9736 0.9575
343 0.9908 0.6047 0.7458 0.6203 0.6900 0.4788 0.6313 0.6371 0.7311 0.7869
235 0.9685 0.7218 0.8667 0.3396 0.4459 0.6850 0.3327 0.3465 0.2740 0.7254
352 0.9902 0.5679 0.6856 0.6086 0.6651 0.5630 0.6052 0.6215 0.7822 0.5097
361 0.9886 0.5598 0.6605 0.6580 0.7092 0.5828 0.6400 0.5528 0.7608 0.7897
244 0.9621 0.7159 0.7677 0.4417 0.5460 0.6642 0.4271 0.5080 0.5141 0.8340
253 0.9282 0.6162 0.7835 0.4330 0.4834 0.6830 0.4530 0.4233 0.4961 0.5221
262 0.9723 0.7770 0.7540 0.4114 0.4262 0.6593 0.4069 0.4533 0.4199 0.5877
mean 0.9711 0.4947 0.7609 0.6930 0.7399 0.5678 0.6807 0.7095 0.7844 0.8391
Table 29. OSNNet-TR-KUc-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUc KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9735 0.5736 0.9857 0.8359 0.8742 0.7121 0.7928 0.8433 0.9333 0.8762
523 0.9827 0.6066 0.9878 0.7197 0.7940 0.6733 0.7075 0.7727 0.8849 0.9355
631 0.9722 0.5317 0.9885 0.8394 0.9054 0.7252 0.8043 0.8253 0.9315 0.9461
424 0.9885 0.6988 0.9914 0.6885 0.7702 0.5813 0.6661 0.7402 0.8425 0.9122
532 0.9821 0.6128 0.9844 0.7663 0.8586 0.7242 0.7331 0.8162 0.8970 0.9197
325 0.9942 0.7516 0.9858 0.5212 0.6288 0.4838 0.5501 0.5639 0.6962 0.8134
433 0.9864 0.6044 0.9829 0.7354 0.7815 0.6249 0.7269 0.7503 0.8506 0.8456
541 0.9822 0.5754 0.9797 0.8027 0.8562 0.7185 0.7806 0.7572 0.8777 0.9176
442 0.9869 0.6378 0.9776 0.7155 0.7843 0.6504 0.6907 0.8111 0.8320 0.9112
334 0.9933 0.7553 0.9857 0.5718 0.6075 0.5523 0.5855 0.5903 0.7014 0.7735
226 0.9958 0.8018 0.9882 0.3829 0.4823 0.4669 0.4109 0.4035 0.5183 0.7799
451 0.9858 0.6030 0.9798 0.7401 0.8083 0.6943 0.7512 0.7430 0.8617 0.9434
343 0.9933 0.7283 0.9796 0.5791 0.6694 0.6386 0.6193 0.5995 0.6586 0.7784
235 0.9958 0.7728 0.9868 0.3673 0.4805 0.5405 0.3648 0.3616 0.3695 0.7041
352 0.9932 0.6998 0.9771 0.6027 0.6544 0.7193 0.6053 0.6371 0.7781 0.7428
361 0.9924 0.7054 0.9713 0.5797 0.6716 0.7815 0.5710 0.5050 0.6941 0.8217
244 0.9962 0.8782 0.9881 0.3962 0.4650 0.6524 0.3847 0.4470 0.5016 0.6612
253 0.9967 0.8027 0.9695 0.3521 0.4509 0.5931 0.4003 0.3893 0.5001 0.6794
262 0.9957 0.8294 0.9475 0.3658 0.4061 0.6461 0.3895 0.3745 0.3704 0.6849
mean 0.9888 0.6931 0.9809 0.6085 0.6815 0.6410 0.6071 0.6280 0.7210 0.8235
Table 30. OSNNet-TR-KUc-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUc KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9918 0.8113 0.6580 0.3417 0.3610 0.2527 0.3127 0.3437 0.4407 0.3276
523 0.9932 0.7930 0.6960 0.2986 0.3405 0.2591 0.3008 0.3362 0.4084 0.4544
631 0.9932 0.8137 0.4867 0.2345 0.2583 0.1729 0.2110 0.1983 0.3046 0.2974
424 0.9919 0.7602 0.9636 0.5296 0.5958 0.4359 0.5113 0.5697 0.6737 0.7591
532 0.9935 0.8196 0.5656 0.2817 0.3509 0.2585 0.2710 0.3181 0.3640 0.4496
325 0.9945 0.7591 0.9233 0.5062 0.6205 0.4612 0.5305 0.5480 0.6721 0.7758
433 0.9917 0.7093 0.8392 0.5258 0.5868 0.4539 0.5153 0.5556 0.6251 0.6522
541 0.9939 0.7989 0.5596 0.3156 0.3443 0.2704 0.2905 0.2617 0.3506 0.4116
442 0.9927 0.7587 0.8374 0.4738 0.5325 0.4319 0.4574 0.5659 0.5864 0.6831
334 0.9943 0.7901 0.9582 0.4747 0.5137 0.4699 0.4909 0.4822 0.5867 0.6997
226 0.9958 0.8018 0.9882 0.3829 0.4823 0.4669 0.4109 0.4035 0.5183 0.7799
451 0.9924 0.7537 0.7692 0.4492 0.5008 0.3970 0.4502 0.4795 0.5439 0.6664
343 0.9950 0.7649 0.8829 0.4764 0.5505 0.5242 0.5048 0.4746 0.5313 0.6192
235 0.9958 0.7728 0.9868 0.3673 0.4805 0.5405 0.3648 0.3616 0.3695 0.7041
352 0.9950 0.7611 0.8273 0.4480 0.4988 0.5560 0.4481 0.5045 0.5822 0.5525
361 0.9934 0.7337 0.9498 0.5264 0.6093 0.7346 0.5189 0.4650 0.6209 0.7775
244 0.9962 0.8782 0.9881 0.3962 0.4650 0.6524 0.3847 0.4470 0.5016 0.6612
253 0.9967 0.8027 0.9695 0.3521 0.4509 0.5931 0.4003 0.3893 0.5001 0.6794
262 0.9957 0.8294 0.9475 0.3658 0.4061 0.6461 0.3895 0.3745 0.3704 0.6849
mean 0.9940 0.7848 0.8314 0.4077 0.4710 0.4514 0.4086 0.4252 0.5027 0.6124

Table 31. OSNNet-TR-KUcn-normal-table_image.png
First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUcn KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9934 0.8304 0.9945 0.3439 0.8605 0.1488 0.3200 0.2209 0.3650 0.2724
523 0.9932 0.7904 0.9949 0.3266 0.9336 0.3416 0.3308 0.3870 0.5529 0.6059
631 0.9920 0.7790 0.9951 0.4926 0.8930 0.4300 0.4709 0.5907 0.5352 0.5546
424 0.9951 0.8114 0.9953 0.4432 0.9005 0.3033 0.4309 0.3902 0.6351 0.5494
532 0.9926 0.7906 0.9952 0.4604 0.8311 0.5132 0.4216 0.4568 0.6941 0.6415
325 0.9955 0.7847 0.9967 0.4076 0.9322 0.4226 0.4524 0.4538 0.4971 0.6703
433 0.9928 0.7793 0.9962 0.5477 0.9161 0.4844 0.5396 0.5472 0.6084 0.6046
541 0.9902 0.7398 0.9959 0.6168 0.9407 0.5535 0.6001 0.5863 0.6718 0.5940
442 0.9912 0.7537 0.9951 0.6919 0.9572 0.5893 0.6478 0.7776 0.9108 0.8865
334 0.9943 0.7636 0.9971 0.6878 0.9454 0.5447 0.7016 0.6908 0.7799 0.7172
226 0.9954 0.7259 0.9985 0.6515 0.9274 0.4696 0.6470 0.6147 0.6554 0.7441
451 0.9908 0.7277 0.9963 0.7379 0.9442 0.6861 0.7754 0.7340 0.7895 0.9336
343 0.9923 0.7028 0.9966 0.7543 0.9781 0.6781 0.7679 0.7985 0.8322 0.7294
235 0.9953 0.7349 0.9983 0.7242 0.9587 0.6558 0.7420 0.6896 0.8659 0.8490
352 0.9894 0.7128 0.9965 0.6950 0.9500 0.8215 0.7119 0.7409 0.7555 0.7519
361 0.9894 0.7343 0.9974 0.7402 0.9784 0.8578 0.7389 0.7249 0.9508 0.8227
244 0.9936 0.7932 0.9977 0.7744 0.9599 0.7098 0.7537 0.7800 0.9559 0.8653
253 0.9899 0.7024 0.9985 0.8055 0.9736 0.8351 0.8250 0.8561 0.9383 0.9300
262 0.9895 0.6801 0.9984 0.9074 0.9767 0.8736 0.8918 0.8538 0.9622 0.8442
mean 0.9924 0.7546 0.9965 0.6215 0.9346 0.5747 0.6194 0.6260 0.7345 0.7140
Table 32. OSNNet-TR-KUcn-threshold-table_image.png
Results with threshold on softmax. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUcn KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9792 0.4951 0.9988 0.8916 0.9889 0.6185 0.8418 0.8747 0.9925 0.9888
523 0.9826 0.4382 0.9983 0.8605 0.9980 0.6919 0.8338 0.9258 0.9946 0.9974
631 0.9754 0.3823 0.9990 0.9437 0.9906 0.7267 0.9082 0.9593 0.9986 0.9967
424 0.9858 0.5518 0.9985 0.8557 0.9916 0.5986 0.8283 0.8846 0.9913 0.9841
532 0.9798 0.4550 0.9985 0.9121 0.9927 0.8232 0.8490 0.9363 0.9982 0.9983
325 0.9885 0.5477 0.9990 0.8053 0.9893 0.6370 0.8271 0.8375 0.9663 0.9893
433 0.9805 0.4859 0.9988 0.9109 0.9933 0.7262 0.8795 0.9250 0.9936 0.9861
541 0.9728 0.3887 0.9986 0.9479 0.9975 0.8038 0.9248 0.9331 0.9993 0.9962
442 0.9763 0.3916 0.9987 0.9514 0.9983 0.8433 0.9005 0.9764 0.9999 0.9993
334 0.9854 0.5039 0.9988 0.9243 0.9931 0.7451 0.9143 0.9476 0.9975 0.9896
226 0.9891 0.5783 0.9995 0.8713 0.9847 0.6424 0.8634 0.8489 0.9712 0.9748
451 0.9730 0.3810 0.9990 0.9603 0.9961 0.8901 0.9498 0.9471 0.9992 0.9993
343 0.9796 0.3886 0.9989 0.9538 0.9991 0.8435 0.9540 0.9667 0.9989 0.9870
235 0.9868 0.4831 0.9993 0.9186 0.9969 0.7823 0.9228 0.9029 0.9966 0.9891
352 0.9737 0.4182 0.9991 0.9364 1.0000 0.9401 0.9169 0.9678 0.9976 0.9931
361 0.9734 0.3751 0.9995 0.9495 0.9987 0.9495 0.9318 0.9547 0.9999 0.9983
244 0.9852 0.5978 0.9992 0.9298 0.9947 0.8302 0.8936 0.9462 0.9992 0.9951
253 0.9763 0.3835 0.9994 0.9598 0.9984 0.9148 0.9521 0.9757 0.9995 0.9966
262 0.9750 0.4043 0.9995 0.9817 0.9984 0.9451 0.9732 0.9510 0.9990 0.9828
mean 0.9799 0.4553 0.9990 0.9192 0.9948 0.7870 0.8982 0.9295 0.9944 0.9917
Table 33. OSNNet-TR-KUcn-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUcn KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.7984 0.2481 0.9188 0.9258 0.9902 0.6041 0.9040 0.9077 0.9882 0.9915
523 0.8745 0.3547 0.7599 0.8967 0.9548 0.5947 0.8495 0.9444 0.9960 0.9739
631 0.7420 0.2325 0.8792 0.9562 0.9870 0.5454 0.9336 0.9852 0.9995 0.9988
424 0.8760 0.4337 0.7875 0.8111 0.8663 0.5836 0.7978 0.8800 0.9250 0.9491
532 0.8780 0.3286 0.7645 0.9075 0.9772 0.7093 0.8648 0.8886 0.9980 0.9716
325 0.9784 0.6275 0.7068 0.6521 0.8187 0.4273 0.6826 0.6936 0.7451 0.8520
433 0.8823 0.4462 0.8389 0.8466 0.9124 0.5325 0.8278 0.8931 0.9687 0.9479
541 0.8711 0.4623 0.8804 0.8307 0.9391 0.6176 0.8077 0.8454 0.9644 0.9345
442 0.8906 0.4679 0.8956 0.8089 0.9174 0.5800 0.7898 0.8956 0.8925 0.8600
334 0.9323 0.5302 0.8369 0.7363 0.8528 0.6558 0.7309 0.7637 0.7943 0.8347
226 0.9676 0.5126 0.6779 0.6752 0.6971 0.4298 0.6409 0.6316 0.7207 0.6731
451 0.9380 0.4947 0.7711 0.8443 0.9439 0.6830 0.8640 0.8272 0.9214 0.9722
343 0.9210 0.5601 0.8463 0.7732 0.9298 0.6604 0.7611 0.8167 0.9288 0.8654
235 0.9659 0.5435 0.7786 0.6970 0.7641 0.6355 0.6899 0.6719 0.8322 0.7777
352 0.8348 0.5095 0.9188 0.8083 0.9027 0.7577 0.7826 0.8139 0.8393 0.7229
361 0.9796 0.5772 0.8477 0.7734 0.8418 0.6604 0.7549 0.6655 0.8650 0.7675
244 0.8239 0.5227 0.8731 0.7315 0.8008 0.7089 0.7211 0.7453 0.8006 0.8392
253 0.9716 0.6123 0.8164 0.6316 0.7103 0.6953 0.6404 0.6718 0.5916 0.5758
262 0.9517 0.5861 0.7788 0.7154 0.7450 0.7459 0.7331 0.7606 0.7439 0.7150
mean 0.8988 0.4763 0.8198 0.7906 0.8711 0.6225 0.7777 0.8054 0.8692 0.8538
Table 34. OSNNet-TR-KUcn-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUcn KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9348 0.2534 0.9111 0.9532 0.9776 0.5398 0.9155 0.9481 0.9985 0.9961
523 0.9439 0.3451 0.7005 0.9028 0.9234 0.5286 0.8544 0.9558 0.9973 0.9741
631 0.9614 0.2320 0.7979 0.9622 0.9875 0.5190 0.9263 0.9408 1.0000 0.9991
424 0.9855 0.5081 0.8003 0.7971 0.9371 0.4647 0.7733 0.8543 0.9129 0.9291
532 0.9828 0.3322 0.6751 0.8952 0.9632 0.6353 0.8523 0.8923 0.9964 0.9696
325 0.9927 0.6573 0.6992 0.6536 0.8294 0.4057 0.6610 0.7066 0.8335 0.8362
433 0.9852 0.4867 0.6010 0.8706 0.9176 0.4474 0.8324 0.8480 0.9919 0.9832
541 0.9831 0.3698 0.7293 0.8811 0.9103 0.5073 0.8612 0.8477 0.9889 0.9763
442 0.9621 0.4440 0.8187 0.8678 0.9195 0.5438 0.8151 0.8811 0.9627 0.9318
334 0.9886 0.6694 0.7040 0.6999 0.8408 0.4989 0.7026 0.7306 0.8707 0.8243
226 0.9676 0.5126 0.6779 0.6752 0.6971 0.4298 0.6409 0.6316 0.7207 0.6731
451 0.9857 0.4312 0.7535 0.8912 0.9486 0.6388 0.8947 0.8194 0.9846 0.9826
343 0.9918 0.6161 0.8002 0.7364 0.9142 0.5077 0.7398 0.7417 0.8920 0.8663
235 0.9659 0.5435 0.7786 0.6970 0.7641 0.6355 0.6899 0.6719 0.8322 0.7777
352 0.9915 0.6888 0.7539 0.6688 0.8386 0.5561 0.6674 0.6963 0.8357 0.7360
361 0.9900 0.6680 0.6691 0.7190 0.8147 0.5751 0.6919 0.6647 0.8508 0.8134
244 0.8239 0.5227 0.8731 0.7315 0.8008 0.7089 0.7211 0.7453 0.8006 0.8392
253 0.9716 0.6123 0.8164 0.6316 0.7103 0.6953 0.6404 0.6718 0.5916 0.5758
262 0.9517 0.5861 0.7788 0.7154 0.7450 0.7459 0.7331 0.7606 0.7439 0.7150
mean 0.9663 0.4989 0.7547 0.7868 0.8652 0.5570 0.7691 0.7899 0.8845 0.8631
Table 35. OSNNet-TR-KUcn-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUcn KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9734 0.5917 0.9858 0.8405 0.9610 0.7222 0.7980 0.8437 0.9394 0.9139
523 0.9822 0.5955 0.9855 0.7834 0.9608 0.6961 0.7562 0.8696 0.9169 0.9555
631 0.9717 0.5358 0.9851 0.8773 0.9649 0.7176 0.8445 0.9037 0.9726 0.9444
424 0.9879 0.6831 0.9905 0.7597 0.9554 0.5841 0.7346 0.8085 0.9052 0.9072
532 0.9822 0.6119 0.9814 0.8015 0.9491 0.7391 0.7473 0.8271 0.9503 0.9432
325 0.9934 0.7063 0.9888 0.6731 0.9314 0.5195 0.7099 0.7194 0.8289 0.8748
433 0.9869 0.6443 0.9841 0.8191 0.9451 0.6283 0.7940 0.8406 0.9085 0.9096
541 0.9818 0.5712 0.9786 0.8249 0.9407 0.6880 0.8162 0.8286 0.8868 0.8937
442 0.9872 0.6462 0.9809 0.8183 0.9614 0.6477 0.7658 0.8525 0.9360 0.9286
334 0.9925 0.7241 0.9813 0.7528 0.9273 0.5552 0.7630 0.7910 0.9069 0.8703
226 0.9953 0.7175 0.9911 0.7321 0.9006 0.4782 0.7098 0.7064 0.8105 0.8766
451 0.9873 0.6687 0.9817 0.8124 0.9395 0.7324 0.8245 0.7701 0.8638 0.9235
343 0.9932 0.6820 0.9773 0.7896 0.9625 0.6440 0.8018 0.8201 0.8370 0.8222
235 0.9959 0.7104 0.9865 0.7468 0.9305 0.5563 0.7412 0.7048 0.8954 0.8458
352 0.9921 0.6927 0.9795 0.7599 0.9200 0.7290 0.7590 0.7720 0.9035 0.8112
361 0.9931 0.7214 0.9632 0.7213 0.8962 0.7501 0.7146 0.5756 0.9140 0.8111
244 0.9968 0.8537 0.9874 0.7128 0.9296 0.6282 0.6954 0.7557 0.8422 0.7618
253 0.9969 0.7715 0.9420 0.5841 0.7824 0.6085 0.6295 0.6951 0.7280 0.7487
262 0.9954 0.8259 0.9802 0.7615 0.8440 0.6929 0.7639 0.7161 0.8530 0.7828
mean 0.9887 0.6818 0.9806 0.7669 0.9264 0.6483 0.7563 0.7790 0.8842 0.8697
Table 36. OSNNet-TR-KUcn-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUcn KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9921 0.8151 0.6480 0.3467 0.4965 0.2345 0.3167 0.3333 0.4314 0.3770
523 0.9923 0.7745 0.7509 0.4246 0.6591 0.3562 0.4087 0.5095 0.5505 0.6289
631 0.9926 0.7986 0.5680 0.3545 0.5039 0.2026 0.3259 0.3827 0.4382 0.3881
424 0.9929 0.7855 0.8982 0.5428 0.7879 0.3753 0.5165 0.5779 0.6535 0.6739
532 0.9938 0.8067 0.5696 0.3243 0.5007 0.2817 0.3126 0.3910 0.4724 0.4579
325 0.9953 0.7693 0.9577 0.5476 0.8314 0.4296 0.5806 0.6076 0.6870 0.7375
433 0.9913 0.7404 0.7943 0.6032 0.7543 0.4224 0.5755 0.6087 0.6591 0.6772
541 0.9932 0.7910 0.6133 0.3961 0.5048 0.2691 0.3703 0.3997 0.4078 0.4225
442 0.9926 0.7579 0.7892 0.5599 0.7189 0.4097 0.5239 0.6088 0.6554 0.6534
334 0.9935 0.7508 0.9496 0.7047 0.8875 0.5114 0.7146 0.7371 0.8606 0.8299
226 0.9953 0.7175 0.9911 0.7321 0.9006 0.4782 0.7098 0.7064 0.8105 0.8766
451 0.9935 0.7797 0.7280 0.4955 0.6235 0.4169 0.5237 0.4176 0.4833 0.6279
343 0.9944 0.7121 0.9600 0.7224 0.9110 0.5759 0.7331 0.7589 0.7882 0.7354
235 0.9959 0.7104 0.9865 0.7468 0.9305 0.5563 0.7412 0.7048 0.8954 0.8458
352 0.9941 0.7578 0.9106 0.6355 0.8148 0.5778 0.6224 0.6399 0.7669 0.6366
361 0.9946 0.7626 0.8487 0.6156 0.7872 0.6200 0.6173 0.4759 0.7810 0.7130
244 0.9968 0.8537 0.9874 0.7128 0.9296 0.6282 0.6954 0.7557 0.8422 0.7618
253 0.9969 0.7715 0.9420 0.5841 0.7824 0.6085 0.6295 0.6951 0.7280 0.7487
262 0.9954 0.8259 0.9802 0.7615 0.8440 0.6929 0.7639 0.7161 0.8530 0.7828
mean 0.9940 0.7727 0.8354 0.5690 0.7457 0.4551 0.5622 0.5804 0.6718 0.6618

Table 37. OSNNet-TR-KUcl-normal-table_image.png
First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUcl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9928 0.4893 0.9954 0.9955 0.9889 0.2187 0.9138 0.9053 1.0000 0.9459
523 0.9933 0.4275 0.9942 0.9956 0.9891 0.3479 0.9222 0.9527 1.0000 0.9911
631 0.9918 0.4952 0.9948 0.9923 0.9755 0.4325 0.9084 0.9513 1.0000 0.9632
424 0.9946 0.5443 0.9953 0.9938 0.9860 0.3206 0.9063 0.9426 0.9999 0.9488
532 0.9924 0.4603 0.9948 0.9953 0.9957 0.5219 0.9251 0.9517 1.0000 0.9403
325 0.9947 0.4690 0.9965 0.9966 0.9862 0.4286 0.9584 0.9778 1.0000 0.9950
433 0.9933 0.4829 0.9965 0.9965 0.9925 0.4951 0.9444 0.9737 1.0000 0.9746
541 0.9920 0.4695 0.9941 0.9936 0.9773 0.5162 0.9284 0.9571 0.9988 0.9580
442 0.9929 0.4688 0.9941 0.9957 0.9880 0.5805 0.9419 0.9751 1.0000 0.9953
334 0.9941 0.5201 0.9973 0.9971 0.9932 0.5528 0.9577 0.9567 0.9999 0.9640
226 0.9961 0.5036 0.9978 0.9970 0.9974 0.4784 0.9691 0.9665 1.0000 0.9838
451 0.9894 0.5172 0.9964 0.9951 0.9858 0.6970 0.9515 0.9626 1.0000 0.9911
343 0.9915 0.5068 0.9966 0.9965 0.9892 0.6762 0.9592 0.9845 0.9998 0.9946
235 0.9945 0.4113 0.9982 0.9984 0.9959 0.6861 0.9807 0.9779 1.0000 0.9979
352 0.9909 0.4220 0.9967 0.9978 0.9955 0.8260 0.9694 0.9483 1.0000 0.9855
361 0.9904 0.4489 0.9972 0.9953 0.9972 0.8686 0.9569 0.9633 1.0000 0.9915
244 0.9937 0.4763 0.9980 0.9983 0.9982 0.7371 0.9590 0.9838 1.0000 0.9926
253 0.9918 0.4583 0.9979 0.9972 0.9853 0.8204 0.9717 0.9924 0.9998 0.9812
262 0.9899 0.5411 0.9976 0.9977 0.9924 0.8500 0.9784 0.9623 1.0000 0.9879
mean 0.9926 0.4796 0.9963 0.9961 0.9900 0.5818 0.9475 0.9624 0.9999 0.9780
Table 38. OSNNet-TR-KUcl-threshold-table_image.png
Results with threshold on softmax. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUcl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9786 0.2279 0.9993 0.9995 0.9990 0.6614 0.9815 0.9948 1.0000 1.0000
523 0.9824 0.1770 0.9982 0.9995 0.9980 0.7060 0.9833 0.9975 1.0000 1.0000
631 0.9756 0.2152 0.9990 0.9996 0.9988 0.7533 0.9868 1.0000 1.0000 0.9999
424 0.9857 0.2705 0.9990 0.9990 0.9958 0.6204 0.9750 0.9944 1.0000 0.9998
532 0.9775 0.1894 0.9988 0.9996 1.0000 0.8184 0.9861 0.9939 1.0000 0.9999
325 0.9880 0.2357 0.9989 0.9996 1.0000 0.6426 0.9895 0.9949 1.0000 1.0000
433 0.9817 0.2547 0.9990 0.9997 0.9981 0.7320 0.9870 0.9967 1.0000 0.9999
541 0.9763 0.2169 0.9986 0.9998 0.9985 0.7941 0.9902 0.9958 1.0000 0.9999
442 0.9775 0.1710 0.9984 0.9995 0.9974 0.8407 0.9925 0.9976 1.0000 1.0000
334 0.9858 0.2816 0.9988 0.9997 1.0000 0.7580 0.9887 0.9921 1.0000 0.9994
226 0.9915 0.2964 0.9993 0.9996 1.0000 0.6524 0.9903 0.9928 1.0000 0.9993
451 0.9713 0.2475 0.9991 0.9997 0.9977 0.8921 0.9931 1.0000 1.0000 1.0000
343 0.9790 0.2389 0.9990 0.9997 0.9992 0.8552 0.9931 0.9990 1.0000 1.0000
235 0.9866 0.2431 0.9993 0.9998 1.0000 0.7998 0.9923 0.9953 1.0000 1.0000
352 0.9767 0.1640 0.9989 0.9998 1.0000 0.9390 0.9962 0.9979 1.0000 0.9998
361 0.9754 0.1660 0.9992 0.9997 1.0000 0.9585 0.9922 1.0000 1.0000 1.0000
244 0.9848 0.2858 0.9992 0.9997 0.9992 0.8489 0.9882 0.9944 1.0000 1.0000
253 0.9808 0.2161 0.9993 0.9996 0.9977 0.9058 0.9926 0.9988 1.0000 0.9998
262 0.9757 0.3111 0.9994 0.9999 0.9995 0.9304 0.9931 0.9915 1.0000 1.0000
mean 0.9806 0.2320 0.9990 0.9996 0.9989 0.7952 0.9890 0.9962 1.0000 0.9999
Table 39. OSNNet-TR-KUcl-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUcl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.7619 0.1632 0.9776 0.9928 0.9974 0.6633 0.9646 0.9862 1.0000 1.0000
523 0.7972 0.2288 0.8384 0.9756 0.9175 0.5474 0.9425 0.9776 0.9980 0.9974
631 0.7662 0.1769 0.9514 0.9896 0.9838 0.6598 0.9695 0.9777 1.0000 0.9998
424 0.9295 0.3937 0.7835 0.9159 0.8787 0.4882 0.8629 0.9196 0.9425 0.9394
532 0.9012 0.2809 0.8038 0.9722 0.9773 0.6299 0.9298 0.9317 0.9988 0.9990
325 0.8494 0.3778 0.8051 0.9105 0.8401 0.4842 0.8377 0.8748 0.9418 0.9228
433 0.9523 0.3771 0.8861 0.9700 0.9436 0.5935 0.9334 0.9434 0.9996 0.9839
541 0.7905 0.3229 0.9271 0.9347 0.8990 0.7425 0.9088 0.9022 0.9926 0.9903
442 0.9327 0.4289 0.8771 0.9606 0.9531 0.6495 0.8865 0.9369 0.9970 0.9732
334 0.9728 0.5208 0.7886 0.8156 0.7721 0.5779 0.7721 0.8008 0.9154 0.8589
226 0.9298 0.2747 0.7912 0.8194 0.7795 0.6069 0.7796 0.8401 0.9155 0.8813
451 0.8094 0.4161 0.9127 0.9516 0.9187 0.6896 0.9209 0.8820 0.9420 0.8829
343 0.8580 0.3694 0.8443 0.9743 0.9600 0.7034 0.9352 0.9547 0.9980 0.9648
235 0.9654 0.3391 0.7379 0.7766 0.7551 0.6977 0.7682 0.7854 0.7662 0.7855
352 0.9101 0.3764 0.8145 0.9098 0.8796 0.7386 0.8917 0.8590 0.9490 0.9546
361 0.9324 0.4344 0.8062 0.9746 0.9486 0.7124 0.9092 0.8734 0.9880 0.8797
244 0.9513 0.2868 0.7656 0.8310 0.8181 0.6705 0.8192 0.8030 0.8600 0.9128
253 0.9495 0.3062 0.8740 0.8869 0.8720 0.7796 0.8552 0.8714 0.8495 0.8122
262 0.9717 0.5842 0.7988 0.7568 0.7435 0.7229 0.7168 0.7124 0.7559 0.6741
mean 0.8911 0.3504 0.8413 0.9115 0.8862 0.6504 0.8739 0.8859 0.9374 0.9164
Table 40. OSNNet-TR-KUcl-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUcl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9624 0.2399 0.8043 0.9867 0.9954 0.5773 0.9497 0.9759 1.0000 0.9999
523 0.9832 0.2814 0.7525 0.9674 0.9125 0.5675 0.9128 0.9676 0.9988 0.9993
631 0.9582 0.1965 0.7822 0.9863 0.9759 0.5016 0.9614 0.9570 0.9999 0.9996
424 0.9877 0.4166 0.7478 0.8756 0.8292 0.4480 0.8375 0.8946 0.9384 0.9476
532 0.9815 0.2866 0.6684 0.9860 0.9861 0.5862 0.9320 0.9336 1.0000 0.9998
325 0.9916 0.5292 0.7090 0.8742 0.8157 0.3582 0.8126 0.8088 0.9400 0.9153
433 0.9666 0.3782 0.7999 0.9600 0.9130 0.4482 0.9195 0.9119 0.9977 0.9807
541 0.9629 0.2966 0.7471 0.9240 0.8911 0.5678 0.9042 0.8627 0.9925 0.9984
442 0.9868 0.4085 0.6808 0.9532 0.9253 0.5336 0.8831 0.9330 0.9969 0.9756
334 0.9900 0.5817 0.6877 0.7946 0.7890 0.4819 0.7466 0.7701 0.8654 0.8763
226 0.9298 0.2747 0.7912 0.8194 0.7795 0.6069 0.7796 0.8401 0.9155 0.8813
451 0.9821 0.4537 0.8115 0.9438 0.9359 0.6449 0.9079 0.8721 0.9939 0.9633
343 0.9871 0.4698 0.7769 0.9136 0.9050 0.4821 0.8836 0.8572 0.9948 0.8893
235 0.9654 0.3391 0.7379 0.7766 0.7551 0.6977 0.7682 0.7854 0.7662 0.7855
352 0.9872 0.5073 0.8145 0.9551 0.9119 0.6601 0.8896 0.8765 0.9323 0.9172
361 0.9889 0.5221 0.6012 0.9249 0.8892 0.5579 0.8458 0.8571 0.9707 0.8627
244 0.9513 0.2868 0.7656 0.8310 0.8181 0.6705 0.8192 0.8030 0.8600 0.9128
253 0.9495 0.3062 0.8740 0.8869 0.8720 0.7796 0.8552 0.8714 0.8495 0.8122
262 0.9717 0.5842 0.7988 0.7568 0.7435 0.7229 0.7168 0.7124 0.7559 0.6741
mean 0.9728 0.3873 0.7553 0.9009 0.8760 0.5733 0.8592 0.8679 0.9352 0.9153
Table 41. OSNNet-TR-KUcl-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUcl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9735 0.5616 0.9872 0.9555 0.9479 0.7156 0.8695 0.9158 0.9958 0.9547
523 0.9819 0.5584 0.9873 0.9617 0.9421 0.6869 0.8802 0.9407 0.9953 0.9313
631 0.9714 0.5311 0.9861 0.9575 0.9496 0.7326 0.8909 0.9482 0.9925 0.9522
424 0.9881 0.6257 0.9907 0.9727 0.9753 0.5724 0.8746 0.9259 0.9938 0.9244
532 0.9801 0.5709 0.9862 0.9582 0.9599 0.7480 0.8726 0.9482 0.9966 0.9558
325 0.9936 0.5972 0.9894 0.9767 0.9244 0.5160 0.9179 0.9199 0.9963 0.9660
433 0.9863 0.6144 0.9894 0.9702 0.9284 0.6375 0.9051 0.9216 0.9894 0.8980
541 0.9810 0.5613 0.9823 0.9482 0.9542 0.7101 0.8873 0.9324 0.9940 0.9225
442 0.9878 0.6370 0.9795 0.9646 0.9477 0.6580 0.8814 0.9339 0.9960 0.9595
334 0.9927 0.6760 0.9847 0.9644 0.9119 0.5615 0.9089 0.9045 0.9888 0.9092
226 0.9962 0.6247 0.9903 0.9885 0.9853 0.4747 0.9388 0.9474 0.9996 0.9496
451 0.9862 0.6182 0.9823 0.9634 0.9287 0.7220 0.9055 0.8917 0.9920 0.9608
343 0.9922 0.6402 0.9784 0.9686 0.9564 0.6474 0.9132 0.9334 0.9981 0.9521
235 0.9958 0.5825 0.9863 0.9860 0.9611 0.5639 0.9421 0.9240 0.9920 0.9412
352 0.9931 0.6358 0.9802 0.9419 0.9091 0.7389 0.8785 0.8725 0.9655 0.8755
361 0.9929 0.6720 0.9712 0.9435 0.9311 0.7850 0.8622 0.7884 0.9883 0.9017
244 0.9968 0.7797 0.9837 0.9661 0.9626 0.5955 0.8699 0.9219 0.9972 0.9304
253 0.9963 0.6680 0.9774 0.9237 0.9105 0.6148 0.8862 0.8905 0.9734 0.8802
262 0.9964 0.7697 0.9755 0.9501 0.9124 0.6497 0.8960 0.8312 0.9765 0.8250
mean 0.9885 0.6276 0.9836 0.9611 0.9420 0.6490 0.8937 0.9101 0.9906 0.9258
Table 42. OSNNet-TR-KUcl-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUcl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9922 0.8129 0.6329 0.4411 0.4726 0.2146 0.3642 0.3725 0.5996 0.4299
523 0.9920 0.7607 0.8047 0.7031 0.6145 0.3505 0.6276 0.6385 0.8430 0.6322
631 0.9931 0.8234 0.3460 0.2892 0.2591 0.1272 0.2519 0.3030 0.3497 0.2801
424 0.9911 0.7173 0.9786 0.9221 0.8999 0.4632 0.8076 0.8329 0.9217 0.8364
532 0.9928 0.7889 0.6458 0.5587 0.5932 0.3227 0.4878 0.5560 0.6336 0.5497
325 0.9945 0.6413 0.8997 0.8858 0.8251 0.4539 0.8262 0.8268 0.9011 0.8725
433 0.9926 0.7477 0.7484 0.6812 0.6544 0.3501 0.6115 0.6318 0.7040 0.5647
541 0.9934 0.7750 0.6985 0.5824 0.5376 0.2991 0.4986 0.4423 0.7437 0.4742
442 0.9917 0.7221 0.8979 0.8894 0.8243 0.4924 0.7857 0.8257 0.9724 0.8590
334 0.9941 0.7267 0.8652 0.8222 0.7694 0.4566 0.7578 0.7523 0.8541 0.7756
226 0.9962 0.6247 0.9903 0.9885 0.9853 0.4747 0.9388 0.9474 0.9996 0.9496
451 0.9934 0.7838 0.8160 0.7356 0.6698 0.4288 0.6741 0.6104 0.8083 0.7315
343 0.9935 0.6912 0.9368 0.9232 0.8747 0.5733 0.8573 0.8562 0.9834 0.9033
235 0.9958 0.5825 0.9863 0.9860 0.9611 0.5639 0.9421 0.9240 0.9920 0.9412
352 0.9943 0.6950 0.8653 0.8104 0.7574 0.5916 0.7377 0.7082 0.8542 0.7174
361 0.9932 0.6799 0.9651 0.9303 0.9185 0.7728 0.8466 0.7688 0.9877 0.8735
244 0.9968 0.7797 0.9837 0.9661 0.9626 0.5955 0.8699 0.9219 0.9972 0.9304
253 0.9963 0.6680 0.9774 0.9237 0.9105 0.6148 0.8862 0.8905 0.9734 0.8802
262 0.9964 0.7697 0.9755 0.9501 0.9124 0.6497 0.8960 0.8312 0.9765 0.8250
mean 0.9939 0.7258 0.8428 0.7889 0.7580 0.4629 0.7193 0.7179 0.8471 0.7382

Table 43. OSNNet-TR-KUcnl-normal-table_image.png
First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUcnl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9924 0.4947 0.9943 0.9961 0.9951 0.2194 0.9185 0.9088 1.0000 0.9606
523 0.9933 0.4284 0.9944 0.9948 0.9961 0.3556 0.9241 0.9703 1.0000 0.9888
631 0.9920 0.4734 0.9950 0.9954 0.9928 0.4700 0.9301 0.9719 1.0000 0.9772
424 0.9945 0.4945 0.9948 0.9962 0.9952 0.3286 0.9309 0.9548 0.9998 0.9626
532 0.9924 0.4516 0.9954 0.9948 0.9955 0.5283 0.9183 0.9552 1.0000 0.9702
325 0.9954 0.5132 0.9968 0.9960 0.9921 0.4278 0.9512 0.9765 0.9999 0.9914
433 0.9929 0.4707 0.9964 0.9973 0.9921 0.5090 0.9512 0.9664 1.0000 0.9648
541 0.9912 0.4406 0.9952 0.9956 0.9927 0.5282 0.9515 0.9806 1.0000 0.9873
442 0.9908 0.4197 0.9959 0.9964 0.9946 0.6209 0.9489 0.9711 1.0000 0.9908
334 0.9933 0.5002 0.9972 0.9977 0.9991 0.5304 0.9618 0.9771 1.0000 0.9865
226 0.9951 0.4906 0.9982 0.9963 0.9990 0.4880 0.9598 0.9628 0.9997 0.9955
451 0.9902 0.4453 0.9964 0.9979 0.9952 0.6835 0.9702 0.9867 0.9999 0.9968
343 0.9933 0.4631 0.9965 0.9970 0.9984 0.6576 0.9752 0.9856 1.0000 0.9959
235 0.9945 0.3758 0.9984 0.9989 0.9987 0.6842 0.9839 0.9821 1.0000 0.9977
352 0.9919 0.3926 0.9966 0.9978 0.9993 0.8271 0.9725 0.9751 1.0000 0.9774
361 0.9889 0.3990 0.9978 0.9966 0.9938 0.8635 0.9633 0.9632 1.0000 0.9963
244 0.9918 0.4739 0.9978 0.9983 0.9962 0.7384 0.9591 0.9728 0.9999 0.9861
253 0.9923 0.3809 0.9977 0.9981 0.9987 0.8179 0.9779 0.9945 1.0000 0.9976
262 0.9892 0.5179 0.9984 0.9989 0.9974 0.8675 0.9843 0.9911 0.9997 0.9913
mean 0.9924 0.4540 0.9965 0.9969 0.9959 0.5866 0.9544 0.9709 0.9999 0.9850
Table 44. OSNNet-TR-KUcnl-threshold-table_image.png
Results with threshold on softmax. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUcnl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9792 0.2252 0.9990 0.9996 1.0000 0.6414 0.9835 0.9926 1.0000 1.0000
523 0.9819 0.1774 0.9980 0.9993 1.0000 0.6979 0.9862 0.9962 1.0000 0.9999
631 0.9753 0.1865 0.9990 0.9997 1.0000 0.7655 0.9909 1.0000 1.0000 1.0000
424 0.9854 0.2394 0.9984 0.9996 1.0000 0.6052 0.9822 0.9914 1.0000 0.9997
532 0.9777 0.1823 0.9988 0.9996 1.0000 0.8243 0.9856 0.9983 1.0000 0.9999
325 0.9882 0.2788 0.9988 0.9993 0.9982 0.6280 0.9857 0.9936 1.0000 0.9999
433 0.9814 0.2378 0.9988 0.9997 0.9989 0.7444 0.9879 0.9980 1.0000 0.9997
541 0.9741 0.1924 0.9988 0.9997 0.9991 0.8011 0.9928 0.9938 1.0000 1.0000
442 0.9744 0.1653 0.9989 0.9996 0.9991 0.8730 0.9896 0.9986 1.0000 0.9998
334 0.9847 0.2585 0.9991 0.9999 1.0000 0.7404 0.9913 0.9933 1.0000 0.9999
226 0.9895 0.3120 0.9993 0.9994 1.0000 0.6557 0.9871 0.9857 1.0000 0.9999
451 0.9721 0.2148 0.9989 0.9997 0.9995 0.8834 0.9963 1.0000 1.0000 1.0000
343 0.9817 0.2187 0.9990 0.9996 1.0000 0.8427 0.9960 1.0000 1.0000 1.0000
235 0.9854 0.2277 0.9995 0.9999 1.0000 0.8073 0.9961 0.9971 1.0000 1.0000
352 0.9787 0.1477 0.9990 0.9997 1.0000 0.9471 0.9966 0.9965 1.0000 1.0000
361 0.9719 0.1567 0.9995 0.9996 0.9995 0.9461 0.9940 0.9960 1.0000 1.0000
244 0.9818 0.2755 0.9993 0.9996 1.0000 0.8554 0.9897 0.9951 1.0000 0.9994
253 0.9823 0.1812 0.9991 0.9998 0.9993 0.9028 0.9929 0.9988 1.0000 1.0000
262 0.9733 0.2951 0.9996 0.9999 0.9995 0.9408 0.9971 0.9974 1.0000 1.0000
mean 0.9799 0.2196 0.9990 0.9996 0.9996 0.7949 0.9906 0.9959 1.0000 0.9999
Table 45. OSNNet-TR-KUcnl-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUcnl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.8807 0.2462 0.8686 0.9885 0.9921 0.6167 0.9448 0.9706 1.0000 0.9974
523 0.8590 0.2569 0.8652 0.9845 0.9502 0.6227 0.9468 0.9662 1.0000 0.9937
631 0.8243 0.2246 0.9575 0.9911 0.9893 0.5463 0.9623 0.9838 0.9999 0.9985
424 0.8728 0.3445 0.8130 0.8837 0.9176 0.4962 0.8574 0.9011 0.9733 0.9941
532 0.9426 0.3382 0.8546 0.9756 0.9799 0.6309 0.9339 0.9386 0.9999 0.9879
325 0.9173 0.4628 0.8815 0.9271 0.9331 0.4791 0.8935 0.8864 0.9503 0.9178
433 0.8827 0.3294 0.7995 0.9726 0.9509 0.5920 0.9224 0.9594 0.9975 0.9868
541 0.8291 0.3286 0.9222 0.9761 0.9743 0.5764 0.9483 0.9277 0.9991 0.9777
442 0.9570 0.4125 0.8231 0.9565 0.9249 0.5486 0.9028 0.9495 0.9882 0.9712
334 0.9339 0.4420 0.8207 0.8550 0.8477 0.6086 0.8482 0.8351 0.8746 0.9410
226 0.9665 0.4193 0.6342 0.5958 0.5657 0.5101 0.5696 0.6089 0.5980 0.7286
451 0.8092 0.4052 0.9084 0.9578 0.9490 0.5856 0.9311 0.8759 0.9799 0.9870
343 0.8489 0.3994 0.9459 0.9676 0.9639 0.7357 0.9407 0.9564 0.9836 0.9184
235 0.9567 0.3236 0.6043 0.6302 0.6039 0.5222 0.5911 0.6165 0.7507 0.8327
352 0.8759 0.3229 0.9496 0.9630 0.9638 0.7920 0.9333 0.8766 0.9634 0.8718
361 0.9128 0.3768 0.7543 0.9019 0.8860 0.6618 0.8749 0.8248 0.8462 0.8530
244 0.9586 0.4895 0.9220 0.9109 0.9124 0.6759 0.8702 0.8752 0.9125 0.8686
253 0.9347 0.4930 0.4923 0.6785 0.6698 0.5351 0.6116 0.6680 0.6191 0.5664
262 0.9529 0.4259 0.9479 0.9210 0.9274 0.8356 0.9055 0.8885 0.9258 0.8443
mean 0.9008 0.3706 0.8297 0.8967 0.8896 0.6090 0.8626 0.8689 0.9138 0.9072
Table 46. OSNNet-TR-KUcnl-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUcnl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9763 0.2637 0.8406 0.9882 0.9953 0.5479 0.9390 0.9642 1.0000 0.9985
523 0.9847 0.2946 0.8971 0.9803 0.9511 0.5434 0.9282 0.9493 0.9991 0.9905
631 0.9578 0.2111 0.8661 0.9849 0.9838 0.4945 0.9547 0.9878 1.0000 0.9990
424 0.9872 0.3856 0.8186 0.9081 0.8806 0.3958 0.8348 0.8962 0.9841 0.9904
532 0.9631 0.3110 0.6987 0.9460 0.9563 0.6188 0.9101 0.9204 0.9961 0.9827
325 0.9880 0.4494 0.7943 0.9169 0.8436 0.4240 0.8602 0.8790 0.9349 0.9117
433 0.9870 0.3922 0.6476 0.9533 0.9176 0.4793 0.9038 0.9151 0.9909 0.9943
541 0.9405 0.3191 0.7331 0.9311 0.9263 0.5600 0.9188 0.9105 0.9954 0.9846
442 0.9836 0.4163 0.7463 0.9576 0.9250 0.6037 0.9050 0.9418 0.9917 0.9895
334 0.9873 0.5389 0.8128 0.8783 0.8496 0.5403 0.8086 0.8729 0.9071 0.8713
226 0.9665 0.4193 0.6342 0.5958 0.5657 0.5101 0.5696 0.6089 0.5980 0.7286
451 0.9861 0.4275 0.7696 0.9519 0.9361 0.4557 0.9236 0.8474 0.9940 0.9844
343 0.9906 0.5072 0.8126 0.9063 0.9246 0.4972 0.8861 0.8372 0.9689 0.9260
235 0.9567 0.3236 0.6043 0.6302 0.6039 0.5222 0.5911 0.6165 0.7507 0.8327
352 0.9917 0.5055 0.7130 0.8684 0.8396 0.4503 0.8274 0.7020 0.9216 0.8352
361 0.9916 0.5892 0.7305 0.8281 0.7707 0.5109 0.7596 0.7485 0.8459 0.7798
244 0.9586 0.4895 0.9220 0.9109 0.9124 0.6759 0.8702 0.8752 0.9125 0.8686
253 0.9347 0.4930 0.4923 0.6785 0.6698 0.5351 0.6116 0.6680 0.6191 0.5664
262 0.9529 0.4259 0.9479 0.9210 0.9274 0.8356 0.9055 0.8885 0.9258 0.8443
mean 0.9729 0.4085 0.7622 0.8808 0.8621 0.5369 0.8373 0.8437 0.9124 0.8989
Table 47. OSNNet-TR-KUcnl-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUcnl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9751 0.5739 0.9863 0.9661 0.9751 0.7068 0.8791 0.9089 0.9974 0.9584
523 0.9816 0.5601 0.9860 0.9665 0.9505 0.7045 0.8847 0.9237 0.9964 0.9383
631 0.9687 0.4940 0.9889 0.9691 0.9793 0.7239 0.9167 0.9626 0.9984 0.9625
424 0.9878 0.6286 0.9898 0.9772 0.9767 0.5581 0.8885 0.9229 0.9962 0.9561
532 0.9797 0.5820 0.9877 0.9488 0.9571 0.7584 0.8719 0.9417 0.9973 0.9499
325 0.9938 0.6171 0.9898 0.9749 0.9653 0.5200 0.9108 0.9229 0.9925 0.9544
433 0.9860 0.6102 0.9893 0.9666 0.9456 0.6433 0.9118 0.9323 0.9916 0.8972
541 0.9808 0.5586 0.9777 0.9418 0.9458 0.6891 0.8937 0.9140 0.9810 0.9302
442 0.9861 0.6230 0.9798 0.9503 0.9376 0.6701 0.8726 0.9481 0.9926 0.9502
334 0.9929 0.6921 0.9761 0.9488 0.9279 0.5622 0.8922 0.9369 0.9729 0.8532
226 0.9958 0.6231 0.9895 0.9854 0.9632 0.4786 0.9379 0.9358 0.9997 0.9678
451 0.9863 0.6200 0.9806 0.9476 0.9425 0.7170 0.9043 0.8826 0.9849 0.9543
343 0.9934 0.6411 0.9822 0.9510 0.9360 0.6344 0.9143 0.9180 0.9889 0.9385
235 0.9957 0.6142 0.9883 0.9805 0.9675 0.5594 0.9448 0.9233 0.9942 0.9364
352 0.9933 0.6790 0.9742 0.9320 0.9112 0.7332 0.8733 0.8880 0.9898 0.8731
361 0.9933 0.6755 0.9664 0.9254 0.9338 0.7528 0.8591 0.8389 0.9811 0.9396
244 0.9966 0.8076 0.9871 0.9500 0.9499 0.6143 0.8618 0.8970 0.9917 0.8991
253 0.9964 0.6925 0.9721 0.9065 0.9111 0.6013 0.8754 0.8570 0.9918 0.8673
262 0.9960 0.7518 0.9671 0.9135 0.8727 0.6781 0.8731 0.8117 0.8654 0.7943
mean 0.9884 0.6339 0.9821 0.9527 0.9447 0.6476 0.8929 0.9087 0.9844 0.9222
Table 48. OSNNet-TR-KUcnl-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. First cell indicates set used for training. Columns refer to the different testing sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUcnl KN KNc KU KUl KUn UU UUl UUn RN RNp
622 0.9902 0.7991 0.7299 0.6364 0.6416 0.3023 0.5411 0.5010 0.7060 0.5672
523 0.9921 0.7565 0.8185 0.7156 0.6239 0.3714 0.6252 0.6144 0.8656 0.6104
631 0.9930 0.8061 0.4816 0.3893 0.3700 0.1616 0.3497 0.3299 0.4804 0.3488
424 0.9934 0.7628 0.9309 0.8525 0.8391 0.3527 0.7414 0.7707 0.9214 0.7539
532 0.9931 0.7914 0.6769 0.4952 0.5296 0.3144 0.4434 0.4355 0.6304 0.4805
325 0.9943 0.6666 0.9643 0.9148 0.8834 0.4726 0.8519 0.8505 0.9748 0.8892
433 0.9918 0.7355 0.8514 0.7875 0.7677 0.4238 0.7268 0.7251 0.8064 0.6171
541 0.9934 0.8010 0.6243 0.4972 0.4939 0.2517 0.4497 0.4509 0.5766 0.4399
442 0.9925 0.7318 0.8610 0.7275 0.7215 0.4183 0.6637 0.7070 0.7569 0.7492
334 0.9931 0.6929 0.9663 0.9182 0.9035 0.5466 0.8571 0.8953 0.9034 0.8287
226 0.9958 0.6231 0.9895 0.9854 0.9632 0.4786 0.9379 0.9358 0.9997 0.9678
451 0.9913 0.7173 0.9033 0.7944 0.7685 0.5279 0.7519 0.6710 0.8445 0.8007
343 0.9949 0.7181 0.8927 0.7813 0.7875 0.4894 0.7566 0.7176 0.8101 0.8040
235 0.9957 0.6142 0.9883 0.9805 0.9675 0.5594 0.9448 0.9233 0.9942 0.9364
352 0.9945 0.7343 0.8752 0.8203 0.7959 0.6146 0.7578 0.7731 0.8819 0.7471
361 0.9946 0.7402 0.8525 0.7973 0.8004 0.6366 0.7401 0.7167 0.8810 0.8196
244 0.9966 0.8076 0.9871 0.9500 0.9499 0.6143 0.8618 0.8970 0.9917 0.8991
253 0.9964 0.6925 0.9721 0.9065 0.9111 0.6013 0.8754 0.8570 0.9918 0.8673
262 0.9960 0.7518 0.9671 0.9135 0.8727 0.6781 0.8731 0.8117 0.8654 0.7943
mean 0.9938 0.7338 0.8596 0.7823 0.7679 0.4640 0.7236 0.7149 0.8359 0.7327

Table 49. OSNNet-TR-all-normal-table_image.png
Rows refer to training sets. Columns refer to the different testing sets. Blue indicates low accuracy (< 0.5).
all KN KNc KU KUl KUn UU UUl UUn RN RNp
KU 0.9924 0.2204 0.9965 0.7911 0.8624 0.5766 0.7861 0.7737 0.7959 0.6975
KUn 0.9927 0.0318 0.9963 0.9900 0.9984 0.5681 0.9841 0.9902 1.0000 0.9006
KUl 0.9924 0.0026 0.9963 0.9995 0.9999 0.5732 0.9978 0.9999 1.0000 0.9811
KUnl 0.9923 0.0023 0.9964 0.9995 0.9998 0.5786 0.9979 0.9998 1.0000 0.9827
KUc 0.9924 0.8630 0.9963 0.0488 0.1512 0.5684 0.0642 0.0705 0.0016 0.3724
KUcn 0.9924 0.7546 0.9965 0.6215 0.9346 0.5747 0.6194 0.6260 0.7345 0.7140
KUcl 0.9926 0.4796 0.9963 0.9961 0.9900 0.5818 0.9475 0.9624 0.9999 0.9780
KUcnl 0.9924 0.4540 0.9965 0.9969 0.9959 0.5866 0.9544 0.9709 0.9999 0.9850
Table 50. OSNNet-TR-all-threshold-table_image.png
Results with threshold on softmax. Rows refer to training sets. Columns refer to the different testing sets. Blue indicates low accuracy (< 0.5).
all KN KNc KU KUl KUn UU UUl UUn RN RNp
KU 0.9800 0.0903 0.9990 0.9670 0.9822 0.7895 0.9571 0.9552 0.9808 0.9832
KUn 0.9803 0.0104 0.9990 0.9977 0.9997 0.7838 0.9951 0.9982 1.0000 0.9976
KUl 0.9801 0.0007 0.9990 0.9999 1.0000 0.7890 0.9992 1.0000 1.0000 0.9998
KUnl 0.9796 0.0001 0.9990 0.9999 1.0000 0.7924 0.9993 1.0000 1.0000 0.9998
KUc 0.9803 0.6018 0.9990 0.6969 0.7887 0.7810 0.6920 0.7245 0.8180 0.9483
KUcn 0.9799 0.4553 0.9990 0.9192 0.9948 0.7870 0.8982 0.9295 0.9944 0.9917
KUcl 0.9806 0.2320 0.9990 0.9996 0.9989 0.7952 0.9890 0.9962 1.0000 0.9999
KUcnl 0.9799 0.2196 0.9990 0.9996 0.9996 0.7949 0.9906 0.9959 1.0000 0.9999
Table 51. OSNNet-TR-all-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. Rows refer to training sets. Columns refer to the different testing sets. Blue indicates low accuracy (< 0.5).
all KN KNc KU KUl KUn UU UUl UUn RN RNp
KU 0.9009 0.1494 0.8261 0.8781 0.8884 0.6303 0.8689 0.8699 0.8987 0.8660
KUn 0.9027 0.0789 0.8562 0.9152 0.9196 0.6526 0.9108 0.9192 0.9485 0.9141
KUl 0.8841 0.0630 0.8674 0.8817 0.8823 0.6491 0.8781 0.8824 0.8882 0.9273
KUnl 0.9138 0.0803 0.8199 0.8322 0.8326 0.6423 0.8272 0.8348 0.8339 0.9063
KUc 0.8918 0.4555 0.8299 0.7142 0.7495 0.6322 0.7047 0.7273 0.7837 0.8541
KUcn 0.8988 0.4763 0.8198 0.7906 0.8711 0.6225 0.7777 0.8054 0.8692 0.8538
KUcl 0.8911 0.3504 0.8413 0.9115 0.8862 0.6504 0.8739 0.8859 0.9374 0.9164
KUcnl 0.9008 0.3706 0.8297 0.8967 0.8896 0.6090 0.8626 0.8689 0.9138 0.9072
Table 52. OSNNet-TR-all-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. Rows refer to training sets. Columns refer to the different testing sets. Blue indicates low accuracy (< 0.5).
all KN KNc KU KUl KUn UU UUl UUn RN RNp
KU 0.9690 0.1629 0.7682 0.8626 0.8703 0.5659 0.8524 0.8507 0.8825 0.8652
KUn 0.9624 0.1029 0.7884 0.8854 0.8869 0.5881 0.8788 0.8916 0.9226 0.9134
KUl 0.9667 0.0940 0.8027 0.8169 0.8147 0.5709 0.8151 0.8179 0.8374 0.9171
KUnl 0.9750 0.0974 0.7696 0.7988 0.7927 0.5679 0.7940 0.7997 0.7923 0.8845
KUc 0.9711 0.4947 0.7609 0.6930 0.7399 0.5678 0.6807 0.7095 0.7844 0.8391
KUcn 0.9663 0.4989 0.7547 0.7868 0.8652 0.5570 0.7691 0.7899 0.8845 0.8631
KUcl 0.9728 0.3873 0.7553 0.9009 0.8760 0.5733 0.8592 0.8679 0.9352 0.9153
KUcnl 0.9729 0.4085 0.7622 0.8808 0.8621 0.5369 0.8373 0.8437 0.9124 0.8989
Table 53. OSNNet-TR-all-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. Rows refer to training sets. Columns refer to the different testing sets. Blue indicates low accuracy (< 0.5).
all KN KNc KU KUl KUn UU UUl UUn RN RNp
KU 0.9886 0.1841 0.9811 0.8958 0.9160 0.6456 0.8850 0.8748 0.9126 0.9063
KUn 0.9884 0.0666 0.9821 0.9856 0.9936 0.6464 0.9779 0.9814 0.9994 0.9488
KUl 0.9886 0.0163 0.9804 0.9983 0.9979 0.6464 0.9957 0.9977 1.0000 0.9722
KUnl 0.9883 0.0181 0.9790 0.9981 0.9974 0.6494 0.9952 0.9962 0.9999 0.9584
KUc 0.9888 0.6931 0.9809 0.6085 0.6815 0.6410 0.6071 0.6280 0.7210 0.8235
KUcn 0.9887 0.6818 0.9806 0.7669 0.9264 0.6483 0.7563 0.7790 0.8842 0.8697
KUcl 0.9885 0.6276 0.9836 0.9611 0.9420 0.6490 0.8937 0.9101 0.9906 0.9258
KUcnl 0.9884 0.6339 0.9821 0.9527 0.9447 0.6476 0.8929 0.9087 0.9844 0.9222
Table 54. OSNNet-TR-all-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. Rows refer to training sets. Columns refer to the different testing sets. Blue indicates low accuracy (< 0.5).
all KN KNc KU KUl KUn UU UUl UUn RN RNp
KU 0.9938 0.2628 0.8693 0.7394 0.7584 0.4772 0.7281 0.7184 0.7552 0.7606
KUn 0.9937 0.1425 0.8573 0.8881 0.8944 0.4698 0.8772 0.8740 0.9226 0.8052
KUl 0.9937 0.0611 0.8554 0.9385 0.9316 0.4703 0.9363 0.9253 0.9490 0.8340
KUnl 0.9937 0.0666 0.8437 0.9136 0.9034 0.4681 0.9075 0.8967 0.9171 0.8144
KUc 0.9940 0.7848 0.8314 0.4077 0.4710 0.4514 0.4086 0.4252 0.5027 0.6124
KUcn 0.9940 0.7727 0.8354 0.5690 0.7457 0.4551 0.5622 0.5804 0.6718 0.6618
KUcl 0.9939 0.7258 0.8428 0.7889 0.7580 0.4629 0.7193 0.7179 0.8471 0.7382
KUcnl 0.9938 0.7338 0.8596 0.7823 0.7679 0.4640 0.7236 0.7149 0.8359 0.7327

Table. Information about training sets. All training sets include samples from MNIST dataset. Variations shown in this table indicate the portion of data included from CHARS74K additional dataset.
KU no training sample from CHARS74K.
KUn numbers for the KU classes.
KUl letters for the KU classes.
KUnl numbers and letters for the KU classes.
KUc numbers for the KN classes.
KUcn numbers for both KN and KU classes.
KUcl numbers for KN classes and letters for KU classes.
KUcnl numbers for both KN and KU classes and letters for KU classes.

Table 55. OSNNet-TE-KN-normal-table_image.png
First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KN KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9935 0.9936 0.9933 0.9928 0.9932 0.9934 0.9928 0.9924
523 0.9941 0.9937 0.9937 0.9935 0.9938 0.9932 0.9933 0.9933
631 0.9916 0.9925 0.9919 0.9913 0.9919 0.9920 0.9918 0.9920
424 0.9941 0.9953 0.9940 0.9935 0.9945 0.9951 0.9946 0.9945
532 0.9932 0.9928 0.9928 0.9927 0.9932 0.9926 0.9924 0.9924
325 0.9951 0.9949 0.9950 0.9951 0.9948 0.9955 0.9947 0.9954
433 0.9922 0.9930 0.9929 0.9922 0.9927 0.9928 0.9933 0.9929
541 0.9918 0.9913 0.9910 0.9921 0.9908 0.9902 0.9920 0.9912
442 0.9910 0.9912 0.9919 0.9920 0.9916 0.9912 0.9929 0.9908
334 0.9943 0.9941 0.9940 0.9943 0.9935 0.9943 0.9941 0.9933
226 0.9941 0.9966 0.9949 0.9958 0.9958 0.9954 0.9961 0.9951
451 0.9891 0.9904 0.9882 0.9896 0.9882 0.9908 0.9894 0.9902
343 0.9924 0.9925 0.9937 0.9926 0.9928 0.9923 0.9915 0.9933
235 0.9940 0.9953 0.9948 0.9942 0.9953 0.9953 0.9945 0.9945
352 0.9907 0.9900 0.9895 0.9892 0.9901 0.9894 0.9909 0.9919
361 0.9910 0.9905 0.9906 0.9905 0.9909 0.9894 0.9904 0.9889
244 0.9929 0.9921 0.9935 0.9923 0.9931 0.9936 0.9937 0.9918
253 0.9921 0.9918 0.9912 0.9905 0.9917 0.9899 0.9918 0.9923
262 0.9879 0.9891 0.9892 0.9894 0.9886 0.9895 0.9899 0.9892
mean 0.9924 0.9927 0.9924 0.9923 0.9924 0.9924 0.9926 0.9924
Table 56. OSNNet-TE-KN-threshold-table_image.png
Results with threshold on softmax. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KN KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9805 0.9802 0.9799 0.9788 0.9799 0.9792 0.9786 0.9792
523 0.9830 0.9828 0.9826 0.9828 0.9835 0.9826 0.9824 0.9819
631 0.9746 0.9762 0.9755 0.9742 0.9752 0.9754 0.9756 0.9753
424 0.9852 0.9867 0.9845 0.9830 0.9862 0.9858 0.9857 0.9854
532 0.9795 0.9790 0.9790 0.9784 0.9810 0.9798 0.9775 0.9777
325 0.9880 0.9882 0.9885 0.9882 0.9883 0.9885 0.9880 0.9882
433 0.9802 0.9803 0.9808 0.9793 0.9799 0.9805 0.9817 0.9814
541 0.9752 0.9739 0.9738 0.9742 0.9742 0.9728 0.9763 0.9741
442 0.9744 0.9747 0.9763 0.9763 0.9760 0.9763 0.9775 0.9744
334 0.9861 0.9856 0.9858 0.9857 0.9852 0.9854 0.9858 0.9847
226 0.9879 0.9907 0.9880 0.9891 0.9902 0.9891 0.9915 0.9895
451 0.9707 0.9729 0.9680 0.9716 0.9694 0.9730 0.9713 0.9721
343 0.9800 0.9809 0.9828 0.9802 0.9815 0.9796 0.9790 0.9817
235 0.9854 0.9879 0.9874 0.9874 0.9871 0.9868 0.9866 0.9854
352 0.9760 0.9744 0.9749 0.9741 0.9740 0.9737 0.9767 0.9787
361 0.9761 0.9751 0.9765 0.9756 0.9749 0.9734 0.9754 0.9719
244 0.9843 0.9822 0.9856 0.9819 0.9843 0.9852 0.9848 0.9818
253 0.9807 0.9811 0.9787 0.9773 0.9807 0.9763 0.9808 0.9823
262 0.9721 0.9726 0.9735 0.9744 0.9744 0.9750 0.9757 0.9733
mean 0.9800 0.9803 0.9801 0.9796 0.9803 0.9799 0.9806 0.9799
Table 57. OSNNet-TE-KN-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KN KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.8183 0.7286 0.7465 0.8354 0.8066 0.7984 0.7619 0.8807
523 0.8453 0.8701 0.7971 0.8274 0.8561 0.8745 0.7972 0.8590
631 0.8603 0.7630 0.6850 0.7517 0.7388 0.7420 0.7662 0.8243
424 0.9020 0.9847 0.9134 0.9363 0.9122 0.8760 0.9295 0.8728
532 0.9012 0.8245 0.8327 0.8749 0.8697 0.8780 0.9012 0.9426
325 0.9460 0.9785 0.8823 0.9265 0.9092 0.9784 0.8494 0.9173
433 0.9385 0.9234 0.8906 0.9290 0.8786 0.8823 0.9523 0.8827
541 0.7335 0.8387 0.8066 0.8898 0.8649 0.8711 0.7905 0.8291
442 0.9384 0.8949 0.9593 0.9789 0.8846 0.8906 0.9327 0.9570
334 0.9424 0.9406 0.9222 0.9060 0.9440 0.9323 0.9728 0.9339
226 0.9447 0.8996 0.9518 0.9783 0.8876 0.9676 0.9298 0.9665
451 0.9862 0.8952 0.8909 0.9039 0.8223 0.9380 0.8094 0.8092
343 0.8685 0.9337 0.9147 0.9320 0.8739 0.9210 0.8580 0.8489
235 0.9605 0.9243 0.9708 0.9536 0.9685 0.9659 0.9654 0.9567
352 0.8651 0.9568 0.9056 0.9411 0.9689 0.8348 0.9101 0.8759
361 0.8230 0.9342 0.9382 0.8996 0.8955 0.9796 0.9324 0.9128
244 0.9167 0.9322 0.9237 0.9765 0.9621 0.8239 0.9513 0.9586
253 0.9667 0.9708 0.9029 0.9689 0.9282 0.9716 0.9495 0.9347
262 0.9604 0.9580 0.9634 0.9514 0.9723 0.9517 0.9717 0.9529
mean 0.9009 0.9027 0.8841 0.9138 0.8918 0.8988 0.8911 0.9008
Table 58. OSNNet-TE-KN-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KN KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9769 0.9642 0.9627 0.9759 0.9622 0.9348 0.9624 0.9763
523 0.9636 0.9850 0.9639 0.9832 0.9837 0.9439 0.9832 0.9847
631 0.9324 0.9172 0.9590 0.9771 0.9731 0.9614 0.9582 0.9578
424 0.9857 0.9847 0.9845 0.9631 0.9882 0.9855 0.9877 0.9872
532 0.9662 0.9124 0.9666 0.9826 0.9646 0.9828 0.9815 0.9631
325 0.9895 0.9921 0.9894 0.9882 0.9862 0.9927 0.9916 0.9880
433 0.9724 0.9744 0.9853 0.9642 0.9861 0.9852 0.9666 0.9870
541 0.9591 0.9593 0.9468 0.9844 0.9621 0.9831 0.9629 0.9405
442 0.9853 0.9829 0.9632 0.9856 0.9830 0.9621 0.9868 0.9836
334 0.9895 0.9911 0.9877 0.9887 0.9889 0.9886 0.9900 0.9873
226 0.9447 0.8996 0.9518 0.9783 0.8876 0.9676 0.9298 0.9665
451 0.9848 0.9816 0.9864 0.9354 0.9849 0.9857 0.9821 0.9861
343 0.9924 0.9904 0.9860 0.9905 0.9908 0.9918 0.9871 0.9906
235 0.9605 0.9243 0.9708 0.9536 0.9685 0.9659 0.9654 0.9567
352 0.9883 0.9828 0.9868 0.9922 0.9902 0.9915 0.9872 0.9917
361 0.9759 0.9822 0.9866 0.9859 0.9886 0.9900 0.9889 0.9916
244 0.9167 0.9322 0.9237 0.9765 0.9621 0.8239 0.9513 0.9586
253 0.9667 0.9708 0.9029 0.9689 0.9282 0.9716 0.9495 0.9347
262 0.9604 0.9580 0.9634 0.9514 0.9723 0.9517 0.9717 0.9529
mean 0.9690 0.9624 0.9667 0.9750 0.9711 0.9663 0.9728 0.9729
Table 59. OSNNet-TE-KN-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KN KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9718 0.9734 0.9712 0.9712 0.9735 0.9734 0.9735 0.9751
523 0.9836 0.9826 0.9830 0.9823 0.9827 0.9822 0.9819 0.9816
631 0.9723 0.9707 0.9732 0.9689 0.9722 0.9717 0.9714 0.9687
424 0.9878 0.9885 0.9878 0.9880 0.9885 0.9879 0.9881 0.9878
532 0.9813 0.9825 0.9810 0.9815 0.9821 0.9822 0.9801 0.9797
325 0.9932 0.9937 0.9935 0.9939 0.9942 0.9934 0.9936 0.9938
433 0.9873 0.9854 0.9866 0.9864 0.9864 0.9869 0.9863 0.9860
541 0.9812 0.9796 0.9813 0.9818 0.9822 0.9818 0.9810 0.9808
442 0.9862 0.9873 0.9864 0.9871 0.9869 0.9872 0.9878 0.9861
334 0.9925 0.9929 0.9927 0.9927 0.9933 0.9925 0.9927 0.9929
226 0.9954 0.9952 0.9950 0.9954 0.9958 0.9953 0.9962 0.9958
451 0.9863 0.9858 0.9863 0.9848 0.9858 0.9873 0.9862 0.9863
343 0.9930 0.9930 0.9930 0.9931 0.9933 0.9932 0.9922 0.9934
235 0.9955 0.9960 0.9959 0.9961 0.9958 0.9959 0.9958 0.9957
352 0.9932 0.9922 0.9929 0.9926 0.9932 0.9921 0.9931 0.9933
361 0.9935 0.9931 0.9935 0.9934 0.9924 0.9931 0.9929 0.9933
244 0.9967 0.9961 0.9965 0.9963 0.9962 0.9968 0.9968 0.9966
253 0.9969 0.9966 0.9969 0.9964 0.9967 0.9969 0.9963 0.9964
262 0.9963 0.9957 0.9958 0.9959 0.9957 0.9954 0.9964 0.9960
mean 0.9886 0.9884 0.9886 0.9883 0.9888 0.9887 0.9885 0.9884
Table 60. OSNNet-TE-KN-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KN KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9914 0.9909 0.9914 0.9907 0.9918 0.9921 0.9922 0.9902
523 0.9926 0.9919 0.9930 0.9926 0.9932 0.9923 0.9920 0.9921
631 0.9917 0.9924 0.9929 0.9917 0.9932 0.9926 0.9931 0.9930
424 0.9936 0.9928 0.9922 0.9925 0.9919 0.9929 0.9911 0.9934
532 0.9925 0.9928 0.9922 0.9932 0.9935 0.9938 0.9928 0.9931
325 0.9934 0.9937 0.9942 0.9944 0.9945 0.9953 0.9945 0.9943
433 0.9918 0.9911 0.9918 0.9917 0.9917 0.9913 0.9926 0.9918
541 0.9921 0.9941 0.9925 0.9921 0.9939 0.9932 0.9934 0.9934
442 0.9924 0.9927 0.9921 0.9926 0.9927 0.9926 0.9917 0.9925
334 0.9938 0.9940 0.9929 0.9937 0.9943 0.9935 0.9941 0.9931
226 0.9954 0.9952 0.9950 0.9954 0.9958 0.9953 0.9962 0.9958
451 0.9931 0.9929 0.9919 0.9936 0.9924 0.9935 0.9934 0.9913
343 0.9938 0.9941 0.9936 0.9941 0.9950 0.9944 0.9935 0.9949
235 0.9955 0.9960 0.9959 0.9961 0.9958 0.9959 0.9958 0.9957
352 0.9941 0.9930 0.9945 0.9931 0.9950 0.9941 0.9943 0.9945
361 0.9947 0.9943 0.9949 0.9942 0.9934 0.9946 0.9932 0.9946
244 0.9967 0.9961 0.9965 0.9963 0.9962 0.9968 0.9968 0.9966
253 0.9969 0.9966 0.9969 0.9964 0.9967 0.9969 0.9963 0.9964
262 0.9963 0.9957 0.9958 0.9959 0.9957 0.9954 0.9964 0.9960
mean 0.9938 0.9937 0.9937 0.9937 0.9940 0.9940 0.9939 0.9938

Table 61. OSNNet-TE-KNc-normal-table_image.png
First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KNc KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.4029 0.0538 0.0000 0.0000 0.8402 0.8304 0.4893 0.4947
523 0.2790 0.0512 0.0007 0.0000 0.8311 0.7904 0.4275 0.4284
631 0.2652 0.0382 0.0000 0.0014 0.8383 0.7790 0.4952 0.4734
424 0.3469 0.0422 0.0000 0.0008 0.8813 0.8114 0.5443 0.4945
532 0.2492 0.0330 0.0014 0.0000 0.8526 0.7906 0.4603 0.4516
325 0.2029 0.0319 0.0000 0.0010 0.8865 0.7847 0.4690 0.5132
433 0.2292 0.0389 0.0036 0.0026 0.8367 0.7793 0.4829 0.4707
541 0.1834 0.0212 0.0021 0.0037 0.8195 0.7398 0.4695 0.4406
442 0.1133 0.0157 0.0000 0.0000 0.8523 0.7537 0.4688 0.4197
334 0.2639 0.0368 0.0042 0.0027 0.8945 0.7636 0.5201 0.5002
226 0.4527 0.0878 0.0053 0.0026 0.8927 0.7259 0.5036 0.4906
451 0.1327 0.0149 0.0033 0.0021 0.8349 0.7277 0.5172 0.4453
343 0.1554 0.0204 0.0040 0.0071 0.8574 0.7028 0.5068 0.4631
235 0.2067 0.0476 0.0038 0.0075 0.8544 0.7349 0.4113 0.3758
352 0.1139 0.0031 0.0000 0.0000 0.8559 0.7128 0.4220 0.3926
361 0.0815 0.0078 0.0033 0.0015 0.8768 0.7343 0.4489 0.3990
244 0.2999 0.0289 0.0028 0.0027 0.9273 0.7932 0.4763 0.4739
253 0.0997 0.0096 0.0019 0.0000 0.8848 0.7024 0.4583 0.3809
262 0.1100 0.0207 0.0129 0.0078 0.8791 0.6801 0.5411 0.5179
mean 0.2204 0.0318 0.0026 0.0023 0.8630 0.7546 0.4796 0.4540
Table 62. OSNNet-TE-KNc-threshold-table_image.png
Results with threshold on softmax. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KNc KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.1320 0.0157 0.0000 0.0000 0.5270 0.4951 0.2279 0.2252
523 0.1140 0.0195 0.0007 0.0000 0.5216 0.4382 0.1770 0.1774
631 0.0824 0.0073 0.0000 0.0000 0.4470 0.3823 0.2152 0.1865
424 0.1587 0.0175 0.0000 0.0000 0.6334 0.5518 0.2705 0.2394
532 0.0811 0.0108 0.0000 0.0000 0.5133 0.4550 0.1894 0.1823
325 0.1024 0.0057 0.0000 0.0000 0.6840 0.5477 0.2357 0.2788
433 0.0895 0.0120 0.0016 0.0000 0.5626 0.4859 0.2547 0.2378
541 0.0554 0.0033 0.0006 0.0000 0.4892 0.3887 0.2169 0.1924
442 0.0311 0.0034 0.0000 0.0000 0.5138 0.3916 0.1710 0.1653
334 0.1345 0.0086 0.0014 0.0000 0.6594 0.5039 0.2816 0.2585
226 0.2393 0.0388 0.0026 0.0026 0.7379 0.5783 0.2964 0.3120
451 0.0557 0.0032 0.0000 0.0000 0.5391 0.3810 0.2475 0.2148
343 0.0557 0.0089 0.0000 0.0000 0.5882 0.3886 0.2389 0.2187
235 0.1272 0.0225 0.0019 0.0000 0.6697 0.4831 0.2431 0.2277
352 0.0293 0.0000 0.0000 0.0000 0.5463 0.4182 0.1640 0.1477
361 0.0236 0.0011 0.0022 0.0000 0.5846 0.3751 0.1660 0.1567
244 0.1377 0.0069 0.0000 0.0000 0.8291 0.5978 0.2858 0.2755
253 0.0347 0.0042 0.0019 0.0000 0.6810 0.3835 0.2161 0.1812
262 0.0309 0.0082 0.0000 0.0000 0.7066 0.4043 0.3111 0.2951
mean 0.0903 0.0104 0.0007 0.0001 0.6018 0.4553 0.2320 0.2196
Table 63. OSNNet-TE-KNc-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KNc KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.0700 0.0399 0.0120 0.0247 0.2145 0.2481 0.1632 0.2462
523 0.0996 0.0597 0.0632 0.0468 0.3026 0.3547 0.2288 0.2569
631 0.0521 0.0399 0.0283 0.0217 0.1529 0.2325 0.1769 0.2246
424 0.1681 0.0656 0.1022 0.1062 0.4688 0.4337 0.3937 0.3445
532 0.0962 0.0475 0.1004 0.0879 0.2795 0.3286 0.2809 0.3382
325 0.1816 0.0797 0.0499 0.0128 0.6155 0.6275 0.3778 0.4628
433 0.1667 0.0351 0.0673 0.0940 0.3849 0.4462 0.3771 0.3294
541 0.1141 0.0683 0.0583 0.0793 0.2927 0.4623 0.3229 0.3286
442 0.1399 0.0474 0.0459 0.0373 0.3844 0.4679 0.4289 0.4125
334 0.1731 0.1070 0.0494 0.0781 0.5840 0.5302 0.5208 0.4420
226 0.2227 0.0540 0.0640 0.1719 0.5450 0.5126 0.2747 0.4193
451 0.1802 0.0636 0.0645 0.0753 0.2808 0.4947 0.4161 0.4052
343 0.0860 0.0646 0.0444 0.0991 0.4632 0.5601 0.3694 0.3994
235 0.2473 0.1725 0.0995 0.1729 0.7218 0.5435 0.3391 0.3236
352 0.1420 0.0371 0.0266 0.0372 0.4580 0.5095 0.3764 0.3229
361 0.1207 0.0499 0.0696 0.0224 0.3971 0.5772 0.4344 0.3768
244 0.3056 0.0571 0.0651 0.1431 0.7159 0.5227 0.2868 0.4895
253 0.1737 0.2973 0.0762 0.0832 0.6162 0.6123 0.3062 0.4930
262 0.0980 0.1126 0.1098 0.1324 0.7770 0.5861 0.5842 0.4259
mean 0.1494 0.0789 0.0630 0.0803 0.4555 0.4763 0.3504 0.3706
Table 64. OSNNet-TE-KNc-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KNc KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.0636 0.0426 0.0250 0.0515 0.2313 0.2534 0.2399 0.2637
523 0.0925 0.0611 0.0713 0.0472 0.3002 0.3451 0.2814 0.2946
631 0.0701 0.0401 0.0375 0.0316 0.2045 0.2320 0.1965 0.2111
424 0.1781 0.0838 0.1510 0.1296 0.5098 0.5081 0.4166 0.3856
532 0.0797 0.0697 0.0814 0.1069 0.3265 0.3322 0.2866 0.3110
325 0.2207 0.1476 0.0796 0.0402 0.6339 0.6573 0.5292 0.4494
433 0.1700 0.0828 0.1040 0.0610 0.4313 0.4867 0.3782 0.3922
541 0.0863 0.0860 0.0813 0.0995 0.2671 0.3698 0.2966 0.3191
442 0.1279 0.0731 0.1217 0.0618 0.3480 0.4440 0.4085 0.4163
334 0.2072 0.1251 0.1125 0.0988 0.6503 0.6694 0.5817 0.5389
226 0.2227 0.0540 0.0640 0.1719 0.5450 0.5126 0.2747 0.4193
451 0.1694 0.1095 0.1632 0.0863 0.3888 0.4312 0.4537 0.4275
343 0.2520 0.1177 0.1441 0.1185 0.6047 0.6161 0.4698 0.5072
235 0.2473 0.1725 0.0995 0.1729 0.7218 0.5435 0.3391 0.3236
352 0.2115 0.0953 0.0766 0.0941 0.5679 0.6888 0.5073 0.5055
361 0.1182 0.1272 0.1223 0.1209 0.5598 0.6680 0.5221 0.5892
244 0.3056 0.0571 0.0651 0.1431 0.7159 0.5227 0.2868 0.4895
253 0.1737 0.2973 0.0762 0.0832 0.6162 0.6123 0.3062 0.4930
262 0.0980 0.1126 0.1098 0.1324 0.7770 0.5861 0.5842 0.4259
mean 0.1629 0.1029 0.0940 0.0974 0.4947 0.4989 0.3873 0.4085
Table 65. OSNNet-TE-KNc-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KNc KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.1123 0.0349 0.0030 0.0039 0.5736 0.5917 0.5616 0.5739
523 0.1426 0.0589 0.0061 0.0076 0.6066 0.5955 0.5584 0.5601
631 0.0809 0.0342 0.0026 0.0019 0.5317 0.5358 0.5311 0.4940
424 0.1827 0.0442 0.0032 0.0061 0.6988 0.6831 0.6257 0.6286
532 0.1314 0.0556 0.0110 0.0084 0.6128 0.6119 0.5709 0.5820
325 0.1799 0.0557 0.0021 0.0071 0.7516 0.7063 0.5972 0.6171
433 0.1465 0.0549 0.0135 0.0133 0.6044 0.6443 0.6144 0.6102
541 0.1056 0.0367 0.0153 0.0157 0.5754 0.5712 0.5613 0.5586
442 0.1059 0.0355 0.0119 0.0103 0.6378 0.6462 0.6370 0.6230
334 0.2034 0.0771 0.0103 0.0210 0.7553 0.7241 0.6760 0.6921
226 0.3676 0.1157 0.0219 0.0087 0.8018 0.7175 0.6247 0.6231
451 0.1517 0.0734 0.0253 0.0238 0.6030 0.6687 0.6182 0.6200
343 0.1751 0.0667 0.0211 0.0277 0.7283 0.6820 0.6402 0.6411
235 0.2908 0.1199 0.0223 0.0211 0.7728 0.7104 0.5825 0.6142
352 0.1667 0.0542 0.0070 0.0032 0.6998 0.6927 0.6358 0.6790
361 0.1205 0.0564 0.0213 0.0243 0.7054 0.7214 0.6720 0.6755
244 0.3555 0.0848 0.0454 0.0241 0.8782 0.8537 0.7797 0.8076
253 0.2106 0.0957 0.0234 0.0355 0.8027 0.7715 0.6680 0.6925
262 0.2686 0.1114 0.0434 0.0809 0.8294 0.8259 0.7697 0.7518
mean 0.1841 0.0666 0.0163 0.0181 0.6931 0.6818 0.6276 0.6339
Table 66. OSNNet-TE-KNc-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KNc KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.3118 0.2137 0.0922 0.0842 0.8113 0.8151 0.8129 0.7991
523 0.2678 0.1521 0.0733 0.0833 0.7930 0.7745 0.7607 0.7565
631 0.2697 0.2667 0.1574 0.1340 0.8137 0.7986 0.8234 0.8061
424 0.3065 0.1071 0.0209 0.0429 0.7602 0.7855 0.7173 0.7628
532 0.3054 0.2472 0.1230 0.1433 0.8196 0.8067 0.7889 0.7914
325 0.2040 0.0557 0.0032 0.0071 0.7591 0.7693 0.6413 0.6666
433 0.2653 0.1462 0.0726 0.0896 0.7093 0.7404 0.7477 0.7355
541 0.2307 0.2549 0.1365 0.0997 0.7989 0.7910 0.7750 0.8010
442 0.2122 0.1137 0.0616 0.0595 0.7587 0.7579 0.7221 0.7318
334 0.2498 0.1338 0.0177 0.0311 0.7901 0.7508 0.7267 0.6929
226 0.3676 0.1157 0.0219 0.0087 0.8018 0.7175 0.6247 0.6231
451 0.2991 0.2215 0.1056 0.1999 0.7537 0.7797 0.7838 0.7173
343 0.2134 0.0988 0.0378 0.0505 0.7649 0.7121 0.6912 0.7181
235 0.2908 0.1199 0.0223 0.0211 0.7728 0.7104 0.5825 0.6142
352 0.1980 0.0616 0.0558 0.0214 0.7611 0.7578 0.6950 0.7343
361 0.1659 0.1076 0.0471 0.0493 0.7337 0.7626 0.6799 0.7402
244 0.3555 0.0848 0.0454 0.0241 0.8782 0.8537 0.7797 0.8076
253 0.2106 0.0957 0.0234 0.0355 0.8027 0.7715 0.6680 0.6925
262 0.2686 0.1114 0.0434 0.0809 0.8294 0.8259 0.7697 0.7518
mean 0.2628 0.1425 0.0611 0.0666 0.7848 0.7727 0.7258 0.7338

Table 67. OSNNet-TE-KU-normal-table_image.png
First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KU KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9939 0.9948 0.9944 0.9948 0.9948 0.9945 0.9954 0.9943
523 0.9948 0.9953 0.9951 0.9944 0.9936 0.9949 0.9942 0.9944
631 0.9947 0.9938 0.9945 0.9952 0.9949 0.9951 0.9948 0.9950
424 0.9944 0.9957 0.9960 0.9960 0.9952 0.9953 0.9953 0.9948
532 0.9953 0.9949 0.9953 0.9955 0.9944 0.9952 0.9948 0.9954
325 0.9968 0.9963 0.9965 0.9958 0.9967 0.9967 0.9965 0.9968
433 0.9974 0.9960 0.9958 0.9966 0.9967 0.9962 0.9965 0.9964
541 0.9955 0.9949 0.9943 0.9951 0.9952 0.9959 0.9941 0.9952
442 0.9954 0.9953 0.9949 0.9945 0.9949 0.9951 0.9941 0.9959
334 0.9970 0.9975 0.9968 0.9970 0.9972 0.9971 0.9973 0.9972
226 0.9984 0.9977 0.9980 0.9977 0.9970 0.9985 0.9978 0.9982
451 0.9966 0.9957 0.9971 0.9965 0.9967 0.9963 0.9964 0.9964
343 0.9969 0.9965 0.9950 0.9961 0.9960 0.9966 0.9966 0.9965
235 0.9983 0.9979 0.9985 0.9978 0.9978 0.9983 0.9982 0.9984
352 0.9970 0.9971 0.9969 0.9973 0.9969 0.9965 0.9967 0.9966
361 0.9969 0.9971 0.9967 0.9967 0.9975 0.9974 0.9972 0.9978
244 0.9982 0.9975 0.9979 0.9979 0.9980 0.9977 0.9980 0.9978
253 0.9978 0.9979 0.9980 0.9980 0.9979 0.9985 0.9979 0.9977
262 0.9982 0.9981 0.9984 0.9977 0.9981 0.9984 0.9976 0.9984
mean 0.9965 0.9963 0.9963 0.9964 0.9963 0.9965 0.9963 0.9965
Table 68. OSNNet-TE-KU-threshold-table_image.png
Results with threshold on softmax. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KU KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9988 0.9988 0.9989 0.9987 0.9988 0.9988 0.9993 0.9990
523 0.9983 0.9985 0.9987 0.9985 0.9979 0.9983 0.9982 0.9980
631 0.9990 0.9987 0.9990 0.9988 0.9992 0.9990 0.9990 0.9990
424 0.9987 0.9988 0.9989 0.9989 0.9985 0.9985 0.9990 0.9984
532 0.9989 0.9987 0.9988 0.9986 0.9987 0.9985 0.9988 0.9988
325 0.9990 0.9988 0.9989 0.9988 0.9992 0.9990 0.9989 0.9988
433 0.9993 0.9990 0.9986 0.9988 0.9990 0.9988 0.9990 0.9988
541 0.9989 0.9986 0.9988 0.9987 0.9989 0.9986 0.9986 0.9988
442 0.9988 0.9985 0.9988 0.9987 0.9987 0.9987 0.9984 0.9989
334 0.9987 0.9990 0.9987 0.9989 0.9989 0.9988 0.9988 0.9991
226 0.9994 0.9992 0.9993 0.9994 0.9989 0.9995 0.9993 0.9993
451 0.9991 0.9990 0.9993 0.9994 0.9991 0.9990 0.9991 0.9989
343 0.9990 0.9989 0.9986 0.9988 0.9987 0.9989 0.9990 0.9990
235 0.9994 0.9992 0.9995 0.9994 0.9991 0.9993 0.9993 0.9995
352 0.9991 0.9991 0.9992 0.9992 0.9992 0.9991 0.9989 0.9990
361 0.9992 0.9993 0.9991 0.9993 0.9992 0.9995 0.9992 0.9995
244 0.9996 0.9993 0.9993 0.9993 0.9993 0.9992 0.9992 0.9993
253 0.9994 0.9992 0.9993 0.9992 0.9993 0.9994 0.9993 0.9991
262 0.9995 0.9995 0.9995 0.9994 0.9997 0.9995 0.9994 0.9996
mean 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990
Table 69. OSNNet-TE-KU-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KU KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.8853 0.9531 0.9532 0.8246 0.9391 0.9188 0.9776 0.8686
523 0.8759 0.7150 0.7342 0.8080 0.8126 0.7599 0.8384 0.8652
631 0.9144 0.9568 0.9279 0.9045 0.9607 0.8792 0.9514 0.9575
424 0.6660 0.8472 0.7719 0.7907 0.7494 0.7875 0.7835 0.8130
532 0.8076 0.7966 0.7300 0.7253 0.8246 0.7645 0.8038 0.8546
325 0.8616 0.8058 0.9494 0.9348 0.7469 0.7068 0.8051 0.8815
433 0.8098 0.8935 0.8488 0.7619 0.7206 0.8389 0.8861 0.7995
541 0.9263 0.7715 0.9017 0.8177 0.8165 0.8804 0.9271 0.9222
442 0.7157 0.8917 0.8598 0.8457 0.7495 0.8956 0.8771 0.8231
334 0.7675 0.9152 0.8670 0.8481 0.9816 0.8369 0.7886 0.8207
226 0.7044 0.9775 0.9194 0.6452 0.9993 0.6779 0.7912 0.6342
451 0.7917 0.8713 0.9164 0.8699 0.8346 0.7711 0.9127 0.9084
343 0.9380 0.8961 0.8788 0.7781 0.8409 0.8463 0.8443 0.9459
235 0.7753 0.7327 0.7997 0.8536 0.8667 0.7786 0.7379 0.6043
352 0.8603 0.8532 0.9617 0.8651 0.7796 0.9188 0.8145 0.9496
361 0.9221 0.8692 0.8665 0.8921 0.8407 0.8477 0.8062 0.7543
244 0.8367 0.9753 0.7732 0.7223 0.7677 0.8731 0.7656 0.9220
253 0.7561 0.6732 0.9154 0.8497 0.7835 0.8164 0.8740 0.4923
262 0.8814 0.8721 0.9060 0.8402 0.7540 0.7788 0.7988 0.9479
mean 0.8261 0.8562 0.8674 0.8199 0.8299 0.8198 0.8413 0.8297
Table 70. OSNNet-TE-KU-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KU KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.8348 0.8345 0.8584 0.8203 0.7846 0.9111 0.8043 0.8406
523 0.6555 0.7407 0.7673 0.8128 0.7914 0.7005 0.7525 0.8971
631 0.8088 0.8526 0.8506 0.8785 0.7959 0.7979 0.7822 0.8661
424 0.7361 0.8192 0.7138 0.8282 0.7451 0.8003 0.7478 0.8186
532 0.7657 0.7370 0.7701 0.6300 0.7528 0.6751 0.6684 0.6987
325 0.8110 0.7717 0.7831 0.8472 0.7805 0.6992 0.7090 0.7943
433 0.6151 0.7757 0.6312 0.8330 0.6307 0.6010 0.7999 0.6476
541 0.7303 0.6908 0.8155 0.6925 0.7490 0.7293 0.7471 0.7331
442 0.7559 0.7467 0.7740 0.7640 0.6691 0.8187 0.6808 0.7463
334 0.7797 0.8600 0.7919 0.6600 0.7397 0.7040 0.6877 0.8128
226 0.7044 0.9775 0.9194 0.6452 0.9993 0.6779 0.7912 0.6342
451 0.8482 0.7402 0.7558 0.8012 0.7548 0.7535 0.8115 0.7696
343 0.7929 0.7216 0.8041 0.7478 0.7458 0.8002 0.7769 0.8126
235 0.7753 0.7327 0.7997 0.8536 0.8667 0.7786 0.7379 0.6043
352 0.7418 0.7850 0.8609 0.8097 0.6856 0.7539 0.8145 0.7130
361 0.7656 0.6736 0.7607 0.5869 0.6605 0.6691 0.6012 0.7305
244 0.8367 0.9753 0.7732 0.7223 0.7677 0.8731 0.7656 0.9220
253 0.7561 0.6732 0.9154 0.8497 0.7835 0.8164 0.8740 0.4923
262 0.8814 0.8721 0.9060 0.8402 0.7540 0.7788 0.7988 0.9479
mean 0.7682 0.7884 0.8027 0.7696 0.7609 0.7547 0.7553 0.7622
Table 71. OSNNet-TE-KU-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KU KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9854 0.9893 0.9905 0.9882 0.9857 0.9858 0.9872 0.9863
523 0.9843 0.9875 0.9861 0.9841 0.9878 0.9855 0.9873 0.9860
631 0.9884 0.9888 0.9837 0.9895 0.9885 0.9851 0.9861 0.9889
424 0.9868 0.9903 0.9901 0.9906 0.9914 0.9905 0.9907 0.9898
532 0.9802 0.9790 0.9851 0.9826 0.9844 0.9814 0.9862 0.9877
325 0.9890 0.9896 0.9894 0.9871 0.9858 0.9888 0.9894 0.9898
433 0.9873 0.9874 0.9863 0.9881 0.9829 0.9841 0.9894 0.9893
541 0.9801 0.9806 0.9757 0.9758 0.9797 0.9786 0.9823 0.9777
442 0.9789 0.9764 0.9774 0.9783 0.9776 0.9809 0.9795 0.9798
334 0.9860 0.9807 0.9853 0.9838 0.9857 0.9813 0.9847 0.9761
226 0.9889 0.9900 0.9825 0.9814 0.9882 0.9911 0.9903 0.9895
451 0.9810 0.9808 0.9784 0.9838 0.9798 0.9817 0.9823 0.9806
343 0.9803 0.9783 0.9761 0.9795 0.9796 0.9773 0.9784 0.9822
235 0.9852 0.9826 0.9813 0.9878 0.9868 0.9865 0.9863 0.9883
352 0.9725 0.9780 0.9740 0.9766 0.9771 0.9795 0.9802 0.9742
361 0.9680 0.9738 0.9699 0.9633 0.9713 0.9632 0.9712 0.9664
244 0.9876 0.9840 0.9642 0.9870 0.9881 0.9874 0.9837 0.9871
253 0.9570 0.9654 0.9807 0.9739 0.9695 0.9420 0.9774 0.9721
262 0.9743 0.9769 0.9716 0.9188 0.9475 0.9802 0.9755 0.9671
mean 0.9811 0.9821 0.9804 0.9790 0.9809 0.9806 0.9836 0.9821
Table 72. OSNNet-TE-KU-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KU KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.6392 0.6941 0.7007 0.7228 0.6580 0.6480 0.6329 0.7299
523 0.7385 0.8141 0.7118 0.5954 0.6960 0.7509 0.8047 0.8185
631 0.6685 0.5455 0.4956 0.5834 0.4867 0.5680 0.3460 0.4816
424 0.8774 0.9390 0.9500 0.9394 0.9636 0.8982 0.9786 0.9309
532 0.7394 0.5769 0.6693 0.5551 0.5656 0.5696 0.6458 0.6769
325 0.9856 0.9896 0.9815 0.9860 0.9233 0.9577 0.8997 0.9643
433 0.7699 0.8716 0.8676 0.7796 0.8392 0.7943 0.7484 0.8514
541 0.7780 0.5996 0.6408 0.7587 0.5596 0.6133 0.6985 0.6243
442 0.8669 0.8638 0.8646 0.8406 0.8374 0.7892 0.8979 0.8610
334 0.9418 0.8813 0.9762 0.9605 0.9582 0.9496 0.8652 0.9663
226 0.9889 0.9900 0.9825 0.9814 0.9882 0.9911 0.9903 0.9895
451 0.8485 0.7996 0.8068 0.6520 0.7692 0.7280 0.8160 0.9033
343 0.9138 0.9608 0.9302 0.9407 0.8829 0.9600 0.9368 0.8927
235 0.9852 0.9826 0.9813 0.9878 0.9868 0.9865 0.9863 0.9883
352 0.9483 0.9686 0.8730 0.9577 0.8273 0.9106 0.8653 0.8752
361 0.9083 0.8862 0.9039 0.9094 0.9498 0.8487 0.9651 0.8525
244 0.9876 0.9840 0.9642 0.9870 0.9881 0.9874 0.9837 0.9871
253 0.9570 0.9654 0.9807 0.9739 0.9695 0.9420 0.9774 0.9721
262 0.9743 0.9769 0.9716 0.9188 0.9475 0.9802 0.9755 0.9671
mean 0.8693 0.8573 0.8554 0.8437 0.8314 0.8354 0.8428 0.8596

Table 73. OSNNet-TE-KUl-normal-table_image.png
First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUl KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.3574 0.9768 0.9989 0.9995 0.0253 0.3439 0.9955 0.9961
523 0.4987 0.9749 0.9993 0.9990 0.0137 0.3266 0.9956 0.9948
631 0.6312 0.9863 0.9995 0.9993 0.0237 0.4926 0.9923 0.9954
424 0.5221 0.9835 0.9991 0.9992 0.0272 0.4432 0.9938 0.9962
532 0.7684 0.9907 0.9992 0.9990 0.0369 0.4604 0.9953 0.9948
325 0.7782 0.9811 0.9995 0.9987 0.0261 0.4076 0.9966 0.9960
433 0.7747 0.9917 0.9994 0.9998 0.0494 0.5477 0.9965 0.9973
541 0.8245 0.9935 0.9997 0.9997 0.0455 0.6168 0.9936 0.9956
442 0.9036 0.9936 0.9996 0.9995 0.0438 0.6919 0.9957 0.9964
334 0.7907 0.9918 0.9995 0.9995 0.0527 0.6878 0.9971 0.9977
226 0.6529 0.9777 0.9993 0.9994 0.0455 0.6515 0.9970 0.9963
451 0.9599 0.9959 0.9998 0.9996 0.0652 0.7379 0.9951 0.9979
343 0.9323 0.9955 0.9995 0.9999 0.0617 0.7543 0.9965 0.9970
235 0.9227 0.9922 0.9998 0.9997 0.0534 0.7242 0.9984 0.9989
352 0.9229 0.9982 0.9997 0.9997 0.0543 0.6950 0.9978 0.9978
361 0.9585 0.9954 0.9997 0.9997 0.0718 0.7402 0.9953 0.9966
244 0.8747 0.9941 0.9998 0.9997 0.0776 0.7744 0.9983 0.9983
253 0.9800 0.9979 0.9999 0.9996 0.0653 0.8055 0.9972 0.9981
262 0.9773 0.9987 1.0000 0.9999 0.0887 0.9074 0.9977 0.9989
mean 0.7911 0.9900 0.9995 0.9995 0.0488 0.6215 0.9961 0.9969
Table 74. OSNNet-TE-KUl-threshold-table_image.png
Results with threshold on softmax. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUl KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9368 0.9962 0.9999 0.9998 0.8309 0.8916 0.9995 0.9996
523 0.9379 0.9953 0.9999 0.9996 0.7506 0.8605 0.9995 0.9993
631 0.9742 0.9979 0.9999 0.9999 0.8713 0.9437 0.9996 0.9997
424 0.8984 0.9957 0.9997 0.9998 0.6918 0.8557 0.9990 0.9996
532 0.9734 0.9979 0.9999 1.0000 0.7979 0.9121 0.9996 0.9996
325 0.9135 0.9959 0.9998 0.9998 0.6052 0.8053 0.9996 0.9993
433 0.9619 0.9984 0.9999 1.0000 0.7563 0.9109 0.9997 0.9997
541 0.9866 0.9990 0.9998 0.9999 0.8428 0.9479 0.9998 0.9997
442 0.9926 0.9985 0.9999 0.9999 0.7905 0.9514 0.9995 0.9996
334 0.9618 0.9981 0.9999 1.0000 0.6919 0.9243 0.9997 0.9999
226 0.9051 0.9923 0.9998 0.9998 0.4670 0.8713 0.9996 0.9994
451 0.9959 0.9993 1.0000 0.9999 0.8068 0.9603 0.9997 0.9997
343 0.9923 0.9992 0.9999 0.9999 0.7078 0.9538 0.9997 0.9996
235 0.9851 0.9976 0.9999 0.9999 0.5040 0.9186 0.9998 0.9999
352 0.9912 0.9997 1.0000 0.9999 0.7356 0.9364 0.9998 0.9997
361 0.9944 0.9990 1.0000 0.9998 0.7525 0.9495 0.9997 0.9996
244 0.9778 0.9977 0.9999 0.9999 0.5342 0.9298 0.9997 0.9996
253 0.9967 0.9996 0.9999 0.9999 0.5175 0.9598 0.9996 0.9998
262 0.9972 0.9998 1.0000 0.9999 0.5861 0.9817 0.9999 0.9999
mean 0.9670 0.9977 0.9999 0.9999 0.6969 0.9192 0.9996 0.9996
Table 75. OSNNet-TE-KUl-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUl KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9735 0.9703 0.9773 0.8932 0.9601 0.9258 0.9928 0.9885
523 0.9231 0.9381 0.7457 0.8605 0.8699 0.8967 0.9756 0.9845
631 0.9742 0.9882 0.9755 0.9115 0.9671 0.9562 0.9896 0.9911
424 0.8646 0.8985 0.7278 0.8418 0.8064 0.8111 0.9159 0.8837
532 0.9474 0.9578 0.6864 0.6629 0.9187 0.9075 0.9722 0.9756
325 0.7708 0.8992 0.9409 0.9892 0.6280 0.6521 0.9105 0.9271
433 0.9008 0.9833 0.8979 0.8013 0.8495 0.8466 0.9700 0.9726
541 0.9654 0.9212 0.8990 0.7685 0.8940 0.8307 0.9347 0.9761
442 0.9006 0.9829 0.9054 0.8995 0.7754 0.8089 0.9606 0.9565
334 0.8660 0.8580 0.9141 0.8360 0.5897 0.7363 0.8156 0.8550
226 0.7408 0.9793 0.8968 0.6510 0.5696 0.6752 0.8194 0.5958
451 0.8956 0.9826 0.9137 0.8057 0.9232 0.8443 0.9516 0.9578
343 0.9366 0.9110 0.9178 0.7210 0.7308 0.7732 0.9743 0.9676
235 0.7743 0.7257 0.8497 0.7486 0.3396 0.6970 0.7766 0.6302
352 0.8403 0.9742 0.9922 0.9599 0.7115 0.8083 0.9098 0.9630
361 0.9324 0.9467 0.9304 0.9889 0.7502 0.7734 0.9746 0.9019
244 0.6864 0.9664 0.9113 0.7908 0.4417 0.7315 0.8310 0.9109
253 0.8338 0.6502 0.8182 0.8510 0.4330 0.6316 0.8869 0.6785
262 0.9568 0.8556 0.8519 0.8308 0.4114 0.7154 0.7568 0.9210
mean 0.8781 0.9152 0.8817 0.8322 0.7142 0.7906 0.9115 0.8967
Table 76. OSNNet-TE-KUl-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUl KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9792 0.9576 0.8866 0.8697 0.9486 0.9532 0.9867 0.9882
523 0.8912 0.9209 0.7893 0.8971 0.8989 0.9028 0.9674 0.9803
631 0.9455 0.9668 0.8021 0.9383 0.9586 0.9622 0.9863 0.9849
424 0.8330 0.9143 0.6511 0.7562 0.7692 0.7971 0.8756 0.9081
532 0.9595 0.9341 0.8890 0.6062 0.9158 0.8952 0.9860 0.9460
325 0.7383 0.8202 0.8978 0.9068 0.5586 0.6536 0.8742 0.9169
433 0.9185 0.9649 0.7780 0.8663 0.8467 0.8706 0.9600 0.9533
541 0.9565 0.9089 0.8542 0.6999 0.9336 0.8811 0.9240 0.9311
442 0.9107 0.9148 0.7692 0.7945 0.8493 0.8678 0.9532 0.9576
334 0.8520 0.9149 0.7436 0.7666 0.5552 0.6999 0.7946 0.8783
226 0.7408 0.9793 0.8968 0.6510 0.5696 0.6752 0.8194 0.5958
451 0.9100 0.9178 0.7366 0.7989 0.8505 0.8912 0.9438 0.9519
343 0.8065 0.8343 0.7053 0.8399 0.6203 0.7364 0.9136 0.9063
235 0.7743 0.7257 0.8497 0.7486 0.3396 0.6970 0.7766 0.6302
352 0.7785 0.9148 0.9569 0.8863 0.6086 0.6688 0.9551 0.8684
361 0.9173 0.7612 0.7341 0.6785 0.6580 0.7190 0.9249 0.8281
244 0.6864 0.9664 0.9113 0.7908 0.4417 0.7315 0.8310 0.9109
253 0.8338 0.6502 0.8182 0.8510 0.4330 0.6316 0.8869 0.6785
262 0.9568 0.8556 0.8519 0.8308 0.4114 0.7154 0.7568 0.9210
mean 0.8626 0.8854 0.8169 0.7988 0.6930 0.7868 0.9009 0.8808
Table 77. OSNNet-TE-KUl-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUl KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9391 0.9888 0.9992 0.9996 0.8359 0.8405 0.9555 0.9661
523 0.8931 0.9866 0.9991 0.9991 0.7197 0.7834 0.9617 0.9665
631 0.9497 0.9937 0.9987 0.9996 0.8394 0.8773 0.9575 0.9691
424 0.8691 0.9876 0.9992 0.9986 0.6885 0.7597 0.9727 0.9772
532 0.9144 0.9846 0.9991 0.9990 0.7663 0.8015 0.9582 0.9488
325 0.8230 0.9798 0.9988 0.9986 0.5212 0.6731 0.9767 0.9749
433 0.9264 0.9927 0.9994 0.9995 0.7354 0.8191 0.9702 0.9666
541 0.9456 0.9913 0.9974 0.9981 0.8027 0.8249 0.9482 0.9418
442 0.9468 0.9885 0.9986 0.9991 0.7155 0.8183 0.9646 0.9503
334 0.9037 0.9851 0.9982 0.9981 0.5718 0.7528 0.9644 0.9488
226 0.7306 0.9766 0.9978 0.9984 0.3829 0.7321 0.9885 0.9854
451 0.9430 0.9902 0.9983 0.9986 0.7401 0.8124 0.9634 0.9476
343 0.9194 0.9882 0.9986 0.9989 0.5791 0.7896 0.9686 0.9510
235 0.8738 0.9753 0.9984 0.9986 0.3673 0.7468 0.9860 0.9805
352 0.9118 0.9901 0.9982 0.9982 0.6027 0.7599 0.9419 0.9320
361 0.9139 0.9863 0.9973 0.9977 0.5797 0.7213 0.9435 0.9254
244 0.8502 0.9833 0.9959 0.9983 0.3962 0.7128 0.9661 0.9500
253 0.8916 0.9724 0.9975 0.9961 0.3521 0.5841 0.9237 0.9065
262 0.8751 0.9862 0.9978 0.9891 0.3658 0.7615 0.9501 0.9135
mean 0.8958 0.9856 0.9983 0.9981 0.6085 0.7669 0.9611 0.9527
Table 78. OSNNet-TE-KUl-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUl KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.5125 0.7442 0.8633 0.8099 0.3417 0.3467 0.4411 0.6364
523 0.5491 0.8425 0.9236 0.7338 0.2986 0.4246 0.7031 0.7156
631 0.4971 0.5950 0.7625 0.6949 0.2345 0.3545 0.2892 0.3893
424 0.6568 0.9294 0.9938 0.9829 0.5296 0.5428 0.9221 0.8525
532 0.5658 0.6894 0.7894 0.6589 0.2817 0.3243 0.5587 0.4952
325 0.8007 0.9798 0.9983 0.9983 0.5062 0.5476 0.8858 0.9148
433 0.6674 0.8968 0.9492 0.8857 0.5258 0.6032 0.6812 0.7875
541 0.6565 0.7002 0.7502 0.8847 0.3156 0.3961 0.5824 0.4972
442 0.7879 0.9259 0.9690 0.9808 0.4738 0.5599 0.8894 0.7275
334 0.8293 0.9285 0.9966 0.9956 0.4747 0.7047 0.8222 0.9182
226 0.7306 0.9766 0.9978 0.9984 0.3829 0.7321 0.9885 0.9854
451 0.7448 0.8762 0.9106 0.7757 0.4492 0.4955 0.7356 0.7944
343 0.8549 0.9635 0.9936 0.9932 0.4764 0.7224 0.9232 0.7813
235 0.8738 0.9753 0.9984 0.9986 0.3673 0.7468 0.9860 0.9805
352 0.8370 0.9854 0.9780 0.9921 0.4480 0.6355 0.8104 0.8203
361 0.8676 0.9235 0.9652 0.9906 0.5264 0.6156 0.9303 0.7973
244 0.8502 0.9833 0.9959 0.9983 0.3962 0.7128 0.9661 0.9500
253 0.8916 0.9724 0.9975 0.9961 0.3521 0.5841 0.9237 0.9065
262 0.8751 0.9862 0.9978 0.9891 0.3658 0.7615 0.9501 0.9135
mean 0.7394 0.8881 0.9385 0.9136 0.4077 0.5690 0.7889 0.7823

Table 79. OSNNet-TE-KUn-normal-table_image.png
First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUn KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.5069 0.9984 1.0000 1.0000 0.1208 0.8605 0.9889 0.9951
523 0.6579 1.0000 1.0000 1.0000 0.1292 0.9336 0.9891 0.9961
631 0.7418 0.9988 1.0000 1.0000 0.1198 0.8930 0.9755 0.9928
424 0.6866 0.9970 1.0000 1.0000 0.1177 0.9005 0.9860 0.9952
532 0.8281 0.9957 1.0000 1.0000 0.1085 0.8311 0.9957 0.9955
325 0.8931 0.9986 1.0000 1.0000 0.2067 0.9322 0.9862 0.9921
433 0.8689 0.9958 1.0000 1.0000 0.1577 0.9161 0.9925 0.9921
541 0.8748 0.9991 0.9991 1.0000 0.1483 0.9407 0.9773 0.9927
442 0.9524 0.9991 1.0000 1.0000 0.1928 0.9572 0.9880 0.9946
334 0.8765 0.9963 0.9988 0.9976 0.1362 0.9454 0.9932 0.9991
226 0.7592 0.9976 1.0000 1.0000 0.1583 0.9274 0.9974 0.9990
451 0.9781 0.9969 1.0000 1.0000 0.1225 0.9442 0.9858 0.9952
343 0.9469 0.9991 1.0000 0.9992 0.1371 0.9781 0.9892 0.9984
235 0.9826 0.9987 1.0000 1.0000 0.1904 0.9587 0.9959 0.9987
352 0.9588 1.0000 1.0000 1.0000 0.1472 0.9500 0.9955 0.9993
361 0.9881 1.0000 1.0000 1.0000 0.1747 0.9784 0.9972 0.9938
244 0.9018 1.0000 1.0000 1.0000 0.1670 0.9599 0.9982 0.9962
253 0.9944 0.9993 1.0000 1.0000 0.1726 0.9736 0.9853 0.9987
262 0.9895 0.9995 1.0000 1.0000 0.1652 0.9767 0.9924 0.9974
mean 0.8624 0.9984 0.9999 0.9998 0.1512 0.9346 0.9900 0.9959
Table 80. OSNNet-TE-KUn-threshold-table_image.png
Results with threshold on softmax. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUn KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9577 1.0000 1.0000 1.0000 0.9060 0.9889 0.9990 1.0000
523 0.9600 1.0000 1.0000 1.0000 0.8388 0.9980 0.9980 1.0000
631 0.9881 1.0000 1.0000 1.0000 0.9161 0.9906 0.9988 1.0000
424 0.9586 1.0000 1.0000 1.0000 0.8047 0.9916 0.9958 1.0000
532 0.9859 1.0000 1.0000 1.0000 0.8661 0.9927 1.0000 1.0000
325 0.9354 0.9986 1.0000 1.0000 0.7713 0.9893 1.0000 0.9982
433 0.9783 0.9989 1.0000 1.0000 0.8304 0.9933 0.9981 0.9989
541 0.9949 1.0000 1.0000 1.0000 0.9003 0.9975 0.9985 0.9991
442 0.9983 1.0000 1.0000 1.0000 0.8966 0.9983 0.9974 0.9991
334 0.9917 1.0000 1.0000 1.0000 0.7676 0.9931 1.0000 1.0000
226 0.9532 0.9976 1.0000 1.0000 0.5664 0.9847 1.0000 1.0000
451 0.9970 1.0000 1.0000 1.0000 0.8712 0.9961 0.9977 0.9995
343 0.9942 1.0000 1.0000 1.0000 0.7985 0.9991 0.9992 1.0000
235 1.0000 1.0000 1.0000 1.0000 0.6498 0.9969 1.0000 1.0000
352 0.9948 1.0000 1.0000 1.0000 0.8174 1.0000 1.0000 1.0000
361 0.9994 1.0000 1.0000 1.0000 0.8349 0.9987 1.0000 0.9995
244 0.9781 1.0000 1.0000 1.0000 0.6638 0.9947 0.9992 1.0000
253 0.9984 1.0000 1.0000 1.0000 0.6409 0.9984 0.9977 0.9993
262 0.9983 1.0000 1.0000 1.0000 0.6446 0.9984 0.9995 0.9995
mean 0.9822 0.9997 1.0000 1.0000 0.7887 0.9948 0.9989 0.9996
Table 81. OSNNet-TE-KUn-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUn KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9740 0.9691 0.9760 0.9147 0.9764 0.9902 0.9974 0.9921
523 0.8790 0.9509 0.7335 0.8247 0.8336 0.9548 0.9175 0.9502
631 0.9674 0.9926 0.9675 0.9072 0.9854 0.9870 0.9838 0.9893
424 0.8858 0.8892 0.7517 0.8627 0.8275 0.8663 0.8787 0.9176
532 0.9556 0.9734 0.6647 0.7227 0.9551 0.9772 0.9773 0.9799
325 0.8463 0.9022 0.9638 0.9950 0.6828 0.8187 0.8401 0.9331
433 0.9287 0.9739 0.8851 0.7370 0.8676 0.9124 0.9436 0.9509
541 0.9608 0.9264 0.9069 0.7843 0.9324 0.9391 0.8990 0.9743
442 0.8817 0.9846 0.8933 0.8799 0.7699 0.9174 0.9531 0.9249
334 0.8788 0.8847 0.9145 0.8471 0.6298 0.8528 0.7721 0.8477
226 0.7489 0.9607 0.8944 0.6378 0.5932 0.6971 0.7795 0.5657
451 0.9053 0.9738 0.9206 0.7806 0.9192 0.9439 0.9187 0.9490
343 0.9435 0.9101 0.9112 0.7361 0.7878 0.9298 0.9600 0.9639
235 0.7915 0.7495 0.8344 0.7453 0.4459 0.7641 0.7551 0.6039
352 0.9038 0.9662 0.9918 0.9479 0.7619 0.9027 0.8796 0.9638
361 0.9301 0.9445 0.9237 0.9764 0.8159 0.8418 0.9486 0.8860
244 0.7124 0.9918 0.9540 0.8095 0.5460 0.8008 0.8181 0.9124
253 0.8294 0.6536 0.8134 0.8760 0.4834 0.7103 0.8720 0.6698
262 0.9566 0.8754 0.8633 0.8339 0.4262 0.7450 0.7435 0.9274
mean 0.8884 0.9196 0.8823 0.8326 0.7495 0.8711 0.8862 0.8896
Table 82. OSNNet-TE-KUn-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUn KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9805 0.9686 0.8906 0.8594 0.9755 0.9776 0.9954 0.9953
523 0.8773 0.9300 0.7940 0.8783 0.8564 0.9234 0.9125 0.9511
631 0.9605 0.9674 0.8398 0.9199 0.9837 0.9875 0.9759 0.9838
424 0.8427 0.8978 0.6686 0.7319 0.8584 0.9371 0.8292 0.8806
532 0.9604 0.9538 0.8829 0.6229 0.9600 0.9632 0.9861 0.9563
325 0.8127 0.8130 0.8978 0.9117 0.6848 0.8294 0.8157 0.8436
433 0.9053 0.9446 0.7368 0.8415 0.8741 0.9176 0.9130 0.9176
541 0.9571 0.9116 0.8607 0.6972 0.9609 0.9103 0.8911 0.9263
442 0.9183 0.9182 0.7176 0.7890 0.8480 0.9195 0.9253 0.9250
334 0.8754 0.9289 0.7463 0.7766 0.6013 0.8408 0.7890 0.8496
226 0.7489 0.9607 0.8944 0.6378 0.5932 0.6971 0.7795 0.5657
451 0.9103 0.9134 0.7127 0.7956 0.8953 0.9486 0.9359 0.9361
343 0.7936 0.8280 0.7061 0.8003 0.6900 0.9142 0.9050 0.9246
235 0.7915 0.7495 0.8344 0.7453 0.4459 0.7641 0.7551 0.6039
352 0.8028 0.8933 0.9492 0.8780 0.6651 0.8386 0.9119 0.8396
361 0.9005 0.7505 0.7173 0.6558 0.7092 0.8147 0.8892 0.7707
244 0.7124 0.9918 0.9540 0.8095 0.5460 0.8008 0.8181 0.9124
253 0.8294 0.6536 0.8134 0.8760 0.4834 0.7103 0.8720 0.6698
262 0.9566 0.8754 0.8633 0.8339 0.4262 0.7450 0.7435 0.9274
mean 0.8703 0.8869 0.8147 0.7927 0.7399 0.8652 0.8760 0.8621
Table 83. OSNNet-TE-KUn-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUn KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9510 0.9952 0.9984 0.9957 0.8742 0.9610 0.9479 0.9751
523 0.8793 0.9917 1.0000 1.0000 0.7940 0.9608 0.9421 0.9505
631 0.9711 0.9965 0.9977 1.0000 0.9054 0.9649 0.9496 0.9793
424 0.9050 0.9971 0.9988 0.9974 0.7702 0.9554 0.9753 0.9767
532 0.9218 0.9885 0.9989 1.0000 0.8586 0.9491 0.9599 0.9571
325 0.8864 0.9958 1.0000 1.0000 0.6288 0.9314 0.9244 0.9653
433 0.9324 0.9969 0.9969 0.9969 0.7815 0.9451 0.9284 0.9456
541 0.9564 0.9927 0.9928 0.9890 0.8562 0.9407 0.9542 0.9458
442 0.9547 0.9974 0.9974 0.9991 0.7843 0.9614 0.9477 0.9376
334 0.9416 0.9912 0.9988 0.9976 0.6075 0.9273 0.9119 0.9279
226 0.7782 0.9976 1.0000 1.0000 0.4823 0.9006 0.9853 0.9632
451 0.9562 0.9900 0.9965 0.9982 0.8083 0.9395 0.9287 0.9425
343 0.9146 0.9949 0.9992 0.9992 0.6694 0.9625 0.9564 0.9360
235 0.9262 0.9956 0.9987 1.0000 0.4805 0.9305 0.9611 0.9675
352 0.9365 0.9962 0.9969 1.0000 0.6544 0.9200 0.9091 0.9112
361 0.9156 0.9901 0.9964 0.9967 0.6716 0.8962 0.9311 0.9338
244 0.8852 0.9949 0.9953 0.9992 0.4650 0.9296 0.9626 0.9499
253 0.8956 0.9822 0.9987 0.9980 0.4509 0.7824 0.9105 0.9111
262 0.8969 0.9944 0.9984 0.9836 0.4061 0.8440 0.9124 0.8727
mean 0.9160 0.9936 0.9979 0.9974 0.6815 0.9264 0.9420 0.9447
Table 84. OSNNet-TE-KUn-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
KUn KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.5015 0.7437 0.8568 0.7712 0.3610 0.4965 0.4726 0.6416
523 0.5392 0.8136 0.8679 0.6707 0.3405 0.6591 0.6145 0.6239
631 0.5045 0.5803 0.7431 0.6749 0.2583 0.5039 0.2591 0.3700
424 0.6810 0.9470 0.9960 0.9697 0.5958 0.7879 0.8999 0.8391
532 0.6036 0.6894 0.8174 0.6841 0.3509 0.5007 0.5932 0.5296
325 0.8742 0.9958 1.0000 1.0000 0.6205 0.8314 0.8251 0.8834
433 0.6757 0.9143 0.9192 0.8753 0.5868 0.7543 0.6544 0.7677
541 0.6538 0.7111 0.7472 0.8572 0.3443 0.5048 0.5376 0.4939
442 0.7825 0.9156 0.9580 0.9612 0.5325 0.7189 0.8243 0.7215
334 0.8593 0.9262 0.9960 0.9924 0.5137 0.8875 0.7694 0.9035
226 0.7782 0.9976 1.0000 1.0000 0.4823 0.9006 0.9853 0.9632
451 0.7544 0.8900 0.9072 0.7644 0.5008 0.6235 0.6698 0.7685
343 0.8539 0.9764 0.9877 0.9891 0.5505 0.9110 0.8747 0.7875
235 0.9262 0.9956 0.9987 1.0000 0.4805 0.9305 0.9611 0.9675
352 0.8662 0.9920 0.9599 0.9930 0.4988 0.8148 0.7574 0.7959
361 0.8779 0.9329 0.9525 0.9803 0.6093 0.7872 0.9185 0.8004
244 0.8852 0.9949 0.9953 0.9992 0.4650 0.9296 0.9626 0.9499
253 0.8956 0.9822 0.9987 0.9980 0.4509 0.7824 0.9105 0.9111
262 0.8969 0.9944 0.9984 0.9836 0.4061 0.8440 0.9124 0.8727
mean 0.7584 0.8944 0.9316 0.9034 0.4710 0.7457 0.7580 0.7679

Table 85. OSNNet-TE-UU-normal-table_image.png
First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
UU KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.1480 0.1553 0.1809 0.1735 0.1708 0.1488 0.2187 0.2194
523 0.3176 0.3404 0.3441 0.3300 0.3030 0.3416 0.3479 0.3556
631 0.4797 0.4670 0.4191 0.4642 0.4353 0.4300 0.4325 0.4700
424 0.3003 0.3362 0.3480 0.3307 0.3206 0.3033 0.3206 0.3286
532 0.5229 0.4718 0.4882 0.5047 0.4695 0.5132 0.5219 0.5283
325 0.4258 0.4173 0.4226 0.4004 0.3966 0.4226 0.4286 0.4278
433 0.5069 0.4815 0.4666 0.5106 0.4863 0.4844 0.4951 0.5090
541 0.5360 0.5162 0.4988 0.5162 0.5347 0.5535 0.5162 0.5282
442 0.6064 0.5992 0.5813 0.6245 0.5970 0.5893 0.5805 0.6209
334 0.5076 0.4969 0.5350 0.5340 0.5305 0.5447 0.5528 0.5304
226 0.4962 0.4401 0.4888 0.4694 0.4552 0.4696 0.4784 0.4880
451 0.6648 0.6717 0.6911 0.7068 0.7300 0.6861 0.6970 0.6835
343 0.6655 0.6660 0.6231 0.6696 0.6586 0.6781 0.6762 0.6576
235 0.6822 0.6420 0.6846 0.6533 0.6374 0.6558 0.6861 0.6842
352 0.8213 0.8523 0.8404 0.8405 0.8426 0.8215 0.8260 0.8271
361 0.8389 0.8651 0.8734 0.8514 0.8592 0.8578 0.8686 0.8635
244 0.7403 0.6975 0.7148 0.7231 0.7216 0.7098 0.7371 0.7384
253 0.8098 0.7984 0.8132 0.8157 0.7951 0.8351 0.8204 0.8179
262 0.8847 0.8793 0.8766 0.8753 0.8564 0.8736 0.8500 0.8675
mean 0.5766 0.5681 0.5732 0.5786 0.5684 0.5747 0.5818 0.5866
Table 86. OSNNet-TE-UU-threshold-table_image.png
Results with threshold on softmax. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
UU KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.6026 0.6167 0.6444 0.6428 0.6211 0.6185 0.6614 0.6414
523 0.6898 0.6799 0.6899 0.6885 0.6703 0.6919 0.7060 0.6979
631 0.7682 0.7784 0.7260 0.7516 0.7497 0.7267 0.7533 0.7655
424 0.6001 0.6287 0.6378 0.6301 0.6036 0.5986 0.6204 0.6052
532 0.8094 0.7709 0.7955 0.8019 0.7816 0.8232 0.8184 0.8243
325 0.6380 0.6416 0.6429 0.6151 0.6113 0.6370 0.6426 0.6280
433 0.7285 0.7230 0.7107 0.7235 0.7072 0.7262 0.7320 0.7444
541 0.8081 0.8076 0.7834 0.8019 0.8026 0.8038 0.7941 0.8011
442 0.8427 0.8512 0.8377 0.8688 0.8406 0.8433 0.8407 0.8730
334 0.7203 0.7260 0.7584 0.7491 0.7409 0.7451 0.7580 0.7404
226 0.6642 0.6200 0.6696 0.6403 0.6292 0.6424 0.6524 0.6557
451 0.8842 0.8775 0.8881 0.9068 0.8978 0.8901 0.8921 0.8834
343 0.8446 0.8461 0.8218 0.8546 0.8419 0.8435 0.8552 0.8427
235 0.8028 0.7729 0.8040 0.7903 0.7736 0.7823 0.7998 0.8073
352 0.9396 0.9515 0.9501 0.9510 0.9492 0.9401 0.9390 0.9471
361 0.9432 0.9435 0.9525 0.9411 0.9486 0.9495 0.9585 0.9461
244 0.8591 0.8236 0.8369 0.8472 0.8464 0.8302 0.8489 0.8554
253 0.9031 0.8872 0.8971 0.9017 0.8868 0.9148 0.9058 0.9028
262 0.9517 0.9468 0.9443 0.9483 0.9373 0.9451 0.9304 0.9408
mean 0.7895 0.7838 0.7890 0.7924 0.7810 0.7870 0.7952 0.7949
Table 87. OSNNet-TE-UU-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
UU KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.6102 0.7278 0.6252 0.6150 0.5843 0.6041 0.6633 0.6167
523 0.6519 0.5574 0.6244 0.6236 0.6173 0.5947 0.5474 0.6227
631 0.5197 0.6537 0.7648 0.6103 0.5963 0.5454 0.6598 0.5463
424 0.4729 0.4528 0.5071 0.5434 0.4663 0.5836 0.4882 0.4962
532 0.6580 0.6430 0.5823 0.6598 0.6651 0.7093 0.6299 0.6309
325 0.4789 0.4363 0.5532 0.5739 0.5041 0.4273 0.4842 0.4791
433 0.5411 0.6178 0.4978 0.5472 0.5506 0.5325 0.5935 0.5920
541 0.6636 0.6220 0.5945 0.6858 0.5910 0.6176 0.7425 0.5764
442 0.5459 0.5865 0.5698 0.5835 0.6363 0.5800 0.6495 0.5486
334 0.5972 0.5835 0.6285 0.6073 0.6103 0.6558 0.5779 0.6086
226 0.5815 0.7407 0.7184 0.4403 0.7431 0.4298 0.6069 0.5101
451 0.5802 0.6124 0.6950 0.6481 0.5932 0.6830 0.6896 0.5856
343 0.7213 0.6976 0.6603 0.6396 0.6856 0.6604 0.7034 0.7357
235 0.7210 0.6097 0.6788 0.7262 0.6850 0.6355 0.6977 0.5222
352 0.7862 0.7531 0.8136 0.7766 0.7042 0.7577 0.7386 0.7920
361 0.7493 0.8666 0.7220 0.8466 0.7724 0.6604 0.7124 0.6618
244 0.5860 0.7977 0.5614 0.5479 0.6642 0.7089 0.6705 0.6759
253 0.7253 0.6826 0.7719 0.8080 0.6830 0.6953 0.7796 0.5351
262 0.7851 0.7590 0.7641 0.7210 0.6593 0.7459 0.7229 0.8356
mean 0.6303 0.6526 0.6491 0.6423 0.6322 0.6225 0.6504 0.6090
Table 88. OSNNet-TE-UU-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
UU KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.5537 0.5532 0.5635 0.5488 0.5445 0.5398 0.5773 0.5479
523 0.5411 0.5973 0.5813 0.5597 0.5648 0.5286 0.5675 0.5434
631 0.4880 0.5320 0.4885 0.4674 0.5187 0.5190 0.5016 0.4945
424 0.4310 0.5205 0.4384 0.4622 0.4349 0.4647 0.4480 0.3958
532 0.6260 0.6689 0.5621 0.6219 0.6285 0.6353 0.5862 0.6188
325 0.4372 0.4005 0.4648 0.4492 0.4036 0.4057 0.3582 0.4240
433 0.4427 0.4586 0.4764 0.5135 0.4756 0.4474 0.4482 0.4793
541 0.5707 0.5165 0.5185 0.5035 0.5065 0.5073 0.5678 0.5600
442 0.5611 0.5236 0.5971 0.6388 0.6167 0.5438 0.5336 0.6037
334 0.4752 0.4530 0.5251 0.4785 0.3878 0.4989 0.4819 0.5403
226 0.5815 0.7407 0.7184 0.4403 0.7431 0.4298 0.6069 0.5101
451 0.5890 0.5458 0.5122 0.6211 0.6484 0.6388 0.6449 0.4557
343 0.4584 0.5459 0.5093 0.5380 0.4788 0.5077 0.4821 0.4972
235 0.7210 0.6097 0.6788 0.7262 0.6850 0.6355 0.6977 0.5222
352 0.4730 0.5964 0.4870 0.6221 0.5630 0.5561 0.6601 0.4503
361 0.7052 0.6724 0.6282 0.5224 0.5828 0.5751 0.5579 0.5109
244 0.5860 0.7977 0.5614 0.5479 0.6642 0.7089 0.6705 0.6759
253 0.7253 0.6826 0.7719 0.8080 0.6830 0.6953 0.7796 0.5351
262 0.7851 0.7590 0.7641 0.7210 0.6593 0.7459 0.7229 0.8356
mean 0.5659 0.5881 0.5709 0.5679 0.5678 0.5570 0.5733 0.5369
Table 89. OSNNet-TE-UU-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
UU KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.7378 0.7129 0.7406 0.7505 0.7121 0.7222 0.7156 0.7068
523 0.6538 0.6819 0.6798 0.6975 0.6733 0.6961 0.6869 0.7045
631 0.7342 0.7342 0.7115 0.7147 0.7252 0.7176 0.7326 0.7239
424 0.5482 0.5995 0.5780 0.5841 0.5813 0.5841 0.5724 0.5581
532 0.7186 0.7274 0.7397 0.7250 0.7242 0.7391 0.7480 0.7584
325 0.5262 0.5205 0.5467 0.5184 0.4838 0.5195 0.5160 0.5200
433 0.6172 0.6240 0.6387 0.6291 0.6249 0.6283 0.6375 0.6433
541 0.7224 0.7198 0.6591 0.6952 0.7185 0.6880 0.7101 0.6891
442 0.6508 0.6562 0.6610 0.6856 0.6504 0.6477 0.6580 0.6701
334 0.5690 0.5464 0.5785 0.5711 0.5523 0.5552 0.5615 0.5622
226 0.4724 0.4599 0.4566 0.4722 0.4669 0.4782 0.4747 0.4786
451 0.7006 0.7036 0.6844 0.7343 0.6943 0.7324 0.7220 0.7170
343 0.6394 0.6500 0.6471 0.6574 0.6386 0.6440 0.6474 0.6344
235 0.5550 0.5536 0.5727 0.5498 0.5405 0.5563 0.5639 0.5594
352 0.7067 0.7251 0.7226 0.7218 0.7193 0.7290 0.7389 0.7332
361 0.7662 0.7654 0.7690 0.7617 0.7815 0.7501 0.7850 0.7528
244 0.6442 0.6242 0.6042 0.6248 0.6524 0.6282 0.5955 0.6143
253 0.6047 0.5866 0.6063 0.6081 0.5931 0.6085 0.6148 0.6013
262 0.6987 0.6913 0.6849 0.6367 0.6461 0.6929 0.6497 0.6781
mean 0.6456 0.6464 0.6464 0.6494 0.6410 0.6483 0.6490 0.6476
Table 90. OSNNet-TE-UU-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
UU KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.2803 0.2803 0.2652 0.3217 0.2527 0.2345 0.2146 0.3023
523 0.3203 0.3681 0.3082 0.3050 0.2591 0.3562 0.3505 0.3714
631 0.2527 0.1995 0.1680 0.1945 0.1729 0.2026 0.1272 0.1616
424 0.3215 0.4436 0.4025 0.4034 0.4359 0.3753 0.4632 0.3527
532 0.3793 0.2918 0.3422 0.2529 0.2585 0.2817 0.3227 0.3144
325 0.5118 0.5205 0.5184 0.4986 0.4612 0.4296 0.4539 0.4726
433 0.4038 0.4144 0.4480 0.3871 0.4539 0.4224 0.3501 0.4238
541 0.3954 0.2464 0.2652 0.3763 0.2704 0.2691 0.2991 0.2517
442 0.4383 0.4490 0.4339 0.4498 0.4319 0.4097 0.4924 0.4183
334 0.4880 0.4364 0.5517 0.5127 0.4699 0.5114 0.4566 0.5466
226 0.4724 0.4599 0.4566 0.4722 0.4669 0.4782 0.4747 0.4786
451 0.4091 0.4249 0.4386 0.3372 0.3970 0.4169 0.4288 0.5279
343 0.5786 0.5861 0.5967 0.5844 0.5242 0.5759 0.5733 0.4894
235 0.5550 0.5536 0.5727 0.5498 0.5405 0.5563 0.5639 0.5594
352 0.6359 0.6914 0.5878 0.6688 0.5560 0.5778 0.5916 0.6146
361 0.6777 0.6571 0.6838 0.7100 0.7346 0.6200 0.7728 0.6366
244 0.6442 0.6242 0.6042 0.6248 0.6524 0.6282 0.5955 0.6143
253 0.6047 0.5866 0.6063 0.6081 0.5931 0.6085 0.6148 0.6013
262 0.6987 0.6913 0.6849 0.6367 0.6461 0.6929 0.6497 0.6781
mean 0.4772 0.4698 0.4703 0.4681 0.4514 0.4551 0.4629 0.4640

Table 91. OSNNet-TE-UUl-normal-table_image.png
First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
UUl KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.3624 0.9639 0.9953 0.9962 0.0352 0.3200 0.9138 0.9185
523 0.5038 0.9642 0.9972 0.9973 0.0220 0.3308 0.9222 0.9241
631 0.6261 0.9766 0.9972 0.9971 0.0382 0.4709 0.9084 0.9301
424 0.5265 0.9739 0.9970 0.9967 0.0392 0.4309 0.9063 0.9309
532 0.7442 0.9789 0.9968 0.9968 0.0415 0.4216 0.9251 0.9183
325 0.7677 0.9730 0.9976 0.9976 0.0330 0.4524 0.9584 0.9512
433 0.7635 0.9848 0.9975 0.9981 0.0622 0.5396 0.9444 0.9512
541 0.8154 0.9870 0.9979 0.9984 0.0698 0.6001 0.9284 0.9515
442 0.8805 0.9876 0.9975 0.9976 0.0521 0.6478 0.9419 0.9489
334 0.7960 0.9864 0.9976 0.9976 0.0836 0.7016 0.9577 0.9618
226 0.6445 0.9734 0.9981 0.9983 0.0588 0.6470 0.9691 0.9598
451 0.9563 0.9931 0.9984 0.9983 0.0901 0.7754 0.9515 0.9702
343 0.9332 0.9944 0.9984 0.9988 0.0762 0.7679 0.9592 0.9752
235 0.9202 0.9901 0.9984 0.9984 0.0669 0.7420 0.9807 0.9839
352 0.9256 0.9959 0.9988 0.9990 0.0792 0.7119 0.9694 0.9725
361 0.9511 0.9922 0.9986 0.9983 0.0783 0.7389 0.9569 0.9633
244 0.8622 0.9892 0.9982 0.9981 0.0907 0.7537 0.9590 0.9591
253 0.9744 0.9956 0.9990 0.9983 0.0949 0.8250 0.9717 0.9779
262 0.9815 0.9977 0.9991 0.9993 0.1088 0.8918 0.9784 0.9843
mean 0.7861 0.9841 0.9978 0.9979 0.0642 0.6194 0.9475 0.9544
Table 92. OSNNet-TE-UUl-threshold-table_image.png
Results with threshold on softmax. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
UUl KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9067 0.9894 0.9988 0.9990 0.7850 0.8418 0.9815 0.9835
523 0.9155 0.9898 0.9990 0.9988 0.7330 0.8338 0.9833 0.9862
631 0.9596 0.9944 0.9988 0.9991 0.8283 0.9082 0.9868 0.9909
424 0.8691 0.9917 0.9986 0.9986 0.6716 0.8283 0.9750 0.9822
532 0.9634 0.9950 0.9990 0.9990 0.7564 0.8490 0.9861 0.9856
325 0.9084 0.9921 0.9992 0.9992 0.6265 0.8271 0.9895 0.9857
433 0.9517 0.9953 0.9990 0.9994 0.7519 0.8795 0.9870 0.9879
541 0.9831 0.9973 0.9992 0.9993 0.8200 0.9248 0.9902 0.9928
442 0.9837 0.9962 0.9992 0.9991 0.7767 0.9005 0.9925 0.9896
334 0.9508 0.9951 0.9991 0.9992 0.7001 0.9143 0.9887 0.9913
226 0.8945 0.9879 0.9992 0.9991 0.4870 0.8634 0.9903 0.9871
451 0.9934 0.9983 0.9995 0.9995 0.8071 0.9498 0.9931 0.9963
343 0.9898 0.9984 0.9996 0.9997 0.7378 0.9540 0.9931 0.9960
235 0.9811 0.9965 0.9995 0.9992 0.4999 0.9228 0.9923 0.9961
352 0.9878 0.9986 0.9997 0.9995 0.7305 0.9169 0.9962 0.9966
361 0.9899 0.9975 0.9995 0.9995 0.7312 0.9318 0.9922 0.9940
244 0.9648 0.9958 0.9992 0.9995 0.5161 0.8936 0.9882 0.9897
253 0.9947 0.9984 0.9995 0.9994 0.5699 0.9521 0.9926 0.9929
262 0.9974 0.9992 0.9999 0.9998 0.6194 0.9732 0.9931 0.9971
mean 0.9571 0.9951 0.9992 0.9993 0.6920 0.8982 0.9890 0.9906
Table 93. OSNNet-TE-UUl-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
UUl KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9620 0.9608 0.9714 0.8686 0.9274 0.9040 0.9646 0.9448
523 0.9118 0.9194 0.7457 0.8578 0.8396 0.8495 0.9425 0.9468
631 0.9674 0.9831 0.9680 0.9112 0.9470 0.9336 0.9695 0.9623
424 0.8355 0.9018 0.7213 0.8409 0.7706 0.7978 0.8629 0.8574
532 0.9278 0.9415 0.6826 0.6457 0.8898 0.8648 0.9298 0.9339
325 0.7573 0.8947 0.9450 0.9850 0.6349 0.6826 0.8377 0.8935
433 0.8945 0.9802 0.8954 0.7770 0.8368 0.8278 0.9334 0.9224
541 0.9651 0.9241 0.9171 0.7749 0.8875 0.8077 0.9088 0.9483
442 0.8887 0.9755 0.9061 0.8907 0.7687 0.7898 0.8865 0.9028
334 0.8593 0.8501 0.9124 0.8449 0.6130 0.7309 0.7721 0.8482
226 0.7255 0.9711 0.8975 0.6429 0.5579 0.6409 0.7796 0.5696
451 0.9097 0.9792 0.9259 0.8184 0.9059 0.8640 0.9209 0.9311
343 0.9329 0.9028 0.8915 0.7137 0.7385 0.7611 0.9352 0.9407
235 0.7739 0.7273 0.8440 0.7586 0.3327 0.6899 0.7682 0.5911
352 0.8571 0.9634 0.9894 0.9447 0.7152 0.7826 0.8917 0.9333
361 0.9164 0.9382 0.9062 0.9810 0.7375 0.7549 0.9092 0.8749
244 0.6758 0.9600 0.9062 0.7849 0.4271 0.7211 0.8192 0.8702
253 0.8082 0.6727 0.8224 0.8446 0.4530 0.6404 0.8552 0.6116
262 0.9407 0.8594 0.8356 0.8317 0.4069 0.7331 0.7168 0.9055
mean 0.8689 0.9108 0.8781 0.8272 0.7047 0.7777 0.8739 0.8626
Table 94. OSNNet-TE-UUl-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
UUl KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9648 0.9386 0.8842 0.8632 0.9117 0.9155 0.9497 0.9390
523 0.8969 0.9088 0.7943 0.8929 0.8673 0.8544 0.9128 0.9282
631 0.9386 0.9571 0.7914 0.9345 0.9203 0.9263 0.9614 0.9547
424 0.8107 0.9054 0.6615 0.7580 0.7446 0.7733 0.8375 0.8348
532 0.9452 0.9309 0.8783 0.5908 0.8761 0.8523 0.9320 0.9101
325 0.7218 0.8079 0.8726 0.8865 0.5928 0.6610 0.8126 0.8602
433 0.9141 0.9588 0.7790 0.8586 0.8307 0.8324 0.9195 0.9038
541 0.9538 0.9127 0.8600 0.7001 0.9078 0.8612 0.9042 0.9188
442 0.8964 0.9061 0.7724 0.7867 0.8256 0.8151 0.8831 0.9050
334 0.8385 0.8992 0.7503 0.7632 0.5813 0.7026 0.7466 0.8086
226 0.7255 0.9711 0.8975 0.6429 0.5579 0.6409 0.7796 0.5696
451 0.9197 0.9101 0.7588 0.8074 0.8213 0.8947 0.9079 0.9236
343 0.7995 0.8178 0.7104 0.8360 0.6313 0.7398 0.8836 0.8861
235 0.7739 0.7273 0.8440 0.7586 0.3327 0.6899 0.7682 0.5911
352 0.7714 0.9073 0.9451 0.8859 0.6052 0.6674 0.8896 0.8274
361 0.9009 0.7455 0.7226 0.6599 0.6400 0.6919 0.8458 0.7596
244 0.6758 0.9600 0.9062 0.7849 0.4271 0.7211 0.8192 0.8702
253 0.8082 0.6727 0.8224 0.8446 0.4530 0.6404 0.8552 0.6116
262 0.9407 0.8594 0.8356 0.8317 0.4069 0.7331 0.7168 0.9055
mean 0.8524 0.8788 0.8151 0.7940 0.6807 0.7691 0.8592 0.8373
Table 95. OSNNet-TE-UUl-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
UUl KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9236 0.9792 0.9969 0.9973 0.7928 0.7980 0.8695 0.8791
523 0.8717 0.9766 0.9976 0.9974 0.7075 0.7562 0.8802 0.8847
631 0.9373 0.9867 0.9963 0.9976 0.8043 0.8445 0.8909 0.9167
424 0.8379 0.9756 0.9966 0.9947 0.6661 0.7346 0.8746 0.8885
532 0.8963 0.9751 0.9969 0.9965 0.7331 0.7473 0.8726 0.8719
325 0.8283 0.9682 0.9962 0.9962 0.5501 0.7099 0.9179 0.9108
433 0.9179 0.9846 0.9973 0.9969 0.7269 0.7940 0.9051 0.9118
541 0.9365 0.9847 0.9974 0.9973 0.7806 0.8162 0.8873 0.8937
442 0.9237 0.9809 0.9965 0.9957 0.6907 0.7658 0.8814 0.8726
334 0.8934 0.9782 0.9961 0.9960 0.5855 0.7630 0.9089 0.8922
226 0.7058 0.9664 0.9946 0.9964 0.4109 0.7098 0.9388 0.9379
451 0.9418 0.9843 0.9962 0.9965 0.7512 0.8245 0.9055 0.9043
343 0.9161 0.9866 0.9962 0.9973 0.6193 0.8018 0.9132 0.9143
235 0.8703 0.9709 0.9953 0.9957 0.3648 0.7412 0.9421 0.9448
352 0.9042 0.9865 0.9969 0.9966 0.6053 0.7590 0.8785 0.8733
361 0.8964 0.9733 0.9917 0.9927 0.5710 0.7146 0.8622 0.8591
244 0.8247 0.9709 0.9888 0.9940 0.3847 0.6954 0.8699 0.8618
253 0.8955 0.9694 0.9964 0.9913 0.4003 0.6295 0.8862 0.8754
262 0.8939 0.9818 0.9942 0.9827 0.3895 0.7639 0.8960 0.8731
mean 0.8850 0.9779 0.9957 0.9952 0.6071 0.7563 0.8937 0.8929
Table 96. OSNNet-TE-UUl-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
UUl KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.4929 0.7153 0.8542 0.7965 0.3127 0.3167 0.3642 0.5411
523 0.5386 0.8230 0.9220 0.7235 0.3008 0.4087 0.6276 0.6252
631 0.4726 0.5832 0.7529 0.6883 0.2110 0.3259 0.2519 0.3497
424 0.6177 0.9135 0.9889 0.9759 0.5113 0.5165 0.8076 0.7414
532 0.5398 0.6654 0.7947 0.6572 0.2710 0.3126 0.4878 0.4434
325 0.8083 0.9682 0.9954 0.9960 0.5305 0.5806 0.8262 0.8519
433 0.6662 0.8777 0.9441 0.8696 0.5153 0.5755 0.6115 0.7268
541 0.6471 0.6992 0.7628 0.8807 0.2905 0.3703 0.4986 0.4497
442 0.7556 0.9009 0.9635 0.9700 0.4574 0.5239 0.7857 0.6637
334 0.8246 0.9227 0.9915 0.9928 0.4909 0.7146 0.7578 0.8571
226 0.7058 0.9664 0.9946 0.9964 0.4109 0.7098 0.9388 0.9379
451 0.7437 0.8825 0.9207 0.7696 0.4502 0.5237 0.6741 0.7519
343 0.8538 0.9638 0.9913 0.9884 0.5048 0.7331 0.8573 0.7566
235 0.8703 0.9709 0.9953 0.9957 0.3648 0.7412 0.9421 0.9448
352 0.8327 0.9818 0.9792 0.9922 0.4481 0.6224 0.7377 0.7578
361 0.8498 0.9093 0.9599 0.9817 0.5189 0.6173 0.8466 0.7401
244 0.8247 0.9709 0.9888 0.9940 0.3847 0.6954 0.8699 0.8618
253 0.8955 0.9694 0.9964 0.9913 0.4003 0.6295 0.8862 0.8754
262 0.8939 0.9818 0.9942 0.9827 0.3895 0.7639 0.8960 0.8731
mean 0.7281 0.8772 0.9363 0.9075 0.4086 0.5622 0.7193 0.7236

Table 97. OSNNet-TE-UUn-normal-table_image.png
First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
UUn KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.2977 0.9825 1.0000 1.0000 0.0000 0.2209 0.9053 0.9088
523 0.4514 0.9908 1.0000 1.0000 0.0367 0.3870 0.9527 0.9703
631 0.6490 0.9818 1.0000 1.0000 0.0534 0.5907 0.9513 0.9719
424 0.4739 0.9828 1.0000 1.0000 0.0248 0.3902 0.9426 0.9548
532 0.7613 0.9868 1.0000 1.0000 0.0275 0.4568 0.9517 0.9552
325 0.7635 0.9850 1.0000 0.9988 0.0491 0.4538 0.9778 0.9765
433 0.7218 0.9776 1.0000 1.0000 0.0692 0.5472 0.9737 0.9664
541 0.8404 0.9958 1.0000 1.0000 0.0866 0.5863 0.9571 0.9806
442 0.9185 0.9948 1.0000 0.9986 0.0864 0.7776 0.9751 0.9711
334 0.7448 0.9940 1.0000 0.9993 0.0487 0.6908 0.9567 0.9771
226 0.6049 0.9696 1.0000 1.0000 0.0480 0.6147 0.9665 0.9628
451 0.9367 1.0000 1.0000 1.0000 0.0767 0.7340 0.9626 0.9867
343 0.9200 0.9962 1.0000 1.0000 0.0900 0.7985 0.9845 0.9856
235 0.9008 0.9929 1.0000 1.0000 0.0407 0.6896 0.9779 0.9821
352 0.9253 0.9986 1.0000 1.0000 0.1194 0.7409 0.9483 0.9751
361 0.9938 1.0000 1.0000 1.0000 0.1282 0.7249 0.9633 0.9632
244 0.8685 0.9935 1.0000 1.0000 0.1214 0.7800 0.9838 0.9728
253 0.9663 0.9988 1.0000 1.0000 0.0952 0.8561 0.9924 0.9945
262 0.9615 0.9921 0.9988 1.0000 0.1366 0.8538 0.9623 0.9911
mean 0.7737 0.9902 0.9999 0.9998 0.0705 0.6260 0.9624 0.9709
Table 98. OSNNet-TE-UUn-threshold-table_image.png
Results with threshold on softmax. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
UUn KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9250 0.9979 1.0000 1.0000 0.8412 0.8747 0.9948 0.9926
523 0.9158 0.9972 1.0000 1.0000 0.8360 0.9258 0.9975 0.9962
631 0.9708 1.0000 1.0000 1.0000 0.8068 0.9593 1.0000 1.0000
424 0.8377 0.9951 1.0000 1.0000 0.7498 0.8846 0.9944 0.9914
532 0.9665 1.0000 1.0000 1.0000 0.8315 0.9363 0.9939 0.9983
325 0.9092 0.9973 1.0000 1.0000 0.6592 0.8375 0.9949 0.9936
433 0.9221 0.9986 1.0000 1.0000 0.7773 0.9250 0.9967 0.9980
541 0.9939 0.9979 1.0000 1.0000 0.8800 0.9331 0.9958 0.9938
442 0.9962 0.9986 1.0000 1.0000 0.8691 0.9764 0.9976 0.9986
334 0.9285 0.9991 1.0000 1.0000 0.7277 0.9476 0.9921 0.9933
226 0.8709 0.9884 1.0000 1.0000 0.4858 0.8489 0.9928 0.9857
451 0.9971 1.0000 1.0000 1.0000 0.8382 0.9471 1.0000 1.0000
343 0.9860 1.0000 1.0000 1.0000 0.7056 0.9667 0.9990 1.0000
235 0.9742 0.9972 1.0000 1.0000 0.5043 0.9029 0.9953 0.9971
352 0.9920 1.0000 1.0000 1.0000 0.7687 0.9678 0.9979 0.9965
361 1.0000 1.0000 1.0000 1.0000 0.6680 0.9547 1.0000 0.9960
244 0.9835 0.9989 1.0000 1.0000 0.6228 0.9462 0.9944 0.9951
253 0.9879 1.0000 1.0000 1.0000 0.5679 0.9757 0.9988 0.9988
262 0.9921 1.0000 1.0000 1.0000 0.6267 0.9510 0.9915 0.9974
mean 0.9552 0.9982 1.0000 1.0000 0.7245 0.9295 0.9962 0.9959
Table 99. OSNNet-TE-UUn-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
UUn KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9589 0.9779 0.9952 0.9152 0.9495 0.9077 0.9862 0.9706
523 0.9395 0.9380 0.7569 0.8758 0.9010 0.9444 0.9776 0.9662
631 0.9852 0.9740 0.9917 0.9064 0.9843 0.9852 0.9777 0.9838
424 0.8079 0.9115 0.7455 0.8472 0.8425 0.8800 0.9196 0.9011
532 0.9603 0.9447 0.6546 0.6286 0.9237 0.8886 0.9317 0.9386
325 0.7692 0.8896 0.9383 0.9708 0.7170 0.6936 0.8748 0.8864
433 0.8698 0.9763 0.8773 0.7734 0.8707 0.8931 0.9434 0.9594
541 0.9412 0.9391 0.9352 0.8535 0.8783 0.8454 0.9022 0.9277
442 0.8791 0.9912 0.9063 0.9147 0.8198 0.8956 0.9369 0.9495
334 0.8501 0.8834 0.9235 0.8498 0.6011 0.7637 0.8008 0.8351
226 0.7405 0.9812 0.9081 0.6615 0.5818 0.6316 0.8401 0.6089
451 0.8752 0.9268 0.9062 0.7194 0.8629 0.8272 0.8820 0.8759
343 0.9181 0.9155 0.9066 0.7388 0.7883 0.8167 0.9547 0.9564
235 0.8121 0.7692 0.8684 0.7682 0.3465 0.6719 0.7854 0.6165
352 0.8339 0.9523 0.9986 0.9700 0.7509 0.8139 0.8590 0.8766
361 0.9260 0.9563 0.9090 0.9896 0.6150 0.6655 0.8734 0.8248
244 0.6991 0.9907 0.8940 0.7863 0.5080 0.7453 0.8030 0.8752
253 0.8203 0.6969 0.8127 0.8578 0.4233 0.6718 0.8714 0.6680
262 0.9424 0.8507 0.8382 0.8348 0.4533 0.7606 0.7124 0.8885
mean 0.8699 0.9192 0.8824 0.8348 0.7273 0.8054 0.8859 0.8689
Table 100. OSNNet-TE-UUn-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
UUn KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9579 0.9641 0.9041 0.8788 0.9588 0.9481 0.9759 0.9642
523 0.8918 0.9335 0.7717 0.8816 0.9451 0.9558 0.9676 0.9493
631 0.9670 0.9699 0.8258 0.9668 0.9770 0.9408 0.9570 0.9878
424 0.7749 0.9128 0.6587 0.7568 0.8282 0.8543 0.8946 0.8962
532 0.9638 0.9296 0.8688 0.5968 0.9314 0.8923 0.9336 0.9204
325 0.7229 0.8152 0.8695 0.8779 0.6046 0.7066 0.8088 0.8790
433 0.8715 0.9459 0.7215 0.8579 0.8556 0.8480 0.9119 0.9151
541 0.9577 0.9450 0.8935 0.7191 0.9058 0.8477 0.8627 0.9105
442 0.9149 0.9315 0.8113 0.7980 0.8808 0.8811 0.9330 0.9418
334 0.8033 0.9225 0.7463 0.7683 0.6128 0.7306 0.7701 0.8729
226 0.7405 0.9812 0.9081 0.6615 0.5818 0.6316 0.8401 0.6089
451 0.8458 0.8259 0.7463 0.7852 0.8567 0.8194 0.8721 0.8474
343 0.7951 0.8285 0.7184 0.8057 0.6371 0.7417 0.8572 0.8372
235 0.8121 0.7692 0.8684 0.7682 0.3465 0.6719 0.7854 0.6165
352 0.7649 0.8927 0.9325 0.8775 0.6215 0.6963 0.8765 0.7020
361 0.9167 0.8354 0.7511 0.7153 0.5528 0.6647 0.8571 0.7485
244 0.6991 0.9907 0.8940 0.7863 0.5080 0.7453 0.8030 0.8752
253 0.8203 0.6969 0.8127 0.8578 0.4233 0.6718 0.8714 0.6680
262 0.9424 0.8507 0.8382 0.8348 0.4533 0.7606 0.7124 0.8885
mean 0.8507 0.8916 0.8179 0.7997 0.7095 0.7899 0.8679 0.8437
Table 101. OSNNet-TE-UUn-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
UUn KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9180 0.9866 1.0000 1.0000 0.8433 0.8437 0.9158 0.9089
523 0.8908 0.9847 0.9992 0.9981 0.7727 0.8696 0.9407 0.9237
631 0.9270 0.9895 1.0000 1.0000 0.8253 0.9037 0.9482 0.9626
424 0.8040 0.9777 1.0000 0.9992 0.7402 0.8085 0.9259 0.9229
532 0.9183 0.9874 0.9970 0.9976 0.8162 0.8271 0.9482 0.9417
325 0.8178 0.9717 1.0000 0.9969 0.5639 0.7194 0.9199 0.9229
433 0.8985 0.9738 0.9980 0.9974 0.7503 0.8406 0.9216 0.9323
541 0.9380 0.9947 1.0000 1.0000 0.7572 0.8286 0.9324 0.9140
442 0.9463 0.9962 1.0000 1.0000 0.8111 0.8525 0.9339 0.9481
334 0.8460 0.9799 0.9961 1.0000 0.5903 0.7910 0.9045 0.9369
226 0.6745 0.9611 0.9988 0.9989 0.4035 0.7064 0.9474 0.9358
451 0.8891 0.9971 1.0000 0.9900 0.7430 0.7701 0.8917 0.8826
343 0.9054 0.9747 0.9944 0.9951 0.5995 0.8201 0.9334 0.9180
235 0.8425 0.9715 0.9977 0.9988 0.3616 0.7048 0.9240 0.9233
352 0.8740 0.9806 0.9985 1.0000 0.6371 0.7720 0.8725 0.8880
361 0.9396 0.9880 0.9979 0.9960 0.5050 0.5756 0.7884 0.8389
244 0.8208 0.9841 0.9926 0.9978 0.4470 0.7557 0.9219 0.8970
253 0.8891 0.9729 0.9946 0.9903 0.3893 0.6951 0.8905 0.8570
262 0.8810 0.9745 0.9906 0.9710 0.3745 0.7161 0.8312 0.8117
mean 0.8748 0.9814 0.9977 0.9962 0.6280 0.7790 0.9101 0.9087
Table 102. OSNNet-TE-UUn-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
UUn KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.4809 0.6894 0.8387 0.8146 0.3437 0.3333 0.3725 0.5010
523 0.5650 0.8331 0.9032 0.7135 0.3362 0.5095 0.6385 0.6144
631 0.4801 0.6111 0.6962 0.7008 0.1983 0.3827 0.3030 0.3299
424 0.5750 0.9054 0.9915 0.9729 0.5697 0.5779 0.8329 0.7707
532 0.5800 0.6685 0.7470 0.5556 0.3181 0.3910 0.5560 0.4355
325 0.8027 0.9717 0.9994 0.9969 0.5480 0.6076 0.8268 0.8505
433 0.6456 0.8604 0.9388 0.8215 0.5556 0.6087 0.6318 0.7251
541 0.6906 0.6912 0.7630 0.8708 0.2617 0.3997 0.4423 0.4509
442 0.8083 0.9300 0.9588 0.9767 0.5659 0.6088 0.8257 0.7070
334 0.7762 0.9129 0.9870 0.9974 0.4822 0.7371 0.7523 0.8953
226 0.6745 0.9611 0.9988 0.9989 0.4035 0.7064 0.9474 0.9358
451 0.6445 0.8292 0.8609 0.6966 0.4795 0.4176 0.6104 0.6710
343 0.8391 0.9529 0.9761 0.9847 0.4746 0.7589 0.8562 0.7176
235 0.8425 0.9715 0.9977 0.9988 0.3616 0.7048 0.9240 0.9233
352 0.7514 0.9729 0.9671 0.9830 0.5045 0.6399 0.7082 0.7731
361 0.9027 0.9140 0.9779 0.9960 0.4650 0.4759 0.7688 0.7167
244 0.8208 0.9841 0.9926 0.9978 0.4470 0.7557 0.9219 0.8970
253 0.8891 0.9729 0.9946 0.9903 0.3893 0.6951 0.8905 0.8570
262 0.8810 0.9745 0.9906 0.9710 0.3745 0.7161 0.8312 0.8117
mean 0.7184 0.8740 0.9253 0.8967 0.4252 0.5804 0.7179 0.7149

Table 103. OSNNet-TE-RN-normal-table_image.png
First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
RN KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.3296 1.0000 1.0000 1.0000 0.0011 0.3650 1.0000 1.0000
523 0.5400 1.0000 1.0000 1.0000 0.0002 0.5529 1.0000 1.0000
631 0.4997 1.0000 1.0000 1.0000 0.0007 0.5352 1.0000 1.0000
424 0.4466 1.0000 1.0000 1.0000 0.0010 0.6351 0.9999 0.9998
532 0.8585 1.0000 1.0000 1.0000 0.0011 0.6941 1.0000 1.0000
325 0.7802 1.0000 1.0000 1.0000 0.0005 0.4971 1.0000 0.9999
433 0.8563 1.0000 1.0000 1.0000 0.0010 0.6084 1.0000 1.0000
541 0.7573 1.0000 1.0000 1.0000 0.0020 0.6718 0.9988 1.0000
442 0.8562 1.0000 1.0000 1.0000 0.0039 0.9108 1.0000 1.0000
334 0.7992 1.0000 1.0000 1.0000 0.0027 0.7799 0.9999 1.0000
226 0.6726 0.9998 1.0000 1.0000 0.0028 0.6554 1.0000 0.9997
451 0.9639 1.0000 1.0000 1.0000 0.0004 0.7895 1.0000 0.9999
343 0.9568 1.0000 1.0000 1.0000 0.0000 0.8322 0.9998 1.0000
235 0.9517 1.0000 1.0000 1.0000 0.0000 0.8659 1.0000 1.0000
352 0.9555 1.0000 1.0000 1.0000 0.0000 0.7555 1.0000 1.0000
361 0.9886 1.0000 1.0000 1.0000 0.0000 0.9508 1.0000 1.0000
244 0.9145 1.0000 1.0000 1.0000 0.0005 0.9559 1.0000 0.9999
253 0.9999 1.0000 1.0000 1.0000 0.0120 0.9383 0.9998 1.0000
262 0.9957 1.0000 1.0000 1.0000 0.0000 0.9622 1.0000 0.9997
mean 0.7959 1.0000 1.0000 1.0000 0.0016 0.7345 0.9999 0.9999
Table 104. OSNNet-TE-RN-threshold-table_image.png
Results with threshold on softmax. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
RN KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9514 1.0000 1.0000 1.0000 0.9791 0.9925 1.0000 1.0000
523 0.9872 1.0000 1.0000 1.0000 0.9529 0.9946 1.0000 1.0000
631 0.9467 1.0000 1.0000 1.0000 0.9870 0.9986 1.0000 1.0000
424 0.9176 1.0000 1.0000 1.0000 0.9263 0.9913 1.0000 1.0000
532 0.9983 1.0000 1.0000 1.0000 0.9744 0.9982 1.0000 1.0000
325 0.9223 1.0000 1.0000 1.0000 0.8151 0.9663 1.0000 1.0000
433 0.9930 1.0000 1.0000 1.0000 0.9130 0.9936 1.0000 1.0000
541 0.9983 1.0000 1.0000 1.0000 0.9508 0.9993 1.0000 1.0000
442 0.9717 1.0000 1.0000 1.0000 0.8900 0.9999 1.0000 1.0000
334 0.9935 1.0000 1.0000 1.0000 0.8551 0.9975 1.0000 1.0000
226 0.9613 1.0000 1.0000 1.0000 0.6301 0.9712 1.0000 1.0000
451 0.9999 1.0000 1.0000 1.0000 0.9111 0.9992 1.0000 1.0000
343 0.9991 1.0000 1.0000 1.0000 0.6983 0.9989 1.0000 1.0000
235 0.9983 1.0000 1.0000 1.0000 0.4757 0.9966 1.0000 1.0000
352 0.9995 1.0000 1.0000 1.0000 0.9068 0.9976 1.0000 1.0000
361 0.9999 1.0000 1.0000 1.0000 0.7644 0.9999 1.0000 1.0000
244 0.9981 1.0000 1.0000 1.0000 0.6845 0.9992 1.0000 1.0000
253 1.0000 1.0000 1.0000 1.0000 0.6842 0.9995 1.0000 1.0000
262 1.0000 1.0000 1.0000 1.0000 0.5423 0.9990 1.0000 1.0000
mean 0.9808 1.0000 1.0000 1.0000 0.8180 0.9944 1.0000 1.0000
Table 105. OSNNet-TE-RN-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
RN KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9994 0.9903 0.9994 0.8844 0.9989 0.9882 1.0000 1.0000
523 0.9622 0.9936 0.7830 0.8575 0.9555 0.9960 0.9980 1.0000
631 0.9863 0.9999 0.9964 0.8999 1.0000 0.9995 1.0000 0.9999
424 0.9409 0.9576 0.7047 0.8915 0.9787 0.9250 0.9425 0.9733
532 0.9310 0.9991 0.6192 0.5845 0.9934 0.9980 0.9988 0.9999
325 0.8014 0.9436 0.9764 0.9984 0.8457 0.7451 0.9418 0.9503
433 0.9843 0.9999 0.9152 0.7683 0.9605 0.9687 0.9996 0.9975
541 0.9756 0.9925 0.9810 0.8483 0.9302 0.9644 0.9926 0.9991
442 0.8595 1.0000 0.9003 0.9037 0.9161 0.8925 0.9970 0.9882
334 0.9282 0.8755 0.9093 0.8956 0.5613 0.7943 0.9154 0.8746
226 0.7808 0.9845 0.9003 0.6484 0.5475 0.7207 0.9155 0.5980
451 0.9555 1.0000 0.9954 0.8339 0.9986 0.9214 0.9420 0.9799
343 0.8891 0.9241 0.8615 0.6965 0.7847 0.9288 0.9980 0.9836
235 0.7957 0.7330 0.8820 0.7287 0.2740 0.8322 0.7662 0.7507
352 0.8772 0.9985 1.0000 0.9572 0.9129 0.8393 0.9490 0.9634
361 0.9659 0.9819 0.8739 0.9858 0.8021 0.8650 0.9880 0.8462
244 0.6643 0.9999 0.9465 0.8058 0.5141 0.8006 0.8600 0.9125
253 0.8247 0.7817 0.8001 0.8421 0.4961 0.5916 0.8495 0.6191
262 0.9540 0.8650 0.8313 0.8128 0.4199 0.7439 0.7559 0.9258
mean 0.8987 0.9485 0.8882 0.8339 0.7837 0.8692 0.9374 0.9138
Table 106. OSNNet-TE-RN-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
RN KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9999 0.9985 0.9030 0.9551 0.9998 0.9985 1.0000 1.0000
523 0.9152 0.9907 0.8001 0.8988 0.9980 0.9973 0.9988 0.9991
631 0.9986 0.9926 0.7829 0.9984 1.0000 1.0000 0.9999 1.0000
424 0.8747 0.9577 0.6956 0.7682 0.9569 0.9129 0.9384 0.9841
532 0.9475 0.9634 0.9006 0.4804 0.9954 0.9964 1.0000 0.9961
325 0.7530 0.8242 0.9119 0.9030 0.7932 0.8335 0.9400 0.9349
433 0.9788 0.9988 0.7999 0.8945 0.9883 0.9919 0.9977 0.9909
541 0.9812 0.9936 0.8805 0.7248 0.9307 0.9889 0.9925 0.9954
442 0.8304 0.9755 0.7679 0.6901 0.9874 0.9627 0.9969 0.9917
334 0.9510 0.9900 0.7708 0.7438 0.7549 0.8707 0.8654 0.9071
226 0.7808 0.9845 0.9003 0.6484 0.5475 0.7207 0.9155 0.5980
451 0.9810 0.9806 0.9264 0.8113 0.9736 0.9846 0.9939 0.9940
343 0.7314 0.7827 0.6723 0.8310 0.7311 0.8920 0.9948 0.9689
235 0.7957 0.7330 0.8820 0.7287 0.2740 0.8322 0.7662 0.7507
352 0.8199 0.9235 0.9985 0.9002 0.7822 0.8357 0.9323 0.9216
361 0.9850 0.7936 0.7406 0.6165 0.7608 0.8508 0.9707 0.8459
244 0.6643 0.9999 0.9465 0.8058 0.5141 0.8006 0.8600 0.9125
253 0.8247 0.7817 0.8001 0.8421 0.4961 0.5916 0.8495 0.6191
262 0.9540 0.8650 0.8313 0.8128 0.4199 0.7439 0.7559 0.9258
mean 0.8825 0.9226 0.8374 0.7923 0.7844 0.8845 0.9352 0.9124
Table 107. OSNNet-TE-RN-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
RN KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9408 1.0000 1.0000 1.0000 0.9333 0.9394 0.9958 0.9974
523 0.9607 1.0000 1.0000 1.0000 0.8849 0.9169 0.9953 0.9964
631 0.8880 1.0000 1.0000 1.0000 0.9315 0.9726 0.9925 0.9984
424 0.8848 1.0000 1.0000 1.0000 0.8425 0.9052 0.9938 0.9962
532 0.8210 0.9992 1.0000 1.0000 0.8970 0.9503 0.9966 0.9973
325 0.8366 0.9965 1.0000 1.0000 0.6962 0.8289 0.9963 0.9925
433 0.9637 1.0000 1.0000 1.0000 0.8506 0.9085 0.9894 0.9916
541 0.9550 0.9995 1.0000 1.0000 0.8777 0.8868 0.9940 0.9810
442 0.8675 1.0000 1.0000 1.0000 0.8320 0.9360 0.9960 0.9926
334 0.9706 1.0000 1.0000 1.0000 0.7014 0.9069 0.9888 0.9729
226 0.8415 0.9970 1.0000 1.0000 0.5183 0.8105 0.9996 0.9997
451 0.9420 1.0000 1.0000 1.0000 0.8617 0.8638 0.9920 0.9849
343 0.8967 0.9998 1.0000 1.0000 0.6586 0.8370 0.9981 0.9889
235 0.9233 0.9995 1.0000 1.0000 0.3695 0.8954 0.9920 0.9942
352 0.9043 1.0000 1.0000 1.0000 0.7781 0.9035 0.9655 0.9898
361 0.8950 1.0000 1.0000 1.0000 0.6941 0.9140 0.9883 0.9811
244 0.9461 1.0000 1.0000 1.0000 0.5016 0.8422 0.9972 0.9917
253 0.9781 0.9985 1.0000 1.0000 0.5001 0.7280 0.9734 0.9918
262 0.9244 0.9990 1.0000 0.9975 0.3704 0.8530 0.9765 0.8654
mean 0.9126 0.9994 1.0000 0.9999 0.7210 0.8842 0.9906 0.9844
Table 108. OSNNet-TE-RN-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
RN KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.4327 0.8218 0.8572 0.8106 0.4407 0.4314 0.5996 0.7060
523 0.5745 0.8955 0.9483 0.7064 0.4084 0.5505 0.8430 0.8656
631 0.3432 0.5870 0.7498 0.6171 0.3046 0.4382 0.3497 0.4804
424 0.6805 0.9878 1.0000 0.9984 0.6737 0.6535 0.9217 0.9214
532 0.5809 0.7751 0.8082 0.7538 0.3640 0.4724 0.6336 0.6304
325 0.8074 0.9965 1.0000 1.0000 0.6721 0.6870 0.9011 0.9748
433 0.7319 0.9324 0.9386 0.8601 0.6251 0.6591 0.7040 0.8064
541 0.6492 0.6887 0.7880 0.9039 0.3506 0.4078 0.7437 0.5766
442 0.6427 0.9839 0.9992 0.9971 0.5864 0.6554 0.9724 0.7569
334 0.9196 0.9859 1.0000 1.0000 0.5867 0.8606 0.8541 0.9034
226 0.8415 0.9970 1.0000 1.0000 0.5183 0.8105 0.9996 0.9997
451 0.8166 0.9809 0.9532 0.7848 0.5439 0.4833 0.8083 0.8445
343 0.8565 0.9980 1.0000 1.0000 0.5313 0.7882 0.9834 0.8101
235 0.9233 0.9995 1.0000 1.0000 0.3695 0.8954 0.9920 0.9942
352 0.8234 1.0000 1.0000 1.0000 0.5822 0.7669 0.8542 0.8819
361 0.8771 0.9028 0.9892 0.9960 0.6209 0.7810 0.9877 0.8810
244 0.9461 1.0000 1.0000 1.0000 0.5016 0.8422 0.9972 0.9917
253 0.9781 0.9985 1.0000 1.0000 0.5001 0.7280 0.9734 0.9918
262 0.9244 0.9990 1.0000 0.9975 0.3704 0.8530 0.9765 0.8654
mean 0.7552 0.9226 0.9490 0.9171 0.5027 0.6718 0.8471 0.8359

Table 109. OSNNet-TE-RNp-normal-table_image.png
First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
RNp KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.2453 0.7266 0.9288 0.9614 0.0551 0.2724 0.9459 0.9606
523 0.5580 0.9370 0.9883 0.9938 0.2562 0.6059 0.9911 0.9888
631 0.5348 0.8778 0.9712 0.9939 0.1697 0.5546 0.9632 0.9772
424 0.3871 0.8202 0.9780 0.9903 0.3235 0.5494 0.9488 0.9626
532 0.6245 0.8899 0.9321 0.9419 0.2818 0.6415 0.9403 0.9702
325 0.7051 0.8067 0.9885 0.9817 0.3105 0.6703 0.9950 0.9914
433 0.5341 0.8570 0.9730 0.9873 0.2335 0.6046 0.9746 0.9648
541 0.6387 0.9280 0.9849 0.9818 0.3124 0.5940 0.9580 0.9873
442 0.8713 0.9772 0.9978 0.9981 0.4767 0.8865 0.9953 0.9908
334 0.7011 0.9009 0.9871 0.9800 0.4126 0.7172 0.9640 0.9865
226 0.7080 0.8147 0.9944 0.9916 0.3610 0.7441 0.9838 0.9955
451 0.9608 0.9885 0.9980 0.9983 0.5658 0.9336 0.9911 0.9968
343 0.7998 0.9199 0.9869 0.9821 0.3695 0.7294 0.9946 0.9959
235 0.9134 0.9498 0.9845 0.9865 0.5080 0.8490 0.9979 0.9977
352 0.8220 0.9485 0.9834 0.9921 0.4118 0.7519 0.9855 0.9774
361 0.8008 0.9392 0.9905 0.9863 0.4786 0.8227 0.9915 0.9963
244 0.8157 0.9320 0.9794 0.9836 0.4299 0.8653 0.9926 0.9861
253 0.8124 0.9682 0.9989 0.9802 0.6134 0.9300 0.9812 0.9976
262 0.8195 0.9295 0.9961 0.9603 0.5057 0.8442 0.9879 0.9913
mean 0.6975 0.9006 0.9811 0.9827 0.3724 0.7140 0.9780 0.9850
Table 110. OSNNet-TE-RNp-threshold-table_image.png
Results with threshold on softmax. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
RNp KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9854 0.9989 0.9997 0.9999 0.9761 0.9888 1.0000 1.0000
523 0.9689 0.9995 1.0000 1.0000 0.9821 0.9974 1.0000 0.9999
631 0.9962 0.9994 0.9999 1.0000 0.9854 0.9967 0.9999 1.0000
424 0.9517 0.9976 0.9999 0.9999 0.9644 0.9841 0.9998 0.9997
532 0.9955 0.9986 0.9991 0.9977 0.9838 0.9983 0.9999 0.9999
325 0.9814 0.9944 0.9999 0.9998 0.9506 0.9893 1.0000 0.9999
433 0.9683 0.9986 0.9993 1.0000 0.9438 0.9861 0.9999 0.9997
541 0.9953 0.9995 1.0000 1.0000 0.9766 0.9962 0.9999 1.0000
442 0.9975 0.9998 1.0000 1.0000 0.9916 0.9993 1.0000 0.9998
334 0.9895 0.9950 0.9999 0.9999 0.8946 0.9896 0.9994 0.9999
226 0.9502 0.9823 0.9998 0.9997 0.8925 0.9748 0.9993 0.9999
451 0.9998 0.9999 1.0000 1.0000 0.9860 0.9993 1.0000 1.0000
343 0.9784 0.9987 1.0000 1.0000 0.9136 0.9870 1.0000 1.0000
235 0.9955 0.9980 0.9998 0.9999 0.8900 0.9891 1.0000 1.0000
352 0.9771 0.9981 0.9998 0.9999 0.9523 0.9931 0.9998 1.0000
361 0.9954 0.9995 1.0000 1.0000 0.9360 0.9983 1.0000 1.0000
244 0.9856 0.9982 0.9989 0.9996 0.9358 0.9951 1.0000 0.9994
253 0.9752 0.9995 1.0000 0.9999 0.8935 0.9966 0.9998 1.0000
262 0.9940 0.9988 1.0000 0.9994 0.9695 0.9828 1.0000 1.0000
mean 0.9832 0.9976 0.9998 0.9998 0.9483 0.9917 0.9999 0.9999
Table 111. OSNNet-TE-RNp-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
RNp KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9972 0.9997 0.9987 0.9993 0.9915 0.9915 1.0000 0.9974
523 0.9788 0.9848 0.9964 0.9955 0.9742 0.9739 0.9974 0.9937
631 0.9934 0.9947 0.9997 0.9986 0.9976 0.9988 0.9998 0.9985
424 0.9357 0.9819 0.9765 0.9827 0.9748 0.9491 0.9394 0.9941
532 0.9941 0.9972 0.9968 0.9860 0.9951 0.9716 0.9990 0.9879
325 0.8255 0.9308 0.9736 0.9810 0.7831 0.8520 0.9228 0.9178
433 0.9167 0.9810 0.9243 0.9735 0.9662 0.9479 0.9839 0.9868
541 0.9603 0.9810 0.9938 0.9788 0.9693 0.9345 0.9903 0.9777
442 0.9196 0.9355 0.9428 0.9676 0.8341 0.8600 0.9732 0.9712
334 0.8963 0.8652 0.9596 0.9198 0.7091 0.8347 0.8589 0.9410
226 0.8220 0.9482 0.8998 0.6254 0.9052 0.6731 0.8813 0.7286
451 0.9862 0.9570 0.9646 0.9693 0.9744 0.9722 0.8829 0.9870
343 0.8880 0.7969 0.9604 0.9006 0.8566 0.8654 0.9648 0.9184
235 0.7241 0.7927 0.8078 0.7639 0.7254 0.7777 0.7855 0.8327
352 0.7692 0.9121 0.9148 0.9519 0.7796 0.7229 0.9546 0.8718
361 0.8187 0.8450 0.9100 0.8421 0.8479 0.7675 0.8797 0.8530
244 0.6268 0.9551 0.8859 0.7989 0.8340 0.8392 0.9128 0.8686
253 0.6116 0.6864 0.7867 0.7524 0.5221 0.5758 0.8122 0.5664
262 0.7902 0.8225 0.7267 0.8326 0.5877 0.7150 0.6741 0.8443
mean 0.8660 0.9141 0.9273 0.9063 0.8541 0.8538 0.9164 0.9072
Table 112. OSNNet-TE-RNp-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
RNp KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9996 0.9989 0.9983 0.9999 0.9971 0.9961 0.9999 0.9985
523 0.9654 0.9962 0.9964 0.9976 0.9382 0.9741 0.9993 0.9905
631 0.9804 0.9964 0.9983 0.9993 0.9943 0.9991 0.9996 0.9990
424 0.9249 0.9529 0.9775 0.9849 0.9302 0.9291 0.9476 0.9904
532 0.9981 0.9956 0.9712 0.9953 0.9677 0.9696 0.9998 0.9827
325 0.7967 0.8948 0.9359 0.9209 0.8527 0.8362 0.9153 0.9117
433 0.9562 0.9580 0.9593 0.9871 0.9728 0.9832 0.9807 0.9943
541 0.9761 0.9976 0.9987 0.9788 0.9842 0.9763 0.9984 0.9846
442 0.9446 0.9875 0.9354 0.9637 0.9381 0.9318 0.9756 0.9895
334 0.7962 0.8606 0.9437 0.9113 0.7493 0.8243 0.8763 0.8713
226 0.8220 0.9482 0.8998 0.6254 0.9052 0.6731 0.8813 0.7286
451 0.9900 0.9697 0.9638 0.9891 0.9575 0.9826 0.9633 0.9844
343 0.7958 0.8455 0.8475 0.7929 0.7869 0.8663 0.8893 0.9260
235 0.7241 0.7927 0.8078 0.7639 0.7254 0.7777 0.7855 0.8327
352 0.8141 0.8893 0.9523 0.8129 0.5097 0.7360 0.9172 0.8352
361 0.9267 0.8074 0.8394 0.6980 0.7897 0.8134 0.8627 0.7798
244 0.6268 0.9551 0.8859 0.7989 0.8340 0.8392 0.9128 0.8686
253 0.6116 0.6864 0.7867 0.7524 0.5221 0.5758 0.8122 0.5664
262 0.7902 0.8225 0.7267 0.8326 0.5877 0.7150 0.6741 0.8443
mean 0.8652 0.9134 0.9171 0.8845 0.8391 0.8631 0.9153 0.8989
Table 113. OSNNet-TE-RNp-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
RNp KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.9890 0.9879 0.9963 0.9950 0.8762 0.9139 0.9547 0.9584
523 0.9576 0.9944 0.9954 0.9965 0.9355 0.9555 0.9313 0.9383
631 0.9841 0.9641 0.9940 0.9990 0.9461 0.9444 0.9522 0.9625
424 0.9465 0.9875 0.9958 0.9868 0.9122 0.9072 0.9244 0.9561
532 0.9632 0.9914 0.9914 0.9918 0.9197 0.9432 0.9558 0.9499
325 0.9299 0.9464 0.9821 0.9850 0.8134 0.8748 0.9660 0.9544
433 0.9330 0.9801 0.9901 0.9962 0.8456 0.9096 0.8980 0.8972
541 0.9577 0.9926 0.9889 0.9843 0.9176 0.8937 0.9225 0.9302
442 0.9230 0.9822 0.9973 0.9960 0.9112 0.9286 0.9595 0.9502
334 0.9532 0.9711 0.9779 0.9861 0.7735 0.8703 0.9092 0.8532
226 0.8873 0.9300 0.9901 0.9612 0.7799 0.8766 0.9496 0.9678
451 0.9733 0.9649 0.9676 0.9921 0.9434 0.9235 0.9608 0.9543
343 0.9114 0.8888 0.9674 0.9797 0.7784 0.8222 0.9521 0.9385
235 0.8763 0.9580 0.9713 0.9463 0.7041 0.8458 0.9412 0.9364
352 0.8693 0.9226 0.9529 0.9672 0.7428 0.8112 0.8755 0.8731
361 0.9243 0.9380 0.9735 0.9154 0.8217 0.8111 0.9017 0.9396
244 0.7387 0.9283 0.8930 0.9037 0.6612 0.7618 0.9304 0.8991
253 0.7347 0.8234 0.9517 0.8257 0.6794 0.7487 0.8802 0.8673
262 0.7677 0.8746 0.8950 0.8016 0.6849 0.7828 0.8250 0.7943
mean 0.9063 0.9488 0.9722 0.9584 0.8235 0.8697 0.9258 0.9222
Table 114. OSNNet-TE-RNp-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. First cell indicates set used for testing. Columns refer to the different training sets. Red indicates low accuracies per column. Blue indicates low accuracy (< 0.5).
RNp KU KUn KUl KUnl KUc KUcn KUcl KUcnl
622 0.6237 0.6214 0.6416 0.7128 0.3276 0.3770 0.4299 0.5672
523 0.6498 0.8015 0.7528 0.6611 0.4544 0.6289 0.6322 0.6104
631 0.5703 0.4582 0.4772 0.6461 0.2974 0.3881 0.2801 0.3488
424 0.7360 0.9069 0.9448 0.8808 0.7591 0.6739 0.8364 0.7539
532 0.6614 0.6155 0.6403 0.5295 0.4496 0.4579 0.5497 0.4805
325 0.9117 0.9464 0.9693 0.9776 0.7758 0.7375 0.8725 0.8892
433 0.6899 0.8242 0.8465 0.7614 0.6522 0.6772 0.5647 0.6171
541 0.6817 0.5968 0.6349 0.7631 0.4116 0.4225 0.4742 0.4399
442 0.7277 0.8607 0.8965 0.8871 0.6831 0.6534 0.8590 0.7492
334 0.8723 0.8927 0.9510 0.9609 0.6997 0.8299 0.7756 0.8287
226 0.8873 0.9300 0.9901 0.9612 0.7799 0.8766 0.9496 0.9678
451 0.7861 0.7077 0.8032 0.6405 0.6664 0.6279 0.7315 0.8007
343 0.8538 0.8418 0.8700 0.8477 0.6192 0.7354 0.9033 0.8040
235 0.8763 0.9580 0.9713 0.9463 0.7041 0.8458 0.9412 0.9364
352 0.8216 0.8821 0.8428 0.8950 0.5525 0.6366 0.7174 0.7471
361 0.8604 0.8291 0.8736 0.8716 0.7775 0.7130 0.8735 0.8196
244 0.7387 0.9283 0.8930 0.9037 0.6612 0.7618 0.9304 0.8991
253 0.7347 0.8234 0.9517 0.8257 0.6794 0.7487 0.8802 0.8673
262 0.7677 0.8746 0.8950 0.8016 0.6849 0.7828 0.8250 0.7943
mean 0.7606 0.8052 0.8340 0.8144 0.6124 0.6618 0.7382 0.7327

Table 115. OSNNet-TE-all-normal-table_image.png
Rows refer to testing sets. Columns refer to the different training sets. Blue indicates low accuracy (< 0.5).
all KU KUn KUl KUnl KUc KUcn KUcl KUcnl
KN 0.9924 0.9927 0.9924 0.9923 0.9924 0.9924 0.9926 0.9924
KNc 0.2204 0.0318 0.0026 0.0023 0.8630 0.7546 0.4796 0.4540
KU 0.9965 0.9963 0.9963 0.9964 0.9963 0.9965 0.9963 0.9965
KUl 0.7911 0.9900 0.9995 0.9995 0.0488 0.6215 0.9961 0.9969
KUn 0.8624 0.9984 0.9999 0.9998 0.1512 0.9346 0.9900 0.9959
UU 0.5766 0.5681 0.5732 0.5786 0.5684 0.5747 0.5818 0.5866
UUl 0.7861 0.9841 0.9978 0.9979 0.0642 0.6194 0.9475 0.9544
UUn 0.7737 0.9902 0.9999 0.9998 0.0705 0.6260 0.9624 0.9709
RN 0.7959 1.0000 1.0000 1.0000 0.0016 0.7345 0.9999 0.9999
RNp 0.6975 0.9006 0.9811 0.9827 0.3724 0.7140 0.9780 0.9850
Table 116. OSNNet-TE-all-threshold-table_image.png
Results with threshold on softmax. Rows refer to testing sets. Columns refer to the different training sets. Blue indicates low accuracy (< 0.5).
all KU KUn KUl KUnl KUc KUcn KUcl KUcnl
KN 0.9800 0.9803 0.9801 0.9796 0.9803 0.9799 0.9806 0.9799
KNc 0.0903 0.0104 0.0007 0.0001 0.6018 0.4553 0.2320 0.2196
KU 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990 0.9990
KUl 0.9670 0.9977 0.9999 0.9999 0.6969 0.9192 0.9996 0.9996
KUn 0.9822 0.9997 1.0000 1.0000 0.7887 0.9948 0.9989 0.9996
UU 0.7895 0.7838 0.7890 0.7924 0.7810 0.7870 0.7952 0.7949
UUl 0.9571 0.9951 0.9992 0.9993 0.6920 0.8982 0.9890 0.9906
UUn 0.9552 0.9982 1.0000 1.0000 0.7245 0.9295 0.9962 0.9959
RN 0.9808 1.0000 1.0000 1.0000 0.8180 0.9944 1.0000 1.0000
RNp 0.9832 0.9976 0.9998 0.9998 0.9483 0.9917 0.9999 0.9999
Table 117. OSNNet-TE-all-mcossvm_ova_gsio-table_image.png
Results with SSVMO open-set classifier. Rows refer to testing sets. Columns refer to the different training sets. Blue indicates low accuracy (< 0.5).
all KU KUn KUl KUnl KUc KUcn KUcl KUcnl
KN 0.9009 0.9027 0.8841 0.9138 0.8918 0.8988 0.8911 0.9008
KNc 0.1494 0.0789 0.0630 0.0803 0.4555 0.4763 0.3504 0.3706
KU 0.8261 0.8562 0.8674 0.8199 0.8299 0.8198 0.8413 0.8297
KUl 0.8781 0.9152 0.8817 0.8322 0.7142 0.7906 0.9115 0.8967
KUn 0.8884 0.9196 0.8823 0.8326 0.7495 0.8711 0.8862 0.8896
UU 0.6303 0.6526 0.6491 0.6423 0.6322 0.6225 0.6504 0.6090
UUl 0.8689 0.9108 0.8781 0.8272 0.7047 0.7777 0.8739 0.8626
UUn 0.8699 0.9192 0.8824 0.8348 0.7273 0.8054 0.8859 0.8689
RN 0.8987 0.9485 0.8882 0.8339 0.7837 0.8692 0.9374 0.9138
RNp 0.8660 0.9141 0.9273 0.9063 0.8541 0.8538 0.9164 0.9072
Table 118. OSNNet-TE-all-mcossvm_ova_gsic-table_image.png
Results with SSVMC open-set classifier. Rows refer to testing sets. Columns refer to the different training sets. Blue indicates low accuracy (< 0.5).
all KU KUn KUl KUnl KUc KUcn KUcl KUcnl
KN 0.9690 0.9624 0.9667 0.9750 0.9711 0.9663 0.9728 0.9729
KNc 0.1629 0.1029 0.0940 0.0974 0.4947 0.4989 0.3873 0.4085
KU 0.7682 0.7884 0.8027 0.7696 0.7609 0.7547 0.7553 0.7622
KUl 0.8626 0.8854 0.8169 0.7988 0.6930 0.7868 0.9009 0.8808
KUn 0.8703 0.8869 0.8147 0.7927 0.7399 0.8652 0.8760 0.8621
UU 0.5659 0.5881 0.5709 0.5679 0.5678 0.5570 0.5733 0.5369
UUl 0.8524 0.8788 0.8151 0.7940 0.6807 0.7691 0.8592 0.8373
UUn 0.8507 0.8916 0.8179 0.7997 0.7095 0.7899 0.8679 0.8437
RN 0.8825 0.9226 0.8374 0.7923 0.7844 0.8845 0.9352 0.9124
RNp 0.8652 0.9134 0.9171 0.8845 0.8391 0.8631 0.9153 0.8989
Table 119. OSNNet-TE-all-osnn2_imc_gseo-table_image.png
Results with OSNNO open-set classifier. Rows refer to testing sets. Columns refer to the different training sets. Blue indicates low accuracy (< 0.5).
all KU KUn KUl KUnl KUc KUcn KUcl KUcnl
KN 0.9886 0.9884 0.9886 0.9883 0.9888 0.9887 0.9885 0.9884
KNc 0.1841 0.0666 0.0163 0.0181 0.6931 0.6818 0.6276 0.6339
KU 0.9811 0.9821 0.9804 0.9790 0.9809 0.9806 0.9836 0.9821
KUl 0.8958 0.9856 0.9983 0.9981 0.6085 0.7669 0.9611 0.9527
KUn 0.9160 0.9936 0.9979 0.9974 0.6815 0.9264 0.9420 0.9447
UU 0.6456 0.6464 0.6464 0.6494 0.6410 0.6483 0.6490 0.6476
UUl 0.8850 0.9779 0.9957 0.9952 0.6071 0.7563 0.8937 0.8929
UUn 0.8748 0.9814 0.9977 0.9962 0.6280 0.7790 0.9101 0.9087
RN 0.9126 0.9994 1.0000 0.9999 0.7210 0.8842 0.9906 0.9844
RNp 0.9063 0.9488 0.9722 0.9584 0.8235 0.8697 0.9258 0.9222
Table 120. OSNNet-TE-all-osnn2_imc_gsec-table_image.png
Results with OSNNC open-set classifier. Rows refer to testing sets. Columns refer to the different training sets. Blue indicates low accuracy (< 0.5).
all KU KUn KUl KUnl KUc KUcn KUcl KUcnl
KN 0.9938 0.9937 0.9937 0.9937 0.9940 0.9940 0.9939 0.9938
KNc 0.2628 0.1425 0.0611 0.0666 0.7848 0.7727 0.7258 0.7338
KU 0.8693 0.8573 0.8554 0.8437 0.8314 0.8354 0.8428 0.8596
KUl 0.7394 0.8881 0.9385 0.9136 0.4077 0.5690 0.7889 0.7823
KUn 0.7584 0.8944 0.9316 0.9034 0.4710 0.7457 0.7580 0.7679
UU 0.4772 0.4698 0.4703 0.4681 0.4514 0.4551 0.4629 0.4640
UUl 0.7281 0.8772 0.9363 0.9075 0.4086 0.5622 0.7193 0.7236
UUn 0.7184 0.8740 0.9253 0.8967 0.4252 0.5804 0.7179 0.7149
RN 0.7552 0.9226 0.9490 0.9171 0.5027 0.6718 0.8471 0.8359
RNp 0.7606 0.8052 0.8340 0.8144 0.6124 0.6618 0.7382 0.7327

Last updated on July 16, 2019.