jung-geun/PSO
1
0
Fork 0
mirror of https://github.com/jung-geun/PSO.git synced 2025-12-20 04:50:45 +09:00
Code Issues Packages Projects Releases Wiki Activity

23-10-25

Browse Source 
version 1.0.3
최고 점수 클래스 변수로 변경
log 저장 위치 고정
bean, seeds 데이터셋 추가 실험
This commit is contained in:
jung-geun
2023-10-25 12:53:23 +09:00
parent c8741dcd6d
commit 389027409d
13 changed files with 19828 additions and 225 deletions
Download Patch File Download Diff File Expand all files Collapse all files

69
bean.py Normal file
View File

@@ -0,0 +1,69 @@
import os
import pandas as pd
import tensorflow as tf
from sklearn.model_selection import train_test_split
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras.layers import Dense
from tensorflow.keras.models import Sequential
from tensorflow.keras.utils import to_categorical
from ucimlrepo import fetch_ucirepo
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
os.environ['TF_FORCE_GPU_ALLOW_GROWTH'] = 'true'
def make_model():
model = Sequential()
model.add(Dense(12, input_dim=16, activation='relu'))
model.add(Dense(8, activation='relu'))
model.add(Dense(7, activation='softmax'))
return model
def get_data():
# fetch dataset
dry_bean_dataset = fetch_ucirepo(id=602)
# data (as pandas dataframes)
X = dry_bean_dataset.data.features
y = dry_bean_dataset.data.targets
x = X.to_numpy()
# object to categorical
x = x.astype('float32')
y_class = to_categorical(y)
# metadata
# print(dry_bean_dataset.metadata)
# variable information
# print(dry_bean_dataset.variables)
# print(X.head())
# print(y.head())
# y_class = to_categorical(y)
x_train, x_test, y_train, y_test = train_test_split(
x, y_class, test_size=0.2, random_state=42, shuffle=True)
return x_train, x_test, y_train, y_test
x_train, x_test, y_train, y_test = get_data()
model = make_model()
early_stopping = keras.callbacks.EarlyStopping(
patience=10, min_delta=0.001, restore_best_weights=True)
model.compile(loss='sparse_categorical_crossentropy',
optimizer='adam', metrics=['accuracy', "mse"])
model.summary()
history = model.fit(x_train, y_train, epochs=150,
batch_size=10, callbacks=[early_stopping])
score = model.evaluate(x_test, y_test, verbose=2)

19020
data/magic+gamma+telescope/magic04.data Executable file
View File

File diff suppressed because it is too large Load Diff

111
data/magic+gamma+telescope/magic04.names Executable file
View File

@@ -0,0 +1,111 @@
1. Title of Database: MAGIC gamma telescope data 2004
2. Sources:
(a) Original owner of the database:
R. K. Bock
Major Atmospheric Gamma Imaging Cherenkov Telescope project (MAGIC)
http://wwwmagic.mppmu.mpg.de
rkb@mail.cern.ch
(b) Donor:
P. Savicky
Institute of Computer Science, AS of CR
Czech Republic
savicky@cs.cas.cz
(c) Date received: May 2007
3. Past Usage:
(a) Bock, R.K., Chilingarian, A., Gaug, M., Hakl, F., Hengstebeck, T.,
Jirina, M., Klaschka, J., Kotrc, E., Savicky, P., Towers, S.,
Vaicilius, A., Wittek W. (2004).
Methods for multidimensional event classification: a case study
using images from a Cherenkov gamma-ray telescope.
Nucl.Instr.Meth. A, 516, pp. 511-528.
(b) P. Savicky, E. Kotrc.
Experimental Study of Leaf Confidences for Random Forest.
Proceedings of COMPSTAT 2004, In: Computational Statistics.
(Ed.: Antoch J.) - Heidelberg, Physica Verlag 2004, pp. 1767-1774.
(c) J. Dvorak, P. Savicky.
Softening Splits in Decision Trees Using Simulated Annealing.
Proceedings of ICANNGA 2007, Warsaw, (Ed.: Beliczynski et. al),
Part I, LNCS 4431, pp. 721-729.
4. Relevant Information:
The data are MC generated (see below) to simulate registration of high energy
gamma particles in a ground-based atmospheric Cherenkov gamma telescope using the
imaging technique. Cherenkov gamma telescope observes high energy gamma rays,
taking advantage of the radiation emitted by charged particles produced
inside the electromagnetic showers initiated by the gammas, and developing in the
atmosphere. This Cherenkov radiation (of visible to UV wavelengths) leaks
through the atmosphere and gets recorded in the detector, allowing reconstruction
of the shower parameters. The available information consists of pulses left by
the incoming Cherenkov photons on the photomultiplier tubes, arranged in a
plane, the camera. Depending on the energy of the primary gamma, a total of
few hundreds to some 10000 Cherenkov photons get collected, in patterns
(called the shower image), allowing to discriminate statistically those
caused by primary gammas (signal) from the images of hadronic showers
initiated by cosmic rays in the upper atmosphere (background).
Typically, the image of a shower after some pre-processing is an elongated
cluster. Its long axis is oriented towards the camera center if the shower axis
is parallel to the telescope's optical axis, i.e. if the telescope axis is
directed towards a point source. A principal component analysis is performed
in the camera plane, which results in a correlation axis and defines an ellipse.
If the depositions were distributed as a bivariate Gaussian, this would be
an equidensity ellipse. The characteristic parameters of this ellipse
(often called Hillas parameters) are among the image parameters that can be
used for discrimination. The energy depositions are typically asymmetric
along the major axis, and this asymmetry can also be used in discrimination.
There are, in addition, further discriminating characteristics, like the
extent of the cluster in the image plane, or the total sum of depositions.
The data set was generated by a Monte Carlo program, Corsika, described in
D. Heck et al., CORSIKA, A Monte Carlo code to simulate extensive air showers,
Forschungszentrum Karlsruhe FZKA 6019 (1998).
The program was run with parameters allowing to observe events with energies down
to below 50 GeV.
5. Number of Instances: 19020
6. Number of Attributes: 11 (including the class)
7. Attribute information:
1. fLength: continuous # major axis of ellipse [mm]
2. fWidth: continuous # minor axis of ellipse [mm]
3. fSize: continuous # 10-log of sum of content of all pixels [in #phot]
4. fConc: continuous # ratio of sum of two highest pixels over fSize [ratio]
5. fConc1: continuous # ratio of highest pixel over fSize [ratio]
6. fAsym: continuous # distance from highest pixel to center, projected onto major axis [mm]
7. fM3Long: continuous # 3rd root of third moment along major axis [mm]
8. fM3Trans: continuous # 3rd root of third moment along minor axis [mm]
9. fAlpha: continuous # angle of major axis with vector to origin [deg]
10. fDist: continuous # distance from origin to center of ellipse [mm]
11. class: g,h # gamma (signal), hadron (background)
8. Missing Attribute Values: None
9. Class Distribution:
g = gamma (signal): 12332
h = hadron (background): 6688
For technical reasons, the number of h events is underestimated.
In the real data, the h class represents the majority of the events.
The simple classification accuracy is not meaningful for this data, since
classifying a background event as signal is worse than classifying a signal
event as background. For comparison of different classifiers an ROC curve
has to be used. The relevant points on this curve are those, where the
probability of accepting a background event as signal is below one of the
following thresholds: 0.01, 0.02, 0.05, 0.1, 0.2 depending on the required
quality of the sample of the accepted events for different experiments.

210
data/seeds/seeds_dataset.txt Executable file
View File

@@ -0,0 +1,210 @@
15.26 14.84 0.871 5.763 3.312 2.221 5.22 1
14.88 14.57 0.8811 5.554 3.333 1.018 4.956 1
14.29 14.09 0.905 5.291 3.337 2.699 4.825 1
13.84 13.94 0.8955 5.324 3.379 2.259 4.805 1
16.14 14.99 0.9034 5.658 3.562 1.355 5.175 1
14.38 14.21 0.8951 5.386 3.312 2.462 4.956 1
14.69 14.49 0.8799 5.563 3.259 3.586 5.219 1
14.11 14.1 0.8911 5.42 3.302 2.7 5 1
16.63 15.46 0.8747 6.053 3.465 2.04 5.877 1
16.44 15.25 0.888 5.884 3.505 1.969 5.533 1
15.26 14.85 0.8696 5.714 3.242 4.543 5.314 1
14.03 14.16 0.8796 5.438 3.201 1.717 5.001 1
13.89 14.02 0.888 5.439 3.199 3.986 4.738 1
13.78 14.06 0.8759 5.479 3.156 3.136 4.872 1
13.74 14.05 0.8744 5.482 3.114 2.932 4.825 1
14.59 14.28 0.8993 5.351 3.333 4.185 4.781 1
13.99 13.83 0.9183 5.119 3.383 5.234 4.781 1
15.69 14.75 0.9058 5.527 3.514 1.599 5.046 1
14.7 14.21 0.9153 5.205 3.466 1.767 4.649 1
12.72 13.57 0.8686 5.226 3.049 4.102 4.914 1
14.16 14.4 0.8584 5.658 3.129 3.072 5.176 1
14.11 14.26 0.8722 5.52 3.168 2.688 5.219 1
15.88 14.9 0.8988 5.618 3.507 0.7651 5.091 1
12.08 13.23 0.8664 5.099 2.936 1.415 4.961 1
15.01 14.76 0.8657 5.789 3.245 1.791 5.001 1
16.19 15.16 0.8849 5.833 3.421 0.903 5.307 1
13.02 13.76 0.8641 5.395 3.026 3.373 4.825 1
12.74 13.67 0.8564 5.395 2.956 2.504 4.869 1
14.11 14.18 0.882 5.541 3.221 2.754 5.038 1
13.45 14.02 0.8604 5.516 3.065 3.531 5.097 1
13.16 13.82 0.8662 5.454 2.975 0.8551 5.056 1
15.49 14.94 0.8724 5.757 3.371 3.412 5.228 1
14.09 14.41 0.8529 5.717 3.186 3.92 5.299 1
13.94 14.17 0.8728 5.585 3.15 2.124 5.012 1
15.05 14.68 0.8779 5.712 3.328 2.129 5.36 1
16.12 15 0.9 5.709 3.485 2.27 5.443 1
16.2 15.27 0.8734 5.826 3.464 2.823 5.527 1
17.08 15.38 0.9079 5.832 3.683 2.956 5.484 1
14.8 14.52 0.8823 5.656 3.288 3.112 5.309 1
14.28 14.17 0.8944 5.397 3.298 6.685 5.001 1
13.54 13.85 0.8871 5.348 3.156 2.587 5.178 1
13.5 13.85 0.8852 5.351 3.158 2.249 5.176 1
13.16 13.55 0.9009 5.138 3.201 2.461 4.783 1
15.5 14.86 0.882 5.877 3.396 4.711 5.528 1
15.11 14.54 0.8986 5.579 3.462 3.128 5.18 1
13.8 14.04 0.8794 5.376 3.155 1.56 4.961 1
15.36 14.76 0.8861 5.701 3.393 1.367 5.132 1
14.99 14.56 0.8883 5.57 3.377 2.958 5.175 1
14.79 14.52 0.8819 5.545 3.291 2.704 5.111 1
14.86 14.67 0.8676 5.678 3.258 2.129 5.351 1
14.43 14.4 0.8751 5.585 3.272 3.975 5.144 1
15.78 14.91 0.8923 5.674 3.434 5.593 5.136 1
14.49 14.61 0.8538 5.715 3.113 4.116 5.396 1
14.33 14.28 0.8831 5.504 3.199 3.328 5.224 1
14.52 14.6 0.8557 5.741 3.113 1.481 5.487 1
15.03 14.77 0.8658 5.702 3.212 1.933 5.439 1
14.46 14.35 0.8818 5.388 3.377 2.802 5.044 1
14.92 14.43 0.9006 5.384 3.412 1.142 5.088 1
15.38 14.77 0.8857 5.662 3.419 1.999 5.222 1
12.11 13.47 0.8392 5.159 3.032 1.502 4.519 1
11.42 12.86 0.8683 5.008 2.85 2.7 4.607 1
11.23 12.63 0.884 4.902 2.879 2.269 4.703 1
12.36 13.19 0.8923 5.076 3.042 3.22 4.605 1
13.22 13.84 0.868 5.395 3.07 4.157 5.088 1
12.78 13.57 0.8716 5.262 3.026 1.176 4.782 1
12.88 13.5 0.8879 5.139 3.119 2.352 4.607 1
14.34 14.37 0.8726 5.63 3.19 1.313 5.15 1
14.01 14.29 0.8625 5.609 3.158 2.217 5.132 1
14.37 14.39 0.8726 5.569 3.153 1.464 5.3 1
12.73 13.75 0.8458 5.412 2.882 3.533 5.067 1
17.63 15.98 0.8673 6.191 3.561 4.076 6.06 2
16.84 15.67 0.8623 5.998 3.484 4.675 5.877 2
17.26 15.73 0.8763 5.978 3.594 4.539 5.791 2
19.11 16.26 0.9081 6.154 3.93 2.936 6.079 2
16.82 15.51 0.8786 6.017 3.486 4.004 5.841 2
16.77 15.62 0.8638 5.927 3.438 4.92 5.795 2
17.32 15.91 0.8599 6.064 3.403 3.824 5.922 2
20.71 17.23 0.8763 6.579 3.814 4.451 6.451 2
18.94 16.49 0.875 6.445 3.639 5.064 6.362 2
17.12 15.55 0.8892 5.85 3.566 2.858 5.746 2
16.53 15.34 0.8823 5.875 3.467 5.532 5.88 2
18.72 16.19 0.8977 6.006 3.857 5.324 5.879 2
20.2 16.89 0.8894 6.285 3.864 5.173 6.187 2
19.57 16.74 0.8779 6.384 3.772 1.472 6.273 2
19.51 16.71 0.878 6.366 3.801 2.962 6.185 2
18.27 16.09 0.887 6.173 3.651 2.443 6.197 2
18.88 16.26 0.8969 6.084 3.764 1.649 6.109 2
18.98 16.66 0.859 6.549 3.67 3.691 6.498 2
21.18 17.21 0.8989 6.573 4.033 5.78 6.231 2
20.88 17.05 0.9031 6.45 4.032 5.016 6.321 2
20.1 16.99 0.8746 6.581 3.785 1.955 6.449 2
18.76 16.2 0.8984 6.172 3.796 3.12 6.053 2
18.81 16.29 0.8906 6.272 3.693 3.237 6.053 2
18.59 16.05 0.9066 6.037 3.86 6.001 5.877 2
18.36 16.52 0.8452 6.666 3.485 4.933 6.448 2
16.87 15.65 0.8648 6.139 3.463 3.696 5.967 2
19.31 16.59 0.8815 6.341 3.81 3.477 6.238 2
18.98 16.57 0.8687 6.449 3.552 2.144 6.453 2
18.17 16.26 0.8637 6.271 3.512 2.853 6.273 2
18.72 16.34 0.881 6.219 3.684 2.188 6.097 2
16.41 15.25 0.8866 5.718 3.525 4.217 5.618 2
17.99 15.86 0.8992 5.89 3.694 2.068 5.837 2
19.46 16.5 0.8985 6.113 3.892 4.308 6.009 2
19.18 16.63 0.8717 6.369 3.681 3.357 6.229 2
18.95 16.42 0.8829 6.248 3.755 3.368 6.148 2
18.83 16.29 0.8917 6.037 3.786 2.553 5.879 2
18.85 16.17 0.9056 6.152 3.806 2.843 6.2 2
17.63 15.86 0.88 6.033 3.573 3.747 5.929 2
19.94 16.92 0.8752 6.675 3.763 3.252 6.55 2
18.55 16.22 0.8865 6.153 3.674 1.738 5.894 2
18.45 16.12 0.8921 6.107 3.769 2.235 5.794 2
19.38 16.72 0.8716 6.303 3.791 3.678 5.965 2
19.13 16.31 0.9035 6.183 3.902 2.109 5.924 2
19.14 16.61 0.8722 6.259 3.737 6.682 6.053 2
20.97 17.25 0.8859 6.563 3.991 4.677 6.316 2
19.06 16.45 0.8854 6.416 3.719 2.248 6.163 2
18.96 16.2 0.9077 6.051 3.897 4.334 5.75 2
19.15 16.45 0.889 6.245 3.815 3.084 6.185 2
18.89 16.23 0.9008 6.227 3.769 3.639 5.966 2
20.03 16.9 0.8811 6.493 3.857 3.063 6.32 2
20.24 16.91 0.8897 6.315 3.962 5.901 6.188 2
18.14 16.12 0.8772 6.059 3.563 3.619 6.011 2
16.17 15.38 0.8588 5.762 3.387 4.286 5.703 2
18.43 15.97 0.9077 5.98 3.771 2.984 5.905 2
15.99 14.89 0.9064 5.363 3.582 3.336 5.144 2
18.75 16.18 0.8999 6.111 3.869 4.188 5.992 2
18.65 16.41 0.8698 6.285 3.594 4.391 6.102 2
17.98 15.85 0.8993 5.979 3.687 2.257 5.919 2
20.16 17.03 0.8735 6.513 3.773 1.91 6.185 2
17.55 15.66 0.8991 5.791 3.69 5.366 5.661 2
18.3 15.89 0.9108 5.979 3.755 2.837 5.962 2
18.94 16.32 0.8942 6.144 3.825 2.908 5.949 2
15.38 14.9 0.8706 5.884 3.268 4.462 5.795 2
16.16 15.33 0.8644 5.845 3.395 4.266 5.795 2
15.56 14.89 0.8823 5.776 3.408 4.972 5.847 2
15.38 14.66 0.899 5.477 3.465 3.6 5.439 2
17.36 15.76 0.8785 6.145 3.574 3.526 5.971 2
15.57 15.15 0.8527 5.92 3.231 2.64 5.879 2
15.6 15.11 0.858 5.832 3.286 2.725 5.752 2
16.23 15.18 0.885 5.872 3.472 3.769 5.922 2
13.07 13.92 0.848 5.472 2.994 5.304 5.395 3
13.32 13.94 0.8613 5.541 3.073 7.035 5.44 3
13.34 13.95 0.862 5.389 3.074 5.995 5.307 3
12.22 13.32 0.8652 5.224 2.967 5.469 5.221 3
11.82 13.4 0.8274 5.314 2.777 4.471 5.178 3
11.21 13.13 0.8167 5.279 2.687 6.169 5.275 3
11.43 13.13 0.8335 5.176 2.719 2.221 5.132 3
12.49 13.46 0.8658 5.267 2.967 4.421 5.002 3
12.7 13.71 0.8491 5.386 2.911 3.26 5.316 3
10.79 12.93 0.8107 5.317 2.648 5.462 5.194 3
11.83 13.23 0.8496 5.263 2.84 5.195 5.307 3
12.01 13.52 0.8249 5.405 2.776 6.992 5.27 3
12.26 13.6 0.8333 5.408 2.833 4.756 5.36 3
11.18 13.04 0.8266 5.22 2.693 3.332 5.001 3
11.36 13.05 0.8382 5.175 2.755 4.048 5.263 3
11.19 13.05 0.8253 5.25 2.675 5.813 5.219 3
11.34 12.87 0.8596 5.053 2.849 3.347 5.003 3
12.13 13.73 0.8081 5.394 2.745 4.825 5.22 3
11.75 13.52 0.8082 5.444 2.678 4.378 5.31 3
11.49 13.22 0.8263 5.304 2.695 5.388 5.31 3
12.54 13.67 0.8425 5.451 2.879 3.082 5.491 3
12.02 13.33 0.8503 5.35 2.81 4.271 5.308 3
12.05 13.41 0.8416 5.267 2.847 4.988 5.046 3
12.55 13.57 0.8558 5.333 2.968 4.419 5.176 3
11.14 12.79 0.8558 5.011 2.794 6.388 5.049 3
12.1 13.15 0.8793 5.105 2.941 2.201 5.056 3
12.44 13.59 0.8462 5.319 2.897 4.924 5.27 3
12.15 13.45 0.8443 5.417 2.837 3.638 5.338 3
11.35 13.12 0.8291 5.176 2.668 4.337 5.132 3
11.24 13 0.8359 5.09 2.715 3.521 5.088 3
11.02 13 0.8189 5.325 2.701 6.735 5.163 3
11.55 13.1 0.8455 5.167 2.845 6.715 4.956 3
11.27 12.97 0.8419 5.088 2.763 4.309 5 3
11.4 13.08 0.8375 5.136 2.763 5.588 5.089 3
10.83 12.96 0.8099 5.278 2.641 5.182 5.185 3
10.8 12.57 0.859 4.981 2.821 4.773 5.063 3
11.26 13.01 0.8355 5.186 2.71 5.335 5.092 3
10.74 12.73 0.8329 5.145 2.642 4.702 4.963 3
11.48 13.05 0.8473 5.18 2.758 5.876 5.002 3
12.21 13.47 0.8453 5.357 2.893 1.661 5.178 3
11.41 12.95 0.856 5.09 2.775 4.957 4.825 3
12.46 13.41 0.8706 5.236 3.017 4.987 5.147 3
12.19 13.36 0.8579 5.24 2.909 4.857 5.158 3
11.65 13.07 0.8575 5.108 2.85 5.209 5.135 3
12.89 13.77 0.8541 5.495 3.026 6.185 5.316 3
11.56 13.31 0.8198 5.363 2.683 4.062 5.182 3
11.81 13.45 0.8198 5.413 2.716 4.898 5.352 3
10.91 12.8 0.8372 5.088 2.675 4.179 4.956 3
11.23 12.82 0.8594 5.089 2.821 7.524 4.957 3
10.59 12.41 0.8648 4.899 2.787 4.975 4.794 3
10.93 12.8 0.839 5.046 2.717 5.398 5.045 3
11.27 12.86 0.8563 5.091 2.804 3.985 5.001 3
11.87 13.02 0.8795 5.132 2.953 3.597 5.132 3
10.82 12.83 0.8256 5.18 2.63 4.853 5.089 3
12.11 13.27 0.8639 5.236 2.975 4.132 5.012 3
12.8 13.47 0.886 5.16 3.126 4.873 4.914 3
12.79 13.53 0.8786 5.224 3.054 5.483 4.958 3
13.37 13.78 0.8849 5.32 3.128 4.67 5.091 3
12.62 13.67 0.8481 5.41 2.911 3.306 5.231 3
12.76 13.38 0.8964 5.073 3.155 2.828 4.83 3
12.38 13.44 0.8609 5.219 2.989 5.472 5.045 3
12.67 13.32 0.8977 4.984 3.135 2.3 4.745 3
11.18 12.72 0.868 5.009 2.81 4.051 4.828 3
12.7 13.41 0.8874 5.183 3.091 8.456 5 3
12.37 13.47 0.8567 5.204 2.96 3.919 5.001 3
12.19 13.2 0.8783 5.137 2.981 3.631 4.87 3
11.23 12.88 0.8511 5.14 2.795 4.325 5.003 3
13.2 13.66 0.8883 5.236 3.232 8.315 5.056 3
11.84 13.21 0.8521 5.175 2.836 3.598 5.044 3
12.3 13.34 0.8684 5.243 2.974 5.637 5.063 3

3
fashion_mnist.py
View File

@@ -109,8 +109,7 @@ best_score = pso_mnist.fit(
save_info=True, save_info=True,
log=2, log=2,
log_name="fashion_mnist", log_name="fashion_mnist",
save_path="./logs/fashion_mnist", renewal="acc",
renewal="mse",
check_point=25, check_point=25,
empirical_balance=False, empirical_balance=False,
dispersion=False, dispersion=False,

12
iris.py
View File

@@ -50,22 +50,24 @@ pso_iris = optimizer(
c1=0.6, c1=0.6,
w_min=0.5, w_min=0.5,
w_max=0.9, w_max=0.9,
negative_swarm=0.1, negative_swarm=0,
mutation_swarm=0.2, mutation_swarm=0.2,
particle_min=-3, convergence_reset=True,
particle_max=3, convergence_reset_patience=10,
convergence_reset_monitor="mse",
convergence_reset_min_delta=0.05,
) )
best_score = pso_iris.fit( best_score = pso_iris.fit(
x_train, x_train,
y_train, y_train,
epochs=200, epochs=500,
save_info=True, save_info=True,
log=2, log=2,
log_name="iris", log_name="iris",
save_path="result/iris",
renewal="acc", renewal="acc",
check_point=25, check_point=25,
validate_data=(x_test, y_test),
) )
gc.collect() gc.collect()

44
mnist.py
View File

@@ -40,10 +40,11 @@ def make_model():
input_shape=(28, 28, 1)) input_shape=(28, 28, 1))
) )
model.add(MaxPooling2D(pool_size=(2, 2))) model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.5))
model.add(Conv2D(64, kernel_size=(3, 3), activation="relu")) model.add(Conv2D(64, kernel_size=(3, 3), activation="relu"))
model.add(MaxPooling2D(pool_size=(2, 2))) model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten()) model.add(Flatten())
model.add(Dropout(0.25)) model.add(Dropout(0.5))
model.add(Dense(256, activation="relu")) model.add(Dense(256, activation="relu"))
model.add(Dense(128, activation="relu")) model.add(Dense(128, activation="relu"))
model.add(Dense(10, activation="softmax")) model.add(Dense(10, activation="softmax"))
@@ -51,23 +52,6 @@ def make_model():
return model return model
def random_state():
with open(
"result/mnist/20230723-061626/mean_squared_error_[0.6384999752044678, 0.0723000094294548].json",
"r",
) as f:
json_ = json.load(f)
rs = (
json_["random_state_0"],
np.array(json_["random_state_1"]),
json_["random_state_2"],
json_["random_state_3"],
json_["random_state_4"],
)
return rs
# %% # %%
model = make_model() model = make_model()
x_train, y_train, x_test, y_test = get_data() x_train, y_train, x_test, y_test = get_data()
@@ -91,17 +75,17 @@ loss = [
pso_mnist = optimizer( pso_mnist = optimizer(
model, model,
loss="categorical_crossentropy", loss="categorical_crossentropy",
n_particles=500, n_particles=100,
c0=0.5, c0=0.5,
c1=1.0, c1=0.8,
w_min=0.7, w_min=0.6,
w_max=0.9, w_max=0.9,
negative_swarm=0.05, negative_swarm=0.0,
mutation_swarm=0.3, mutation_swarm=0.2,
convergence_reset=True, convergence_reset=True,
convergence_reset_patience=10, convergence_reset_patience=10,
convergence_reset_monitor="mse", convergence_reset_monitor="loss",
convergence_reset_min_delta=0.0005, convergence_reset_min_delta=0.05,
) )
best_score = pso_mnist.fit( best_score = pso_mnist.fit(
@@ -111,13 +95,13 @@ best_score = pso_mnist.fit(
save_info=True, save_info=True,
log=2, log=2,
log_name="mnist", log_name="mnist",
save_path="./logs/mnist", renewal="loss",
renewal="acc",
check_point=25, check_point=25,
empirical_balance=False, empirical_balance=True,
dispersion=False, dispersion=False,
batch_size=5000, batch_size=2048,
back_propagation=True, back_propagation=False,
validate_data=(x_test, y_test),
) )
print("Done!") print("Done!")

67
mnist_tf.py
View File

@@ -39,20 +39,51 @@ def get_data_test():
return x_test, y_test return x_test, y_test
class _batch_generator: class _batch_generator_:
def __init__(self, x, y, batch_size: int = 32): def __init__(self, x, y, batch_size: int = None):
self.batch_size = batch_size
self.index = 0 self.index = 0
dataset = tf.data.Dataset.from_tensor_slices((x, y)) self.x = x
self.dataset = list(dataset.batch(batch_size)) self.y = y
self.max_index = len(dataset) // batch_size self.setBatchSize(batch_size)
def next(self): def next(self):
self.index += 1 self.index += 1
if self.index >= self.max_index: if self.index >= self.max_index:
self.index = 0 self.index = 0
self.__getBatchSlice(self.batch_size)
return self.dataset[self.index][0], self.dataset[self.index][1] return self.dataset[self.index][0], self.dataset[self.index][1]
def getMaxIndex(self):
return self.max_index
def getIndex(self):
return self.index
def setIndex(self, index):
self.index = index
def getBatchSize(self):
return self.batch_size
def setBatchSize(self, batch_size: int = None):
if batch_size is None:
batch_size = len(self.x) // 10
elif batch_size > len(self.x):
batch_size = len(self.x)
self.batch_size = batch_size
print(f"batch size : {self.batch_size}")
self.dataset = self.__getBatchSlice(self.batch_size)
self.max_index = len(self.dataset)
def __getBatchSlice(self, batch_size):
return list(
tf.data.Dataset.from_tensor_slices(
(self.x, self.y)).shuffle(len(self.x)).batch(batch_size)
)
def getDataset(self):
return self.dataset
def make_model(): def make_model():
model = Sequential() model = Sequential()
@@ -60,11 +91,13 @@ def make_model():
Conv2D(32, kernel_size=(5, 5), activation="relu", Conv2D(32, kernel_size=(5, 5), activation="relu",
input_shape=(28, 28, 1)) input_shape=(28, 28, 1))
) )
model.add(MaxPooling2D(pool_size=(3, 3))) model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.5))
model.add(Conv2D(64, kernel_size=(3, 3), activation="relu")) model.add(Conv2D(64, kernel_size=(3, 3), activation="relu"))
model.add(MaxPooling2D(pool_size=(2, 2))) model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten()) model.add(Flatten())
model.add(Dropout(0.5))
model.add(Dense(256, activation="relu"))
model.add(Dense(128, activation="relu")) model.add(Dense(128, activation="relu"))
model.add(Dense(10, activation="softmax")) model.add(Dense(10, activation="softmax"))
@@ -76,21 +109,25 @@ x_train, y_train, x_test, y_test = get_data()
y_train = tf.one_hot(y_train, 10) y_train = tf.one_hot(y_train, 10)
y_test = tf.one_hot(y_test, 10) y_test = tf.one_hot(y_test, 10)
dataset = _batch_generator(x_train, y_train, 64) batch = 64
dataset = _batch_generator_(x_train, y_train, batch)
model.compile(optimizer="adam", loss="mse", metrics=["accuracy"]) model.compile(optimizer="adam", loss="categorical_crossentropy",
metrics=["accuracy", "mse", "mae"])
count = 0 count = 0
print(f"batch size : {batch}")
while count < 20: print("iter " + str(dataset.getMaxIndex()))
print("Training model...")
while count < dataset.getMaxIndex():
x_batch, y_batch = dataset.next() x_batch, y_batch = dataset.next()
count += 1 count += 1
print("Training model...") print(f"iter {count}/{dataset.getMaxIndex()}")
model.fit(x_batch, y_batch, epochs=1, batch_size=1, verbose=1) model.fit(x_batch, y_batch, epochs=1, batch_size=batch, verbose=1)
print(count) print(count)
print("Evaluating model...") print("Evaluating model...")
model.evaluate(x_test, y_test, verbose=2) model.evaluate(x_test, y_test, verbose=1)
weights = model.get_weights() weights = model.get_weights()

2
pso/__init__.py
View File

@@ -1,7 +1,7 @@
from .optimizer import Optimizer as optimizer from .optimizer import Optimizer as optimizer
from .particle import Particle as particle from .particle import Particle as particle
__version__ = "1.0.2" __version__ = "1.0.3"
__all__ = [ __all__ = [
"optimizer", "optimizer",

276
pso/optimizer.py
View File

@@ -18,6 +18,8 @@ if gpus:
except RuntimeError as r: except RuntimeError as r:
print(r) print(r)
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"
class Optimizer: class Optimizer:
""" """
@@ -107,12 +109,9 @@ class Optimizer:
self.w_max = w_max # 최대 관성 수치 self.w_max = w_max # 최대 관성 수치
self.negative_swarm = negative_swarm # 최적해와 반대로 이동할 파티클 비율 - 0 ~ 1 사이의 값 self.negative_swarm = negative_swarm # 최적해와 반대로 이동할 파티클 비율 - 0 ~ 1 사이의 값
self.mutation_swarm = mutation_swarm # 관성을 추가로 사용할 파티클 비율 - 0 ~ 1 사이의 값 self.mutation_swarm = mutation_swarm # 관성을 추가로 사용할 파티클 비율 - 0 ~ 1 사이의 값
self.g_best_score = [np.inf, 0, np.inf] # 최고 점수 - 시작은 0으로 초기화
self.g_best = model.get_weights() # 최고 점수를 받은 가중치
self.avg_score = 0 # 평균 점수 self.avg_score = 0 # 평균 점수
# self.sigma = 1.0 # self.sigma = 1.0
self.save_path = None # 저장 위치
self.renewal = "acc" self.renewal = "acc"
self.dispersion = False self.dispersion = False
self.day = datetime.now().strftime("%Y%m%d-%H%M%S") self.day = datetime.now().strftime("%Y%m%d-%H%M%S")
@@ -139,13 +138,12 @@ class Optimizer:
if i < self.negative_swarm * self.n_particles: if i < self.negative_swarm * self.n_particles:
negative_count += 1 negative_count += 1
# del m, init_weights, w_, sh_, len_
gc.collect() gc.collect()
tf.keras.backend.reset_uids() tf.keras.backend.reset_uids()
tf.keras.backend.clear_session() tf.keras.backend.clear_session()
# del model_ self.particles[0].update_global_best()
print(f"negative swarm : {negative_count} / {n_particles}") print(f"negative swarm : {negative_count} / {n_particles}")
print(f"mutation swarm : {mutation_swarm * 100}%") print(f"mutation swarm : {mutation_swarm * 100}%")
@@ -171,15 +169,13 @@ class Optimizer:
del self.w_min del self.w_min
del self.w_max del self.w_max
del self.negative_swarm del self.negative_swarm
del self.g_best_score
del self.g_best
del self.avg_score del self.avg_score
gc.collect() gc.collect()
tf.keras.backend.reset_uids() tf.keras.backend.reset_uids()
tf.keras.backend.clear_session() tf.keras.backend.clear_session()
def _encode_(self, weights): def _encode(self, weights):
""" """
가중치를 1차원으로 풀어서 반환 가중치를 1차원으로 풀어서 반환
@@ -205,7 +201,7 @@ class Optimizer:
def _decode_(self, weight, shape, length): def _decode_(self, weight, shape, length):
""" """
_encode_ 로 인코딩된 가중치를 원본 shape으로 복원 _encode 로 인코딩된 가중치를 원본 shape으로 복원
파라미터는 encode의 리턴값을 그대로 사용을 권장 파라미터는 encode의 리턴값을 그대로 사용을 권장
Args: Args:
@@ -243,17 +239,19 @@ class Optimizer:
""" """
self.model.set_weights(weights) self.model.set_weights(weights)
score = self.model.evaluate(x, y, verbose=0) score = self.model.evaluate(x, y, verbose=0)
if self.renewal == "acc": if self.renewal == "loss":
score_ = score[1]
else:
score_ = score[0] score_ = score[0]
elif self.renewal == "acc":
score_ = score[1]
elif self.renewal == "mse":
score_ = score[2]
if score_ > 0: if score_ > 0:
return 1 / (1 + score_) return 1 / (1 + score_)
else: else:
return 1 + np.abs(score_) return 1 + np.abs(score_)
def __weight_range__(self): def __weight_range(self):
""" """
가중치의 범위를 반환 가중치의 범위를 반환
@@ -261,7 +259,7 @@ class Optimizer:
(float): 가중치의 최소값 (float): 가중치의 최소값
(float): 가중치의 최대값 (float): 가중치의 최대값
""" """
w_, w_s, w_l = self._encode_(self.g_best) w_, w_s, w_l = self._encode(Particle.g_best_weights)
weight_min = np.min(w_) weight_min = np.min(w_)
weight_max = np.max(w_) weight_max = np.max(w_)
@@ -269,9 +267,8 @@ class Optimizer:
return weight_min, weight_max return weight_min, weight_max
class _batch_generator_: class batch_generator:
def __init__(self, x, y, batch_size: int = 32): def __init__(self, x, y, batch_size: int = None):
self.batch_size = batch_size
self.index = 0 self.index = 0
self.x = x self.x = x
self.y = y self.y = y
@@ -281,7 +278,7 @@ class Optimizer:
self.index += 1 self.index += 1
if self.index >= self.max_index: if self.index >= self.max_index:
self.index = 0 self.index = 0
self.__getBatchSlice__(self.batch_size) self.__getBatchSlice(self.batch_size)
return self.dataset[self.index][0], self.dataset[self.index][1] return self.dataset[self.index][0], self.dataset[self.index][1]
def getMaxIndex(self): def getMaxIndex(self):
@@ -296,15 +293,20 @@ class Optimizer:
def getBatchSize(self): def getBatchSize(self):
return self.batch_size return self.batch_size
def setBatchSize(self, batch_size): def setBatchSize(self, batch_size: int = None):
if batch_size is None:
batch_size = len(self.x) // 10
elif batch_size > len(self.x):
batch_size = len(self.x)
self.batch_size = batch_size self.batch_size = batch_size
if self.batch_size > len(self.x):
self.batch_size = len(self.x)
print(f"batch size : {self.batch_size}")
self.dataset = self.__getBatchSlice__(self.batch_size)
self.max_index = len(self.dataset)
def __getBatchSlice__(self, batch_size): print(f"batch size : {self.batch_size}")
self.dataset = self.__getBatchSlice(self.batch_size)
self.max_index = len(self.dataset)
if batch_size % len(self.x) != 0:
self.max_index -= 1
def __getBatchSlice(self, batch_size):
return list( return list(
tf.data.Dataset.from_tensor_slices( tf.data.Dataset.from_tensor_slices(
(self.x, self.y)).shuffle(len(self.x)).batch(batch_size) (self.x, self.y)).shuffle(len(self.x)).batch(batch_size)
@@ -321,13 +323,12 @@ class Optimizer:
log: int = 0, log: int = 0,
log_name: str = None, log_name: str = None,
save_info: bool = False, save_info: bool = False,
save_path: str = "./logs",
renewal: str = "mse", renewal: str = "mse",
empirical_balance: bool = False, empirical_balance: bool = False,
dispersion: bool = False, dispersion: bool = False,
check_point: int = None, check_point: int = None,
batch_size: int = None, batch_size: int = None,
validate_data: any = None, validate_data: tuple = None,
back_propagation: bool = False, back_propagation: bool = False,
): ):
""" """
@@ -344,6 +345,8 @@ class Optimizer:
check_point : int - 저장할 위치 - None : 저장 안함 check_point : int - 저장할 위치 - None : 저장 안함
batch_size : int - batch size default : None => len(x) // 10 batch_size : int - batch size default : None => len(x) // 10
batch_size > len(x) : auto max batch size batch_size > len(x) : auto max batch size
validate_data : tuple - (x, y) default : None => (x, y)
back_propagation : bool - True : back propagation, False : not back propagation
""" """
try: try:
if x.shape[0] != y.shape[0]: if x.shape[0] != y.shape[0]:
@@ -352,47 +355,53 @@ class Optimizer:
if log not in [0, 1, 2]: if log not in [0, 1, 2]:
raise ValueError("log not in [0, 1, 2]") raise ValueError("log not in [0, 1, 2]")
if save_info and save_path is None:
raise ValueError("save_path is None")
if renewal not in ["acc", "loss", "mse"]: if renewal not in ["acc", "loss", "mse"]:
raise ValueError("renewal not in ['acc', 'loss', 'mse']") raise ValueError("renewal not in ['acc', 'loss', 'mse']")
if check_point is not None and save_path is None: if validate_data is not None:
raise ValueError("save_path is None") if validate_data[0].shape[0] != validate_data[1].shape[0]:
raise ValueError("validate_data shape error")
if validate_data is None:
validate_data = (x, y)
if batch_size is not None and batch_size < 1:
raise ValueError("batch_size < 1")
if batch_size is None or batch_size > len(x):
batch_size = len(x)
except ValueError as ve: except ValueError as ve:
sys.exit(ve) sys.exit(ve)
except Exception as e:
sys.exit(e)
self.save_path = save_path
self.empirical_balance = empirical_balance self.empirical_balance = empirical_balance
self.dispersion = dispersion self.dispersion = dispersion
if batch_size is None:
batch_size = len(x) // 10
self.renewal = renewal self.renewal = renewal
particle_sum = 0 # x_j particle_sum = 0 # x_j
try: try:
train_log_dir = "logs/fit/" + self.day if log_name is None:
log_name = "fit"
self.log_path = f"logs/{log_name}/{self.day}"
if log == 2: if log == 2:
assert log_name is not None, "log_name is None" assert log_name is not None, "log_name is None"
train_log_dir = f"logs/{log_name}/{self.day}/train" train_log_dir = self.log_path + "/train"
for i in range(self.n_particles): for i in range(self.n_particles):
self.train_summary_writer[i] = tf.summary.create_file_writer( self.train_summary_writer[i] = tf.summary.create_file_writer(
train_log_dir + f"/{i}" train_log_dir + f"/{i}"
) )
elif check_point is not None or log == 1: elif check_point is not None or log == 1:
if save_path is None: if not os.path.exists(self.log_path):
raise ValueError("save_path is None") os.makedirs(self.log_path, exist_ok=True)
else:
self.save_path = save_path
if not os.path.exists(f"{save_path}/{self.day}"):
os.makedirs(f"{save_path}/{self.day}", exist_ok=True)
except ValueError as ve: except ValueError as ve:
sys.exit(ve) sys.exit(ve)
except Exception as e:
sys.exit(e)
if back_propagation: if back_propagation:
model_ = keras.models.model_from_json(self.model.to_json()) model_ = keras.models.model_from_json(self.model.to_json())
@@ -404,19 +413,19 @@ class Optimizer:
model_.fit(x, y, epochs=1, verbose=0) model_.fit(x, y, epochs=1, verbose=0)
score = model_.evaluate(x, y, verbose=1) score = model_.evaluate(x, y, verbose=1)
self.g_best_score = score Particle.g_best_score = score
self.g_best = model_.get_weights() Particle.g_best_weights = model_.get_weights()
del model_ del model_
dataset = self._batch_generator_(x, y, batch_size=batch_size) dataset = self.batch_generator(x, y, batch_size=batch_size)
try: try:
epoch_sum = 0 epoch_sum = 0
epochs_pbar = tqdm( epochs_pbar = tqdm(
range(epochs), range(epochs),
desc=f"best - loss: {self.g_best_score[0]:.4f} - acc: {self.g_best_score[1]:.4f} - mse: {self.g_best_score[2]:.4f}", desc=f"best - loss: {Particle.g_best_score[0]:.4f} - acc: {Particle.g_best_score[1]:.4f} - mse: {Particle.g_best_score[2]:.4f}",
ascii=True, ascii=True,
leave=True, leave=True,
position=0, position=0,
@@ -429,6 +438,8 @@ class Optimizer:
max_acc = 0 max_acc = 0
min_loss = np.inf min_loss = np.inf
min_mse = np.inf min_mse = np.inf
# 한번의 실행 동안 최고 점수를 받은 파티클의 인덱스
best_particle_index = 0
# epoch_particle_sum = 0 # epoch_particle_sum = 0
part_pbar = tqdm( part_pbar = tqdm(
range(len(self.particles)), range(len(self.particles)),
@@ -437,33 +448,34 @@ class Optimizer:
leave=False, leave=False,
position=1, position=1,
) )
w = self.w_max - (self.w_max - self.w_min) * epoch / epochs w = self.w_max - (self.w_max - self.w_min) * epoch / epochs
for i in part_pbar: for i in part_pbar:
part_pbar.set_description( part_pbar.set_description(
f"loss: {min_loss:.4f} acc: {max_acc:.4f} mse: {min_mse:.4f}" f"loss: {min_loss:.4f} acc: {max_acc:.4f} mse: {min_mse:.4f}"
) )
g_best = Particle.g_best_weights
g_best = self.g_best
x_batch, y_batch = dataset.next() x_batch, y_batch = dataset.next()
weight_min, weight_max = self.__weight_range__() weight_min, weight_max = self.__weight_range()
if dispersion: if dispersion:
ts = weight_min + np.random.rand() * (weight_max - weight_min) ts = weight_min + np.random.rand() * (weight_max - weight_min)
g_, g_sh, g_len = self._encode_(self.g_best) g_, g_sh, g_len = self._encode(
Particle.g_best_weights)
decrement = (epochs - epoch + 1) / epochs decrement = (epochs - epoch + 1) / epochs
g_ = (1 - decrement) * g_ + decrement * ts g_ = (1 - decrement) * g_ + decrement * ts
g_best = self._decode(g_, g_sh, g_len) g_best = self._decode_(g_, g_sh, g_len)
if empirical_balance: if empirical_balance:
if np.random.rand() < np.exp(-(epoch) / epochs): if np.random.rand() < np.exp(-(epoch) / epochs):
w_p_ = self._f( w_p_ = self._f(
x, y, self.particles[i].get_best_weights() x_batch, y_batch, self.particles[i].get_best_weights(
)
) )
w_g_ = self._f(x, y, g_best) w_g_ = self._f(x_batch, y_batch, g_best)
w_p = w_p_ / (w_p_ + w_g_) w_p = w_p_ / (w_p_ + w_g_)
w_g = w_p_ / (w_p_ + w_g_) w_g = w_p_ / (w_p_ + w_g_)
@@ -473,7 +485,7 @@ class Optimizer:
else: else:
p_b = self.particles[i].get_best_score() p_b = self.particles[i].get_best_score()
g_a = self.avg_score g_a = self.avg_score
l_b = p_b - g_a l_b = p_b[1] - g_a
sigma_post = np.sqrt(np.power(l_b, 2)) sigma_post = np.sqrt(np.power(l_b, 2))
sigma_pre = ( sigma_pre = (
1 1
@@ -517,7 +529,6 @@ class Optimizer:
self.c0, self.c0,
self.c1, self.c1,
w, w,
g_best,
w_p, w_p,
w_g, w_g,
renewal=renewal, renewal=renewal,
@@ -526,7 +537,7 @@ class Optimizer:
else: else:
score = self.particles[i].step( score = self.particles[i].step(
x_batch, y_batch, self.c0, self.c1, w, g_best, renewal=renewal x_batch, y_batch, self.c0, self.c1, w, renewal=renewal
) )
if log == 2: if log == 2:
@@ -537,98 +548,84 @@ class Optimizer:
) )
tf.summary.scalar("mse", score[2], step=epoch + 1) tf.summary.scalar("mse", score[2], step=epoch + 1)
if renewal == "acc": if renewal == "loss":
# 최고 점수 보다 높거나 같을 경우
if score[1] >= max_acc:
# 각 점수 갱신
min_loss, max_acc, min_mse = score
# 최고 점수 보다 같거나 높을 경우
if score[1] >= self.g_best_score[1]:
# 최고 점수 보다 높을 경우
if score[1] > self.g_best_score[1]:
# 최고 점수 갱신
self.g_best_score = score
# 최고 weight 갱신
self.g_best = self.particles[i].get_best_weights(
)
# 최고 점수 와 같을 경우
else:
# 최저 loss 보다 낮을 경우
if score[0] < self.g_best_score[0]:
self.g_best_score[0] = score[0]
self.g_best = self.particles[i].get_best_weights(
)
epochs_pbar.set_description(
f"best - loss: {self.g_best_score[0]:.4f} - acc: {self.g_best_score[1]:.4f} - mse: {self.g_best_score[2]:.4f}"
)
elif renewal == "loss":
# 최저 loss 보다 작거나 같을 경우 # 최저 loss 보다 작거나 같을 경우
if score[0] <= min_loss: if score[0] < min_loss:
# 각 점수 갱신 # 각 점수 갱신
min_loss, max_acc, min_mse = score min_loss, max_acc, min_mse = score
# 최저 loss 와 같거나 작을 경우 best_particle_index = i
if score[0] <= self.g_best_score[0]: elif renewal == "acc":
# 최저 loss 보다 작을 경우 # 최고 점수 보다 높거나 같을 경우
if score[0] < self.g_best_score[0]: if score[1] > max_acc:
# 최고 점수 갱신 # 각 점수 갱신
self.g_best_score = score min_loss, max_acc, min_mse = score
# 최고 weight 갱신
self.g_best = self.particles[i].get_best_weights( best_particle_index = i
)
# 최저 loss 와 같을 경우
else:
# 최고 acc 보다 높을 경우
if score[1] > self.g_best_score[1]:
self.g_best_score[1] = score[1]
self.g_best = self.particles[i].get_best_weights(
)
epochs_pbar.set_description(
f"best - loss: {self.g_best_score[0]:.4f} - acc: {self.g_best_score[1]:.4f} - mse: {self.g_best_score[2]:.4f}"
)
elif renewal == "mse": elif renewal == "mse":
if score[2] <= min_mse: if score[2] < min_mse:
min_loss, max_acc, min_mse = score min_loss, max_acc, min_mse = score
if score[2] <= self.g_best_score[2]: best_particle_index = i
if score[2] < self.g_best_score[2]:
self.g_best_score[0] = score[0]
self.g_best_score[1] = score[1]
self.g_best_score[2] = score[2]
self.g_best = self.particles[i].get_best_weights(
)
else:
if score[1] > self.g_best_score[1]:
self.g_best_score[1] = score[1]
self.g_best = self.particles[i].get_best_weights(
)
epochs_pbar.set_description(
f"best - loss: {self.g_best_score[0]:.4f} - acc: {self.g_best_score[1]:.4f} - mse: {self.g_best_score[2]:.4f}"
)
particle_sum += score[1] particle_sum += score[1]
if log == 1: if log == 1:
with open( with open(
f"./{save_path}/{self.day}/{self.n_particles}_{epochs}_{self.c0}_{self.c1}_{self.w_min}_{renewal}.csv", f"./logs/{log_name}/{self.day}/{self.n_particles}_{epochs}_{self.c0}_{self.c1}_{self.w_min}_{renewal}.csv",
"a", "a",
) as f: ) as f:
f.write(f"{score[0]}, {score[1]}") f.write(f"{score[0]}, {score[1]}, {score[2]}")
if i != self.n_particles - 1: if i != self.n_particles - 1:
f.write(", ") f.write(", ")
else: else:
f.write("\n") f.write("\n")
part_pbar.refresh() part_pbar.refresh()
# 한번 epoch 가 끝나고 갱신을 진행해야 순간적으로 높은 파티클이 발생해도 오류가 생기지 않음
if renewal == "loss":
if min_loss <= Particle.g_best_score[0]:
if min_loss < Particle.g_best_score[0]:
self.particles[best_particle_index].update_global_best(
)
else:
if max_acc > Particle.g_best_score[1]:
self.particles[best_particle_index].update_global_best(
)
elif renewal == "acc":
if max_acc >= Particle.g_best_score[1]:
# 최고 점수 보다 높을 경우
if max_acc > Particle.g_best_score[1]:
# 최고 점수 갱신
self.particles[best_particle_index].update_global_best(
)
# 최고 점수 와 같을 경우
else:
# 최저 loss 보다 낮을 경우
if min_loss < Particle.g_best_score[0]:
self.particles[best_particle_index].update_global_best(
)
elif renewal == "mse":
if min_mse <= Particle.g_best_score[2]:
if min_mse < Particle.g_best_score[2]:
self.particles[best_particle_index].update_global_best(
)
else:
if max_acc > Particle.g_best_score[1]:
self.particles[best_particle_index].update_global_best(
)
# 최고 점수 갱신
epochs_pbar.set_description(
f"best - loss: {Particle.g_best_score[0]:.4f} - acc: {Particle.g_best_score[1]:.4f} - mse: {Particle.g_best_score[2]:.4f}"
)
if check_point is not None: if check_point is not None:
if epoch % check_point == 0: if epoch % check_point == 0:
os.makedirs(f"./{save_path}/{self.day}", exist_ok=True) os.makedirs(
f"./logs/{log_name}/{self.day}", exist_ok=True)
self._check_point_save( self._check_point_save(
f"./{save_path}/{self.day}/ckpt-{epoch}") f"./logs/{log_name}/{self.day}/ckpt-{epoch}")
tf.keras.backend.reset_uids() tf.keras.backend.reset_uids()
tf.keras.backend.clear_session() tf.keras.backend.clear_session()
@@ -644,13 +641,13 @@ class Optimizer:
print(e) print(e)
finally: finally:
self.model_save(x, y, save_path) self.model_save(validate_data)
print("model save") print("model save")
if save_info: if save_info:
self.save_info(save_path) self.save_info()
print("save info") print("save info")
return self.g_best_score return Particle.g_best_score
def get_best_model(self): def get_best_model(self):
""" """
@@ -660,7 +657,7 @@ class Optimizer:
(keras.models): 모델 (keras.models): 모델
""" """
model = keras.models.model_from_json(self.model.to_json()) model = keras.models.model_from_json(self.model.to_json())
model.set_weights(self.g_best) model.set_weights(Particle.g_best_weights)
model.compile( model.compile(
loss=self.loss, loss=self.loss,
optimizer="adam", optimizer="adam",
@@ -676,7 +673,7 @@ class Optimizer:
Returns: Returns:
(float): 점수 (float): 점수
""" """
return self.g_best_score return Particle.g_best_score
def get_best_weights(self): def get_best_weights(self):
""" """
@@ -685,9 +682,9 @@ class Optimizer:
Returns: Returns:
(float): 가중치 (float): 가중치
""" """
return self.g_best return Particle.g_best_weights
def save_info(self, path: str = "./result"): def save_info(self):
""" """
학습 정보를 저장 학습 정보를 저장
@@ -697,7 +694,7 @@ class Optimizer:
json_save = { json_save = {
"name": f"{self.day}/{self.n_particles}_{self.c0}_{self.c1}_{self.w_min}.h5", "name": f"{self.day}/{self.n_particles}_{self.c0}_{self.c1}_{self.w_min}.h5",
"n_particles": self.n_particles, "n_particles": self.n_particles,
"score": self.g_best_score, "score": Particle.g_best_score,
"c0": self.c0, "c0": self.c0,
"c1": self.c1, "c1": self.c1,
"w_min": self.w_min, "w_min": self.w_min,
@@ -716,7 +713,7 @@ class Optimizer:
} }
with open( with open(
f"./{path}/{self.day}/{self.loss}_{self.g_best_score}.json", f"./{self.log_path}/{self.loss}_{Particle.g_best_score}.json",
"a", "a",
) as f: ) as f:
json.dump(json_save, f, indent=4) json.dump(json_save, f, indent=4)
@@ -731,7 +728,7 @@ class Optimizer:
model = self.get_best_model() model = self.get_best_model()
model.save_weights(save_path) model.save_weights(save_path)
def model_save(self, x, y, save_path: str = "./result"): def model_save(self, valid_data: tuple = None):
""" """
최고 점수를 받은 모델 저장 최고 점수를 받은 모델 저장
@@ -741,11 +738,12 @@ class Optimizer:
Returns: Returns:
(keras.models): 모델 (keras.models): 모델
""" """
x, y = valid_data
model = self.get_best_model() model = self.get_best_model()
score = model.evaluate(x, y, verbose=1) score = model.evaluate(x, y, verbose=1)
print( print(
f"model score - loss: {score[0]} - acc: {score[1]} - mse: {score[2]}") f"model score - loss: {score[0]} - acc: {score[1]} - mse: {score[2]}")
model.save( model.save(
f"./{save_path}/{self.day}/model_{score[0 if self.renewal == 'loss' else 1 if self.renewal == 'acc' else 2 ]}.h5" f"./{self.log_path}/model_{score[0 if self.renewal == 'loss' else 1 if self.renewal == 'acc' else 2 ]}.h5"
) )
return model return model

126
pso/particle.py
View File

@@ -1,5 +1,4 @@
import numpy as np import numpy as np
from tensorflow import keras from tensorflow import keras
@@ -13,11 +12,14 @@ class Particle:
4. 가중치 업데이트 4. 가중치 업데이트
5. 2번으로 돌아가서 반복 5. 2번으로 돌아가서 반복
""" """
g_best_score = [np.inf, 0, np.inf]
g_best_weights = None
count = 0
def __init__( def __init__(
self, self,
model: keras.models, model: keras.models,
loss, loss: any = None,
negative: bool = False, negative: bool = False,
mutation: float = 0, mutation: float = 0,
converge_reset: bool = False, converge_reset: bool = False,
@@ -50,7 +52,7 @@ class Particle:
print(e) print(e)
exit(1) exit(1)
self.__reset_particle__() self.__reset_particle()
self.best_weights = self.model.get_weights() self.best_weights = self.model.get_weights()
self.before_best = self.model.get_weights() self.before_best = self.model.get_weights()
self.negative = negative self.negative = negative
@@ -62,6 +64,7 @@ class Particle:
self.converge_reset_patience = converge_reset_patience self.converge_reset_patience = converge_reset_patience
self.converge_reset_monitor = converge_reset_monitor self.converge_reset_monitor = converge_reset_monitor
self.converge_reset_min_delta = converge_reset_min_delta self.converge_reset_min_delta = converge_reset_min_delta
Particle.count += 1
def __del__(self): def __del__(self):
del self.model del self.model
@@ -70,6 +73,7 @@ class Particle:
del self.negative del self.negative
del self.best_score del self.best_score
del self.best_weights del self.best_weights
Particle.count -= 1
def _encode(self, weights: list): def _encode(self, weights: list):
""" """
@@ -135,35 +139,37 @@ class Particle:
score = self.model.evaluate(x, y, verbose=0) score = self.model.evaluate(x, y, verbose=0)
if renewal == "loss": if renewal == "loss":
if score[0] < self.best_score[0]: if score[0] < self.best_score[0]:
self.best_score[0] = score[0] self.best_score = score
self.best_weights = self.model.get_weights() self.best_weights = self.model.get_weights()
elif renewal == "acc": elif renewal == "acc":
if score[1] > self.best_score[1]: if score[1] > self.best_score[1]:
self.best_score[1] = score[1] self.best_score = score
self.best_weights = self.model.get_weights() self.best_weights = self.model.get_weights()
elif renewal == "mse": elif renewal == "mse":
if score[2] < self.best_score[2]: if score[2] < self.best_score[2]:
self.best_score[2] = score[2] self.best_score = score
self.best_weights = self.model.get_weights() self.best_weights = self.model.get_weights()
else: else:
raise ValueError("renewal must be 'acc' or 'loss' or 'mse'") raise ValueError("renewal must be 'acc' or 'loss' or 'mse'")
return score return score
def __check_converge_reset__(self, score, monitor="loss", patience: int = 10, min_delta: float = 0.0001): def __check_converge_reset(self, score, monitor: str = None, patience: int = 10, min_delta: float = 0.0001):
""" """
early stop을 구현한 함수 early stop을 구현한 함수
Args: Args:
score (float): 현재 점수 [0] - loss, [1] - acc score (float): 현재 점수 [0] - loss, [1] - acc
monitor (str, optional): 감시할 점수. Defaults to "loss". monitor (str, optional): 감시할 점수. Defaults to acc. | "acc" or "loss" or "mse"
patience (int, optional): early stop을 위한 기다리는 횟수. Defaults to 10. patience (int, optional): early stop을 위한 기다리는 횟수. Defaults to 10.
min_delta (float, optional): early stop을 위한 최소 변화량. Defaults to 0.0001. min_delta (float, optional): early stop을 위한 최소 변화량. Defaults to 0.0001.
""" """
if monitor in ["acc", "accuracy"]: if monitor is None:
self.score_history.append(score[1]) monitor = "acc"
elif monitor in ["loss"]: if monitor in ["loss"]:
self.score_history.append(score[0]) self.score_history.append(score[0])
elif monitor in ["acc", "accuracy"]:
self.score_history.append(score[1])
elif monitor in ["mse"]: elif monitor in ["mse"]:
self.score_history.append(score[2]) self.score_history.append(score[2])
else: else:
@@ -176,7 +182,7 @@ class Particle:
return True return True
return False return False
def __reset_particle__(self): def __reset_particle(self):
self.model = keras.models.model_from_json(self.model.to_json()) self.model = keras.models.model_from_json(self.model.to_json())
self.model.compile( self.model.compile(
optimizer="adam", optimizer="adam",
@@ -184,13 +190,13 @@ class Particle:
metrics=["accuracy", "mse"] metrics=["accuracy", "mse"]
) )
i_w_, i_s, i_l = self._encode(self.model.get_weights()) i_w_, i_s, i_l = self._encode(self.model.get_weights())
i_w_ = np.random.uniform(-0.05, 0.05, len(i_w_)) i_w_ = np.random.uniform(-0.05, 0.1, len(i_w_))
self.velocities = self._decode(i_w_, i_s, i_l) self.velocities = self._decode(i_w_, i_s, i_l)
del i_w_, i_s, i_l del i_w_, i_s, i_l
self.score_history = [] self.score_history = []
def _update_velocity(self, local_rate, global_rate, w, g_best): def _update_velocity(self, local_rate, global_rate, w):
""" """
현재 속도 업데이트 현재 속도 업데이트
@@ -198,19 +204,20 @@ class Particle:
local_rate (float): 지역 최적해의 영향력 local_rate (float): 지역 최적해의 영향력
global_rate (float): 전역 최적해의 영향력 global_rate (float): 전역 최적해의 영향력
w (float): 현재 속도의 영향력 - 관성 | 0.9 ~ 0.4 이 적당 w (float): 현재 속도의 영향력 - 관성 | 0.9 ~ 0.4 이 적당
g_best (list): 전역 최적해
""" """
encode_w, w_sh, w_len = self._encode(weights=self.model.get_weights()) encode_w, w_sh, w_len = self._encode(weights=self.model.get_weights())
encode_v, v_sh, v_len = self._encode(weights=self.velocities) encode_v, v_sh, v_len = self._encode(weights=self.velocities)
encode_p, p_sh, p_len = self._encode(weights=self.best_weights) encode_p, p_sh, p_len = self._encode(weights=self.best_weights)
encode_g, g_sh, g_len = self._encode(weights=g_best) encode_g, g_sh, g_len = self._encode(weights=Particle.g_best_weights)
encode_before, before_sh, before_len = self._encode( encode_before, before_sh, before_len = self._encode(
weights=self.before_best weights=self.before_best
) )
r_0 = np.random.rand() r_0 = np.random.rand()
r_1 = np.random.rand() r_1 = np.random.rand()
# 이전 전역 최적해와 현재 전역 최적해가 다르면 관성을 순간적으로 증가 - 값이 바뀔 경우 기존 관성을 특정 기간동안 유지
if not np.array_equal(encode_before, encode_g, equal_nan=True): if not np.array_equal(encode_before, encode_g, equal_nan=True):
# 이전 가중치 중요도의 1.5 배로 관성을 증가
self.before_w = w * 0.5 self.before_w = w * 0.5
w = w + self.before_w w = w + self.before_w
else: else:
@@ -218,13 +225,14 @@ class Particle:
w = w + self.before_w w = w + self.before_w
if self.negative: if self.negative:
# 지역 최적해와 전역 최적해를 음수로 사용하여 전역 탐색을 유도
new_v = ( new_v = (
w * encode_v w * encode_v
+ local_rate * r_0 * (encode_p - encode_w) - local_rate * r_0 * (encode_p - encode_w)
+ -1 * global_rate * r_1 * (encode_g - encode_w) - global_rate * r_1 * (encode_g - encode_w)
) )
if len(self.score_history) > 10 and max(self.score_history[-10:]) - min(self.score_history[-10:]) < 0.01: if len(self.score_history) > 10 and max(self.score_history[-10:]) - min(self.score_history[-10:]) < 0.01:
self.__reset_particle__() self.__reset_particle()
else: else:
new_v = ( new_v = (
@@ -246,7 +254,7 @@ class Particle:
del encode_before, before_sh, before_len del encode_before, before_sh, before_len
del r_0, r_1 del r_0, r_1
def _update_velocity_w(self, local_rate, global_rate, w, w_p, w_g, g_best): def _update_velocity_w(self, local_rate, global_rate, w, w_p, w_g):
""" """
현재 속도 업데이트 현재 속도 업데이트
기본 업데이트의 변형으로 지역 최적해와 전역 최적해를 분산시켜 조기 수렴을 방지 기본 업데이트의 변형으로 지역 최적해와 전역 최적해를 분산시켜 조기 수렴을 방지
@@ -257,12 +265,11 @@ class Particle:
w (float): 현재 속도의 영향력 - 관성 | 0.9 ~ 0.4 이 적당 w (float): 현재 속도의 영향력 - 관성 | 0.9 ~ 0.4 이 적당
w_p (float): 지역 최적해의 분산 정도 w_p (float): 지역 최적해의 분산 정도
w_g (float): 전역 최적해의 분산 정도 w_g (float): 전역 최적해의 분산 정도
g_best (list): 전역 최적해
""" """
encode_w, w_sh, w_len = self._encode(weights=self.model.get_weights()) encode_w, w_sh, w_len = self._encode(weights=self.model.get_weights())
encode_v, v_sh, v_len = self._encode(weights=self.velocities) encode_v, v_sh, v_len = self._encode(weights=self.velocities)
encode_p, p_sh, p_len = self._encode(weights=self.best_weights) encode_p, p_sh, p_len = self._encode(weights=self.best_weights)
encode_g, g_sh, g_len = self._encode(weights=g_best) encode_g, g_sh, g_len = self._encode(weights=Particle.g_best_weights)
encode_before, before_sh, before_len = self._encode( encode_before, before_sh, before_len = self._encode(
weights=self.before_best weights=self.before_best
) )
@@ -275,11 +282,12 @@ class Particle:
else: else:
self.before_w *= 0.75 self.before_w *= 0.75
w = w + self.before_w w = w + self.before_w
if self.negative: if self.negative:
new_v = ( new_v = (
w * encode_v w * encode_v
+ local_rate * r_0 * (w_p * encode_p - encode_w) - local_rate * r_0 * (w_p * encode_p - encode_w)
+ -1 * global_rate * r_1 * (w_g * encode_g - encode_w) - global_rate * r_1 * (w_g * encode_g - encode_w)
) )
else: else:
new_v = ( new_v = (
@@ -289,7 +297,7 @@ class Particle:
) )
if np.random.rand() < self.mutation: if np.random.rand() < self.mutation:
m_v = np.random.uniform(-0.1, 0.1, len(encode_v)) m_v = np.random.uniform(-0.05, 0.05, len(encode_v))
new_v = m_v new_v = m_v
self.velocities = self._decode(new_v, w_sh, w_len) self.velocities = self._decode(new_v, w_sh, w_len)
@@ -313,7 +321,7 @@ class Particle:
del encode_w, w_sh, w_len del encode_w, w_sh, w_len
del encode_v, v_sh, v_len del encode_v, v_sh, v_len
def step(self, x, y, local_rate, global_rate, w, g_best, renewal: str = "acc"): def step(self, x, y, local_rate, global_rate, w, renewal: str = "acc"):
""" """
파티클의 한 스텝을 진행합니다. 파티클의 한 스텝을 진행합니다.
@@ -329,19 +337,35 @@ class Particle:
Returns: Returns:
list: 현재 파티클의 점수 list: 현재 파티클의 점수
""" """
self._update_velocity(local_rate, global_rate, w, g_best) self._update_velocity(local_rate, global_rate, w)
self._update_weights() self._update_weights()
score = self.get_score(x, y, renewal) score = self.get_score(x, y, renewal)
if self.converge_reset and self.__check_converge_reset__( if self.converge_reset and self.__check_converge_reset(
score, self.converge_reset_monitor, self.converge_reset_patience, self.converge_reset_min_delta): score, self.converge_reset_monitor, self.converge_reset_patience, self.converge_reset_min_delta):
self.__reset_particle__() self.__reset_particle()
score = self.get_score(x, y, renewal)
while np.isnan(score[0]) or np.isnan(score[1]) or np.isnan(score[2]) or score[0] == 0 or score[1] == 0 or score[2] == 0 or np.isinf(score[0]) or np.isinf(score[1]) or np.isinf(score[2]) or score[0] > 1000 or score[1] > 1 or score[2] > 1000:
self.__reset_particle()
score = self.get_score(x, y, renewal)
# # score 가 inf 이면 가중치를 초기화
# # score 가 nan 이면 가중치를 초기화
# # score 가 0 이면 가중치를 초기화
# if np.isinf(score[0]) or np.isinf(score[1]) or np.isinf(score[2]) or np.isnan(score[0]) or np.isnan(score[1]) or np.isnan(score[2]) or score[0] == 0 or score[1] == 0 or score[2] == 0:
# self.__reset_particle()
# score = self.get_score(x, y, renewal)
# # score 가 상식적인 범위를 벗어나면 가중치를 초기화
# if score[0] > 1000 or score[1] > 1 or score[2] > 1000:
# self.__reset_particle()
# score = self.get_score(x, y, renewal)
return score return score
def step_w( def step_w(
self, x, y, local_rate, global_rate, w, g_best, w_p, w_g, renewal: str = "acc" self, x, y, local_rate, global_rate, w, w_p, w_g, renewal: str = "acc"
): ):
""" """
파티클의 한 스텝을 진행합니다. 파티클의 한 스텝을 진행합니다.
@@ -361,10 +385,32 @@ class Particle:
Returns: Returns:
float: 현재 파티클의 점수 float: 현재 파티클의 점수
""" """
self._update_velocity_w(local_rate, global_rate, w, w_p, w_g, g_best) self._update_velocity_w(local_rate, global_rate, w, w_p, w_g)
self._update_weights() self._update_weights()
return self.get_score(x, y, renewal) score = self.get_score(x, y, renewal)
if self.converge_reset and self.__check_converge_reset(
score, self.converge_reset_monitor, self.converge_reset_patience, self.converge_reset_min_delta):
self.__reset_particle()
score = self.get_score(x, y, renewal)
while np.isnan(score[0]) or np.isnan(score[1]) or np.isnan(score[2]) or score[0] == 0 or score[1] == 0 or score[2] == 0 or np.isinf(score[0]) or np.isinf(score[1]) or np.isinf(score[2]) or score[0] > 1000 or score[1] > 1 or score[2] > 1000:
self.__reset_particle()
score = self.get_score(x, y, renewal)
# # score 가 inf 이면 가중치를 초기화
# # score 가 nan 이면 가중치를 초기화
# # score 가 0 이면 가중치를 초기화
# if np.isinf(score[0]) or np.isinf(score[1]) or np.isinf(score[2]) or np.isnan(score[0]) or np.isnan(score[1]) or np.isnan(score[2]) or score[0] == 0 or score[1] == 0 or score[2] == 0:
# self.__reset_particle()
# score = self.get_score(x, y, renewal)
# # score 가 상식적인 범위를 벗어나면 가중치를 초기화
# if score[0] > 1000 or score[1] > 1 or score[2] > 1000:
# self.__reset_particle()
# score = self.get_score(x, y, renewal)
return score
def get_best_score(self): def get_best_score(self):
""" """
@@ -383,3 +429,19 @@ class Particle:
list: 가중치 리스트 list: 가중치 리스트
""" """
return self.best_weights return self.best_weights
def set_global_score(self):
"""전역 최고점수를 현재 파티클의 최고점수로 설정합니다
"""
Particle.g_best_score = self.best_score
def set_global_weights(self):
"""전역 최고점수를 받은 가중치를 현재 파티클의 최고점수를 받은 가중치로 설정합니다
"""
Particle.g_best_weights = self.best_weights
def update_global_best(self):
"""현재 파티클의 점수와 가중치를 전역 최고점수와 가중치로 설정합니다
"""
self.set_global_score()
self.set_global_weights()

112
seeds.py Normal file
View File

@@ -0,0 +1,112 @@
# %%
import json
import os
import sys
import numpy as np
import pandas as pd
import tensorflow as tf
from keras.layers import Dense
from keras.models import Sequential
from keras.utils import to_categorical
from sklearn.model_selection import train_test_split
from tensorflow import keras
from pso import optimizer
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"
def get_data():
with open("data/seeds/seeds_dataset.txt", "r", encoding="utf-8") as f:
data = f.readlines()
df = pd.DataFrame([d.split() for d in data])
df.columns = [
"area",
"perimeter",
"compactness",
"length_of_kernel",
"width_of_kernel",
"asymmetry_coefficient",
"length_of_kernel_groove",
"target",
]
df = df.astype(float)
df["target"] = df["target"].astype(int)
x = df.iloc[:, :-1].values.round(0).astype(int)
y = df.iloc[:, -1].values
y_class = to_categorical(y)
x_train, x_test, y_train, y_test = train_test_split(
x, y_class, test_size=0.2, shuffle=True
)
return x_train, y_train, x_test, y_test
def make_model():
model = Sequential()
model.add(Dense(16, activation="relu", input_shape=(7,)))
model.add(Dense(32, activation="relu"))
model.add(Dense(4, activation="softmax"))
return model
# %%
model = make_model()
x_train, y_train, x_test, y_test = get_data()
loss = [
"mean_squared_error",
"categorical_crossentropy",
"sparse_categorical_crossentropy",
"binary_crossentropy",
"kullback_leibler_divergence",
"poisson",
"cosine_similarity",
"log_cosh",
"huber_loss",
"mean_absolute_error",
"mean_absolute_percentage_error",
]
# rs = random_state()
pso_mnist = optimizer(
model,
loss="categorical_crossentropy",
n_particles=100,
c0=0.5,
c1=1.0,
w_min=0.7,
w_max=1.2,
negative_swarm=0.0,
mutation_swarm=0.3,
convergence_reset=True,
convergence_reset_patience=10,
convergence_reset_monitor="mse",
convergence_reset_min_delta=0.0005,
)
best_score = pso_mnist.fit(
x_train,
y_train,
epochs=500,
save_info=True,
log=2,
log_name="seeds",
renewal="acc",
check_point=25,
empirical_balance=False,
dispersion=False,
back_propagation=False,
validate_data=(x_test, y_test),
)
print("Done!")
sys.exit(0)

1
xor.py
View File

@@ -67,7 +67,6 @@ best_score = pso_xor.fit(
save_info=True, save_info=True,
log=2, log=2,
log_name="xor", log_name="xor",
save_path="./result/xor",
renewal="acc", renewal="acc",
check_point=25, check_point=25,
) )