Advanced Deep Learning with Keras

Chapter 5
WGAN ( D)
L =− E D x + E D G
( ) ( ( z)
)
x~ p data w z w
( G)
L =−E z
D w ( G( z)
)
( − )
w ← clip w, 0.01,0.01
5.1.21
5.1.22
5.1.20
Algorithm 5.1.1 WGAN
Table 5.1.1: A comparison between the loss functions of GAN and WGAN
The values of the parameters are α = 0.00005 , c = 0.01 m = 64, and n critic
= 5.
Require: a , the learning rate. c, the clipping parameter. m, the batch size. n critic
, the
number of the critic (discriminator) iterations per generator iteration.
Require: w 0
, initial critic (discriminator) parameters. θ
0
, initial generator parameters
1. while θ has not converged do
2. for t = 1, …, n critic
do
m
()
{ } ~
1
i
3. Sample a batch x
i=
pdata
from the real data
m
() i
4. Sample a batch { z } ~ p( z)
from the uniform noise distribution
i=
1
()
5.
⎡ 1 m
()
( i 1 m
i
w w i 1 w )
i 1 w ( ( ⎤
←∇ − +
= = θ ))
⎢
∑ D ∑ D G
⎣ m
m
⎥⎦
, compute the
discriminator gradients
6. w ← w− α× RMSProp( w, gw)
, update the discriminator parameters
7. w ← clip( w, − c,
c)
, clip discriminator weights
8. end for
m
() i
{ z } ~ p( z)
i=
1
1 m () i
θ θ i 1 w θ
z
=
9. Sample a batch
10.
( ( ))
from the uniform noise distribution
g ←−∇ ∑ D G , compute the generator gradients
m
θ ← θ − α× RMSProp θ,
G , update generator parameters
11. ( )
12. end while
θ
[ 133 ]