Target Sequence
0
0
Train
Test Input Sequence
0
0
Run Inference Step 1
→
↓
LSTM Cell (1 unit)
x
t
(current bit) + h
t−1
(prev hidden)
↺
h
t−1
= previous h
t
(or 0 at start)
↓
linear projections
f
t
· i
t
· g
t
→ Cell State c
t
Forget f
t
σ(W
f
·[h,x]+b)
Input i
t
σ(W
i
·[h,x]+b)
Cand g
t
tanh(W
g
·[h,x]+b)
↓
f
t
⊙ c
t−1
+ i
t
⊙ g
t
↺
c
t−1
(prev step or 0)
Cell State c
t
↓
c
t
→ tanh
o
t
(from input) ⊙ tanh(c
t
) → h
t
Output o
t
σ(W
o
·[h,x]+b)
tanh(c
t
)
–
↓
o
t
⊙ tanh(c
t
)
Hidden State h
t
↺
h
t
→ h
t−1
next step
↓
Dense(1) → σ
–
→
↓
Output
0