home *** CD-ROM | disk | FTP | other *** search
- csv3.1 ! Control Strategy Version 3.1 - Boltzmann I/O model
- L_saRi_a cset recall,0 ! Initialize Recall Counter
- !
- !
- ! Read input. Assume 1 = ON, -1 = OFF, 0 = doesn't matter.
- ! Note: In I/O model there will typically be no 0's.
- !
- L_saRi_a lset in,0 ! Input layer
- L___Ri_a io read ! Recall Input
- L_saR___ io lrnin ! Learn Input
- !
- ! Propagate raw inputs in order to flag clamped units. Clamped
- ! units are flagged in the hidden layer by means of a non-zero
- ! current error term. Note: "set" operation zeros out all hidden units
- ! which don't have set connections.
- !
- L_saRi_a lset in,1 ! Boltzmann layer
- L_saRi_a math set|w=i|e=w|fire ! Inject Input
- !
- ! Read desired output - I/O model and phase+ only.
- ! Back-propagate raw errors to flag clamped units.
- !
- L_saR_sa lset out,0 ! Output layer
- L_saR___ io lrnout ! Desired Output
- L___R_sa io rcltst ! Desired Output (Test)
- L_saR___ math backp|fire ! Backprop output
- !
- ! At this point, the error fields of the hidden layer units are
- ! non-zero if and only if unit is clamped; transfer this to
- ! current error field and then process inputs (and desired outputs) and
- ! and initialize hidden unit activations.
- !
- L_saRi_a lset in,0 ! Input Layer
- L_saRi_a math tran|output|fire ! 1,-1's to 1,0's
- L_saRi_a lset in,1 ! Boltzman layer
- L_saRi_a math set|w=i|ce=e|e=w|fire ! Read true inputs
-
- L_saR___ lset out,0 ! Output Layer
- L_saR___ math tran|output|e=w|backp|fire ! True outputs
- !
- ! Error fields of hidden layer units now contain proper values (0's or
- ! 1's) for the clamped units, and 0's for the unclamped units.
- ! Transfer these values to the tran field using "e=w" and "tran". The
- ! "e=w" operation will transfer the proper clamped values; the "tran"
- ! operation will not affect the clamped values but will randomize the
- ! unclamped values ( assuming a non zero noise ) to be 0 or 1 with
- ! equal probability ( see Boltzman transfer function ).
-
- L_saRisa lset in,1 ! Boltzman layer
- L_saRi_a math i=e|w=e|tran|output|rnoise|e=0|fire
- L_saRi_a cadd recall,1 ! Increment Recall counter
- !
- ! Annealing for phase+ and recall
- !
- @ann1 ! "Iteration" loop, indexed by recall counter
- L_saR_sa fcmp lflag,eors ! End of recall schedule?
- L_saR___ banz @stats1 ! Collect statistics (Learn!)
- L___R_sa banz @wrapup ! wrap up cycle
- L_saR_sa fcmp lflag,ueors ! User End of recall schedule?
- L_saR_sa banz @end
- L_saR_sa cset wrk1,aux2 ! Initialize inner loop
- @ann2 ! Inner loop - # math steps per iteration
- L_saR_sa math sum|tran|rnoise|output|setb ! Fire
- L_saR_sa csub wrk1,1 ! Decrement Inner counter
- L_saR_sa ccmp wrk1,0
- L_saR_sa bgt @ann2
- L_saR_sa cadd recall,1 ! Increment Recall counter
- L___R_s_ eos ! End of recall step
- L_saR__a br @ann1 ! Continue recall
- !
- @stats1 ! Now accumulate co-occurrence statistics for phase+
- L_saR___ cset wrk2,aux3 ! Initialize outer loop
- @stats2
- L_saR___ cset wrk1,aux2 ! Initialize inner loop
- @stats3 ! Inner stats loop - # math steps per iteration
- L_saR___ math sum|tran|lnoise|output|setb ! Fire
- L_saR___ csub wrk1,1 ! Decrement Inner counter
- L_saR___ ccmp wrk1,0
- L_saR___ bgt @stats3
- L_saR___ math learn|setb|fire ! Accumulate stats
- L_saR___ cadd recall,1 ! Increment Recall counter
- L_saR___ csub wrk2,1 ! Decrement Outer counter
- L_saR___ ccmp wrk2,0
- L_saR___ bgt @stats2 ! Continue stats
- !
- ! If end of phase+ go on to do phase-
- ! Do clamping again. For I/O version of the Boltzman machine, this
- ! entails unclamping the "output" units.
- !
- L_saR___ cset recall,0 ! Initialize Recall Counter
- L_saR___ lset in,0 ! Input Layer
- L_saR___ math w=i|output|fire ! Raw inputs
- L_saR___ lset in,1 ! Boltzmann layer
- L_saR___ math set|w=i|e=w|fire ! Inject Input
- !
- ! Read input info again, save clamping info in "ce" field,
- ! initialize clamped units, and randomize non-clamped units.
- !
- L_saR___ lset in,0 ! Input Layer
- L_saR___ math tran|output|fire ! 1,-1's to 1,0's
- L_saR___ lset in,1 ! Boltzman layer
- L_saR___ math set|w=i|ce=e|tran|output|rnoise|e=0|fire
- L_saR___ cadd recall,1 ! Increment Recall counter
- !
- ! Annealing for phase-
- !
- @ann3 ! "Iteration" loop, indexed by recall counter
- L_saR___ fcmp lflag,eors ! End of recall schedule?
- L_saR___ banz @stats4 ! Collect statistics (Learn!)
- L_saR___ fcmp lflag,ueors ! User End of recall schedule?
- L_saR___ banz @end ! Collect statistics (Learn!)
- L_saR___ cset wrk1,aux2 ! Initialize inner loop
- @ann4 ! Inner loop - # math steps per iteration
- L_saR___ math sum|tran|rnoise|output ! Fire
- L_saR___ csub wrk1,1 ! Decrement Inner counter
- L_saR___ ccmp wrk1,0
- L_saR___ bgt @ann4
- L_saR___ cadd recall,1 ! Increment Recall counter
- L_saR___ br @ann3 ! Continue recall
- !
- @stats4 ! Now accumulate co-occurrence statistics for phase-
- L_saR___ cset wrk2,aux3 ! Initialize outer loop
- @stats5
- L_saR___ cset wrk1,aux2 ! Initialize inner loop
- @stats6 ! Inner stats loop - # math steps per iteration
- L_saR___ math sum|tran|lnoise|output ! Fire
- L_saR___ csub wrk1,1 ! Decrement Inner counter
- L_saR___ ccmp wrk1,0
- L_saR___ bgt @stats6
- L_saR___ math learn|fire ! Accumulate stats
- L_saR___ cadd recall,1 ! Increment Recall counter
- L_saR___ csub wrk2,1 ! Decrement Outer counter
- L_saR___ ccmp wrk2,0
- L_saR___ bgt @stats5 ! Continue stats
-
- ! Special call to learn with wrk2 = 0, instructing learn
- ! function to actually update the weights if at the end
- ! of an epoch
- L_saR___ math learn|fire ! Accumulate stats
- !
- ! Show output etc
- !
- @wrapup
- L_saR_sa lset out,0 ! Output layer
- L___R_sa math sum|ce=e|tran|output|e-=w|swap !15 Recall output
- L_saR___ math sum|tran|output|fire ! Learn output
- L_saR___ io lrnrslt ! Show learn result
- L___R_sa io write ! Show Recall Output
- @end
-
