File: landmark.el.html
*Note: This package has now moved to elpa.gnu.org.*
To try this, just type: M-x landmark-test-run
Landmark is a relatively non-participatory game in which a robot attempts to maneuver towards a tree at the center of the window based on unique olfactory cues from each of the 4 directions. If the smell of the tree increases, then the weights in the robot's brain are adjusted to encourage this odor-driven behavior in the future. If the smell of the tree decreases, the robots weights are adjusted to discourage that odor-driven behavior.
In laymen's terms, the search space is initially flat. The point of training is to "turn up the edges of the search space" so that the robot rolls toward the center.
Further, do not become alarmed if the robot appears to oscillate back and forth between two or a few positions. This simply means it is currently caught in a local minimum and is doing its best to work its way out.
The version of this program as described has a small problem. a move in a net direction can produce gross credit assignment. for example, if moving south will produce positive payoff, then, if in a single move, one moves east,west and south, then both east and west will be improved when they shouldn't
The source code was developed as part of a course on Brain Theory and Neural Networks at the University of Southern California. The original problem description and solution appeared in 1981 in the paper "Landmark Learning: An Illustration of Associative Search" authored by Andrew G. Barto and Richard S. Sutton and published to Biological Cybernetics.
Many thanks to Yuri Pryadkin <yuri@rana.usc.edu> for this concise problem description.
;_* Require
Defined variables (53)
landmark-8-directions | The full 8 possible directions. |
landmark-board | Vector recording the actual state of the Landmark board. |
landmark-board-height | Number of lines on the Landmark board. |
landmark-board-width | Number of columns on the Landmark board. |
landmark-bound | The maximum that w0j may be. |
landmark-c | A factor applied to modulate the increase in wij. |
landmark-c-naught | A factor applied to modulate the increase in w0j. |
landmark-cx | This is the x coordinate of the center of the board. |
landmark-cy | This is the y coordinate of the center of the board. |
landmark-debug | If non-nil, debugging is printed. |
landmark-directions | The cardinal directions. |
landmark-draw-limit | After how many moves will Emacs offer a draw? |
landmark-emacs-is-computing | Non-nil if Emacs is in the middle of a computation. |
landmark-emacs-played-first | Non-nil if Emacs played first. |
landmark-emacs-won | For making font-lock use the winner’s face for the line. |
landmark-ew | Used when doing something relative to the east and west axes. |
landmark-font-lock-keywords | Font lock rules for Landmark. |
landmark-game-history | A record of all moves that have been played during current game. |
landmark-game-in-progress | Non-nil if a game is in progress. |
landmark-human-refused-draw | Non-nil if Human refused Emacs offer of a draw. |
landmark-human-took-back | Non-nil if Human took back a move during the game. |
landmark-losing-threshold | Threshold score beyond which a human move is winning. |
landmark-m | This is the x dimension of the playing board. |
landmark-max-stall-time | The maximum number of cycles that the robot can remain stuck in a place. |
landmark-mode-abbrev-table | Abbrev table for ‘landmark-mode’. |
landmark-mode-hook | If non-nil, its value is called on entry to Landmark mode. |
landmark-mode-map | Local keymap to use in Landmark mode. |
landmark-mode-syntax-table | Syntax table for ‘landmark-mode’. |
landmark-n | This is the y dimension of the playing board. |
landmark-nil-score | Score of an empty qtuple. |
landmark-no-payoff | The amount of simulation cycles that have occurred with no movement. |
landmark-ns | Used when doing something relative to the north and south axes. |
landmark-number-of-draws | Number of games already drawn in this session. |
landmark-number-of-emacs-wins | Number of games Emacs won in this session. |
landmark-number-of-human-moves | Number of moves already played by human in current game. |
landmark-number-of-human-wins | Number of games you won in this session. |
landmark-number-of-moves | Number of moves already played in current game. |
landmark-number-of-trials | The number of times that landmark has been run. |
landmark-nvar | Not used. |
landmark-one-moment-please | If non-nil, print "One moment please" when a new board is generated. |
landmark-output-moves | If non-nil, output number of moves so far on a move-by-move basis. |
landmark-saved-board-height | Recorded value of previous board height. |
landmark-saved-board-width | Recorded value of previous board width. |
landmark-saved-score-table | Recorded initial value of previous score table. |
landmark-score-table | Vector recording the actual score of the free squares. |
landmark-score-trans-table | Vector associating qtuple contents to their score. |
landmark-square-height | Vertical spacing between squares on the Landmark board. |
landmark-square-width | Horizontal spacing between squares on the Landmark board. |
landmark-sum-of-moves | The total number of moves made in all games. |
landmark-vector-length | Length of landmark-board vector. |
landmark-winning-threshold | Threshold score beyond which an Emacs move is winning. |
landmark-x-offset | Number of columns between the Landmark board and the side of the window. |
landmark-y-offset | Number of lines between the Landmark board and the top of the window. |
Defined functions (69)
Defined faces (2)
landmark-font-lock-face-O | Face to use for Emacs's O. |
landmark-font-lock-face-X | Face to use for your X. |