From Lazer

Starcat And Robocode Genetic Algorithm Thesis

This project would be to implement a Robocode robot with the Starcat framework. The benefit of having a Robocode Starcat robot it that it gives Starcat researchers a testbed environment where they can experiment with new Starcat functionality. Also in this project after the Robocode Starcat robot is created, robot learning will be experimented with. Learning of the framework will be implemented by the means of dynamically creating composite codelets. A composite codelet is the grouping together of basic codelets to make a new codelet.

Contents 1 Robocode Description 2 Starcat Description 3 Starcat Added Functionality 3.1 Fuzzy Codelets 3.2 Codelets Add different values to different slipnet nodes 3.3 Dynamic Codelets 3.3.1 Building Composite Codelets 3.3.2 Rating Composite Codelets 3.4 Dynamically Restructuring of Slipnet 4 Testing 5 Goals 5.1 Create a functional Robocode Starcat robot 5.2 Experiment with the composite codelet functionality 5.3 Robocode Genetic Creation 6 Hosteders Thoughts 7 On Intelligence Thoughts 8 Look into 9 References 10 Code 11 Book Reviews 12 Questions 13 Problems 13.1 Reflection 13.1.1 Solution 13.2 Digester 13.2.1 Solutions 13.2.1.1 I could manually preform the steps that the Digester class preforms 13.2.1.2 Figure out why the Digester class can not be used 13.2.1.3 Force robocode to look at the Java code base 13.2.1.4 Rename the org to localorg

Robocode Description

Robocode home site

Robocode is an environment where multiple robot agents interact with each other by shooting and ramming each other. The main objective for each robot is to stay alive and to destroy the other robots. This environment is set up to allow individuals to write code in Java to control an individual robot. Therefore, each robot is a collection of java class files that must implement curtain interfaces to be able to interact with the Robocode environment.

Each robot has three separately moving parts (body, gun, and radar). The body of the robot is moved by the tracks on the surface. The body is capable of turning left and right and moving forward and backward. The movements of turning a direction and moving forward or backward can be done at the same time to create an arc movement. The gun piece on the robot can be turned left and right 360 degrees. The radar component is used to view the location of other robots and bullets in its environment. The radar can be turned left and right 360 degrees.

• robocodeAPI

Starcat Description

Starcat

Other Cats

• Madcat

Starcat Added Functionality

Fuzzy Codelets

A fuzzy codelet is a codelet that does not have a clear failed or success boolean it has a number from 0 to 1 of failure and success. The fuzzy codelet class inherits off of the BehavioralCodelet class, where it has all the same functionality except for the failed or success of the codelet. When a BehavioralCodelet runs in the workspace it calls the execute method which only discernible outcome is a change to the workspaceSuccess boolean variable that only allows the codelet to fail or success. If the boolean flag is true only the successful slipnet node recipients get their activation added to in the full amount. When the workspaceSuccess is false only the failure slipnet node recipients get their activation added to in the full amount. In the case of deciding if there is an obstacle in a certain direction a distinct answer of true or false is not the best solution. For this example, the robot would look one hundred pixels in front and if it found a wall it was decided to be an obstacle. So what if the robot was 101 pixels from the wall then it would believe that there was not an obstacle in front of it. But, it would be helpful for the robot to know that there was somewhat of an obstacle in front of it.

The fuzzy codelet gives more of an approximate answer to an asked question. Then when asked if there is an obstacle in the front a range of answer could be returned. For example, partially an obstacle, somewhat of an obstacle, there is an obstacle, or an unavoidable obstacle could be possible answers. A fuzzy codelet with a wall at 101 pixels would believe there is somewhat of a obstacle in front instead of saying that there is no obstacle at all. These approximate answer are given in an value from 0 to 1. Where 1 in the old way would be true and 0 false, but now there are unlimited amount of different answers that can be returned from between 0 and 1.

Figure 1 shows the fuzzy obstacle codelet's fuzzy sets. The success fuzzy set is the blue and yellow areas from 0 to 150. The failure fuzzy set is the red and yellow areas from 50 to 200. Therefore when the distance to a wall is 100 pixels the return value for failure and success is 0.5, meaning it is somewhat of a obstacle and somewhat not an obstacle.

Codelets Add different values to different slipnet nodes

This gives the ability to add different values to different slipnet nodes on a success or failure of the codelet. The code currently only allows the successful recipients to receive one value and the failure recipients to receive one value. If a codelet had more than one successful recipient slipnet nodes to add activation to it could only give the same value to each. This new functionally allows one to add different amounts of activation to each successful and failure recipients.

Dynamic Codelets

Experimenting with allowing the Starcat framework to be able to create new codelets (learning). The process of creating the new codelets is through combining existing codelets.

The basic idea of creating the new codelets for the agent is to create composite codelet. First we will have to create all the basic codelets for the system. These basic codelets will be the building blocks of composite codelet.

Building Composite Codelets

There are many different ways that composite codelet will be created.

• Codelets that are grouped together the most often
• Currently attached codelets when the temperature is low
  • Randomly only take some of the codelets

Rating Composite Codelets

This section describes the method of rating or selecting which composite codelets to keep or remove from the population.

Types of ways to rate codelets:

• Bucket brigade
• genetic (involve test)
• Composite Codelets most used
• Composite Codelets most recently used
• Composite Codelets with the lowest temperature

These are the issues with rating

• When to rate codelets
• How to rate codelets
• Size of population
• Size of population to remove

Dynamically Restructuring of Slipnet

The slipnet is a network comprised of nodes, which represent permanent concepts, and links, which are relations, between them.

Testing

Create a program that runs a given set of robots against each other a set about of times. Find a couple already created robots from Robotcode and place them in the system with a Starcat robot. With this system, we run all the rating and building combinations over a thousand of times to compare what method works the best and why.

Goals

Create a functional Robocode Starcat robot

This robot does not contain the composite codelet functionality. All codelets are hard coded into the robot.

A functional robocode robot is:

• Can move around the environment without running into walls.
  • Starcat Robocode: Experiment I
  • Starcat Robocode: Experiment II
• Give the robot propose
  • Randomly create a waypoint for the robot to go to.
  • Train the robot to catch a moving waypoint.

Experiment with the composite codelet functionality

Testing the composite codelet robot against the basic Robocode Starcat robot.

Robocode Genetic Creation

Place 10 or more agents in the robocode battle field and allow agents to asexually produce offspring based off their fitness score. The offspring would obtain the DNA of their parent with random mutations.

The fitness could be based off of the agent's time alive, health, amount of rams, successful shots, and wall rams compared to agent rams at the end of the battle. The next battle would be between the offspring of the previous battle.

The hard part would be defining what the DNA does to the agent. The first guess would be that the DNA would define the connections between the slipnet nodes in the slipnet. The DNA would generate the weights and the links between the slipnet nodes.

Hosteders Thoughts

On Intelligence Thoughts

The book On Intelligence by Jeff Hawkins is a book about what is believed to be the framework of how the brain works. This framework is the idea that the brain is always predicting its environment and then checking to ensure that its predictions are correct. The main idea is that your senses work in a bidirectional way sensing and predicting. For example your sensor input for your vision is getting data from your eyes, as raw data, and your brain, as predicting data.

Jeff Hawkins describes four separate regions that are involved in processing visual data. The four regions are called IT, V4, V2, and V1. The V1 region is the lowest region and works with low concrete raw data directly from the eyes. The IT region is the highest conceptual region and deals with abstract ideas. Each region is only connected to the regions above and below it, but data can go both ways.

First we will describe how V1 region works. There are many different sub-regions on the V1 that all have different specific tasks to each other. For example one sub-region is activated when a vertical line is in the field of vision, another activates when a straight line leaning 30 degrees right of vertical is seen and so on. Therefore each sub-region has it specific basic pattern that it looks for in the environment.

The IT region has the same basic idea as the V1 region, but its sub-regions are activated with abstract ideas. These abstract idea are built from lower regions activations. The way one could think about how the IT region works is that each sub-region is attached to many sub-regions in the V4 region. If the sub-regions that the IT region is attached to are activated then the IT sub-region activates. An example when one sub-region of the IT is activated is when a picture, a drawing, even a charactater of George W. Bush is seen.

This idea matches well with Douglas Hofstadter's ideas of the brain. Describe How They are the same

• Describe what match
• Describe what does not match

Look into

• Creating structures representing the other robots in the environment.
• The effects of having the robot not take action into for each codelet that is activated.
• Work more on creating the initial structure of the Master Thesis

References

1. Joseph A. Lewis, CS 652 Evolutionary and Adaptive Computation,CS 652
2. M. Mitchell. Analogy-Making as a Complex Adaptive System
3. M. Mitchell. A Complex-Systems Perspective on the Computation vs. Dynamics" Debate in Cognitive Science
4. Joseph A. Lewis, Jamie R. Lawson: Computational Adaptive Autonomy: A Generalization of the Copycat Architecture. IC-AI 2004: 657-663
5. Joseph A. Lewis, Laura Demange: An Implementation of the Copycat Architecture Using the Starcat Adaptive Computational Framework
6. Joseph A. Lewis, Jamie R. Lawson: Representation Emerges from Coupled Behavior
7. Joseph A. Lewis, Jamie R. Lawson: Starcat: An Architecture for Autonomous Adaptive Behavior
8. Joseph A. Lewis: Emergent Representation and Adaptive Behavior Using the Starcat Framework
9. Joseph A. Lewis, Chris Ansoff: Starcat Core Documentation

Code

• Current code

Book Reviews

• The Web of Life : A New Scientific Understanding of Living Systems
• The Selfish Gene

Questions

• On page 8 of the Alife paper, what are "Lateral link, which represent semantic relationships between nodes and are not Slip links"
• Should I read "Analogy-Making as Perception: A Computer Model"
• What is salience as to the entities in the workspace
• Where do the result concept come in to play in the Copycat framework. For example the concept replace the last letter with "d".
• What is the use of creating the identitylink when it length is 100, because no activation is spread when the link is 100.
• What is the SlipnetNode named "Relation"
• What does the PublicStarcatObject do
• What does the interface Keyable used for

Problems

Reflection

Because of security on robocode my the starcat cannot read a file outside of my directory. The error message is

localorg.robocode.BotCat: Exception: java.security.AccessControlException: Preventing localorg.robocode.BotCat from access: (java.io.FilePermission  C:\robocode\robots\org\starcat\slipnet\LateralLink.class read): You may only read files in your own root package directory. 
java.security.AccessControlException: Preventing localorg.robocode.BotCat from access: (java.io.FilePermission C:\robocode\robots\org\starcat\slipnet\LateralLink.class read): You may only   read files in your own root package directory. 
   at robocode.security.RobocodeSecurityManager.checkPermission(Unknown Source)
   at java.lang.SecurityManager.checkRead(Unknown Source)
   at java.io.File.length(Unknown Source)
   at robocode.security.RobocodeClassLoader.loadRobotClass(Unknown Source)
   at robocode.security.RobocodeClassLoader.loadClass(Unknown Source)
   at java.lang.ClassLoader.loadClass(Unknown Source)
   at java.lang.ClassLoader.loadClassInternal(Unknown Source)
   at java.lang.Class.forName0(Native Method)
   at java.lang.Class.forName(Unknown Source)
   at  localorg.starcat.configuration.SlipnetLinkConfiguration.buildLink(SlipnetLinkConfiguration.java:140)
   at localorg.apache.commons.digester.Digester.createLink(Digester.java:352)
   at localorg.apache.commons.digester.Digester.intializeSystem(Digester.java:271)
   at localorg.robocode.RobocodeRunner.<init>(RobocodeRunner.java:37)
   at localorg.robocode.BotCat.run(BotCat.java:20)
   at robocode.peer.RobotPeer.run(Unknown Source)
   at java.lang.Thread.run(Unknown Source)

The startcat code can only read files from directory C:\robocode\robots\localorg\robocode\BotCat.data. Therefore I can not create classes through reflection.

Solution

Take out all the parts that create classes through reflection.

Classes that create classes through reflection are:

• CodeletConfiguration
  • Added method setClazz(BehaviorCodelet codeletBeingConstructed)
• SlipnetLinkConfiguration
  • Added method buildLink(Link linkBegingBuilt)

Digester

There is a problem when adding the apache Digester classes to the source code. At first I got the system to work outside of the robocode system. I did this though adding the jar files to the classpath. When running the system inside of the robocode system, the code does not find the need code inside the jar files that were attached. I have found evidence this is a know robocode error and has not fixed it. Robocode's suggested workaround was to bring all the needed code local to the system. Therefore I extracted all the code and placed it local to robocode. Then when running the system with the all the code local I ran into the problem of LinkageError. Which I believe is because, one section of robocode wants you to bring in classes from the package org.xml.sax for the Digester class, but then another sections finds that this package is defined in the Java JRE. Therefore the system finds duplicate definition for the classes. The only workaround that I could find for this problem was to remove the Digester code.

Errors are

javax.xml.parsers.SAXParser: Cannot set javax.xml.parsers.SAXParser to resolved, did not know it was referenced.
javax.xml.parsers.SAXParser: Got an error with this class: java.lang.LinkageError: loader (instance of  robocode/security/RobocodeClassLoader): attempted  duplicate class definition for name: "org/xml/sax/SAXException"

Solutions

I could manually preform the steps that the Digester class preforms

It looks like I might have to use the starcat framework with out the apache Digester code.

Nine classes use Digester class:

• CodeletConfiguration
• Configuration
• DescriptionTypeConfiguration
• DescriptorConfiguration
• ObjectConfiguration
• RegularPulseConfiguration
• RegularPulseConfguration
• SlipnetLinkConfiguration
• SystemConfiguration

Now re-architect the Digester method.

1. Create a SlipnetNode with the paramtors below for each configuration\object\node\
   1. Add the configuration\object\node\name to the SlipnetNode created above
   2. Add the configuration\object\node\conceptualDepth to the SlipnetNode created above
   3. Add the configuration\object\node\activation to the SlipnetNode created above
   4. Add the configuration\object\node\activationThreshold to the SlipnetNode created above
2. Create nodes
3. Call Slipnet.addSlipnetNode for each of the above SlipnetNode created above.
4. Clear buffer
5. Create a slipnetNode foreach configuration/descriptionType
   1. configuration/descriptionType/name to the SlipnetNode created above
   2. configuration/descriptionType/conceptualDepth to the SlipnetNode created above
   3. configuration/descriptionType/activation to the SlipnetNode created above
   4. configuration/descriptionType/activationThreshold to the SlipnetNode created above
6. Call Slipnet.addSlipnetNode for each of the above SlipnetNode created above.
7. Create nodes and add them to the slipnet
8. Clear buffer
9. Create a slipnetNode foreach configuration/descriptionType
   1. configuration/descriptionType/name to the SlipnetNode created above
   2. configuration/descriptionType/conceptualDepth to the SlipnetNode created above
   3. configuration/descriptionType/objectClass to the SlipnetNode created above
   4. configuration/descriptionType/activation to the SlipnetNode created above
   5. configuration/descriptionType/activationThreshold to the SlipnetNode created above
10. Call Slipnet.addSlipnetNode for each of the above SlipnetNode created above.
11. Create nodes and add them to the slipnet
12. Clear buffer
13. Foreach link/class that match below Call SlipnetLinkConfiguration.buildLink and pass the class name provided
    1. link/name
    2. */link/intrinsicLength
14. Call SlipnetLinkConfiguration.setNodes */link/node with 2 paramtors
    1. First param is "from"
    2. Seconde param is "to"
    3. */link/minShrunkLength
    4. */link/labelNode
15. Create nodes and add them to the slipnet
16. Clear buffer
17. Foreach configuration/codelet
    1. configuration/codelet/clazz
    2. configuration/codelet/name
    3. call method setSourceNode configuration/codelet/sourceNode param 0
    4. configuration/codelet/urgency
    5. configuration/codelet/successAmountToAdd
    6. configuration/codelet/failureAmountToAdd
    7. configuration/codelet/numberToEmit
    8. call method addSuccessActivator configuration/codelet/successActivators/name param 0
    9. call method addFailureActivator configuration/codelet/failureActivators/name param 0
18. Create nodes and add them to the slipnet
19. Clear buffer
20. System Configuration
    1. configuration/system/wsBehaviorAdaptiveExecute
    2. configuration/system/wsControlAdaptiveExecute
    3. configuration/system/crBehaviorAdaptiveExecute
    4. configuration/system/crControlAdaptiveExecute
    5. configuration/system/snBehaviorAdaptiveExecute
    6. configuration/system/snControlAdaptiveExecute
    7. configuration/system/wsBehaviorExecuteFactor
    8. configuration/system/wsControlExecuteFactor
    9. configuration/system/crBehaviorExecuteFactor
    10. configuration/system/crControlExecuteFactor
    11. configuration/system/snBehaviorExecuteFactor
    12. configuration/system/snControlExecuteFactor
    13. configuration/system/wsBehaviorReductionFactor
    14. configuration/system/wsControlReductionFactor
    15. configuration/system/crBehaviorReductionFactor
    16. configuration/system/crControlReductionFactor
    17. configuration/system/snBehaviorReductionFactor
    18. configuration/system/snControlReductionFactor
    19. configuration/system/wsBehaviorSleeper
    20. configuration/system/wsControlSleeper
    21. configuration/system/crBehaviorSleeper
    22. configuration/system/crControlSleeper
    23. configuration/system/snBehaviorSleeper
    24. configuration/system/snControlSleeper
    25. configuration/system/wsBehaviorSleepTime
    26. configuration/system/wsControlSleepTime
    27. configuration/system/crBehaviorSleepTime
    28. configuration/system/crControlSleepTime
    29. configuration/system/snBehaviorSleepTime
    30. configuration/system/snControlSleepTime
21. Create nodes and add them to the SystemConfiguration
22. Clear buffer
23. System Configuration

Figure out why the Digester class can not be used

I am having a problem where the robocode wants the org.xml.sax.SAXException local but when I bring it local it has problems creating org.xml.sax.XMLReader.

I believe that robocode wants the org.xml.sax.SAXException local is because there is already code that has the package of org.***

Got a sample of the system to work with the XMLReader with only adding SAXParseException.class and SAXException.class. When adding the use of DTDHandler the robot prints out

lwf.MyFirstRobot: Got an error with this class: java.lang.ClassNotFoundException: Could not find: org.xml.sax.XMLReader: java.io.FileNotFoundException: C:\robocode\robots\org\xml\sax\XMLReader.class (The system cannot find the file specified)

and the robot will not appear.

Force robocode to look at the Java code base

Cannot find a way to perform force robocode to use the JRE code only.

Rename the org to localorg

Renaming of the org to localorg did fix the error. After making the change the below error still appears localorg.apache.commons.digester.SetRootRule: Got an error with this class: java.lang.ClassNotFoundException: Could not find: org.xml.sax.SAXParseException: java.io.FileNotFoundException: C:\robocode\robots\org\xml\sax\SAXParseException.class (The system cannot find the path specified)