Java Reflection to populate object

In the project we needed a way to create an object after the parsing of the xml file was done. The planned object for creation was a Project object which has the basic information about a Catroid project. We focuses on two possible alternatives to do this. One was use setter methods to populate the project object, the other was using Java reflection to do it.
We conducted another performance experiment to do this. One method used setters to populate the project object. The other used reflection method invocation to do it. Project object did not have setters already on it. So instead of changing the existing Project code I created a ProjectProxy class for the sake of the test. The test proved that there was no drastic change in performance in the two methods. But we decided to go with reflection because it was useful in many other ways to do repetitive things in a loop.

The reflection method was enhanced to set private fields from the parsed document values. That enabled me to create a project object successfully without much change to the existing Project class code. I only had to add a default constructor. The written tests for these tasks passed. 

 
try {
   project = Project.class.newInstance();

   Field[] projectClassFields = Project.class.getDeclaredFields();

   for (Field field : projectClassFields) {
    boolean isCurrentFieldTransient = Modifier.isTransient(field.getModifiers());

    if (isCurrentFieldTransient) {
     continue;
    }
    String tagName = extractTagName(field);
    Object value = null;
    String className = field.getType().getCanonicalName();
    if (className.equals("int")) {
     value = new Integer(Integer.valueOf(headerValues.get(tagName)));
    } else if (field.getType().getCanonicalName().equals("java.lang.String")) {
     value = headerValues.get(tagName);
    }
    if (value != null) {
     field.setAccessible(true);
     field.set(project, value);
    }
   }

  } catch (Throwable e) {
   System.err.println(e);

  }

Catroid coding standards

From having various discussions with the mentors I got very important feedback. It is better to have method names/ variable names represent what they do. That eliminated fully the requirement of having comments explaining every small code snippet. And the code is very readable that way too.

The first sub task of parsing header was completed in this way. 

First few Sub tasks.

My mentor for the project is Thomas of Catroid. First we communicated through mail and after we came to an agreement to use Skype. We have weekly meetings and additional meeting to discuss progress and problems.

First of all I was not very familiar with using Mercurial as a version control system. However I have used SVN before during my internship. Thomas introduced to me a very helpful tutorial by Joel Spolsky that helped me get the idea of Mercurial and how it differs from SVN. The tutorial is very easy to understand and fun to read.

• Mercurial Tutorial

Mercurial CVS is a distributed version control system (In contrast, SVN is a centralized version control) which means each contributor gets his own repository locally. i.e. the repo get's distributed. And the user can commit their local changed to their local repository before pushing them to the main external repository. Another distinctive change is that the changes in Mercurial are kept as changesets, which means information on what and where stuff changed gets captured and stored in revisions. There are a lot of specific characteristics to Mercurial you can learn from the tutorial.

My forst subtask related to the project was buiding a way of parsing a project related xml and extract important information from the header (first part) of the document.

Meanwhile Catroid is newly named as Catrobat programming language. And Catroid is the IDE for android devices.

The Catroid Project code XML:-

Catroid projects are saved in XML file named projectcode.xml in the device sd card. This project code xml file has the root <Content.Project>. And what follows are some general information regarding the project and the version. These are called hereafter by headertags. The headertags information can be helpful to list diplay the project metadata in the project listings. 

Extraction

There are several ways to parse an XML document. SAX and DOM are well known methods of doing so in Java. Because of the nature of extracting in this subtask in to obtain only the header information we have to look at efficiency in getting only the header information. For this I implemented two parsers, A SAX parser and a normal string parser.

String parser:-

The string parser uses a buffered reader to read the xml document. And for each line it will check for the known header tags. It will begin populating a String list with the values only after encountering the <Content.Project> tag. And will stop at the encounter of the <SpriteList> tag which marks the end of the header. The code for this is shown below. 

 
fReader = new FileReader(XMLFileStream);
   BufferedReader bReader = new BufferedReader(fReader);

   String currentLine = "";
   boolean inHeader = true;
   boolean tagCheck = false;

   while (inHeader) {
    currentLine = bReader.readLine();

    if (currentLine.trim().equals(OtherTags.CONTENTPROJECT.getOtherXMLTagString())) {
     tagCheck = true;
    }
    if (tagCheck) {
     if (currentLine.trim().equals(OtherTags.SPRITELIST.getOtherXMLTagString())) {
      inHeader = false;
      tagCheck = false;
     } else {
      for (int i = 0; i < 7; i++) {
       String currentTag = tagIndexes[i].getXmlTagString();
       if (currentLine.contains(currentTag)) {
        parsedStrings.add(currentLine.trim().replace("<" + currentTag + ">", "")
          .replace("", ""));
       }
      }
     }
    }

   }
 

This faces a problem if the elements are not seperated by new lines. That won't be a problem if the XML creating process adds new tags on new lines. HeaderTags is a enum declared to store the 7 headertag values for organized easy access anywhere.

SAX parser:

I also implemented a simple SAX parser that gets the values when the defined header tags are found in the parsed doucument in the endElement() method.

 
private List saxParser(File projectCodeXML) {
  parsedStrings = new ArrayList();
  SAXParserFactory parserFactory = SAXParserFactory.newInstance();

  try {
   SAXParser parser = parserFactory.newSAXParser();

   parser.parse(projectCodeXML, this);

  } catch (ParserConfigurationException e) {
   Log.e("SimpleParser.saxparser", "parserConfiguration exception");
   e.printStackTrace();
  } catch (SAXException e) {
   Log.e("SimpleParser.saxparser", "SAX exception");
   e.printStackTrace();
  } catch (IOException e) {
   Log.e("SimpleParser.saxparser", "IO exception");
   e.printStackTrace();
  }

  return parsedStrings;

 }

 @Override
 public void characters(char[] ch, int start, int length) throws SAXException {
  tempVal = new String(ch, start, length);
 }

 @Override
 public void endElement(String uri, String localName, String qName) throws SAXException {

  for (int i = 0; i < 7; i++) {
   String currentTag = tagIndexes[i].getXmlTagString();
   if (qName.equalsIgnoreCase(currentTag)) {
    parsedStrings.add(tempVal);
   }
  }

 }

An also a simple method was added to get the value of any header tag that was parsed by the SAX parser.

 
public String getvalueof(HeaderTags tag, File XMLFile) {
  List parsedValues = this.Parse(XMLFile);

  return parsedValues.get(tag.ordinal());

 }

The tag.ordinal returns the integer value of the HeaderTag enum.

Althought the first target was to create a Project pbject after parsing the document it has been proven difficult becuase of private attributes of that class and absence of a default constructor. So we are looking into using reflection to overcome this.

Replacing the XML serializer of Catroid: The plan.

(The following is the project proposal data I submitted to GSoC for the project.)

Replace Current XML serializer for Catroid

Name :                 Samitha Jayathilaka.

University :         University of Moratuwa, Sri Lanka

Country :             Sri Lanka.

Contact data :    Email =                  samjayathilaka@gmail.com

                                Phone =               +94715445739

                                Twitter =              @SamJayathilaka

                                Irc_nick=              SamJay

Problem description

In Catroid the project is saved as an XML file. This is a core logic in Catroid. All the essential data related to the project like sprites, scripts, bricks, sound files (metadata), image files (metadata) etc. is stored in an XML schema in a very organized way.

             

figure 1.1 The structuring of the XML created from the Project object.

To do this Catroid uses an xml serializer library called Xstream. Xstream is easy to use for serializing and deserializing object from and to XML.

However there are some limitations when it comes to using XStream. One proven limitation the Catroid community has with XStream now is the difficulty of synchronizing between two versions of Catroid. In different versions some changes may be done to the object structure and thereby the XML schema created by XStream. Those changes can happen due to newer Catroid versions having added new brick functionalities and removing old obsolete script elements.  So it has been proven difficult to make older version projects compatible with newer versions of Catroid.  There are some other flexibility issues as well.

Current Usage

The XML for a project is used to load and save projects. This is handled in the at.tugraz.ist.catroid.io.StorageHandlerclass. XStream is used simply in the public boolean saveProject(Project project) method and the public Project loadProject(String projectName) method. It is done in a single line like xstream. toXML(obj) and xstream.fromXML(stream). The serialized object is at.tugraz.ist.catroid.content.Project which has a spriteList which in turn contains all the relevant data that follows that is related to a particular project. The Project class has @XtreamAlias annotations to define additional fields as well.

Additionally XStream is also used in at.tugraz.ist.catroid.io.CatroidFieldKeySorter class to get the field keys, compare and sort them and store the Field keys in a map structure.

The basic functional requirements of the newly implemented XML serializer are to do these as easily or more easily.

Furthermore the new XML serializer would be more suitable if it had some other features of XStream such as speed and low memory footprint, Clean XML, supporting circular reference, detailed error detection in XML and handling etc.

Solution

The new XML serializer would minimally have to have the same existing support provided by XStream.

So I suggest having a newly added library which contains the XML serializing/deserializing features. And this library would contain all the classes and methods required to store a project and load a project by creating and deserializing XML files.

Like XStream this new library would have to have methods that take a Project object as an input and creates an XML file in the file system. We already have a good XML schema that represents a Project object.

The required functions of the new library would have to do are

1.       Create a new file in the file system.

2.       Identify different attributes in an object.

3.       Create XML tags for the identified attributes. This has to be done uniquely so as not to make two contextually different attributes in the whole xml document have the same name.

4.       Iterate through lists in the serialized object.

5.       Go down the hierarchy of the object model.

6.       Read an input stream from the file system when loading a project.

7.       Quickly identify if the tags on the read XML file is contextually identical to the Project object.

8.       Create new Project object and populate it with the data read from the XML file.

9.       Ignore any minor additional tags that are found when deserializing. 

10.   Handle exceptions like FileNotFound, UnrecogizedTag etc. This exception handling must be made flexible.

In addition a new focus of the new library is the conversion of XML to another XML schema that is structurally similar. This would solve the problem of projects from different versions having compatibility issues. Furthermore the new XML converter should come hand in hand with an XML validator, which can validate a particular converted/generator with an XML schema. All these requirements are explained in a graphical form below for clarity. The green parts are the parts that the library would have to take care of.

 

                               figure 1.2 Functions of the XML serializer

Testing

The development will be done hand in hand with the mentor provided by Catroid. And the development method will be made of Extreme programming and Test driven Development. So development will happen in iterations. Every iteration will give a functional outcome. And unit tests will be written constantly in the ways of Test Driven Development. Testing the new functionalities will be crucial to get the outcome right. Testing will also include every version changes of Catroid that significantly done changes to the project creation as a whole.
And after integrating the new XML serialize/deserialize library additional integration tests would be required. Integration should not harm the functionalities of the other functionalities in any way.

I emphasize on the usage on logging and javadoc commenting. It is important to have good logs of important functionality done by the new library and in turn it will help in debugging as well. And In a team project like this good commenting is a must in my opinion. Because this is a core functionality of Catroid (project storage handling), it is required to have a comprehensive structure and code.

Schedule:

This schedule is in terms of what milestone we must achieve to complete this project. Those are.

           ·         Research and discovery of a suitable method to use to serialize/deserialize    objects.

           ·         Initialization of the new library.

           ·         Saving a project using the new library.

           ·         Loading a project from an XML file from the library.

           ·         Load a project from a different version.

           ·         Define a XML schema for XML conversion.

           ·          Define Convertible tags.

           ·         Conversion of an XML done.

           ·         Create the XML validator.

           ·         Validate a generated XML.

           ·         Handle validation errors.

           ·         Integrate to Catroid.

           ·         Test integration. Tests pass.

The timing of those objectives will have to be discussed more. Any modifications can happen to this schedule in the actual progression of the project.

I think this is a feasible project that can be successfully within the period of GSoC 2012. It is an essential addition for the whole Catroid functionality. I will give high priority for the completion of the Xml serializer/Deserializer. 

Let the games begin!

First a little bit about myself.

I'm Samitha Priyanath Jayathilaka, a Computer Science and Engineering undergraduate from University of Moratuwa. Currently I am in my final year. I love everything about programming. Recently I've developed an interest in Android development. It is developing applications for Android devices. I find this development so natural and easy, it feels like almost playing a game.

Google Summer of Code 2012

Google holds a annual program called "Google Summer of Code" in the times of April to August. This program enables students from around the world to get in contact with an Open Source organization and voluntarily do a project for that particular organization.
The program is being held this year too (2012). I applied to participate in this years program doing a project for Catroid.

Catroid
http://code.google.com/p/catroid/
Catroid is a application that helps Android users to create user programs using a "visual programming language". This is mainly targeted for kids and it is inspired by Scratch. The application is still in the beta stages and there is a great potential for the application to develop.

I have tried Catroid to develop some of my own animations/games and I personally found it very entertaining and highly usable. This is an application with a bright future. These Catroid projects can be downloaded for use in the following address.
http://www.catroid.org/catroid/index/1

The Project
I found out about Catroid from the GSoC organizations list. I liked the idea of the project on the get go. There were a list of ideas provided by Catroid in their project wiki. I got interested in the idea of replacing the XML serializer used by Catroid to save and load project. Having worked a bit with XML handling before and seeing how inportant this is to the functioning of the whole app I decided to apply with this project. And I got selected to carry on with the project!