Monday, June 26, 2006

Spring MVC Forms

Some notes about Spring MVC and forms. In simple situations when the HTML forms maps well onto the domain model validation can be performed directly on the domain model and the domain model can be used as the command class. This reduces the ammount of code that needs to be written and also means that validators can be used across multiple controllers. The problem is that this only works so long as the form maps well to the existing model. In more complicated situations you will need a seperate command class to handle the processing of the form submission and will then want to validate that so that you can easily map errors in validation back to the form fields. If you map the command class onto the domain model and then validate, discovering which errors relate to which parts of the form becomes tricky. The problem is that then you may end up having duplicate validation code for multiple controllers. When mapping a text input box <input type="text" name="number"> you have several options as to how the mapping is performed.
  • Don't attempt any parsing of the field when mapping the field to the command class and leave it as at String. This then means that your controller has to perform the conversion to an integer and generate the apropriate error message. This complicates your controller with code that could be performed better elsewhere.
  • Have it mapped to a primative int. This means that no matter what the input in the form as long as the conversion and validation succeeds you will always get a number back from the command object even is there wasn't one present in the form, and this is the problem, you can't tell when the user left the field empty.
  • Have it mapped to an Integer. This has the advantage that you can now tell if the user left the field blank as you will get back null. You can only do this though if you change to use a custom property editor as the default is to attempt to parse the empty string which fails and raises an error. I do this in my controller: protected void initBinder(HttpServletRequest request, ServletRequestDataBinder binder) {   // Map numbers to Integer but allow them to be empty.   binder.registerCustomEditor(Integer.class, null, new CustomNumberEditor(Integer.class, true)); }
  • If you use property editors to perform your conversion from strings to integers then you will probably also want to change the error messages that are generated when a problem occurs. Have a look at DefaultMessageCodesResolver for the messages to map. As a starting point I have a property: convert to a number. which is displayed when the data binder can't convert a string to a number.

Locking Issues in Bodington

Currently most of the locking that is performed in Bodington is done using synchronized blocks in the Java code. Examples of this can be found in org.bodington.server.resources.ResoureTree and org.bodington.server.resources.UploadedFileManager. This works reasonably well except when you want to scale your application across multiple JVMs as the locking only applies to one JVM. The two ways around this are either to use explicit locks in the database or to code optimistic locks in your application. As deploying Bodington across multiple JVMs isn't important at the moment we are continuing to develop using the existing pattern of synchronized blocks. This is important to me because in my quota implementation I need to lock sections of the tree while I calculate the current quota usage. While I am calculating the usage I don't want that section of the tree to change. As long as the locking is ok the quota calculations should stay in sync with the usage.

Monday, June 19, 2006

Uploaded File Quotas

The implementation of uploaded file quotas is progressing reasonably well in WebLearn. They are now at a state where they seem to work reasonably well and I think the locking should prevent errors occurring when two people are uploading files at the same time. Implementing quotas on uploaded files was reasonably easy as there is one API through which all uploaded files are created/deleted. Next I have to implement quotas for resources which will be much harder as at the moment there isn't a standard API for creating resources although ResourceTree is a starting point. The user importer is now in a reasonable state for next years students and I am just waiting for there to be a larger dataset and then will finish off the testing/debugging. Hopefully there shouldn't be too many bugs in it as most of the logic for group creation was taken from a previous imported and that was working reasonably well.

The Observer | Business | Telecoms pray for time when the Skype finally falls in

The Observer | Business | Telecoms pray for time when the Skype finally falls in talks about how the telecoms companies are worried about Skype and mentions Oxford University as an example of an institution that no longer bans Skype. The reason for this is that in order to use Skype at Oxford you have to configure it so that the peer-to-peer (P2P) part of the client can be filtered by the University firewall preventing "leaching" of bandwidth. We have a page documenting Oxfords Skype setup. It seems that the news never has the full story.

Monday, June 12, 2006

Phone Line Repair and MaxDSL Speed

Last wednesday evening our phone line went dead along with our ADSL service. So the next day we called our provider and asked them to test to line and after a little nagging they got on to BT and had the line tested on Friday, they say there is a fault on the line and an engineer needs to be called. They said an engineer would be out to us on Wednesday of the following week. I was not impressed. Anyway it turns out that a BT engineer appears around lunch time on Sunday and after a while fixes the phone line. After he's left I reconnect the ADSL and find that it isn't working, thankfully he gave us his mobile and so we drop him a call. He says that it will probably need another engineer to look at it tomorrow as nobody who knows about the ADSL stuff is about on a Sunday. Monday morning I get up and discover that the ADSL is working, great. However looking on the router it is syncing at 3584 Kbps/640 Kbps. As I mentioned in previous post I was getting allot more (it settle down to around 5500Kbs/448Kbs), so although I now have working internet access again I have lost 2Mbps in the process. I suspect that there isn't anything I can do about this as the line works fine, just not as well as it used to.

Building Tomcat 5.5 from SVN problem

I was attempting to build Tomcat 5.5 from SVN using the instructions from tomcat pages, however I was getting an error of: /home/buckett/src/tomcat/build.xml:49: The following error occurred while executing this line: /home/buckett/src/tomcat/build/build.xml:791: The following error occurred while executing this line: /home/buckett/src/tomcat/container/webapps/docs/build.xml:88: java.lang.ClassNotFoundException: after a little head scratching I discovered that I had installed ant as an Ubuntu package and I was missing the ant-optional package which provided this extra functionality. After intalling this additional package everything went smoothly.

Bodington Database Objects and User Importer

I am currently working on some new user import code for Oxfords installation of Bodington (WebLearn) which takes a feed of about 35000 users and creates accounts and groups for them. Due to the large number of users involved and the small number of changes that are made each day it is important to try and keep the import running fast. At the moment the bottleneck seems to be the database which eats up most of the CPU during the import, a quick look at a process listing shows PostgreSQL using 80% or the CPU and Java using 10%. Looking at the statement logging it seemed that for each user it was always updating the user in the database even if nothing had changed. Update statements are expensive for a database compared to select statements so cutting down the number of updates seemed an obvious way to improve the performance. Every object that is stored in the database by Bodington extends org.bodington.database.PersistentObject which provides the skeleton for database objects One feature of PersistentObject is that it keeps track of wether an object has unsaved changes or not. This is done through calling the setUnsaved() method which is done by most setters of the subclasseds. Eg: public void setName(String name) { = name;   setUnsaved(); } Now this means that even if the name is set to the same value the object is flagged as being unsaved. One option would have been to have add checking to the userimporter so that it only updateded the name of a user if it was different to the current value: if (!user.getName().equals(newName))   user.setName(newName); However there is no reason why other sections of Bodington shouldn't make use of this checking and it is also a database issue so I moved this code into the setter and ended up with: public void setName( String name ) {   if (name == null)     throw new IllegalArgumentException("Name cannot be null");   if (name.equals(     return; = name;   setUnsaved(); } The checking to see that if name == null is because the database has a NOT NULL constraint and this also prevents a NullPointerException from being thrown if setName(null) is called. I added this checking to org.bodington.server.realm.Users and org.bodington.server.realm.Aliases and a quick test of updating 1000 users gave the results:
  • Always Saving: 1minute 3 seconds.
  • Saving Changes: 30 seconds.
which isn't a bad improvement in performance. One side note is that always saves the object and I could have changed it so that it only saved changed objects but I'm not sure that all the setters of the subclasses call setUnsaved() so I am doing my checking in the importer. The group mapping has come wholesale from some old user import code with a few changes and just needs a little tweaking and it should be nearly ready for some proper testing.

Monday, June 05, 2006

Glassfish and Invalid URL Pattern

I was attempting to deploy WebLearn to the Glassfish application server and was gettting the error "Invalid URL Pattern" for /site/* which is used in various places in our web.xml configuration file. It turns out Glassfish doesn't like extra whitespace in it's url-patterns and we had things like: <url-pattern>   /site/* </url-pattern> when they need to be: <url-pattern>/site/*</url-pattern> I've got the web.xml fixed on WebLearn and will push it across to Bodington at shortly.

Ubuntu Dapper Drake Upgrade

With Ubuntu Dapper Drake being released on the 1st of June I decided to upgrade my work desktop machine today. I just edited my sources.list file and then ran apt-get update; apt-get dist-upgrade. Most of the upgrade went fine, but here are a few things that other people may fin helpful were. nVidia Monitor Detection It seems that the new nVidia driver was autodetecting that my monitor could do 76Hz vertical refresh rate at 1280x1024. Not while it can display this it didn't seem to be able to adjust the screen enough and as a result I was unable to see the two left most columns of pixels. I attempted to fix the vertical refresh rate in my configuration file ( /etc/X11/xorg.conf ) by setting the line VerticalRefresh 50-60 but every time I started X up it would continue to use 76Hz which was outside the range. After quite a bit of Googling I discovered that the nvidia driver attempts to autodetect the monitor settings by default and the way to turn it off is to add an option to the file: Section "Device"   Identifier "NVIDIA Corporation NV18 [GeForce4 MX 440 AGP 8x]"   Driver "nvidia"   Option "UseEdidFreqs" "false" EndSection Pinned Rhythmbox Somehow rhythmbox was pinned in the Synaptic Package Manager, this didn't seem to be affecting apt-get which was upgrading it without any problems. The problem turned out to be that synaptic can have its own list of pinned applications which are stored in /usr/lib/synaptic/preferences. As this only entry in this file was the pinning of rhythmbox I deleted the file and everything worked fine. Packaged Java Ubuntu now packages the Sun JDK which can be install just like any other package. I had a problem though in that all my alternatives (/etc/alternatives) were pointing to the wrong versions. I found a good post on galternatives that allows you to easily edit them.

User Import and Bodington Developers Meeting

The user import code is coming along, last week I added parsing of colleges, courses and departments and some tests against small bits of data. The status handling has also been sorted out so rather than working with strings they are converted to an enumeration. Most of last week was taken up by a Bodington developers meeting in Leeds. The meeting went over two days with the first day being for discussion of short term Bodington developments and the second day for the longer term Bodington vision. Out of the first day we decided to release Bodington 2.8 shortly, most of the code is already in CVS for Bodington 2.8 so once a few extras have been added and some tested has been performed 2.8 should go out the door. Other things discussed included a catchup on what developments have been happening at the various sites that run Bodington and the JISC projects that are using it. The second day we discussed longer term direction including Portals, SOA, SOAP, REST. At the end of the day I just want to build a better product.

Saturday, June 03, 2006

I owe PayPal money

I owe PayPal 56 pence as a result of a currency conversion charge from a long time ago I think. Today I recieved an email nagging me to correct my balance and decided to attempt todo something about it, so I login to my account and attempt to add funds but it refuses to allow it just returning me to the form highlighting the fields that I have filled in sucessfully. So I looked at the email that PayPal had sent me and noticed it ended with: We appreciate your cooperation and invite you to reply to this email with any questions. Yours sincerely, PayPal Please do not reply to this email. This mailbox is not monitored and you will not receive a response...... So I replied to the mail and we'll see how we get on. They do accept cheques to fix my balance but I signed up with PayPal to get away from cheques....