blog buckett

General Logging in MySQL

2011-12-31T10:32:00.001Z

I was looking to investigate some connection problems we were having to a MySQL database and wanted to be sure that the client was connecting to correct machine. MySQL 5.1 doesn't provide this information on it's own, but it is contained in the general log. However the log contains all statements executed against MySQL so will have a performance impact on a production machine. The general log can be enabled/disabled while MySQL is still running which means you can enable the logging, capture the events your interested in and then disable it again. Todo this connect to MySQL as a user who can set GLOBALS (typically root) and run:

mysql> SET GLOBAL general_log_file='/var/log/mysql.log';
Query OK, 0 rows affected (0.00 sec)

mysql> SET GLOBAL general_log = 1;
Query OK, 0 rows affected (0.00 sec)

Then the file /var/log/mysql.log should start filling up with statements. Once your done disable the general log:

mysql> SET GLOBAL general_log = 0;
Query OK, 0 rows affected (0.01 sec)

In my case I was looking for connection messages so a simple grep pulled out the lines I was interested in:

grep Connect /var/log/mysql.log | less

Using a maven SNAPSHOT plugin from repository.apache.org

2010-06-01T13:25:00.004Z

I was working on a project and hitting a bug in the maven eclipse plugin which was causing the JRE to be placed incorrectly in the Eclipse .classpath. If it was just a single project I'd have gone and edited the resulting file manually but it affected a few, so I thought I'd try and grab a newer copy of the maven eclipse plugin. I did this by first adding a profile to my maven setttings.xml file ($HOME/.m2/settings.xml on UNIX).


<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">



   <profiles>



     <!-- This is the additional bit you need -->

     <profile>

       <id>apache-plugin-snapshots</id>

       <pluginRepositories>

         <pluginRepository>

           <id>apache-snapshots</id>

           <name>Apache Snapshot Repository</name>

           <url>https://repository.apache.org/content/groups/snapshots-group/</url>

           <releases>

             <enabled>false</enabled>

           </releases>

           <snapshots>

             <enabled>true</enabled>

           </snapshots>

         </pluginRepository>

       </pluginRepositories>

     </profile>



   </profiles>



</settings>

and then run the mvn eclipse command:


mvn -Papache-plugin-snapshots org.apache.maven.plugins:maven-eclipse-plugin:2.9-SNAPSHOT:eclipse

which downloaded the newer plugin and then ran it against the project I had checked out.

Java host caching (sun.net.inetaddr.ttl/networkaddress.cache.ttl) on OS X

2009-03-23T13:40:00.002Z

On my Mac OS X box I was trying to get some round robin DNS working from Java (1.5) but I was the host was always resolving to the same IP. I'd set sun.net.inetaddr.ttl=0, but that didn't seem to help. It seems out of the box a Mac will cache lookups, but you can clean out the cache with the command dscacheutil -flushcache

gitweb - 500 - HEAD ref not found for project

2009-02-02T10:55:00.002Z

We have some git repositories here at Oxford and a gitweb interface to view them. For some of our projects I was seeing:

500 - HEAD ref not found for project

It turns out this was because the HEAD file ( {git-directoy}/HEAD ) was pointing to ref: refs/heads/master and these projects didn't have a master branch. Changing the HEAD file to point to the correct branch fixed gitweb.

Trimpath error "context._MODIFIERS is undefined"

2009-01-22T16:24:00.003Z

I was doing a little JavaScript templating with Trimpath today and at one point I was getting a nice error in my console of:


context._MODIFIERS is undefined

Looking through the trimpath code it turned out this was because I was passing in a string rather than an object as my context for the generation of the template. So I was effectively doing:


TrimPath.processDOMTemplate("someNodeId", "some string");

rather than:


TrimPath.processDOMTemplate("someNodeId", {key: "value", otherKey: "value"});

In my case I was getting the error because I forgot to eval some JSON before passing it to trimpath.

Pooling in Sakai LDAP

2008-12-24T11:52:00.002Z

Here at Oxford we connect to a LDAP server to get details about users for our Sakai service. Previously we had reports of some pages taking a long time to load when lots of users details where requested from the LDAP. One obvious change we could make was to switch to use pooling on our connections to the LDAP server. To check that this was improving the performance I wrote a little test case which attempts to get details about 100 users individually from the LDAP.

Without pooling - 100 users in 10 sec = 10 users/sec.
With pooling - 100 users in 5 secs = 20 users/sec.

This is a nice doubling of performance, although I was still expecting to see better performance generally and while the test is running the local load on the machine isn't very high so I suspect any further performance improvements will have to come on the server.

Tomcat UNIX aliases

2008-12-23T15:05:00.002Z

To make managing development instances of tomcat a little easier I have a few bash aliases (you can add these lines to your ~/.bash_profile):

alias tchome='export CATALINA_HOME=`pwd`; echo CATALINA_HOME=$CATALINA_HOME'
alias tctail='tail -f ${CATALINA_HOME}/logs/catalina.out'
alias tcless='less ${CATALINA_HOME}/logs/catalina.out'
alias tcshow='echo ${CATALINA_HOME}'
alias tc='${CATALINA_HOME}/bin/catalina.sh'

so my typical workflow is cd {tomcat-directory}, tchome and then I move back around the filesystem and use tc start and tctail.

Keep on rolling...

2008-09-29T20:52:00.002Z

Well after a quite a few weeks of waiting my new set of wheels turned up. A Brompton S6L in apple green. The gears needs a little adjusting but otherwise it's a great ride.

Error: sec_error_bad_signature with Firefox 3

2008-09-24T21:06:00.008Z

Recently when most of our team upgraded their browsers to Firefox 3 we noticed then when we tried to visit on of our internal websites over a secure connection we would get presented with an error page saying "sec_error_bad_signature". Most of the existing comments on the web seem to be saying that they are seeing this error when visiting a site with an untrusted certificate. However in our case it was a correctly signed certificate obtained through the JANET Server Certificate Service.

The server is just a Ubuntu Linux server with a standard installation of Apache Tomcat 5.5 and another very similar setup was working correctly. But after close inspection it became clear that the server had a public key using the DSA algorithm but the machine that was working correctly was using the RSA algorithm. The certificate work was done with the Java keytool command and I think I probably didn't specify the -keyalg RSA argument. After recreating with RSA the certificate and installing it everything was working correctly. The screenshots are taken with Safari (which worked fine with DSA) and shows the two different certificates.

Jack Buckett

2008-08-16T08:45:00.002Z

Well it's happened again and our second son Jack Buckett was born on Wednesday the 6th of August at 8:31 weighing 4.09kgs. We are all home and doing well now.

RSF Error

2008-07-15T09:04:00.005Z

I've just been doing a little web development with RSF and it seemed that sometimes when accessing the RSF webapp I would get an error:


[An error occurred handling this RSF request]

[Context has not been started properly]

with a stack trace starting with:


WARN: Error initialising SakaiRSF servlet:  (2008-07-15 10:07:43,909 http-9080-Processor21_PonderUtilCore)

java.lang.IllegalStateException: Error acquiring web application context - servlet context not configured correctly

 at uk.ac.cam.caret.sakai.rsf.servlet.ReasonableSakaiServlet.init(ReasonableSakaiServlet.java:41)

 at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1139)

 at org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:791)

It seems that this occurs if you access the RSF webapp while it is still starting up. The only way to fix it is to reload the webapp again and give it a little more time.

Removing Eclipse team provider information

2008-04-15T10:20:00.005Z

At the moment there are two competing plugins for Eclipse that can provide SVN integration, Subversive and Subclipse. At various times I have tried each of them and they both have their plus points however if I have a project checked out with one plugin and the Eclipse installation I am using doesn't have that plugin installed I don't get any options under the team menu to share the project.

Rather than getting the old plugin back running and "un-sharing" the project you can just delete a file from the workspace metadata to allow you to re-share the project.
workspace/.metadata/.plugins/org.eclipse.core.resources/.projects/{projectname}/.index.properties
This seems to work for me under Eclipse 3.3.2.

tailor for svn to hg

2008-03-19T09:28:00.002Z

Well after attempting to convert the portal part of the sakai subversion repository to mercurial I have discovered that tailor (the tool I used) doesn't understand svn tags and branches (or doesn't convert them to mercurial tags and branches). So although is seems to have converted the portal ok. I now have a huge checkout:

buckett@oucs-matthewb:~/tmp/portal $ du -sh *
228M    branches
4.0K    project.log
739M    tags
23M     trunk

So I think tailor would well for converting a branch (eg 2.5-x) or trunk but not for keeping details about when those branches were made from trunk.

Converting Subversion repository to Mercurial with tailor (Ubuntu 7.10)

2008-03-17T17:49:00.003Z

I'm looking at managing our local builds in a better way than a set of patches against a point on the Sakai tree. Currently I am looking at placing a copy of the sakai subversion repository into a mercurial one. Mercurial doesn't seem to have good subversion support out of the box (unlike git) but the suggested route seems to be to use a genral purpose tool called tailor. Subversion, mercurial and tailor are packaged for Ubuntu (7.10) so after installing them I attempted a quick conversion of part of the Sakai repository.

To do this I generated a tailor configuration file with a command similar to:
tailor --verbose --source-kind svn --repository https://source.sakaiproject.org/svn/ --module portal --target-kind hg > sakai.tailor
and then attempted to run the conversion with:
tailor -c sakai.tailor
which failed with a nice python stack trace:
File "/var/lib/python-support/python2.4/vcpx/repository/hg.py", line 438, in _prepareWorkingDirectory message = 'Tailor preparing to convert repo by adding .hgignore') File "/var/lib/python-support/python2.4/vcpx/repository/hg.py", line 339, in _hgCommand allopts = self._defaultOpts(cmd) File "/var/lib/python-support/python2.4/vcpx/repository/hg.py", line 333, in _defaultOpts findcmd = commands.find AttributeError: 'module' object has no attribute 'find'
after a little puzzling I found a bug report about this "not compatible with mercurial 0.9.4", which contained a small patch to fix the issue. Applying this patch to my local machine allowed tailor to run fine. It seems that Ubuntu 8.04 (which will be released in a month) contains a later version of tailor so people shouldn't see this issue.

Following Sakai developments

2008-02-25T11:54:00.003Z

Some of my current work involves developing the Sakai portal and collaborating with others in this space. To keep abreast of what is happening I follow the SVN commit messages and have a filter setup in the Sakai bug tracking system (JIRA).

I filter the SVN commit messages (as I get notified of every single Sakai commit) so that I only see the ones that relate to the /portal part of the repository. My filter (GMail) for this is to:(source@collab.sakaiproject.org) (portal/trunk OR portal/branchces). In JIRA I have a filter which emails me any tickets that have changed in the last 24 hours and have it run once a day. The trick for this is to have the Updated From field set to -1d

This all allows me to know about stuff that is changing without being overwhelmed with email.

Disabled input buttons in Sakai

2008-01-07T12:43:00.000Z

In HTML you can disable the ability for an input button to be clicked typically you just add the disabled attribute to a button with some value. Eg:


<input value="Click" disabled="true" type="button">

Now in Sakai if you do this for an active button (default one for a form) then under Firefox the button loses some of it's 3D look but remains blue (default default button colour). IE always renders a disabled button grey (allowed in W3C spec). To get a consistent look across the browsers and make it clearer to a Firefix user that the button is disabled you have to have the disabled attribute equal disabled:


<input class="active" value="Click" disabled="disabled" type="button">

the button is made grey by the following CSS (from /reference/library/src/webapp/skin/default/tool.css)


.act .active[disabled="disabled"] {
   color:grey
}

Nicer URLs?

2007-12-06T17:48:00.000Z

After quite a bit of head bashing nicer URLs in Sakai are coming along, here the site has nice name (/working_lu) and the page has a nice name (siteinfo). It's still at the very early stages but its getting there.

SVN Tip for Sakai

2007-11-28T08:51:00.000Z

When your working on a checkout of Sakai and editing files across multiple projects it can be handy to see what you have changed, svn status is normally the way todo this but due to the number of external definitions in Sakai it produces allot of output which can't easily be suppressed. Piping this through egrep and tr can clean things up.

svn status | egrep -v "^(Perform|X)" | tr -s "\n"

Gives a nice concise output as though those externals didn't exist. Aliasing this to something like svnst makes it easy to get back to.

NoMachine Client in Fullscreen

2007-11-21T09:56:00.001Z

I have recently discovered the machine that is NoMachine and am using it to connect to my work computer from outside the office. So far it seems to work really well. The only thing is that I like to run it in fullscreen mode and was struggling to find out how to return to my local desktop. Eventually I found this post about NX which provided all the solutions. In summary the options are:

Clicking on the top right pixel: Minimises the NX Client and returns you to your client desktop.
CTRL-ALT-M: Same as above
CTRL-ALT-T: Brings up a popup asking if I want to terminate the session or just disconnect from it.

Now if only they had a client of Mac OSX which supported fullscreen mode.

Sakai Sessions and UsageSession

2007-09-24T16:33:00.000Z

Having just been looking at these for a little while I thought I'd document them a little. The basic setup is that the Sakai Session is your session with the application and is very similar to a HttpSession. Your Session is setup for each request by the Sakai RequestFilter. The UsageSession on the other hand is your logged usage of the application amd is mainly there so that you can enumerate all the active UsageSessions. The UsageSession is actually store in the Session and is mainly used for presence.

However just to confuse things a little the UsageSessionService (manages UsageSessions) has some very important methods, login() and logout(). These setup and destroy your Session and UsageSession at the same time. Login tools in Sakai should call UsageSession.login() to setup the Sakai session with the authenticated user details.

Formatting with Styles

2007-09-23T07:17:00.000Z

There is a very clear post from darcusblog about how having a good UI can encourage people to use styles in their document. Unsurprisingly the company with the good UI is Apple, but it shows that a good UI can guide users into good practices.

More Kerberos Puzzlers

2007-07-23T09:41:00.000Z

Ok so when trying to kerberos enable another machine for ssh connections things were almost working (getting offered kerberos for authentication by sshd) but connections were always failing with the following error message in the logs:

Jul 17 12:33:46 machine sshd[5020]: debug1: An invalid name was supplied\nHostn
ame cannot be canonicalized\n

Now after looking through pages of google it seemed to be that there was something wrong with the hostname but looking on other machines they had very similar setups and were working correctly. The reason for this message was that I think that kerberos was using the hostname for kerberos and couldn't find an entry for the hostname in DNS. The reason it couldn't find a hostname was the there wasn't a search entry in /etc/resolv.conf specifying the domain to look in, adding search oucs.ox.ac.uk to /etc/resolv.conf fixed the problem and everything worked fine.

Hopefully this post might help someone else from scratching hole in their head. But it would have been much easier to debug if the invalid hostname was logged in the message.

svn_load_dirs.pl and Ubuntu

2007-07-18T10:12:00.000Z

I need to do some vendor drops with Subversion and the documentation seems to point to svn_load_dirs.pl as being the best place to start. So I install subversion and subversion-tools onto my Ubuntu system only to find that I don't seem to have a svn_load_dirs.pl installed. After a little bit of digging it seems that svn_load_dirs.pl was removed from the Debian package (which the Ubuntu one is based) due to licensing concerns although this isn't very clear as the changelog doesn't explain that this was one of the affected files. So it looks like I need to get myself a copy of the source distribution.

Ethiopian Food in Amsterdam

2007-06-14T09:54:00.000Z

Had some really nice food in Amsterdam last night, after a drink we ended up at an Ethiopian Restaurant suggested by Adam. I had some Gored Gored but we all ended up sharing as it came on a big plate. Good company and good food.

Loading Resources in Sakai Components

2007-06-14T09:04:00.000Z

I meant to blog about this a while ago but it has recently come up again and so I though I should write my ideas down. Ok so the problem is that you have a Sakai component which is created by the component manager and it wants to load something from inside the component, in my case it was an iBatis configuration file specified through a spring configuration file. So in my spring config I have something like:

  
<property name="configLocation">
 classpath:/sql-maps-config.xml
</property>

Now the problem is that this property is loaded through a PropertyEditor which sets the property to a ClassPathResource. In Spring the classloading policy is that resources should be loaded through the thread's classloader. When the component manager is starting up and so when my component is create this is caused by the fact that a web application has asked for it so the thread classloader points to the webapp and the resource in your component won't be found.

The solution to this is to have a way to load resources using the standard JVM Class.getResourceAsStream() which will use the classloader used to load the current class. So I created an class:

package org.sakaiproject.util;

import org.springframework.core.io.ClassPathResource;

/**
* Resource implementation for class path resources.
* Loads out of the same classpath as this class was loaded through.
*
* Supports resolution as java.io.File if the class path
* resource resides in the file system, but not for resources in a JAR.
* Always supports resolution as URL.
*
* @author buckett
*/
public class LocalClassPathResource extends ClassPathResource {

 public LocalClassPathResource(String path) {
  super(path, LocalClassPathResource.class.getClassLoader());
 }
}

and put this in my component. I also then change the configuration in my components.xml to use the resource directly:

<property name="configLocation">
<bean class="org.sakaiproject.util.LocalClassPathResource">
 <constructor-arg type="java.lang.String">
  <value>/sql-maps-config.xml</value>
 </constructor-arg>
</bean>
</property>

Now the configuration file gets loaded without any problems. Hopefully future Sakai developments in the component manager will mean this hack isn't needed later on.

Safari 3 Beta and FCKEditor

2007-06-12T12:42:00.000Z

Just saw in the news that Apple has release a Safari 3 Beta but include a Windows build, now this probably isn't going to take on IE and Firefox but does allow developers on Windows so have a better idea about how websites will render on it. Now it also allow people to see that FCKEditor nightly is working reasonably well in Safari.

Plane on Google Maps

2007-06-09T10:23:00.000Z

Found a plane on Google Maps today. You can even read the writing on it showing it's an Air Canada plane.

Sakai Maven Plugin Install

2007-06-08T13:37:00.000Z

Always losing the command to install the Sakai Maven Plugin, Dr Chuck has some instructions but they rely on already having configured your maven repository. The one stop command is:

maven plugin:download -DgroupId=sakaiproject -DartifactId=sakai -Dversion=2.2 -Dmaven.repo.remote=http://source.sakaiproject.org/maven

This only works with maven 1. If nothing else maybe this will give Chucks second post a little more Google karma and push it up in search results.

Custom Logging in Sakai

2007-06-07T08:24:00.000Z

If you are debugging something in Sakai and want to setup a custom log configuration you can just edit your sakai.properties. Here's an example of from a sakai.properties file:

log.config.count=2
log.config.1 = DEBUG.org.sakaiproject.portal.charon.handlers.HierarchyHandler
log.config.2 = DEBUG.org.sakaiproject.portal.charon.handlers.SiteHandler

Don't forget Resource normally same as Entity

2007-06-06T12:03:00.000Z

It seems originally org.sakaiproject.entity.api.Entity interface was called Resource and although most things seem to have been updated there are still some references across the codebase to Resources. An example is the org.sakaiproject.event.api.Event#getResource() method which actually returns a Entity reference.

Sakai Portal Handlers

2007-06-05T15:03:00.000Z

Sakai has a reworked portal in 2.4 called the Charon Skinnable Portal and this portal has the concept of handlers which can be used to handle different requests. Most of this code is in portal/portal-impl. The PortalHandler interface has a useful base class called BasePortalHandler and then some of the usable implementations are:

PageHandler - Just displays a page which will load the tool in an iframe without anything else by default. URL: /portal/page/placementId
WorksiteHandler - Just displays a site with the list of tools but no inter site navigation, supports supplying a additional page reference. URL: /portal/worksite/siteId
GalleryHandler - Display the site as well as the inter site navigation but still no logo or login box, supports supplying an additional page reference. URL: /portal/gallery/siteId
SiteHandler - Display a normal Sakai screen with site and inter site navigation, including login/logout links.

All the handlers are setup in SkinnableCharonPortal.init() which calls through to the portal service. The list of handlers if configurable at runtime through the portal service. Just posting this as a note for future reference really.

Bash History Expansion

2007-06-05T08:42:00.000Z

I've alway been quite fond of bash history expansion and in particular !! (previous command) and !$ (last argument of previous command) but today I discovered a new trick, the :h modifier. This takes a word and removes file file name, here I am using it in action.

buckett@oucs-matthewb:~ $ sudo vi /opt/tivoli/tsm/client/ba/bin/incl.excl.SAVE 
buckett@oucs-matthewb:~ $ cd !$:h
cd /opt/tivoli/tsm/client/ba/bin
buckett@oucs-matthewb:/opt/tivoli/tsm/client/ba/bin $ mv incl.excl.SAVE incl.excl
mv: cannot move `incl.excl.SAVE' to `incl.excl': Permission denied
buckett@oucs-matthewb:/opt/tivoli/tsm/client/ba/bin $ sudo !!
sudo mv incl.excl.SAVE incl.excl

Now I just have to remember it for more than a few days for it to seep into my normal usage of the shell.

Sakai Startup Logs

2007-05-29T08:53:00.000Z

Often after I've started up Sakai I'll tail(1) the logs so I can see how the startup is going, however by default Sakai spits out quite a few informational messages and it is easy to miss the important stuff so now I tend to watch the logs with the following command:

tail -f logs/catalina.out | grep -v INFO

which doesn't display all the INFO messages but still shows me anything serious, and all the internal tomcat messages about how the startup is progressing. I know I could change the logging (log4j) but this is a simple trick which still allows me to look back at the info messages to check things.

Installing Grouper Quickstart

2007-05-25T13:00:00.000Z

Here are a few of my notes on the Internet2 Grouper project. I'm just attempting to get the quickstart up and running to get more of a feel for the application. First of all I grabbed a copy of Grouper 1.2.0RC2 quickstart and started following the instructions. After unpacking the archive and unpacking a clean copy of tomcat I copied across the tomcat-users.xml as I didn't have any existing users I wanted to keep.

After a quick scan of the grouper-ui/build.properties file I attempted to build the UI with ant ui and selected option 6 (war) however this fell over with an ugly ant error. Looking at the build.xml it seemed to be failing on the <propertyfile> task. A quick google and it seems this is an optional task, from previous experience I knew that the nice Ubuntu people split off the optional ant tasks into a seperate packages and sure enough that was missing on my system. An apt-get later and I had the ant build successfully running.

I copies the newly created WAR into Tomcat and started it up. 14 seconds later I could access the Grouper UI. After a delay I realised that all I had was a redirect to populateIndex.do and a blank page. Then I remembered a mention at the start of the document saying I needed an HSQL database to be running. As it turns out this is actually reasonably easy, just ant db at the toplevel of the quickstart. Then a trip back to the webserver and yey, a working grouper installation. Now time for a little play.

Upgrading Sakai to 2.4

2007-05-22T11:17:00.000Z

Ok, I've just upgraded our test sakai instance to the just released 2.4.0. Here are the notes of how I did it, this isn't a recommended pattern:

Get source checkout from svn of 2.4.0
Set maven.repo.remote in master/build.properties
Check build works with maven bld
Get patches used on 2.3.1 release
Apply patches to login-tool and providers and fix rejections
maven cln bld
maven sakai:deploy-zip
Copy sakai.tar.gz to server
Stop existing tomcat and move out of the way
Expand a clean tomcat 5.5.23
Copy across conf/server.xml
Copy across sakai folder from 2.3.1
Copy across mysql connector from 2.3.1
Upgrade the schema
Copy across /portal redirection
Start up tomcat

Of course it didn't go this well, and it took just over a couple of hours to get it all working. We need to patch login-tool for WebAuth authentication and the providers to configure the LDAP provider of user account information, but otherwise it is a fairly standard sakai install.

Sakai maven 1 & 2 problems

2007-05-21T12:51:00.000Z

I had a checkout of sakai trunk and had a look at using maven 2 to build and deploy it. After a little experimentation I changed back to maven 1. Then I did an svn update and rebuilt with maven 1 and after deploy I started to see errors at the tomcat startup, after allot of checking I finally realised that I had the maven 1 and maven 2 artifacts deloyed into the same tomcat instance as as they have different versions by default they both existed in parallel, which means I had both sakai-site-api-dev.jar (from maven 1) and sakai-site-api-M2.jar (from maven 2).

Maybe the version of maven 2 artifacts should be dev by default, rather than M2, to match maven 1? What I'm not yet sure of is why you would ever want more than copy of a particular JAR in a classloader, so maybe the version shouldn't be used in the filename when the deployment is made. Although in production it is useful to have the versions for working out what versions of a file you have.

Google Apps at Oxford

2007-04-26T11:01:00.001Z

Here in OUCS we recently had a presentation by Barry Cornelius about Google Apps. Barry had setup Google Apps to use the Oxford Single Signon solution WebAuth to allow members of Oxford University to login to Google Apps on a domain, in his example he used his own domain but it could easily be something like docs.ox.ac.uk. This meant that anyone with an Oxford Account could use Google Apps without having to create another account.

Now when you signup for Google Apps you are asked to provide a domain onto which these apps will run and Google then allows you to provide branding of that domain. Google has the option to provide lots of applications (Word Processing, Spreadsheets, Calendaring, etc) and here I am talking about providing about everything except GMail which is a much more complicated issue.

My personal thinking is that OUCS should provide this service to its users and view it in a similar way to how it will provide a Shibboleth service. We provide the authentication, but the actual application is run and managed by someone else and the users should be aware of that. The possible problem with running Google Apps on an Oxford domain (docs.ox.ac.uk) is that it looks like an Oxford endorsed service and that if you have issues with it you should visit the Oxford helpdesk. However if Oxford could use the domain oxford.google.co.uk (or similar) then I believe it is much clearer that the service is provided by Google and Oxford is just providing the integration with SSO.

For a little effort we have the ability to make users lives easier in that they only need to sign on to WebAuth and they automatically get access to Google Apps. As Google already provides this service free to individuals there is nothing to stop any member of Oxford University just visiting Google and signing up for an account.

This also shows members of the University how SSO makes lives easier and how OUCS is looking to embrace new tools. But I think that we need to be clear that OUCS isn't endorsing the use of Google Apps and isn't supporting it (Google is). And why stop at Google Apps, if other companies provide similar authentication integration service why not integrate with them?

Sakai LDAP Provider

2007-03-30T09:08:00.000Z

Ok so I was attempting to connect Sakai up to our LDAP server to get user details and after a successful user login was seeing an error in the logs:

WARN: getUser() from LDAP directory exceptionnull (2007-03-29 17:32:34,597 http-
8443-Processor25_edu.amc.sakai.user.JLDAPDirectoryProvider)

Not the most helpful error so I edited the source so that it logged the stack trace as well:

java.lang.NullPointerException
       at com.novell.ldap.asn1.ASN1OctetString.<init>(Unknown Source)
       at com.novell.ldap.rfc2251.RfcLDAPString.<init>(Unknown Source)
       at com.novell.ldap.rfc2251.RfcAttributeDescription.<init>(Unknown Source
)
       at com.novell.ldap.rfc2251.RfcAttributeDescriptionList.<init>(Unknown So
urce)
       at com.novell.ldap.LDAPSearchRequest.<init>(Unknown Source)
       at com.novell.ldap.LDAPConnection.search(Unknown Source)
       at com.novell.ldap.LDAPConnection.search(Unknown Source)
       at edu.amc.sakai.user.JLDAPDirectoryProvider.getEntryFromDirectory(JLDAP
DirectoryProvider.java:360)
       at edu.amc.sakai.user.JLDAPDirectoryProvider.getUser(JLDAPDirectoryProvi
der.java:247)

Now we are getting a null pointer exception thrown from inside the novell LDAP library (not a good sign or very helpful). After a little bit of digging around I found the call line:

conn.search(getBasePath(), LDAPConnection.SCOPE_SUB, searchFilter, attribs, false, cons);

and it turns out (half an hour later) that one of the elements in the attribs array was set to null. This was caused by having a sakai configuration where I had removed the attribute mapping for the group memberships as I wasn't using it but it turns out that the provider was still attempting to retreive this value from the configuration and then using it. It really shouldn't be this hard to fix simple configuration problems.

I believe that this LDAP provider has had some work done on it for 2.4 and I'll check to see if this bug still exists in 2.4 and if it does send in a patch.

Are we there yet?

2007-03-29T14:27:00.000Z

buckett@oucs-matthewb:~/sakai/sakai-2.3.1/checkout $ maven cln bld 
 __  __
|  \/  |__ _Apache__ ___
| |\/| / _` \ V / -_) ' \  ~ intelligent projects ~
|_|  |_\__,_|\_/\___|_||_|  v. 1.0.2

[...snipped...]

BUILD SUCCESSFUL
Total time: 11 minutes 2 seconds
Finished at: Thu Mar 29 15:24:07 BST 2007

Miscellaneous failure No principal in keytab matches desired name

2007-03-28T12:44:00.000Z

I was getting this error in the server logs when trying to ssh into a kerberos enabled box and although my ssh client was getting a principal for the machine host/machine.ox.ac.uk@OX.AC.UK I wasn't getting logged in. Now looking on the net it seemed that it was likely that I didn't have the matching principal in the keytab file. But a klist -k /etc/krb5.keytab showed all the correct keys. The next thing I checked was that the DNS records matched host machine.ox.ac.uk and host 192.0.2.0. After running round in circles I checked /etc/hosts to find the line 192.0.2.0 machine and because my machine used the hosts file before DNS it was looking for the principal host/machine@OX.AC.UK which of course didn't exist.

This whole saga would have been much easier to debug if the error message had just included the principal that wasn't present in the keytab file. The machine was a server Ubuntu 6.10 install. Strangely it seems that my desktop Ubuntu machine has the machine name only associated with 127.0.0.1 and not the external IP, although this machine uses DHCP to get its IP.

pam_env, /etc/environment and expansion

2007-03-27T11:59:00.000Z

Rather than setting environment variables for a system in shell startup files (/etc/profile) you can use pam_env to set them (both ssh and login by default use pam_env). By default on Ubuntu this appears to be where the path is set, now to add something extra to the path I wanted to define a new value (JAVA_HOME) and then include $JAVA_HOME/bin in the path. To start with I edited /etc/environment but it seems that expansion doesn't occur in this file. However you can edit /etc/security/pam_env.conf to set the PATH and expansion does happen here. This is what I ended up adding to /etc/security/pam_env.conf :


JAVA_HOME       DEFAULT="/opt/jdk1.5.0_11"
PATH            DEFAULT="/usr/local/sbin:/usr/local/bin:/usr/sbin:\
/usr/bin:/sbin:/bin:/usr/bin/X11:/usr/games:${JAVA_HOME}/bin"

Firebug and Net View

2007-02-27T13:28:00.000Z

It seems that Firebug has a nice little bug. This caught me out when using it to monitor if some of our files were getting cached. Basically it seems that firebug doesn't notice when some files come from the cache and reports them as being downloaded again.

Microsoft, Patnets and BlueJ (Java IDE)

2007-01-29T09:58:00.000Z

Through Planet Eclipse I found a post detailing how Microsoft first stole and then patented an idea from a Educational Java IDE called BlueJ developed by Deakin University and the University of Kent. Although this may have been done unintentionally I hope this blows up into a bit of a PR disaster for them as their behavior in this matter doesn't seem very good.

ProxyUserDirectoryProvider

2007-01-12T11:59:00.000Z

As part of the work I have been doing with Sakai and Bodington I needed to allow Sakai to access the user details that are stored in Bodington. The normal way to do this would be to write a UserDirectoryProvider (UDP) that sits in the components folder. The problem is that I didn't want to put the whole of Bodington into the components classloader and tomcat was going to load Bodington anyway as a webapp. The solution was to create a proxy that could be injected with the real Bodington UDP when Bodington started up. This needed to be an API that extended the standard UDP:


public interface ProxyUserDirectoryProvider extends UserDirectoryProvider {

 UserDirectoryProvider getUdp();

 void setUdp(UserDirectoryProvider udp);

}

This ProxyUserDirectoryProvider could then be requested from the component manager at Bodington startup and setUdp() called with the full implementation of the Bodington UDP. The proxy I created is currently sitting in https://svn.oucs.ox.ac.uk/projects/vle/sakai/user-proxy/trunk/.

element.getAttribute("class")

2006-12-13T14:40:00.000Z

Why are even the simple things sometimes hard? Attempting to get a element attribute. Here is a table of what the various browsers return when trying to get the class attribute having already found an element:

var element = document.getElementById("theSpan");

in a document containing the HTML:

<span id="theSpan" class="something">

Browser	element. getAttribute( "class" );	element. getAttribute( "CLASS" );	element. getAttribute( "className" );	element. attributes[ "class" ];	element. className;
Firefox 2.0	something	something	null	[object Attr]	something
Opera 9.0	something	something	null	[object Attr]	something
Safari 1.3.2	something	something	null	undefined	something
IE 5.5 (Mac)	null	null	something	something	something
IE 7.0	null	null	something	[object]	something

Updated: Thanks to anonymous for pointing out that you can just use element.className to get the class name used and it seems to work correctly across all browsers I tested.

It seems that there is no simple way to get the class attribute across browsers. The solution seems to be to get the class attribute and if it undefined try the className attribute. Most other attributes should work fine, it is just that IE doesn't like the class attribute that throws the spanner in the works. Here is the quick testing page.

Label Based Feeds

2006-12-12T14:18:00.000Z

When I was at the Sakai conference I was asked if I wanted my blog added to Sakai Planet, the thing is that I blog about more than just Sakai and so only the posts for Sakai should end up in the planet. After a little googling I discovered you can have feeds for labels and it seems that my sakai feed works. I wonder why Google don't make this the default RSS feed when viewing all the posts for a tag?

Marketing through last.fm

2006-11-28T14:10:00.000Z

I have had an account on last.fm for a while now. I send most of music listening to it. A few days ago I got a friend invite from TheHijacker. Looking at the profile it seems this is the profile for a previous member of Dodgy called Nigel Clark who decided to send me a friend invite because I was one of the top listeners to Dodgy in a last week (25 listens). I also a day later got a private message from him saying I should visit his website. I believe he it trying to promote his new album 21st Century Man. To be honest I don't mind this sort of promotion much, as it is obvious that I will quite happily listen to Dodgy and so will probably like his new album. This sort of promotion also isn't at the moment automated and so he obviously feels it is worth his time sending me the message. However I signed up with last.fm because it allows me to find other music rather than have it pushed on me. Although I'm guessing that Nigel Clark hasn't yet reached a big enough critical mass opn last.fm to be recommended to me as other people aren't yet listening to it.

Sakai Site.equals()

2006-11-03T11:01:00.000Z

Took a little while to figure this one out but in Sakai the org.sakaiproject.site.api.Site equals() method will return true when passed a string if it matches the ID of the site. The reason they do this is so that they can save a list of site IDs and then have a collection of sites and call siteList.removeAll(stringList);. This can be seen is action in the CharonPortal where is is used to hide sites that the user isn't interested in.

Starting Rhythmbox in the Notification Area

2006-11-01T08:00:00.000Z

I normally have Rhythmbox running on my desktop to provide me with music. However normally when you start up Rhythmbox it displays its window, now as I added Rhythmbox to the list of programs to automatically startup when I login I wanted it in the notification area (tray). The trick for doing this is to start Rhythmbox with the command:


  rhythmbox-client --hide

Now if only gaim would have some way todo the same thing.

Terta & Bodington "Review"

2006-10-27T09:07:00.000Z

Michael Feldstein has written a couple of very good posts about the Bodington VLE following the Tetra annoucement. His posts aren't a "review" of Bodington but more of an exploration of how Bodington does access control which is often over simplified in other tools.

2006-10-27T08:34:00.000Z

Here at Oxford we use WebAuth for web based Single Sign On (SSO) and when looking at tools we like to see how easily they can be WebAuth enabled. Thankfully Sakai makes it all pretty easy. Sakai has the ability to support two authentication methods at the same time so we can have a login route for both Oxford users (WebAuth) and for internal Sakai users.

Previously we have used the Apache WebAuth module but this time I decided to attempt to use the Java WebAuth Filter developed as part of the SIPE project here at Oxford. One note is that the Standford WebAuth download pages have a newer version of the Java WebAuth filter than the SIPE pages. The Java WebAuth implementation only works with Java 1.5.

The first thing is to configure Kerberos on the machine so that it points to the correct servers, on my Linux box at Oxford this means having a configuration file called /etc/krb5.conf containing:

[libdefaults]


default_realm = OX.AC.UK





[realms]


OX.AC.UK = {


kdc = kdc0.ox.ac.uk


kdc = kdc2.ox.ac.uk


kdc = kdc1.ox.ac.uk


admin_server = kdc-admin.ox.ac.uk


}





[domain_realm]


.ox.ac.uk = OX.AC.UK


ox.ac.uk = OX.AC.UK

I also needed the kerberos tools too, on Ubuntu these come as part of the krb5-user package (sudo apt-get install krb5-user). Once my machine was running with Kerboros I needed to visit the systems development team and get a kerberos principal, this involved saying hello to the nice people upstairs and then typing a password for my new principal (buckett/itss). This principal then had rights over my webauth principal (webauth/oucs-matthewb.oucs.ox.ac.uk) so that I could key a keytab for my webauth principal. I get a keytab with the commands:

buckett@oucs-matthewb:~ $ kadmin -p buckett/itss


Authenticating as principal buckett/itss with password.


Password for buckett/itss@OX.AC.UK:


kadmin: ktadd -k /home/buckett/.webauth.keytab
webauth/oucs-matthewb.oucs.ox.ac.uk


Entry for principal webauth/oucs-matthewb.oucs.ox.ac.uk with kvno 4,
encryption type Triple DES cbc mode with HMAC/sha1 added to keytab
WRFILE:/home/buckett/.webauth.keytab.


Entry for principal webauth/oucs-matthewb.oucs.ox.ac.uk with kvno 4,
encryption type DES cbc mode with CRC-32 added to keytab
WRFILE:/home/buckett/.webauth.keytab.

This gives me a keytab file that allows the WebAuth filter to authenticate with the Kerberos server without having to ask me for a password every time I start it up. I also need to create a keyring file.

touch /home/buckett/.webauth.keyring

and set the permissions on both of the files to be as restrictive as possible (chmod 400 ~/.webauth.*).

The the sakai login tool needs to be changed to include the WebAuth filter.
This means dropping all the JARs from the WebAuth distribution (

bcprov-jdk15-132.jar,
commons-httpclient-3.0.jar, commons-logging-api-1.0.4.jar,
commons-codec-1.3.jar, commons-logging-1.0.4.jar, webauth-java-1.2.jar

) into the WEB-INF/lib folder.
The web.xml then needs some extra sections added (in bold).

  <description>Sakai 2 sample tools:
login</description>





  <!-- Webauth Filter Start -->


  <filter>


   <filter-name>Webauth
Filter</filter-name>


  
<filter-class>uk.ac.ox.webauth.Filter</filter-class>


   <init-param>


    
<param-name>WebAuthDebug</param-name>


    
<param-value>true</param-value>


   </init-param>


   <init-param>


      
<param-name>WebAuthServicePrincipal</param-name>


      
<param-value>webauth/oucs-matthewb.oucs.ox.ac.uk</param-value>


   </init-param>


   <init-param>


      
<param-name>WebAuthKeytab</param-name>


      
<param-value>/home/buckett/.webauth.keytab</param-value>


   </init-param>


   <init-param>


      
<param-name>WebAuthWebKdcPrincipal</param-name>


      
<param-value>service/webkdc@OX.AC.UK</param-value>


   </init-param>


   <init-param>


      
<param-name>WebAuthWebKdcURL</param-name>


      
<param-value>https://webauth.ox.ac.uk:8443/webkdc-service/</param-value>


   </init-param>


   <init-param>


      
<param-name>WebAuthLoginURL</param-name>


      
<param-value>https://webauth.ox.ac.uk/login</param-value>


   </init-param>


   <init-param>


      
<param-name>WebAuthKeyring</param-name>


      
<param-value>/home/buckett/.webauth.keyring</param-value>


   </init-param>


   <init-param>


      
<param-name>AutoAddKeys</param-name>


      
<param-value>true</param-value>


   </init-param>


   <init-param>


      
<param-name>AutoRemoveKeys</param-name>


      
<param-value>true</param-value>


   </init-param>


   <init-param>


      
<param-name>WebAuthExtraRedirect</param-name>


      
<param-value>true</param-value>


   </init-param>


 </filter>


 <!-- Webauth filter end -->





   <filter>


      
<filter-name>sakai.request</filter-name>


      
<filter-class>org.sakaiproject.util.RequestFilter</filter-class>


   </filter>





   <filter>


      
<filter-name>sakai.request.container</filter-name>


      
<filter-class>org.sakaiproject.util.RequestFilter</filter-class>


     
<init-param>


          
<param-name>tool.placement</param-name>


          
<param-value>true</param-value>


      
</init-param>


      
<init-param>


          
<param-name>remote.user</param-name>


          
<param-value>false</param-value>


      
</init-param>


   </filter>





   <filter-mapping>


      
<filter-name>sakai.request</filter-name>


      
<servlet-name>sakai.login</servlet-name>


      
<dispatcher>REQUEST</dispatcher>


      
<dispatcher>FORWARD</dispatcher>


      
<dispatcher>INCLUDE</dispatcher>


   </filter-mapping>





   <filter-mapping>


      
<filter-name>sakai.request.container</filter-name>


      
<servlet-name>sakai.login.container</servlet-name>


      
<dispatcher>REQUEST</dispatcher>


   </filter-mapping>


 





   <!-- Webauth Filter Mapping Start -->


   <filter-mapping>


       <filter-name>Webauth Filter</filter-name>


       <servlet-name>sakai.login.container</servlet-name>


   </filter-mapping>


   <!-- Webauth Filter Mapping End -->





   <servlet>


      
<servlet-name>sakai.login</servlet-name>


      
<servlet-class>org.sakaiproject.login.tool.LoginTool</servlet-class>


      
<init-param>


          
<param-name>container</param-name>


          
<param-value>/sakai-login-tool/container</param-value>


      
</init-param>


      
<load-on-startup>1</load-on-startup>


   </servlet>

You will probably need to change the WebAuth filter configuration to point to the correct keyring/keytab files for you local installation.

The just edit your sakai.properties, making sure these are set.

# LOGIN/LOGOUT





# to include the user id and password for login on the gateway site


top.login=false





# to let the container handle login or not (set to true for
single-signon type setups, false for just internal login)


container.login=true





xlogin.enabled=true


xlogin.text=Guests

Now if you startup Sakai it should provide you with two login buttons in the top right of the portal. One that uses WebAuth and one that uses the internal Sakai authentication.

Sakai and Log4J

2006-10-25T14:54:00.000Z

In my digging around with Sakai I came across the fact that the Sakai logging JARs are deployed to tomcat/common, the extra JARs are deployed there are sakai-util-log-dev.jar and log4j-1.2.8.jar. Placing these files in the common classloader means that they are available to both the deployed applications as well as the servlet container (Tomcat) itself. Now often log4j along with commons-logging JARs are placed into the common classloader so that Tomcat (5.5) will use log4j for its own internal logging. If other log4j JARs don't exist in any of the other classloaders all logging will go through the same log4j class and configuration. Applications are still free to provide their own log4j implmenetation along with configuration which should be insulated from the container log4j. This is because the common classloader is checked last (after the webapp and shared ones). Sakai would only need to deploy its logging code to common if it wanted to control the logging of the container as well as it's own logging. Some Sakai log messages may end up in the container logs if any of the Sakai tools log against the servlet context (getServletContext().log(String)). I don't think there is any harm to Sakai by placing the JARs in shared as all that happens is you can't control the container logging. To get Sakai logging working correctly there would also need to have commons-logging in the common classloader, so I'm not convinced that this setup works to control the container logging. As a side note Tomcat ships with a commons-logging-api but this doesn't provide the full logging framework, just enough to get going. Helpful web documents include:

Sakai Classloaders

2006-10-20T13:12:00.000Z

I am beginning todo a little bit of work with Sakai and was trying to deal with uploaded files from a web form when inside a tool. Now Sakai uses commons-filupload to handle uploaded files and parses them automatically and adds the results back to the request as attributes. I was then trying to access this attribute in my tool with a line:


FileItem fileItem = (FileItem)request.getAttribute("file");

This supprisingly was giving me a ClassCastException complaining that it was unable to cast a DefaultFileItem to a FileItem despite the fact that DefaultFileItem implements the FileItem interface. After a little head scratching Alexis suggested classloader issues we found the problem. Sakai has the parsing of the uploaded files in one classloader(the portal, getting commons-fileupload from shared) which then hands control off to the tool in another classloader (the tool, done by dispatching the request across servlet contexts). The problem was that both classloaders had a copy of the commons-fileupload jar and so when the FileItem class what loaded it wasn't in the same classloader as the DefaultFileItem and so couldn't be cast. Removing the copy of commons-fileupload from the tool fixed the problem. This issue would have been a little easier to debug if when the ClassCastException occured they provided the ID of the classloader that the two classes had come from in the exception message.

Globalsign and Java (again)

2006-10-20T13:08:00.000Z

It seems that I keep having to contact servers with Globalsign certificates and have Java throw a wobbily (java.security.cert.CertPathValidatorException). There are a whole load of globalsign certificates under http://secure.globalsign.net/cacert/ and this time I needed the Server.cacert one. Now my JDK has two extra Globalsign root certificates installed...

Plusnet LLU Disconnection

2006-10-05T21:45:00.000Z

Well it seems that Plusnet have finally sorted out the disconnects on the LLU platform as I have been connected for almost 16 hours without a disconnect. The whole issue has been badly dealt with but hey. In side news my exchange has been ADSL2+ enabled as my router now syncs at: Bandwidth (Up/Down) [kbps/kbps]: 619 / 17,509 In speedtests I don't get anywhere near 17mbs but this is probably due to still having 802.11b (11mb) wireless clients. I should really attach a PC directly to the router and see how it does.

Plusnet Disconnects Continue

2006-09-02T20:53:00.000Z

Well I am still getting disconnected every so often from Plusnet, this has now been going on for more than a week and becomming beyond a joke. For example today I have already been disconnected 18 times. Maybe its time to look for a new ISP.

Plusnet LLU (Tiscalli)

2006-09-01T15:16:00.000Z

I currently have my ADSL providied by plusnet and they currently have a problem with thier LLU provider (Tiscalli) in that authentication is failing, one useful bit of information in this posting is the test username for LLU lines (testing@dslconnect.co.uk) as the standard BT username (bt_test@startup_domain) doesn't work. In other plusnet news my LLU unbundled line is currently getting random disconnection every few hours which is very annoying when having Skype calls. Generally I am very unimpressed with the LLU line, but am not sure I want the hassle of trying to switch back to a BT one.

Blogger Beta

2006-08-31T07:48:00.000Z

When I logged into Blogger this morning they offered me the option of migrating to the new Blogger Beta and so I accepted. So far everything seems ok although my old template (Rounders 4) was very slighly broken (image didn't fill the header) so I have switched to a new minimalist theme. Everything else should be pretty much the same.

jmap and heap dumps on 1.4.2

2006-08-22T14:08:00.000Z

At work we have what might be a memory leak on our production server (it is running the Sun JDK 1.4.2 Update 12 on a Linux server), now it seemed that we might be able to use jmap which has been backported to 1.4.2 to get a heap dump. When I tried this on my desktop I kept getting the error:


buckett@oucs-matthewb:~ $ jmap
Exception in thread "main" java.lang.NoClassDefFoundError: sun/jvm/hotspot/tools/JMap

Now initially I though that I had a problem with JAVA_HOMEs and PATHs. But it turns out that Sun only backported jmap for Solaris (Windows is no luckier), but continue to ship everyone the binary just tempt them. The other option is -Xrunhprof but as we didn't start the JVM orginally with this option there doesn't seem to be a way to get a heap dump without restarting the service. gcore could give us a 1.5Gb core file for the process but there doesn't seem to be a nice way to process this under 1.4. Maybe it is time to switch to 1.5?

xserver-xorg-core upgrade broken

2006-08-22T08:58:00.000Z

This morning I installed a new version of xserver-xorg-core on my Ubuntu desktop and so did my colleage Alexis, he then restarted his computer only to find that X11 (the Windowing System) would not start and was giving an error of:


(EE) No devices detected.

Fatal server error:
no screens found

There is a bug report in the Ubuntu bug database about this: https://launchpad.net/distros/ubuntu/+source/xorg-server/+bug/57153 The easiest way to fix this is to download the previous version of the package and install it. The packages are in http://gb.archive.ubuntu.com/ubuntu/pool/main/x/xorg-server/ and I have a tinyurl to the i386 binary http://tinyurl.com/hshot, version 1.0.2-0ubuntu10.1 as 1.0.2-0ubuntu10.3 is broken. Here is how to downgrade, the stuff I type is in bold:


Ubuntu 6.06.1 LTS oucs-matthewb tty1

oucs-matthewb login: buckett
Password:
Last login: Tue Aug 22 10:00:09 2006 on pts/0
Linux oucs-matthewb 2.6.15-26-686 #1 SMP PREEMPT Thu Aug 3 03:13:28 UTC 2006 i686 GNU/Linux

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
No mail.
1 failure since last login.
Last was Tue 22 Aug 2006 10:01:12 BST on pts/0.
buckett@oucs-matthewb:~ $ cd /tmp
buckett@oucs-matthewb:/tmp $ wget http://tinyurl.com/hshot
--10:01:34--  http://tinyurl.com/hshot
          => `hshot'
Resolving tinyurl.com... 85.255.210.131, 195.66.135.131
Connecting to tinyurl.com|85.255.210.131|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://gb.archive.ubuntu.com/ubuntu/pool/main/x/xorg-server/xserver-xorg-core_1.0.2-0ubuntu10.1_i386.deb [following]
--10:01:34--  http://gb.archive.ubuntu.com/ubuntu/pool/main/x/xorg-server/xserver-xorg-core_1.0.2-0ubuntu10.1_i386.deb
          => `xserver-xorg-core_1.0.2-0ubuntu10.1_i386.deb'
Resolving gb.archive.ubuntu.com... 82.211.81.182
Connecting to gb.archive.ubuntu.com|82.211.81.182|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3,529,260 (3.4M) [application/x-debian-package]

100%[====================================>] 3,529,260      9.78M/s

10:01:34 (9.78 MB/s) - `xserver-xorg-core_1.0.2-0ubuntu10.1_i386.deb' saved [3529260/3529260]

buckett@oucs-matthewb:/tmp $ sudo dpkg -i xserver-xorg-core_1.0.2-0ubuntu10.1_i386.deb
dpkg - warning: downgrading xserver-xorg-core from 1.0.2-0ubuntu10.3 to 1.0.2-0ubuntu10.1.
(Reading database ... 107659 files and directories currently installed.)
Preparing to replace xserver-xorg-core 1:1.0.2-0ubuntu10.3 (using xserver-xorg-core_1.0.2-0ubuntu10.1_i386.deb) ...
Unpacking replacement xserver-xorg-core ...
Setting up xserver-xorg-core (1.0.2-0ubuntu10.1) ...

Maybe this might help someone, you will continue to be prompted to upgrade to the broken version but hopefuly Ubuntu will soon produce a fixed version and you should be able to skip the broken version. The way to check what version you are uprgading to is to use the Update Manager (in Administration menu).

Speedtouch and NAT Loopback

2006-08-07T19:06:00.000Z

My Speedtouch 780WL has an option called NAT Loopback that allows you to access the external IP address from inside your home network. This is useful because if you run a server inside your network you can use the same IP address (and so hostname) to address the server both at home and when you are outside. To enabled NAT Loopback login to the command line (through telnet) and enter:


ip config natloopback=enabled
saveall

Then it should all work (maybe after a reboot). The only gotcha is that it doesn't seem to loopback ICMP packets so you won't be able to ping the external IP.

Permissions in a Tree

2006-08-04T14:34:00.000Z

One of our users of WebLearn here at Oxford pointed out one of the current problems we have with the permission based model Bodington uses. If for example you have a course that contains a couple of pigeon holes (drop boxes). To allow students to use the pigeon holes you need to grant them upload rights. Now rather than granting the students upload rights to each pigeon hole it would be useful if you could just grant the permission once. The obvious solution is to have the pigeon holes inherit permissions from the containing course and then grant the permissions on the course. This way you only have to manage the permissions in one place which reduces the administration and chances of mistakes being make. The problem is that upload permissions in the course allow students to upload content to the course container which you probably don't want. This is what happens when you reuse permission in different tools for slightly different things. The ways around this are either to have a special permission for pigeon hole submission which doesn't mean anything in the course container or the have the idea of roles and a student role means diffrent things in different locations. As for which solution we will move towards I'm not sure.

Gem of CSS

2006-08-01T10:34:00.000Z

I found this little gem of a CSS definition the other day: .red{ color:red; } which just made me shudder. At least the formatting is good.

Reloading log4j.properties

2006-08-01T10:19:00.000Z

Bright-Green.com: How To Re-read log4j.properties is a nice short post on how to re-read a log4j properties file. On our java project we were calling PropertyConfigurator.configure(filename); but never calling LogManager.resetConfiguration(); which meant if you removed a logging property from the configuration and reloaded it it carried on logging. It is a shame that the log4j APIs are so badly documented. LogManager being an example.

Busy Weekend

2006-07-31T22:05:00.000Z

Had a busy weekend, on Saturday we went to Gemma and James's Wedding which was magical and then visited old neighbours Beth, Daniel and Benji on the way home. Along the way we took a few photos. I need another weekend now.

Enabling Ping on Speedtouch WL780

2006-07-31T21:52:00.000Z

By default you can't ping the speedtouch WL780 from the internet, I'm not paranoid about security so I wanted to enable it. Todo this telnet to the router and login. The execute the command: service system ifadd name=PING_RESPONDER group=wan This should enabled ICMP from the internet which then means you can use a service such as l8nc to monitor your internet connection. I currently have l8nc graphing my connection.

MaxDSL and Poor Router

2006-07-27T21:35:00.000Z

Recently we had been getting allot of disconnects on our ADSL line with the router (Safecom SWAMRU-54108) failing to reconnect and requiring a reboot or even being powered off for a short while. After trying to fix it unsuccessfully and with it being out of warranty I decided to get a new router. I opted for a SpeedTouch 780WL from DSL Source which is an ADSL router, wireless access point and voice over IP gateway all in one. They are currently selling it for 75 pounds with free delivery and it arrived the next day. After a very easy configuration it was connecting at 8,157kbs down and 636kbs up and has been rock stable for the past few days. This is much better than the old router which was only managing to sync at about 3,600kbs down. I don't know if my old router had developed a fault recently or was just a bad design. However the old router was very poor in comparison to the new router, it didn't have any polish and things never worked quite as they should.

Nintendo DS Lite (White)

2006-07-12T18:26:00.000Z

I ended up trading in my PS2 for a white Nintendo DS Lite. Although there isn't the breadth of games there are some great ones:

MarioKart DS - Having played MarioKart 64 as a student I felt right at home and this has to be one of the best games for the DS. Friend Code: 257771 280248
New Super Mario Bros. - Never played the orginal so this is all a little new, but very enjoyable.
Advanced Wars Dual Strike - A great turn based strategy game, although the characters are a little annoying the gameplay is excelent and well balanced.
Tony Hawk's American Sk8land - Having enjoyed Tony Hawks on the PS2 I picked this one up, although it is probably the worst game I own, although still good. Friend Code: 068792 703458
Metroid Prime Hunters - A great 1st person shooter which makes very good use of the touchscreen. I'm not very good at it but it is still very enjoyable. Friend Code: 2534 7629 8211

I'm really impressed with the DS lite, it looks good, is nice and small, has some great games and the battery life is excellent.

Spring MVC Forms

2006-06-26T09:58:00.000Z

Some notes about Spring MVC and forms. In simple situations when the HTML forms maps well onto the domain model validation can be performed directly on the domain model and the domain model can be used as the command class. This reduces the ammount of code that needs to be written and also means that validators can be used across multiple controllers. The problem is that this only works so long as the form maps well to the existing model. In more complicated situations you will need a seperate command class to handle the processing of the form submission and will then want to validate that so that you can easily map errors in validation back to the form fields. If you map the command class onto the domain model and then validate, discovering which errors relate to which parts of the form becomes tricky. The problem is that then you may end up having duplicate validation code for multiple controllers. When mapping a text input box <input type="text" name="number"> you have several options as to how the mapping is performed.

Don't attempt any parsing of the field when mapping the field to the command class and leave it as at String. This then means that your controller has to perform the conversion to an integer and generate the apropriate error message. This complicates your controller with code that could be performed better elsewhere.
Have it mapped to a primative int. This means that no matter what the input in the form as long as the conversion and validation succeeds you will always get a number back from the command object even is there wasn't one present in the form, and this is the problem, you can't tell when the user left the field empty.
Have it mapped to an Integer. This has the advantage that you can now tell if the user left the field blank as you will get back null. You can only do this though if you change to use a custom property editor as the default is to attempt to parse the empty string which fails and raises an error. I do this in my controller: protected void initBinder(HttpServletRequest request, ServletRequestDataBinder binder) { // Map numbers to Integer but allow them to be empty. binder.registerCustomEditor(Integer.class, null, new CustomNumberEditor(Integer.class, true)); }

DefaultMessageCodesResolver


typeMismatch.java.lang.Integer=Cannot convert to a number.

Locking Issues in Bodington

2006-06-26T08:47:00.000Z

Currently most of the locking that is performed in Bodington is done using synchronized blocks in the Java code. Examples of this can be found in org.bodington.server.resources.ResoureTree and org.bodington.server.resources.UploadedFileManager. This works reasonably well except when you want to scale your application across multiple JVMs as the locking only applies to one JVM. The two ways around this are either to use explicit locks in the database or to code optimistic locks in your application. As deploying Bodington across multiple JVMs isn't important at the moment we are continuing to develop using the existing pattern of synchronized blocks. This is important to me because in my quota implementation I need to lock sections of the tree while I calculate the current quota usage. While I am calculating the usage I don't want that section of the tree to change. As long as the locking is ok the quota calculations should stay in sync with the usage.

Uploaded File Quotas

2006-06-19T11:06:00.000Z

The implementation of uploaded file quotas is progressing reasonably well in WebLearn. They are now at a state where they seem to work reasonably well and I think the locking should prevent errors occurring when two people are uploading files at the same time. Implementing quotas on uploaded files was reasonably easy as there is one API through which all uploaded files are created/deleted. Next I have to implement quotas for resources which will be much harder as at the moment there isn't a standard API for creating resources although ResourceTree is a starting point. The user importer is now in a reasonable state for next years students and I am just waiting for there to be a larger dataset and then will finish off the testing/debugging. Hopefully there shouldn't be too many bugs in it as most of the logic for group creation was taken from a previous imported and that was working reasonably well.

The Observer | Business | Telecoms pray for time when the Skype finally falls in

2006-06-19T09:12:00.000Z

The Observer | Business | Telecoms pray for time when the Skype finally falls in talks about how the telecoms companies are worried about Skype and mentions Oxford University as an example of an institution that no longer bans Skype. The reason for this is that in order to use Skype at Oxford you have to configure it so that the peer-to-peer (P2P) part of the client can be filtered by the University firewall preventing "leaching" of bandwidth. We have a page documenting Oxfords Skype setup. It seems that the news never has the full story.

Phone Line Repair and MaxDSL Speed

2006-06-12T20:55:00.000Z

Last wednesday evening our phone line went dead along with our ADSL service. So the next day we called our provider and asked them to test to line and after a little nagging they got on to BT and had the line tested on Friday, they say there is a fault on the line and an engineer needs to be called. They said an engineer would be out to us on Wednesday of the following week. I was not impressed. Anyway it turns out that a BT engineer appears around lunch time on Sunday and after a while fixes the phone line. After he's left I reconnect the ADSL and find that it isn't working, thankfully he gave us his mobile and so we drop him a call. He says that it will probably need another engineer to look at it tomorrow as nobody who knows about the ADSL stuff is about on a Sunday. Monday morning I get up and discover that the ADSL is working, great. However looking on the router it is syncing at 3584 Kbps/640 Kbps. As I mentioned in previous post I was getting allot more (it settle down to around 5500Kbs/448Kbs), so although I now have working internet access again I have lost 2Mbps in the process. I suspect that there isn't anything I can do about this as the line works fine, just not as well as it used to.

Building Tomcat 5.5 from SVN problem

2006-06-12T16:47:00.000Z

I was attempting to build Tomcat 5.5 from SVN using the instructions from tomcat pages, however I was getting an error of: /home/buckett/src/tomcat/build.xml:49: The following error occurred while executing this line: /home/buckett/src/tomcat/build/build.xml:791: The following error occurred while executing this line: /home/buckett/src/tomcat/container/webapps/docs/build.xml:88: java.lang.ClassNotFoundException: org.apache.tools.ant.taskdefs.optional.TraXLiaison after a little head scratching I discovered that I had installed ant as an Ubuntu package and I was missing the ant-optional package which provided this extra functionality. After intalling this additional package everything went smoothly.

Bodington Database Objects and User Importer

2006-06-12T09:10:00.000Z

I am currently working on some new user import code for Oxfords installation of Bodington (WebLearn) which takes a feed of about 35000 users and creates accounts and groups for them. Due to the large number of users involved and the small number of changes that are made each day it is important to try and keep the import running fast. At the moment the bottleneck seems to be the database which eats up most of the CPU during the import, a quick look at a process listing shows PostgreSQL using 80% or the CPU and Java using 10%. Looking at the statement logging it seemed that for each user it was always updating the user in the database even if nothing had changed. Update statements are expensive for a database compared to select statements so cutting down the number of updates seemed an obvious way to improve the performance. Every object that is stored in the database by Bodington extends org.bodington.database.PersistentObject which provides the skeleton for database objects One feature of PersistentObject is that it keeps track of wether an object has unsaved changes or not. This is done through calling the setUnsaved() method which is done by most setters of the subclasseds. Eg: public void setName(String name) { this.name = name; setUnsaved(); } Now this means that even if the name is set to the same value the object is flagged as being unsaved. One option would have been to have add checking to the userimporter so that it only updateded the name of a user if it was different to the current value: if (!user.getName().equals(newName)) user.setName(newName); However there is no reason why other sections of Bodington shouldn't make use of this checking and it is also a database issue so I moved this code into the setter and ended up with: public void setName( String name ) { if (name == null) throw new IllegalArgumentException("Name cannot be null"); if (name.equals(this.name) return; this.name = name; setUnsaved(); } The checking to see that if name == null is because the database has a NOT NULL constraint and this also prevents a NullPointerException from being thrown if setName(null) is called. I added this checking to org.bodington.server.realm.Users and org.bodington.server.realm.Aliases and a quick test of updating 1000 users gave the results:

Always Saving: 1minute 3 seconds.
Saving Changes: 30 seconds.

which isn't a bad improvement in performance. One side note is that PersistentObject.save() always saves the object and I could have changed it so that it only saved changed objects but I'm not sure that all the setters of the subclasses call setUnsaved() so I am doing my checking in the importer. The group mapping has come wholesale from some old user import code with a few changes and just needs a little tweaking and it should be nearly ready for some proper testing.

Glassfish and Invalid URL Pattern

2006-06-05T16:56:00.000Z

I was attempting to deploy WebLearn to the Glassfish application server and was gettting the error "Invalid URL Pattern" for /site/* which is used in various places in our web.xml configuration file. It turns out Glassfish doesn't like extra whitespace in it's url-patterns and we had things like: <url-pattern> /site/* </url-pattern> when they need to be: <url-pattern>/site/*</url-pattern> I've got the web.xml fixed on WebLearn and will push it across to Bodington at shortly.

Ubuntu Dapper Drake Upgrade

2006-06-05T14:18:00.000Z

With Ubuntu Dapper Drake being released on the 1st of June I decided to upgrade my work desktop machine today. I just edited my sources.list file and then ran apt-get update; apt-get dist-upgrade. Most of the upgrade went fine, but here are a few things that other people may fin helpful were. nVidia Monitor Detection It seems that the new nVidia driver was autodetecting that my monitor could do 76Hz vertical refresh rate at 1280x1024. Not while it can display this it didn't seem to be able to adjust the screen enough and as a result I was unable to see the two left most columns of pixels. I attempted to fix the vertical refresh rate in my configuration file ( /etc/X11/xorg.conf ) by setting the line VerticalRefresh 50-60 but every time I started X up it would continue to use 76Hz which was outside the range. After quite a bit of Googling I discovered that the nvidia driver attempts to autodetect the monitor settings by default and the way to turn it off is to add an option to the file: Section "Device" Identifier "NVIDIA Corporation NV18 [GeForce4 MX 440 AGP 8x]" Driver "nvidia" Option "UseEdidFreqs" "false" EndSection Pinned Rhythmbox Somehow rhythmbox was pinned in the Synaptic Package Manager, this didn't seem to be affecting apt-get which was upgrading it without any problems. The problem turned out to be that synaptic can have its own list of pinned applications which are stored in /usr/lib/synaptic/preferences. As this only entry in this file was the pinning of rhythmbox I deleted the file and everything worked fine. Packaged Java Ubuntu now packages the Sun JDK which can be install just like any other package. I had a problem though in that all my alternatives (/etc/alternatives) were pointing to the wrong versions. I found a good post on galternatives that allows you to easily edit them.

User Import and Bodington Developers Meeting

2006-06-05T09:33:00.000Z

The user import code is coming along, last week I added parsing of colleges, courses and departments and some tests against small bits of data. The status handling has also been sorted out so rather than working with strings they are converted to an enumeration. Most of last week was taken up by a Bodington developers meeting in Leeds. The meeting went over two days with the first day being for discussion of short term Bodington developments and the second day for the longer term Bodington vision. Out of the first day we decided to release Bodington 2.8 shortly, most of the code is already in CVS for Bodington 2.8 so once a few extras have been added and some tested has been performed 2.8 should go out the door. Other things discussed included a catchup on what developments have been happening at the various sites that run Bodington and the JISC projects that are using it. The second day we discussed longer term direction including Portals, SOA, SOAP, REST. At the end of the day I just want to build a better product.

I owe PayPal money

2006-06-03T19:52:00.000Z

I owe PayPal 56 pence as a result of a currency conversion charge from a long time ago I think. Today I recieved an email nagging me to correct my balance and decided to attempt todo something about it, so I login to my account and attempt to add funds but it refuses to allow it just returning me to the form highlighting the fields that I have filled in sucessfully. So I looked at the email that PayPal had sent me and noticed it ended with: We appreciate your cooperation and invite you to reply to this email with any questions. Yours sincerely, PayPal Please do not reply to this email. This mailbox is not monitored and you will not receive a response...... So I replied to the mail and we'll see how we get on. They do accept cheques to fix my balance but I signed up with PayPal to get away from cheques....

Quotas in WebLearn and User Import

2006-05-30T09:10:00.000Z

Last week I started working on supporting quotas in WebLearn. This involved adding support to BuildingSession to allow the getting and setting of the resource and file quota. So now for any resource you can check and set the quota. I also built some pages to allow users to view the quotas and for sysadmins to set them. Next I need to add support to resource creation so that it always checks there is enough quota before allowing he creation, but this is where it gets a little more messy as there isn't on API for resource creation. I had to stop work on the quotas to start work on the new user import tool as we had the new user import files available. At the moment we get our user details from LDAP but due to a redesign in the data contained in LDAP we need to go direct to the source. The files we get will be processed once a day and any new users created and any new group memberships added. It would be nice if we supported the removal of group memberships this time but I think that will have to wait until the end. So far I have got basic parsing of the users going and need to work on the parsing of the colleges, departments and courses. Then I can start doing user creation and then the mapping of attributes to group memberships.

Trip to Wales

2006-05-28T22:27:00.000Z

Had our first proper short holiday away with Wilf last weekend when we went for an extended weekend to Wales (Tenby). It was really just an escape from the South East for us and so we didn't have a bit list of things todo. Just visited castles, beaches and galleries. Took a few photos and bought a print of a painting to brighten up our house. Traveling was fine with Wilf with the main travelling taking about 4 hours and him sleeping through most of it (waking for a feed half way through each time). As with everything else now things take so long, previously we would have crammed 4 outings into a day and now with Wilf we struggle to get 2, but I don't mind one bit. My brother commented "Your becoming old" when I described our holiday to him, but you can't really go windsurfing, climbing and kite flying with a 10 week old baby, I'll just have to wait a few years. Wilf didn't seem to mind being in a different place and I think we prevented him from throwing up on anyone else's furniture.

MyWebLearn Bookmarking

2006-05-23T10:33:00.000Z

Some more progress on the bookmarking functionality of MyWebLearn has been made which will allow users to bookmark any WebLearn page, adding it to their own space. This is very similar to del.icio.us but is tightly integrated into the WebLearn interface the access controls. Further work needs doing on the internal links inside Bodington to improve the way the links work and are displayed but the basics are there. The reason for developing this is that people often don't want to navigate the whole tree to find the resources that they regulaly use and often don't have a fixed PC so local bookmarks dont work. Hopefully this help solve some of those problems.

Everyones Blogging

2006-05-14T21:09:00.000Z

Well it seems even my other half Anna has started a blog although I think the subject matter is going to be slightly different to mine.

Trends in VLE/LMS

2006-05-13T12:24:00.000Z

Using Google Trends you can get an idea about how interested general Google users are in different VLE products. A graph of several popular VLEs shows that while Blackboard leads the race the product that is gaing ground in all of this is Moodle which is just about to overtake WebCT. The sharp dips for all the products are due to everyone stopping work around Christmas and I believe spike each year is just as the new intake of students happens and maybe it is also at that time that people start looking at other products for their own institutions. Interestingly although Sakai has a reasonable ammount of news coverage (Moodle has no news) general users don't seem all that interested in it. Maybe this is due to the type of users/teachers that Moodle attracts compared to the people that are interested in Sakai. Sakai is aimed squarely at large instituinal deployment compared to small departmental systems that Moodle supports well. Despite the annountment that Blackboard and WebCT (12th October 2005) will merge there doesn't seem to be much change to the trends, does this show that the majority of people searching are users of the systems rather than administrators? The VLE that I work (Bodingont) on doesn't really feature at all but maybe it will get better?

MyWebLearn Progress

2006-05-12T16:32:00.000Z

I am currently working on a development here at Oxford called MyWebLearn which is looking to allow all users to have their own space in the VLE to create content. As WebLearn (Bodington) has very flexible access controls they can also control access to the resource they create. Currently I have written the signup process and the basics are working. The next task is to tidy up all the rough edges.

Spring and Bodington

2006-05-08T10:32:00.000Z

Currently Bodington doesn't use the typical MVC model and each web request maps directly to a controlling template. This makes handling things like errors in forms and redirects difficult. To try and improve matters I have attempted to integrate the Spring MVC framework into Bodington. This is a quick summary of the current state of play.

When a request comes in the normal request handling of loading the resource and permission checking takes place.
Extra URL handling has been added that means that any file beginning with bs_spring is handed off to the Spring servlet.
The Spring servlet has a Bodington specific mapper that looks for the resource in the request and then attempts for find a bean matching /facilityname/page.
Control then passes to the bean which is a normal Spring controller.
The controller then returns its model and view which are mapped to a JSP.

One problem is that the Spring code depends on a facility name, maybe it should depend on the resource directly? We also have alot of duplication of the facilityname all over the code, it's in the spring configuration for the bean names and it's in the view names that are returned by the controllers. Maybe we could use the package names to infer the facilityname? However this depends on having utility code that is called as you can't have the view resolved do the work as it only knows about the view string and the locale.

Plusnet and 8MB ADSL (MaxDSL)

2006-05-02T21:47:00.000Z

I a couple of weeks ago I received an email from Plusnet saying that if I wanted to I could ask to be upgraded to 8MB ADSL as soon as possible. Looking forward to the extra upstream bandwidth I sent a note to Plusnet support confirming that I would like to be upgraded as soon as possible. I got a confirmation back saying it had been received and I would be contacted when the order was placed. Well when I got home to my computer today I found that the internet connection was down so I logged into the ADSL router and was pleasantly suppressed to see the ADSL line sync at DownStream: 6112 Kbps, UpStream: 448 Kbps. This was a nice increase from 2Mbs and all done for free. Looking at the logs I can see that the router has been attempting several speeds and according to Plusnet it could take up to 10 days to settle down. Speeds it has tried so far since about 17:00 today are: * DownStream: 6400 Kbps, UpStream: 448 Kbps * DownStream: 6560 Kbps, UpStream: 448 Kbps * DownStream: 6560 Kbps, UpStream: 448 Kbps * DownStream: 6496 Kbps, UpStream: 448 Kbps * DownStream: 6560 Kbps, UpStream: 448 Kbps * DownStream: 6784 Kbps, UpStream: 448 Kbps * DownStream: 6656 Kbps, UpStream: 448 Kbps * DownStream: 6560 Kbps, UpStream: 448 Kbps * DownStream: 6112 Kbps, UpStream: 448 Kbps * DownStream: 6208 Kbps, UpStream: 448 Kbps * DownStream: 6016 Kbps, UpStream: 448 Kbps While I have been composing this post it has tried the last two on this list. As this adjustment is going on I am seeing a little more packet loss on the line but I'm guessing this is normal. If your looking at this post today you can see a graph of the disconnects I've been experiencing. This graph shows data from the current day will probably be useless shortly... Although the line speed seems to have increased using Plusnets speed tester I am only getting the expected data rate for a 1Mbs line but this may be due to the fact that BT don't adjust the ATM line restrictions as quickly, but it should happen in the next few days. The only thing that would have been useful is if Plusnet would have sent the emails warning me that this would be happening as they said they would.

Create Patch (Eclipse 3.2)

2006-05-02T13:51:00.000Z

I have been using Eclipse 3.2M6 for a little while now and I finding it a nice evolution of the 3.1 product. One simple feature that I had been wishing for was the ability to create patches but select individual files which the patch should contain. In 3.1 you could only create a patch for a folder (and all the files under it) or just one file. This was never a big problem as I could just drop back to the command prompt and do it manually, but it just means I can spend more of my life in Eclipse. This first appeared in Milestone 3 under the title of Improved Patch Support. I end up using patch support quite a bit as I work on both WebLearn and Bodington which are similar codebase but stored in separate CVS repositories so using patches is the best way to move changes across. Although looking in the new and noteworthy the ability to record refactoring should help to allow changes to easily be pushed across.

Bodington DB Layer and Empty Subclasses

2006-04-28T10:08:00.000Z

The Bodington VLE has its own DB layer that allows Java objects to be stored and loaded from the database without having to touch JDBC and SQL. If you have an object that can already be persisted in the database layer and want to subclass it without adding any instance fields (so you don't need an extra table) you can store both the orginal class and the subclass in the same table. The reason this can work is that for every object stored in the database the DB layer keeps track (through the objects table) which Java type it should map to. The SQL to allow this subclassed object to be stored is something like:


INSERT INTO classes (type, super_type, db_name, table_name, java_class)
 VALUES(109, 10, null, null, 'org.bodington.server.resources.NewSubclass')

The type being the new ID for this type. And the super_type being the persisted class that you are subclassing. Using this we could have a new resource class for every type of resource and then we can use polymophism, rather than having switch statements and constants.

GNER and Wifi

2006-04-27T15:11:00.000Z

GNER Wifi service is designed to provide a Wifi service on trains. On Monday I was travelling back from a meeting on the 16:05 from Leeds to London Kings Cross (in the quiet coach) and I opened up my iBook, straight away it detected the network and I associated with it. However no matter what I did it refused to give me a DHCP lease. So all I can report is don't rely on Wifi being available as it may be broken.

Save Parliament! Stop the Legislative and Regulatory Reform Bill

2006-04-11T09:48:00.000Z

I hope the Save Parliament campain gains a little more media attention again. If you're in the UK please spend 5 minutes reading the site.

GeoURL

2006-04-10T13:16:00.000Z

Thanks to Joe for pointing me to GeoURL which basically says how you should put location metadata into your web pages. After a quick edit of my blog templates you'll now know that I live in High Wycombe, it would be nice if there were some support from blogger for this so that when you set you location in your profile it could exposes the geourl metadata as well, but hey.

Compiling for Older JVMs

2006-04-05T19:09:00.000Z

I've been doing some development on WebLearn from home today and had to perform build to be deployed on our test server. Now normally I develop on the same software as we run on the production machines but I just happened to only have the Java 1.5 SDK installed on my home machine. Not really thinking I set the target to 1.4 so that the outputted class files were compatible with the 1.4 JVM that we use on our testing and production machines. This is done through our ant build file with something like:


<javac srcdir="src" destdir="build"  source="1.4" target="1.4"/>

The cruital bit I thought was the target attibute. However after shipping the build to the sysadmin guys they come back saying they're getting a stack trace when deploying the application:


java.lang.NoSuchMethodError:
java.lang.StringBuffer.insert (ILjava/lang/CharSequence;) Ljava/lang/StringBuffer;
       org.bodington.servlet.BuildingServlet.init (Unknown Source)

which shows that although the class files could be run by the 1.4 JVM they used API calls that only exist in the 1.5 JVM. Now at first this is comfusing because the code compiles fine with a 1.4 compiler and classes so what has changed as all the 1.4 API calls should exist in 1.5 (apart from the deprecated ones that were removed). The problem comes from some code similar to this:


StringBuffer str1 = new StringBuffer("123456");
StringBuffer str2 = new StringBuffer("  ");

str1.insert(2, str2);

Under the 1.4 StringBuffer API there isn't the method insert(int offset, StringBuffer str) so the compiler uses the method insert(int offset, Object str). In the 1.5 API again there isn't a perfect match but there is a better one than the Object one, it is the method insert(int offset, CharSequence str) and as StringBuffer implements CharSequence this closer API call is used. We could have fixed the code by changing it to:


str1.insert(2, (Object)str2);

but this just fixes one call and there maybe others that we didn't find. The real solution to this is to compile against the 1.4 classes when you are using the 1.5 compiler. Todo this you can use something like the following:


<javac srcdir="src" destdir="build"  source="1.4" target="1.4" bootclasspath="/home/buckett/j2sdk1.4.2_11/jre/lib/rt.jar" extdirs=""/>

But of course for this you need to have downloaded the 1.4 SDK so I might as well have used the 1.4 compiler. However it is very likely that the 1.5 compiler contains better optimisations so WebLearn should run faster. However in future I think I'll just use the 1.4 compiler as it isn't worth the hassel. Sun does have a document on this this topic but labels it Cross Compiling. Why both the Sun and the ant documentation don't mention that when using the target option you probably also want to use the bootclasspath option. It seems obvious looking back but it wasn't at the time.

Thunderbird Address Book and LDAP

2006-04-03T11:37:00.000Z

Here at Oxford University we have an LDAP server that contains details of all the staff and students, although it isn't yet a production service it is available to computing services (OUCS) staff members. At first I tried configuring Thunderbird to access the LDAP server but found that searches took far too long so I just forgot about it and carried on using Thunderbird without LDAP support. Then one of my colleagues Paul Trafford was trying to switch from Eudora to Thunderbird and was complaining that Thunderbird was taking ages to perform searches so I had another look into it. The reason that the searches were taking ages is that it was performing queries on attributes that don't have good indexes and so these required full scans taking a long time. Unfortunately finding and changing the queries used by Thunderbird isn't an easy task. After a little Googling the first thing to come up was a page explaining how LDAP attributes map onto Address Book properties and although this was useful it didn't explain how to change the search that was begin run. A few people have blogged this and someone else's experience with Thunderbird and LDAP was helpful although it didn't go all the way to solving my problems. After a little more Googling I found another page that have details of hidden preferences for LDAP searches. Now using this and the search that Paul had used in Eudora I was able to recreate a useful and fast LDAP search. Here are the lines for pref.js in your Thunderbird profile directory. You should only edit this file when Thunderbird is not running as it gets saved when Thunderbird closes.


user_pref("ldap_2.autoComplete.directoryServer", "ldap_2.servers.Oxford");
user_pref("ldap_2.servers.Oxford.description", "Oxford");
user_pref("ldap_2.servers.Oxford.filename", "abook-1.mab");
// Oxford uses an objectClass of oucsOrganizationPerson to show a person.
user_pref("ldap_2.servers.Oxford.uri", "ldap://ldap.ox.ac.uk:389/ou=people,dc=ox,dc=ac,dc=uk??sub?(objectClass=oucsOrganizationalPerson)");
// Our LDAP stores some other useful data that should be mapped to attributes so we can search on it.
user_pref("ldap_2.servers.default.attrmap.Custom1", "universityBarcode");
user_pref("ldap_2.servers.default.attrmap.Custom2", "oucsUsername");
user_pref("ldap_2.servers.default.attrmap.Custom3", "uniqueIdentifier");
user_pref("ldap_2.servers.default.attrmap.Department", "oucsDivision");
// Although we have a displayname field it isn't indexed so search and diplay common name.
user_pref("ldap_2.servers.default.attrmap.DisplayName", "cn");
user_pref("ldap_2.servers.default.attrmap.PreferMailFormat", "preferredMail");
// Search on Oxford stuff
user_pref("mail.addr_book.quicksearchquery.format", "?(or(Custom1,=,@V)(Custom2,=,@V)(Custom3,=,@V)(DisplayName,c,@V))");

Adding this to your config means that you can search on the common name, barcode, uniqueID and username and it is fast. It doesn't mean that you will be able to use the LDAP server when composing mails though as it seems that some of these preferences aren't used by that code and so the search still looks at the default attributes and as a result very slow here. This has only been tested with Thunderbird 1.5

New Eclipse Milestone (3.2M6)

2006-04-02T21:41:00.000Z

A couple of days ago Eclipse 3.2M6 was released with some more nice features. It's getting to the point that I might switch to the new version (currently using the stable build, 3.1.2) for my main version as long as it is reasonably stable. I'll probably have a go next week and see how it goes.

Skipping WebAuth Confirmation Screen

2006-03-29T13:57:00.000Z

At my employer Oxford University we use Stanford WebAuth for web based Single Sign On (SSO). One of my particular problems with the current setup at Oxford is that for each service you sign into you have to confirm that you wish to signon using your credentials to the site. At Oxford this is known as the green tick page. Thanks to the wonderfull Firefox plugin called greasemonkey I can have some JavaScript run when I visit the green tick page to automatically follow the link. The result is a greasemonkey script that means I don't have to click on the link. I've put the script up for anyone else who might find it useful. WebAuth Tick Jumper to use it you should already have greasemonkey installed.

SSH Tunnels and Multiple Copies of Firefox

2006-03-29T10:20:00.000Z

I'm currently working from home and need todo some testing of a website that is only accessable from some restricted IPs inside my work offices. To get around this I use and ssh tunnel: ssh -D 1080 -N machine.ox.ac.uk &

-D 1080 sets up a SOCKS server listen on localhost, port 1080
-N mean no command gets run when the connection is established so it can be run in the background safely (SSH 2)

Once I have connected and authenticate I want to start another copy of Firefox but keep my existing instance of Firefox running for normal browsing. Firefox by default tries to limit you to only one copy being running at any one time and so just starting Firefox up you end up with just another window for your existing instance. There are some other blog posts about running multiple copies of Firefox but they don't have a nice one liner. Mine is: ( export MOZ_NO_REMOTE=1; firefox -profilemanager ) & The brackets mean that a subshell is started so that the MOZ_NO_REMOTE only affects the single firefox instance. This then brings up the profile manager as when running multiple copies of Firefox you have to use a different profile for each one. As I am wanting to test a remote website through the ssh connection I create a new profile called SOCKS and start it. Then once the extra copy of Firefox is running I edit the connection settings (Edit -> Preferences) and specify Manual Proxy Settings with the SOCKS host set to localhost and the port set to 1080. After clicking Ok all my browsing from this copy of Firefox goes through the ssh connection so I can test the IP restricted website without having to run a VPN. The only thing left todo is to install a theme in the SOCKS profile so it is more obvious which profile I am using.

Wilf J Buckett

2006-03-15T21:23:00.000Z

Anna gave brith to Wilf J Buckett on Friday the 10th of March 2006, he weighed 6lb 12oz and mother and baby are doing well. It doesn't feel all that different being a father I just see to be getting less sleep. The one change I have noticed is that now rather than watching TV we spend time watching Wilf. I just hope I can remember that not everyone want to hear about all the things Wilf has been doing that day.

Clipboard and the Web

2006-03-08T13:42:00.000Z

Through a post by Phil Wilson I found a really beautiful idea about how the clipboard should work on the web. It show a rather slick way of allowing data to be transfered between web applications and also between desktop applications and the web. To understand it best just watch the screencasts. This sort of feature is one that needs consitent implementation across applications to ease the learning curve for users but with features like this the future does look good for applications being delivered through the web.

e-Literate: Stephen Downes Missed the Point

2006-02-09T22:29:00.000Z

e-Literate: Stephen Downes Missed the Point has some very good discussion on VLE/LMS architecture/development/direction. It seems to have some parrallels with the questions I have regarding the project Tetra. And what do I think? I'm very skeptical about this idea that we are close to a framework where we can take tools from different places and plug them together. If the tools are written against an existing product that defines the layers and APIs then everything works (like plugins for Firefox), but we are trying to get Firefox plugins to work in IE. For me the biggest problem in the VLE world is dealing with authentication and authorization which is often ignored by other web applications as they don't have materials that they need to restrict access to. Eg, Amazon doesn't have some books that only some people can see, so it's web APIs can be simple. In the VLE most of the current material is restricted and authorization checks have to be peformed.