Nicer URLs?

After quite a bit of head bashing nicer URLs in Sakai are coming along, here the site has nice name (/working_lu) and the page has a nice name (siteinfo). It's still at the very early stages but its getting there.

SVN Tip for Sakai

When your working on a checkout of Sakai and editing files across multiple projects it can be handy to see what you have changed, svn status is normally the way todo this but due to the number of external definitions in Sakai it produces allot of output which can't easily be suppressed. Piping this through egrep and tr can clean things up.

svn status | egrep -v "^(Perform|X)" | tr -s "\n"

Gives a nice concise output as though those externals didn't exist. Aliasing this to something like svnst makes it easy to get back to.

NoMachine Client in Fullscreen

I have recently discovered the machine that is NoMachine and am using it to connect to my work computer from outside the office. So far it seems to work really well. The only thing is that I like to run it in fullscreen mode and was struggling to find out how to return to my local desktop. Eventually I found this post about NX which provided all the solutions. In summary the options are:

Clicking on the top right pixel

Minimises the NX Client and returns you to your client desktop.

CTRL-ALT-M

Same as above

CTRL-ALT-T

Brings up a popup asking if I want to terminate the session or just disconnect from it.

Now if only they had a client of Mac OSX which supported fullscreen mode.

Sakai Sessions and UsageSession

Having just been looking at these for a little while I thought I'd document them a little. The basic setup is that the Sakai Session is your session with the application and is very similar to a HttpSession. Your Session is setup for each request by the Sakai RequestFilter. The UsageSession on the other hand is your logged usage of the application amd is mainly there so that you can enumerate all the active UsageSessions. The UsageSession is actually store in the Session and is mainly used for presence.

However just to confuse things a little the UsageSessionService (manages UsageSessions) has some very important methods, login() and logout(). These setup and destroy your Session and UsageSession at the same time. Login tools in Sakai should call UsageSession.login() to setup the Sakai session with the authenticated user details.

Formatting with Styles

There is a very clear post from darcusblog about how having a good UI can encourage people to use styles in their document. Unsurprisingly the company with the good UI is Apple, but it shows that a good UI can guide users into good practices.

More Kerberos Puzzlers

Ok so when trying to kerberos enable another machine for ssh connections things were almost working (getting offered kerberos for authentication by sshd) but connections were always failing with the following error message in the logs:

Jul 17 12:33:46 machine sshd[5020]: debug1: An invalid name was supplied\nHostn
ame cannot be canonicalized\n

Now after looking through pages of google it seemed to be that there was something wrong with the hostname but looking on other machines they had very similar setups and were working correctly. The reason for this message was that I think that kerberos was using the hostname for kerberos and couldn't find an entry for the hostname in DNS. The reason it couldn't find a hostname was the there wasn't a search entry in /etc/resolv.conf specifying the domain to look in, adding search oucs.ox.ac.uk to /etc/resolv.conf fixed the problem and everything worked fine.

Hopefully this post might help someone else from scratching hole in their head. But it would have been much easier to debug if the invalid hostname was logged in the message.

svn_load_dirs.pl and Ubuntu

I need to do some vendor drops with Subversion and the documentation seems to point to svn_load_dirs.pl as being the best place to start. So I install subversion and subversion-tools onto my Ubuntu system only to find that I don't seem to have a svn_load_dirs.pl installed. After a little bit of digging it seems that svn_load_dirs.pl was removed from the Debian package (which the Ubuntu one is based) due to licensing concerns although this isn't very clear as the changelog doesn't explain that this was one of the affected files. So it looks like I need to get myself a copy of the source distribution.

Ethiopian Food in Amsterdam

Had some really nice food in Amsterdam last night, after a drink we ended up at an Ethiopian Restaurant suggested by Adam. I had some Gored Gored but we all ended up sharing as it came on a big plate. Good company and good food.

Loading Resources in Sakai Components

I meant to blog about this a while ago but it has recently come up again and so I though I should write my ideas down. Ok so the problem is that you have a Sakai component which is created by the component manager and it wants to load something from inside the component, in my case it was an iBatis configuration file specified through a spring configuration file. So in my spring config I have something like:

  
<property name="configLocation">
 classpath:/sql-maps-config.xml
</property>

Now the problem is that this property is loaded through a PropertyEditor which sets the property to a ClassPathResource. In Spring the classloading policy is that resources should be loaded through the thread's classloader. When the component manager is starting up and so when my component is create this is caused by the fact that a web application has asked for it so the thread classloader points to the webapp and the resource in your component won't be found.

The solution to this is to have a way to load resources using the standard JVM Class.getResourceAsStream() which will use the classloader used to load the current class. So I created an class:

package org.sakaiproject.util;

import org.springframework.core.io.ClassPathResource;

/**
* Resource implementation for class path resources.
* Loads out of the same classpath as this class was loaded through.
*
* Supports resolution as java.io.File if the class path
* resource resides in the file system, but not for resources in a JAR.
* Always supports resolution as URL.
*
* @author buckett
*/
public class LocalClassPathResource extends ClassPathResource {

 public LocalClassPathResource(String path) {
  super(path, LocalClassPathResource.class.getClassLoader());
 }
}

and put this in my component. I also then change the configuration in my components.xml to use the resource directly:

<property name="configLocation">
<bean class="org.sakaiproject.util.LocalClassPathResource">
 <constructor-arg type="java.lang.String">
  <value>/sql-maps-config.xml</value>
 </constructor-arg>
</bean>
</property>

Now the configuration file gets loaded without any problems. Hopefully future Sakai developments in the component manager will mean this hack isn't needed later on.

Safari 3 Beta and FCKEditor

Just saw in the news that Apple has release a Safari 3 Beta but include a Windows build, now this probably isn't going to take on IE and Firefox but does allow developers on Windows so have a better idea about how websites will render on it. Now it also allow people to see that FCKEditor nightly is working reasonably well in Safari.

Plane on Google Maps

Found a plane on Google Maps today. You can even read the writing on it showing it's an Air Canada plane.

Sakai Maven Plugin Install

Always losing the command to install the Sakai Maven Plugin, Dr Chuck has some instructions but they rely on already having configured your maven repository. The one stop command is:

maven plugin:download -DgroupId=sakaiproject -DartifactId=sakai -Dversion=2.2 -Dmaven.repo.remote=http://source.sakaiproject.org/maven

This only works with maven 1. If nothing else maybe this will give Chucks second post a little more Google karma and push it up in search results.

Custom Logging in Sakai

If you are debugging something in Sakai and want to setup a custom log configuration you can just edit your sakai.properties. Here's an example of from a sakai.properties file:

log.config.count=2
log.config.1 = DEBUG.org.sakaiproject.portal.charon.handlers.HierarchyHandler
log.config.2 = DEBUG.org.sakaiproject.portal.charon.handlers.SiteHandler

Don't forget Resource normally same as Entity

It seems originally org.sakaiproject.entity.api.Entity interface was called Resource and although most things seem to have been updated there are still some references across the codebase to Resources. An example is the org.sakaiproject.event.api.Event#getResource() method which actually returns a Entity reference.

Sakai Portal Handlers

Sakai has a reworked portal in 2.4 called the Charon Skinnable Portal and this portal has the concept of handlers which can be used to handle different requests. Most of this code is in portal/portal-impl. The PortalHandler interface has a useful base class called BasePortalHandler and then some of the usable implementations are:

• PageHandler - Just displays a page which will load the tool in an iframe without anything else by default. URL: /portal/page/placementId
• WorksiteHandler - Just displays a site with the list of tools but no inter site navigation, supports supplying a additional page reference. URL: /portal/worksite/siteId
• GalleryHandler - Display the site as well as the inter site navigation but still no logo or login box, supports supplying an additional page reference. URL: /portal/gallery/siteId
• SiteHandler - Display a normal Sakai screen with site and inter site navigation, including login/logout links.

All the handlers are setup in SkinnableCharonPortal.init() which calls through to the portal service. The list of handlers if configurable at runtime through the portal service. Just posting this as a note for future reference really.

Bash History Expansion

I've alway been quite fond of bash history expansion and in particular !! (previous command) and !$ (last argument of previous command) but today I discovered a new trick, the :h modifier. This takes a word and removes file file name, here I am using it in action.

buckett@oucs-matthewb:~ $ sudo vi /opt/tivoli/tsm/client/ba/bin/incl.excl.SAVE 
buckett@oucs-matthewb:~ $ cd !$:h
cd /opt/tivoli/tsm/client/ba/bin
buckett@oucs-matthewb:/opt/tivoli/tsm/client/ba/bin $ mv incl.excl.SAVE incl.excl
mv: cannot move `incl.excl.SAVE' to `incl.excl': Permission denied
buckett@oucs-matthewb:/opt/tivoli/tsm/client/ba/bin $ sudo !!
sudo mv incl.excl.SAVE incl.excl

Now I just have to remember it for more than a few days for it to seep into my normal usage of the shell.

Sakai Startup Logs

Often after I've started up Sakai I'll tail(1) the logs so I can see how the startup is going, however by default Sakai spits out quite a few informational messages and it is easy to miss the important stuff so now I tend to watch the logs with the following command:

tail -f logs/catalina.out | grep -v INFO

which doesn't display all the INFO messages but still shows me anything serious, and all the internal tomcat messages about how the startup is progressing. I know I could change the logging (log4j) but this is a simple trick which still allows me to look back at the info messages to check things.

Installing Grouper Quickstart

Here are a few of my notes on the Internet2 Grouper project. I'm just attempting to get the quickstart up and running to get more of a feel for the application. First of all I grabbed a copy of Grouper 1.2.0RC2 quickstart and started following the instructions. After unpacking the archive and unpacking a clean copy of tomcat I copied across the tomcat-users.xml as I didn't have any existing users I wanted to keep.

After a quick scan of the grouper-ui/build.properties file I attempted to build the UI with ant ui and selected option 6 (war) however this fell over with an ugly ant error. Looking at the build.xml it seemed to be failing on the <propertyfile> task. A quick google and it seems this is an optional task, from previous experience I knew that the nice Ubuntu people split off the optional ant tasks into a seperate packages and sure enough that was missing on my system. An apt-get later and I had the ant build successfully running.

I copies the newly created WAR into Tomcat and started it up. 14 seconds later I could access the Grouper UI. After a delay I realised that all I had was a redirect to populateIndex.do and a blank page. Then I remembered a mention at the start of the document saying I needed an HSQL database to be running. As it turns out this is actually reasonably easy, just ant db at the toplevel of the quickstart. Then a trip back to the webserver and yey, a working grouper installation. Now time for a little play.

Upgrading Sakai to 2.4

Ok, I've just upgraded our test sakai instance to the just released 2.4.0. Here are the notes of how I did it, this isn't a recommended pattern:

• Get source checkout from svn of 2.4.0
• Set maven.repo.remote in master/build.properties
• Check build works with maven bld
• Get patches used on 2.3.1 release
• Apply patches to login-tool and providers and fix rejections
• maven cln bld
• maven sakai:deploy-zip
• Copy sakai.tar.gz to server
• Stop existing tomcat and move out of the way
• Expand a clean tomcat 5.5.23
• Copy across conf/server.xml
• Copy across sakai folder from 2.3.1
• Copy across mysql connector from 2.3.1
• Upgrade the schema
• Copy across /portal redirection
• Start up tomcat

Of course it didn't go this well, and it took just over a couple of hours to get it all working. We need to patch login-tool for WebAuth authentication and the providers to configure the LDAP provider of user account information, but otherwise it is a fairly standard sakai install.

Sakai maven 1 & 2 problems

I had a checkout of sakai trunk and had a look at using maven 2 to build and deploy it. After a little experimentation I changed back to maven 1. Then I did an svn update and rebuilt with maven 1 and after deploy I started to see errors at the tomcat startup, after allot of checking I finally realised that I had the maven 1 and maven 2 artifacts deloyed into the same tomcat instance as as they have different versions by default they both existed in parallel, which means I had both sakai-site-api-dev.jar (from maven 1) and sakai-site-api-M2.jar (from maven 2).

Maybe the version of maven 2 artifacts should be dev by default, rather than M2, to match maven 1? What I'm not yet sure of is why you would ever want more than copy of a particular JAR in a classloader, so maybe the version shouldn't be used in the filename when the deployment is made. Although in production it is useful to have the versions for working out what versions of a file you have.

Google Apps at Oxford

Here in OUCS we recently had a presentation by Barry Cornelius about Google Apps. Barry had setup Google Apps to use the Oxford Single Signon solution WebAuth to allow members of Oxford University to login to Google Apps on a domain, in his example he used his own domain but it could easily be something like docs.ox.ac.uk. This meant that anyone with an Oxford Account could use Google Apps without having to create another account.

Now when you signup for Google Apps you are asked to provide a domain onto which these apps will run and Google then allows you to provide branding of that domain. Google has the option to provide lots of applications (Word Processing, Spreadsheets, Calendaring, etc) and here I am talking about providing about everything except GMail which is a much more complicated issue.

My personal thinking is that OUCS should provide this service to its users and view it in a similar way to how it will provide a Shibboleth service. We provide the authentication, but the actual application is run and managed by someone else and the users should be aware of that. The possible problem with running Google Apps on an Oxford domain (docs.ox.ac.uk) is that it looks like an Oxford endorsed service and that if you have issues with it you should visit the Oxford helpdesk. However if Oxford could use the domain oxford.google.co.uk (or similar) then I believe it is much clearer that the service is provided by Google and Oxford is just providing the integration with SSO.

For a little effort we have the ability to make users lives easier in that they only need to sign on to WebAuth and they automatically get access to Google Apps. As Google already provides this service free to individuals there is nothing to stop any member of Oxford University just visiting Google and signing up for an account.

This also shows members of the University how SSO makes lives easier and how OUCS is looking to embrace new tools. But I think that we need to be clear that OUCS isn't endorsing the use of Google Apps and isn't supporting it (Google is). And why stop at Google Apps, if other companies provide similar authentication integration service why not integrate with them?

Sakai LDAP Provider

Ok so I was attempting to connect Sakai up to our LDAP server to get user details and after a successful user login was seeing an error in the logs:

WARN: getUser() from LDAP directory exceptionnull (2007-03-29 17:32:34,597 http-
8443-Processor25_edu.amc.sakai.user.JLDAPDirectoryProvider)

Not the most helpful error so I edited the source so that it logged the stack trace as well:

java.lang.NullPointerException
       at com.novell.ldap.asn1.ASN1OctetString.<init>(Unknown Source)
       at com.novell.ldap.rfc2251.RfcLDAPString.<init>(Unknown Source)
       at com.novell.ldap.rfc2251.RfcAttributeDescription.<init>(Unknown Source
)
       at com.novell.ldap.rfc2251.RfcAttributeDescriptionList.<init>(Unknown So
urce)
       at com.novell.ldap.LDAPSearchRequest.<init>(Unknown Source)
       at com.novell.ldap.LDAPConnection.search(Unknown Source)
       at com.novell.ldap.LDAPConnection.search(Unknown Source)
       at edu.amc.sakai.user.JLDAPDirectoryProvider.getEntryFromDirectory(JLDAP
DirectoryProvider.java:360)
       at edu.amc.sakai.user.JLDAPDirectoryProvider.getUser(JLDAPDirectoryProvi
der.java:247)

Now we are getting a null pointer exception thrown from inside the novell LDAP library (not a good sign or very helpful). After a little bit of digging around I found the call line:

conn.search(getBasePath(), LDAPConnection.SCOPE_SUB, searchFilter, attribs, false, cons);

and it turns out (half an hour later) that one of the elements in the attribs array was set to null. This was caused by having a sakai configuration where I had removed the attribute mapping for the group memberships as I wasn't using it but it turns out that the provider was still attempting to retreive this value from the configuration and then using it. It really shouldn't be this hard to fix simple configuration problems.

I believe that this LDAP provider has had some work done on it for 2.4 and I'll check to see if this bug still exists in 2.4 and if it does send in a patch.

Are we there yet?

buckett@oucs-matthewb:~/sakai/sakai-2.3.1/checkout $ maven cln bld 
 __  __
|  \/  |__ _Apache__ ___
| |\/| / _` \ V / -_) ' \  ~ intelligent projects ~
|_|  |_\__,_|\_/\___|_||_|  v. 1.0.2

[...snipped...]

BUILD SUCCESSFUL
Total time: 11 minutes 2 seconds
Finished at: Thu Mar 29 15:24:07 BST 2007

Miscellaneous failure No principal in keytab matches desired name

I was getting this error in the server logs when trying to ssh into a kerberos enabled box and although my ssh client was getting a principal for the machine host/machine.ox.ac.uk@OX.AC.UK I wasn't getting logged in. Now looking on the net it seemed that it was likely that I didn't have the matching principal in the keytab file. But a klist -k /etc/krb5.keytab showed all the correct keys. The next thing I checked was that the DNS records matched host machine.ox.ac.uk and host 192.0.2.0. After running round in circles I checked /etc/hosts to find the line 192.0.2.0 machine and because my machine used the hosts file before DNS it was looking for the principal host/machine@OX.AC.UK which of course didn't exist.

This whole saga would have been much easier to debug if the error message had just included the principal that wasn't present in the keytab file. The machine was a server Ubuntu 6.10 install. Strangely it seems that my desktop Ubuntu machine has the machine name only associated with 127.0.0.1 and not the external IP, although this machine uses DHCP to get its IP.

pam_env, /etc/environment and expansion

Rather than setting environment variables for a system in shell startup files (/etc/profile) you can use pam_env to set them (both ssh and login by default use pam_env). By default on Ubuntu this appears to be where the path is set, now to add something extra to the path I wanted to define a new value (JAVA_HOME) and then include $JAVA_HOME/bin in the path. To start with I edited /etc/environment but it seems that expansion doesn't occur in this file. However you can edit /etc/security/pam_env.conf to set the PATH and expansion does happen here. This is what I ended up adding to /etc/security/pam_env.conf : JAVA_HOME DEFAULT="/opt/jdk1.5.0_11" PATH DEFAULT="/usr/local/sbin:/usr/local/bin:/usr/sbin:\ /usr/bin:/sbin:/bin:/usr/bin/X11:/usr/games:${JAVA_HOME}/bin"

Firebug and Net View

It seems that Firebug has a nice little bug. This caught me out when using it to monitor if some of our files were getting cached. Basically it seems that firebug doesn't notice when some files come from the cache and reports them as being downloaded again.

Microsoft, Patnets and BlueJ (Java IDE)

Through Planet Eclipse I found a post detailing how Microsoft first stole and then patented an idea from a Educational Java IDE called BlueJ developed by Deakin University and the University of Kent. Although this may have been done unintentionally I hope this blows up into a bit of a PR disaster for them as their behavior in this matter doesn't seem very good.

ProxyUserDirectoryProvider

As part of the work I have been doing with Sakai and Bodington I needed to allow Sakai to access the user details that are stored in Bodington. The normal way to do this would be to write a UserDirectoryProvider (UDP) that sits in the components folder. The problem is that I didn't want to put the whole of Bodington into the components classloader and tomcat was going to load Bodington anyway as a webapp. The solution was to create a proxy that could be injected with the real Bodington UDP when Bodington started up. This needed to be an API that extended the standard UDP:

public interface ProxyUserDirectoryProvider extends UserDirectoryProvider {
UserDirectoryProvider getUdp();
void setUdp(UserDirectoryProvider udp);
}


This ProxyUserDirectoryProvider could then be requested from the component manager at Bodington startup and setUdp() called with the full implementation of the Bodington UDP. The proxy I created is currently sitting in https://svn.oucs.ox.ac.uk/projects/vle/sakai/user-proxy/trunk/.