Metadata Bug Fixed

My theory about the cause of the metadata bug was correct, but it took several tries to get the right fix. I've requested that the fix be moved to test. While I was researching this bug, I noticed that the code we use in several places to recurse into subfolders only handles ST Folders and not other ST Folderish objects. So my next job will be to fix this problem.

Metadata Madness

I studies the code in STDublinCoreConfigure.py and came to the conclusion that the cause of the problem she reported is that the metadta items to be deleted are stored in two separate places: metadataMap and myMetadataMap. The current code only deletes items from myMetadataMap. So I modified the code and I am in the process of testing the fix.

Broken Metadata

In the morning I did one last change (I swear, no more changes) to the MS Word cleaning code. I changed the string replace in a loop into a regular expression search and replace in order to make the code faster. Leigh told me about a problem where metdatadata isn't being cleanly deleted from subfolders. I added some diagnostic writes and it looks like a lambda definition is causing the problem. Since I find such things impenetrable, I'm rewriting the lambda as an honest to goodness function and we'll see where we go from there.

Newsbot

I finished testing the stand alone version of the MS Word cleaner. I looked at integrating it into STObjects, but haven't done so yet. After that, I wrote a prototype version of an RSS feed generator for the Hubble News Site, which I called newsbot.

Microsoft Cleaner

I revisited my Python code to clean up Word files and added code to convert the Windpws 1252 character set into html character entities. I tested the code and it seems to be stumbling over some unicode conversions. I'll have to investigate that before adding the code to STObject.

Radio, Radio

Today I found the proep conditionals to get the radio buttons on the search form set properly both on the initial form and on the search results page.

More Search

Mike Watterson asked that the search results contain the search form. So I modified WebSearch.dtml. The result works fine, except that the radio button to select the search database is left unset in the result page. The logic to set this is tricky and I haven't figured out how to modify it to correctly handle all cases yet.

Search Revisited

The new Zope search software was installed today and unfortunately there was a bug. It wouldn't return results from a search of the Procedures Catalog. I traced the bug to a failure to convert the blank in the database name to a plus. So the problem is fixed, but it's embarrassing that we didn't catch it during testing. Bobby Edwards asked if the Category field on the LDAP management screen is necessary. The field maps to BusinessCategory in LDAP. I found that it was being used (sometimes) to hold the user's division name and told Bobby I didn't think it should be deleted now.

Twiki Patch

I patached a security hole in TWiki that was announced on their mailing list. Then I turned to converting the demoroniser script into Python so that I can insert it into STObjects. I have a working standalone version that incorporates some but not all of the demoroniser code. It also looks for the generator metatag to see if Microsoft produced the document. I'll think some more about what fixes we want to incorporate tomorrow before finishing up.

Demoroniser

Mike got a problem report where a user had saved a Microsoft web document as html and uploaded it to our web site. The page displayed badly. Surprisingly this problem only happened in Microsoft IE. I pointed him to a Perl script called demoroniser that fixes various formatting problems in Microsoft Office documents. However, running the script on the page didn't fix the problem. I looked at the script and added another subroutine that does additional fixes, including stripping out Microsoft conditional comments and Microsoft specific tags. The resulting script fixed the original problem. I was interested in this problem because I'm thinking of rewriting the script in Python and incorporating it in STObjects.