Problem Solved

The problem with dropXSL turned out to be a misconfiguration of the server. Mike took time out from his vacation to fix it. I incorporated the code I wrote into the STWebSearch product and started testing. The code is complete except for the following:

The dtml to display the results isn't written yet.
The code that selects the internal or external database is wrong. (It copies the existing code, which is also wrong.)
There's no logic to handle localized searches.

So next week I'll test the existing code and then add the stuff mentioned above.

Problems

I converted the code I had written to a Python class with the same interface as the existing search class. When I started testing the code I ran into a problem: the parameter which creates xml output (dropXSL) doesn't seem to be working. I'll have to figure out why tomorrow.

More XML Parsing

I did a significant rewrite of the xml parsing code, making it recursive. Before I was "special casing" some tags to work around the need for recursion. Recursion always makes my brain hurt, so I took most of the day to get the code working right. Then I started shoe horning the functions I wrote into the existing search class. I'll test it outside of Zope as much as possible and then rewrite the dtml to display the results and test it inside Zope.

XML Parsing

I added code to parse the XML output from Texis. I've been testing it and some things work, others don't, but there's nothing that can't be solved with a clear head and a new day. Space Shuttle launched successfully.

On to Texis

I wrapped up my work on TWiki by backing out the changes I had made to the look and feel and then notifying Claus. I started my next assignment, interfacing the new Texis search engine to Zope. I reviewed the existing code, looked up some info on parsing xml in Python and started coding. So far I've coded two simple functions: buildquery and sendquery, which do what they say. The harder function is next: parsedoc. The code lives in ~/ubik/python/zope/texisquery.py.

TWiki Work

I spent the day getting TWiki configured. The first problem was setting up authorization and I seem to have that working. The second problem was customizing the look and feel and I seem to have made a hash of that. I'll clean up Monday and turn the site over to Claus.

TWiki Debugging

I spent most of the day debugging the TWiki installation. The first problem is that I had failed to update the vhost file. After I fixed that, I found the TWiki pages didn't render properly. The source of that problem was permissions on the subdirectories of data weren't set properly. After fixing that problem and making a few minor tweaks, I called it a day. Tomorrow I wanted to add a rewrite rul and finish the configuration.

I also did more work on webspider.pl. I fixed a bug in double checking against production and another bug in using relative links. I added code to do authorization. Tomorrow I will test it after finishing the TWiki configuration.

TWiki

I wasn't satisfied with the results webspider.pl was giving so I did a couple of tweaks and debugging runs. It seems to work, but i'm not sure I trust the results. Then I continued the configuration of TWiki. I completed all the steps outlined in the installation instructions, but the twiki is not found. Sounds like a problem with the vhosts file. I'll check tomorrow.

Stop and Go

Webspider.pl continued to crash with an out of memory error so I converted the queue of pending links so it is also a dbm file. I refactored the code so both the visited and queue hashes share the same open and close routines. I started the revised script and will check the results tomorrow. I started installing and configuring TWiki. The TWiki tar file was corrupted, so after several tries I downloaded the zip file, which installed properly. I edites the sage.stsci.edu vhosts file and started editing the several TWiki configuration files. I had a discussion with Claus about what he wanted, which basically seems to be a plain install and they will take it from there.

Cleaning Up

I revised the section of the absolute/relative links document to reflect Greg's criticisms. I also finished the revision of webspider.pl. The hash of visited pages is now a DBM file, which I hope will solve the out of memory problem. Tomorrow, TWiki for sure.