Cache Sizing
Today was devoted to figuring out the optimum cache size for Zope. I reconfigured Zope to do cache tracing ran a (too big) test case, and found out that we have a 57% cache hit rate with the default cache size of 20 megabytes. Tomorrow I plan is to find the 90% cache hit rate by binary search and then determine the difference in response time between the default cache size and it.
Posted on Wed, 28 Feb 2007 Tags: zope
Frustration
I finished the testing for large files versus many files for Zope performance and determined that a Zope system with a few large files performs somewhere between a Zope system with many small files and a Zope system with a few small files. Unfortunately, because of the variance in the results, I can't say which it is closer to. I'll have to construct a much larger Data.fs in order to tell. But for now, I'm switching my attention to cache size tuning.
Glen continues to ask about Mhonarc search performance, so I revised the logging code a bit and deployed the search engine with logging on happy. There was a small delay because of Unix permission issues.
Marty wanted a status report on the PAR project, so I gave one to him. I have to get serious about doing a project schedule, because I didn't have one to show him.
Posted on Tue, 27 Feb 2007 Tags: mhonarc zope par
More Performance Testing
I went back to the performance tests and redid them so I could be absolutely sure I was generating the correct results. I had to stop around 3:30 because someone else was using the server and I didn't want to bias the results. I'll summarize the results when they're all in.
Posted on Mon, 26 Feb 2007 Tags: zope
More Performance Testing
I created a Zope instance with a few large objects and ran some performance tests. What I found is that performance scales with the number of objects and not the size of the Data.fs.Posted on Fri, 23 Feb 2007 Tags: zope
Performance Issues
Glenn raised performance issues with the Mhonarc search. I tried the search numerous times during the day. Return times went from 6 to 14 seconds. So I could not duplicate the problem, although I am sure it's real.
I discussed the PAR project with Greg and the issues he's having with the Java code.
We discussed the Zope performance problems at the web domain meeting and decided to look at the effects of caching and number of Zope database objects on performance.
Posted on Thu, 22 Feb 2007 Tags: zope par
ExternalFile
I've got most of the ST metadata wrapper for ExtFile done. What work remains is the method to create a new instance and the conversion of the dtml methods.
Posted on Tue, 20 Feb 2007 Tags: zope
Wrapping ExtFile
I started work on wrapping ExtFile to support ST metadata.
Posted on Mon, 19 Feb 2007 Tags: zope
Come and Go
The profiling I started last night never finished. The Mhonarc slowdown is inconsistent. It was happening this morning, stopped, happened again a bit this afternoon, and then stopped again. I sent Dan a message asking if NFS keeps any log files. I added logging prints to my version of the code, but I will need permission before deploying on happy.
Posted on Fri, 16 Feb 2007 Tags: mhonarc
Profiling
The Mhonarc search engine performance problem has reared its ugly head again and I'm searching for its cause. We had the kickoff meeting on how to authenticate users on our NAS backend and discussed the pros and cons of the different options.
Posted on Thu, 15 Feb 2007 Tags: mhonarc
Tmeouts
Mike told me that he couldn't run my solution to rename the center web site to itsd because every time he tried to run it the Zope process started thrashing. So I looked at the Zope source code to see if it could be modified so the renaming could be done in parts. The code runs recursively and would have to be rewritten to run iteratively for a while and then suspend itself. The obvious way to do this is to use a queue and write it out to disk when the process suspends.
Posted on Tue, 13 Feb 2007 Tags: zope