Thought you all might be interested in this thread from the Evergreen discussion list.  I pulled out the end of Dan's message in my reply.  Take a moment to read the whole message, it is about the transaction time of the Evergreen client and how it can be improved.  Might be a good place for this group to focus some energy.  

I really like the idea of using our resources to get some kind of independent benchmarking system in place too.

Your thoughts?

Lori

----  My post to the Evergreen list today:

Dan Scott said:  I've advocated several times in the past that the library world really should invest some resources in creating independent benchmarks for settling these sorts of questions (and giving library system developers measurable results to work with) without FUD-slinging. One can dream.

Lori Ayre says:  Dan, we may be able to do this as part of the King County IMLS grant project.  I'll put it in the bag of possible to do's over the next three years.
Lori

On Mon, Feb 15, 2010 at 8:41 AM, Dan Scott <dan@coffeecode.net> wrote:
On Sun, 2010-02-14 at 23:13 -0800, Jess Cook wrote:
> Hi All,
> I've heard that since Evergreen is http based, that typical
> circulation transaction
> speeds are slow, an average of 3 seconds from the time a staffer wands
> the barcode to the time a response arrives.  This seems slow.

Yes, this sounds slow to me too.

> Could any of the larger libraries out there, particularly those with
> their
> servers in another location, comment of this slowness issue?  Is it
> simply FUD?

FUD? Nah, that would never happen! If the source is an ILS vendor I know
of who runs their client-server communication over telnet, you might
want to ask them what happens when the network drops a packet or two
(answer: their staff client crashes completely and you need to restart
it from the Start menu...)

I've run a few tests with our 1.6.0.1 test server (2 million items, 16GB
RAM server, staff client running on Windows XP inside a virtual machine)
and a script-based checkout is taking less than two seconds, and a
check-in is taking less than a second. The test server is located about
450 km away from me, and I'm doing this from home, so it's far from a
dedicated high-speed network.

You can get some information from running the staff client in debug mode
by adding "-console" to the end of the staff client command in the
shortcut
(http://www.open-ils.org/dokuwiki/doku.php?id=scratchpad:debug_console).
That will show you lines like:

request 13: duration = 330 ms

which gives you an idea of how long each command takes, as well as all
of the data being sent back and forth between server and client for each
action. It doesn't seem to time each request, however, and I don't think
it groups related requests together into one big timer, which would be
helpful for answering questions like these. It would be nice to have an
instrumented mode for the staff client that summarizes the totals and
averages over a complete session to get a better picture of performance
over time.

>  Is it true that every time a barcode is sent to the base computer
> and back that there is a lot of "overhead" and that this causes
> delays?

There isn't much network overhead in the requests themselves, which is
what the HTTP comment was about. For a checkout, for example, you're
basically passing an authentication token, the user ID, and the barcode
to the server. You could argue that the server sends back too much
information (you probably don't need the subjects or table of contents
for a given book in the checkout results summary row), but that's part
of the reason you can display so many different columns in the results
summary table.

With script-based checkouts, there is some overhead in that the request
pulls objects from the database, those objects get passed into a
JavaScript environment and the scripts return a checkout response
(success/fail, duration, fine type, etc), and then the action gets
applied to the database and the result objects are returned to the staff
client. As of Evergreen 1.6, it is possible to run all circulation
within the database, avoiding the database-to-JavaScript-to-database
dance, which should speed things up. We're not running in-database
circulation yet in Conifer, but it's one of our goals - and we're
looking forward to comparing circulation times accordingly.

I've advocated several times in the past that the library world really
should invest some resources in creating independent benchmarks for
settling these sorts of questions (and giving library system developers
measurable results to work with) without FUD-slinging. One can dream.