Login

Repoze Blog

2007-12-07 00:00:00-05:00

Highway Jam: Faster Application Restart

While driving back from North Carolina, Chris McDonough and I spent the time talking about ways to make WSGI developers more productive.

We discussed, and then discarded, the idea of trying to fix Python's reload story: it seems just too painful and error-prone (like the refresh.txt story in Zope). Instead, we realized that making restarting the application go really fast would be a big win: this observation was highlighted especially by the restart-inducde delays during the demos we did at UNc and CPCC.

In no particular order, here are the ideas we came up with:

Python-based

  • Move the current profiling support out of Zope-the-application and make it middleware, so we can measure the various bits more cleanly. Chris says Ian Bicking has some support already, but only for one request at a time.
  • When doing a "fast" restart, signal first an event which allows listeners to dump any global state to a cache on the filesystem; then, on restarting, arrange to signal another event which allows listeners to reload that state, short-circuiting the whole application initialization phase.
  • In development mode, maybe we can trigger the fast restart whenever any of [getattr(x, '__file__', None) for x in sys.modules.values()] changes. Can we somehow arrange for the triggering request to be retried after the restart? Does it matter?
    Hmm, it looks like this is what paste --reloadr does already. Cool. Score that: NIH, 0; Ian Rocks!, 1

Zope-related Changes

  • Maybe find a way to add all parsed ZCML files, to the "watched files" list too, so that changing ZCML will automagically restart the application.
  • Finish making ZCML parsing side-effect free, so that we can cache the action list (to a ".pyc"-like file) across application runs (a "pickle file").
  • Removing ZCML-parse-time side-effects implies removing the "eager validation / resolution" of dotted names. Instead, the parser would create "deferreds" representing the target objects, which would then be resolved / replaced at the point of use. We would presumably be able to serialize the deferreds in the "pickle file" mentioned above.
  • One particularly interesting bit of global state is the set of pickles cached by each ZODB connection. Repopulating those caches after a restart is one reason that Plone, for instance, takes so long to render the homepage after a restart. So, check into whether it is feasible to dump and reload the pickles maintained by each ZODB database connection to disk during a "fast restart".
  • As an alternative to dumping / reloading the ZODB Connection pickles, investigate whether multiple connections (or even appserver processes) might be able to shart them via a setup using memcached
  • Look into whether we can replace the ZEO disk cache with memcahed.
  • Measure "slow" product initializations, and find ways to speed or bypass them on restart (e.g., PlacelessTranslationService).

posted at: 00:00 | permalink