Highway Jam: Faster Application Restart
While driving back from North Carolina, Chris McDonough and I spent the time talking about ways to make WSGI developers more productive.
We discussed, and then discarded, the idea of trying to fix Python's
reload story: it seems just too painful and error-prone (like the
refresh.txt story in Zope). Instead, we realized that
making restarting the application go really fast would
be a big win: this observation was highlighted especially by the
restart-inducde delays during the demos we did at UNc and CPCC.
In no particular order, here are the ideas we came up with:
- Move the current profiling support out of Zope-the-application and make it middleware, so we can measure the various bits more cleanly. Chris says Ian Bicking has some support already, but only for one request at a time.
- When doing a "fast" restart, signal first an event which allows listeners to dump any global state to a cache on the filesystem; then, on restarting, arrange to signal another event which allows listeners to reload that state, short-circuiting the whole application initialization phase.
In development mode, maybe we can trigger the fast restart whenever
[getattr(x, '__file__', None) for x in sys.modules.values()]changes. Can we somehow arrange for the triggering request to be retried after the restart? Does it matter?
Hmm, it looks like this is what
paste --reloadrdoes already. Cool. Score that: NIH, 0; Ian Rocks!, 1
- Maybe find a way to add all parsed ZCML files, to the "watched files" list too, so that changing ZCML will automagically restart the application.
- Finish making ZCML parsing side-effect free, so that we can cache the action list (to a ".pyc"-like file) across application runs (a "pickle file").
- Removing ZCML-parse-time side-effects implies removing the "eager validation / resolution" of dotted names. Instead, the parser would create "deferreds" representing the target objects, which would then be resolved / replaced at the point of use. We would presumably be able to serialize the deferreds in the "pickle file" mentioned above.
- One particularly interesting bit of global state is the set of pickles cached by each ZODB connection. Repopulating those caches after a restart is one reason that Plone, for instance, takes so long to render the homepage after a restart. So, check into whether it is feasible to dump and reload the pickles maintained by each ZODB database connection to disk during a "fast restart".
- As an alternative to dumping / reloading the ZODB Connection pickles, investigate whether multiple connections (or even appserver processes) might be able to shart them via a setup using memcached
- Look into whether we can replace the ZEO disk cache with memcahed.
- Measure "slow" product initializations, and find ways to speed or bypass them on restart (e.g., PlacelessTranslationService).
posted at: 00:00 | permalink