The second episode (mp3) is now up. It contains ramblings about Repoze and how it relates to:

• A wrapup of the Plone Symposium in New Orleans 2008
• The current state of Zope-related Repoze software
• Info about repoze.who
• Recent Zope/Plone/Python user group talks
• Thoughts about Deliverance

Enjoy!

-Chris

Version 1.0 has optional support for middleware configuration via a config file (thanks to Tres). Being a framework, repoze.who is configuration-heavy, and it can provide a better separation of concerns and more convenience to wire it up via a configuration file than via Python code. So rather than configuring the middleware and attendant plugins via straight Python code, you can now wire who configuration up in an .ini file:

    # who.ini

    [plugin:form]
    # identification and challenge
    use = repoze.who.plugins.form:make_plugin
    login_form_qs = __do_login
    rememberer_name = cookie
    form = %(here)s/login_form.html

    [plugin:auth_tkt]
    # identification
    use = repoze.who.plugins.auth_tkt:make_plugin
    secret = s33kr1t
    cookie_name = oatmeal
    secure = False
    include_ip = False

    [plugin:basicauth]
    # identification and challenge
    use = repoze.who.plugins.basicauth:make_plugin
    realm = 'sample'

    [plugin:htpasswd]
    # authentication
    use = repoze.who.plugins.htpasswd:make_plugin
    filename = %(here)s/passwd
    check_fn = repoze.who.plugins.htpasswd:crypt_check

    [plugin:sqlusers]
    # authentication
    use = repoze.who.plugins.sql:make_authenticator_plugin
    query = "SELECT userid, password FROM users where login = %(login)s;"
    conn_factory = repoze.who.plugins.sql:make_psycopg_conn_factory
    compare_fn = repoze.who.plugins.sql:default_password_compare

    [plugin:sqlproperties]
    name = properties
    use = repoze.who.plugins.sql:make_metadata_plugin
    query = "SELECT firstname, lastname FROM users where userid = %(__userid)s;"
    filter = my.package:filter_propmd
    conn_factory = repoze.who.plugins.sql:make_psycopg_conn_factory

    [general]
    request_classifier = repoze.who.classifiers:default_request_classifier
    challenge_decider = repoze.who.classifiers:default_challenge_decider

    [identifiers]
    # plugin_name;classifier_name:.. or just plugin_name (good for any)
    plugins =
          form;browser
          auth_tkt
          basicauth

    [authenticators]
    # plugin_name;classifier_name.. or just plugin_name (good for any)
    plugins =
          htpasswd
          sqlusers

    [challengers]
    # plugin_name;classifier_name:.. or just plugin_name (good for any)
    plugins =
          form;browser
          basicauth

    [mdproviders]
    plugins =
          sqlproperties

Then you can use a constructor to create the configuration based on the .ini file, e.g.:

from repoze.who.config import WhoConfig

app = {next app in pipeline}
here = os.path.dirname(__file__)
config_file = os.path.join(here, 'who.ini')
parser = WhoConfig(here)
parser.parse(open(config_file))
middleware = PluggableAuthenticationMiddleware(app,
             parser.identifiers,
             parser.authenticators,
             parser.challengers,
             parser.mdproviders,
             parser.request_classifier,
             parser_challenge_decider,
             log_stream = sys.stdout,
             log_level = logging.DEBUG,
             )

There is a PasteScript-compatible constructor available via the egg name egg:repoze.who#config that does just this, so you can also just wire it up via a paste config file equivalently, ala:

[filter:who]
use = egg:repoze.who#config
config_file = %(here)s/etc/who.ini
log_level = debug
log_stream = stdout

You can read the documentation for more information about configuration.

- Chris

• Chris McDonough's "Intro to WSGI" talk.
• Tres Seaver's "Turplango: Playing Nice with Others" talk.
• Tres Seaver's "GenericSetup for Fun and Profit (redux)" lightning talk.

Installing Plone 2.5.5 (implies Zope 2.9.8)

Install repoze.plone 2.5.5 using Buildout

$ svn co http://svn.repoze.org/buildouts/repoze.plone/branches/2.5.5
$ cd 2.5.5
$ python2.4 bootstrap.py
$ bin/buildout
.. follow instructions here on in ..

... or install repoze.plone 2.5.5 using Virtualenv

$ virtualenv24 --no-site-packages plone255
$ cd plone255
$ bin/easy_install -i http://dist.repoze.org/plone/2.5.5/simple repoze.plone
$ bin/mkzope2instance .
$ bin/addzope2user admin admin
$ bin/paster serve etc/zope2.ini

Installing Zope 2.9.8 only (if you don't use Plone)

Install repoze.zope2 2.9.8 using Buildout

$ svn co http://svn.repoze.org/buildouts/repoze.zope2/branches/2.9
$ cd 2.9
$ python2.4 bootstrap.py
$ bin/buildout
.. follow instructions here on in ..

Install repoze.zope2 2.9.8 using Virtualenv

$ virtualenv24 --no-site-packages zope298
$ cd zope298
$ bin/easy_install -i http://dist.repoze.org/zope2/2.9/simple repoze.zope2
$ bin/mkzope2instance .
$ bin/addzope2user admin admin
$ bin/paster serve etc/zope2.ini

One good thing about doing this work is that it's becoming a little clearer how we need to lay out our distribution site (dist.repoze.org) to support multiple build revisions of the same piece of software. I haven't managed yet to clean it up entirely but I think I know how to now.

Have fun!

-C

$ bin/easy_install -i http://dist.repoze.org/plone/3.1.1/simple repoze.plone
$ bin/addzope2user {username} {password}
.. ignore deprecation warnings ..
$ bin/paster serve etc/zope2.ini

To get it via zc.buildout, check out http://svn.repoze.org/buildouts/repoze.plone/branches/3.1.1/ and run:

$ python2.4 bootstrap.py
$ bin/buildout
$ bin/supervisord
$ bin/addzope2user {username} {password}

In either case, visit the result on http://localhost:8080/manage

- C

The TurboGears trunk (which will at some point be released as TurboGears 2) is very different from the released 1.X version of TurboGears. Instead of using CherryPy as a controller dispatch mechanism as TG 1.X does; instead it uses Pylons. One of the side effects of this change is that the venerable TG 1.X authentication and authorization code known as identity doesn't work anymore under the TurboGears trunk. So the identification, authentication, and authorization features that TurboGears 1.X people have become accustomed to are largely missing for people working off the TurboGears trunk. An effort named Authority was established to provide the TG trunk with these features, and code exists in that project to that end. We've worked a bit with the folks who are creating Authority, and I hope there's some way to coalesce the two efforts into one in the future.

In the meantime, I created a sample set of repoze.who plugins and a repoze.who middleware configuration "on spec" for TG2 just for proof-of-concept. This eventually served as a template for Florent Aide, who subsequently developed a more-or-less feature complete set of repoze.who plugins and separate authorization facilities named tgrepozewho to provide functionality equivalent to TG 1.X's "identity" package for TG2 users. Florent also created a sample TG2 application named whotg that can make use of the authentication and authorization features provided by the tgrepozewho configuration package.

The fundamentals of Florent's TG2 application which makes use of repoze.who are these:

• Sqlalchemy models for users, groups, and permissions were created in the whotg package's model.__init__.py, which is represented below in its entirety. These models are used by the plugins to repoze.who that exist in the "tgrepozewho" package (shown later) which perform identification and authorization. These are loaded into a database within the demo package by doing paster setup-app development.ini within a checkout of whotg. Some sample user and group data exists in the generated SQL when you load it this way.

  import datetime
  from pylons import config
  
  from sqlalchemy import Column, MetaData, Table, Integer, Unicode, DateTime
  from sqlalchemy import ForeignKey
  
  from sqlalchemy.orm import scoped_session, sessionmaker
  from sqlalchemy.orm import mapper, relation
  
  import md5
  import sha
  
  # Global session manager.  DBSession() returns the session object
  # appropriate for the current web request.
  DBSession = scoped_session(sessionmaker(autoflush=True, transactional=True))
  
  # Global metadata. If you have multiple databases with overlapping table
  # names, you'll need a metadata for each database.
  metadata = MetaData()
  
  def init_model(engine):
      """Call me before using any of the tables or classes in the model."""
      
  groups_table = Table('tg_group', metadata,
      Column('group_id', Integer, primary_key=True),
      Column('group_name', Unicode(16), unique=True),
      Column('display_name', Unicode(255)),
      Column('created', DateTime, default=datetime.datetime.now)
  )
  
  users_table = Table('tg_user', metadata,
      Column('user_id', Integer, primary_key=True),
      Column('user_name', Unicode(16), unique=True),
      Column('email_address', Unicode(255), unique=True),
      Column('display_name', Unicode(255)),
      Column('password', Unicode(40)),
      Column('created', DateTime, default=datetime.datetime.now)
  )
  
  permissions_table = Table('tg_permission', metadata,
      Column('permission_id', Integer, primary_key=True),
      Column('permission_name', Unicode(16), unique=True),
      Column('description', Unicode(255))
  )
  
  user_group_table = Table('tg_user_group', metadata,
      Column('user_id', Integer, ForeignKey('tg_user.user_id',
          onupdate="CASCADE", ondelete="CASCADE")),
      Column('group_id', Integer, ForeignKey('tg_group.group_id',
          onupdate="CASCADE", ondelete="CASCADE"))
  )
  
  group_permission_table = Table('tg_group_permission', metadata,
      Column('group_id', Integer, ForeignKey('tg_group.group_id',
          onupdate="CASCADE", ondelete="CASCADE")),
      Column('permission_id', Integer, ForeignKey('tg_permission.permission_id',
          onupdate="CASCADE", ondelete="CASCADE"))
  )
  
  # identity model
  class Group(object):
      """An ultra-simple group definition.
      """
      def __repr__(self):
          return '' % self.group_name
  
  class User(object):
      """Reasonably basic User definition. Probably would want additional
      attributes.
      """
      def __repr__(self):
          return '' % (
                  self.email_address, self.display_name)
  
      def permissions(self):
          perms = set()
          for g in self.groups:
              perms = perms | set(g.permissions)
          return perms
      permissions = property(permissions)
  
      def by_email_address(klass, email):
          """A class method that can be used to search users
          based on their email addresses since it is unique.
          """
          session = DBSession()
          return session.query(klass).filter(klass.email_address==email).first()
  
      by_email_address = classmethod(by_email_address)
  
      def by_user_name(klass, username):
          """A class method that permits to search users
          based on their user_name attribute.
          """
          session = DBSession()
          return session.query(klass).filter(klass.user_name==username).first()
  
      by_user_name = classmethod(by_user_name)
  
      def _set_password(self, password):
          """encrypts password on the fly using the encryption
          algo defined in the configuration
          """
          algorithm = config.get('authorize.hashmethod', None)
          self._password = self.__encrypt_password(algorithm, password)
  
      def _get_password(self):
          """returns password
          """
          return self._password
  
      password = property(_get_password, _set_password)
  
      def __encrypt_password(self, algorithm, password):
          """Hash the given password with the specified algorithm. Valid values
          for algorithm are 'md5' and 'sha1'. All other algorithm values will
          be essentially a no-op."""
          hashed_password = password
  
          if isinstance(password, unicode):
              password_8bit = password.encode('UTF-8')
  
          else:
              password_8bit = password
  
          if "md5" == algorithm:
              hashed_password = md5.new(password_8bit).hexdigest()
  
          elif "sha1" == algorithm:
              hashed_password = sha.new(password_8bit).hexdigest()
  
          if not isinstance(hashed_password, unicode):
              hashed_password = hashed_password.decode('UTF-8')
  
          return hashed_password
  
      def validate_password(self, password):
          """Check the password against existing credentials.
          """
          algorithm = config.get('authorize.hashmethod', None)
          return self.password == self.__encrypt_password(algorithm, password)
  
  class Permission(object):
      """A relationship that determines what each Group can do
      """
      pass
  
  mapper(User, users_table,
          properties=dict(_password=users_table.c.password))
  
  mapper(Group, groups_table,
          properties=dict(users=relation(User,
                  secondary=user_group_table, backref='groups')))
  
  mapper(Permission, permissions_table,
          properties=dict(groups=relation(Group,
                  secondary=group_permission_table, backref='permissions')))
  
  
  

• Florent created a set of repoze.who plugins that work with SQLAlchemy model data to do identification and authentication. In this case, Florent created an "authenticator" plugin that examines SQLAlchemy model data to compare the identity information found by repoze.who against SQL data, and a "metadata" plugin which adds permission and group information to the identity. He reuses stock repoze.who plugins for "identificaton" duties, including an auth_tkt based cookie identification plugin and a form login identification plugin. The code that defines and configures these plugins can be viewed here. In that module is also a helper function named make_who_middleware which serves as a constructor for a configuration of middleware which uses these plugins.
• Florent also coded up some functions in the authorize.py module of tgrepozewho that can serve as decorators for controller methods that perform authorization against repoze.who-provided data. repoze.who doesn't perform any authorization itself, it leaves that up to the web framework, so this was required.
• He then added the middleware generated by his helper function to his whotg demo package's middleware pipeline in in "config/middleware.py". This sets up the repoze.who middleware in the TG2 WSGI stack, so that when you start the TG2 server, it's present in the call chain.

      ...
      # CUSTOM MIDDLEWARE HERE (filtered by error handling middlewares)
      from whotg.model import User, DBSession
      user_criterion = User.user_name
      # the name of the column which is used as a unique identifier
      user_id_col = 'user_id'
      from tgrepozewho.middleware import make_who_middleware
      app = make_who_middleware(app, config, User, user_criterion, user_id_col,
              DBSession)
      ...
  

• Then he created two methods on his root controller's class named manage_permission_only and editor_user_only. He protects the manage_permission_only method with a decorator which ensures that the user provided by the middleware has the 'manage' permission (as per the data in the model tables, via his group). He protects the editor_user_only with a decorator that ensures that the user provided by the middleware has the username 'editor'. When these methods are asked for (e.g. via http://locahost:8080/manage_permission_only or via http://localhost:8080/editor_user_only), if the user possesses the proper permission or username (respectively), the TG "about" page will be shown; if he does not, the user will be redirected to a login form. Only the relevant parts of the entire module are shown below.

  from datetime import datetime
  now=datetime.now()
  
  from tg import expose
  from whotg.lib.base import BaseController
  
  from tgrepozewho import authorize
  
  class RootController(BaseController):
  
     ...
  
      @expose('whotg.templates.about')
      @authorize.require(authorize.has_permission('manage'))
      def manage_permission_only(self, **kw):
          return dict(now=now, page='about')
  
      @expose('whotg.templates.about')
      @authorize.require(authorize.is_user('editor'))
      def editor_user_only(self, **kw):
          return dict(now=now, page='about')
  
  

The result is a functional, customizable authentication and authorization system for TurboGears 2 that reuses the repoze.who framework. Hats off to Florent! This appears to be real evidence that we can move towards a "fourth generation" of Python web frameworks where coding framework-specific subsystems from scratch isn't always the norm, because the work that Florent did could be recast pretty easily for just about any TurboGears 2 or Pylons application (or Django application, etc).

- Chris

0.3.6 (2008-04-16)

  - "http" exceptions (like Redirect and Unauthorized) weren't handled
    properly anywhere except in the repoze.zope2 obob helper's
    'invoke' step.  In-the-wild code uses these exceptions before the
    published object has been located (e.g. during traverse() or
    before_traverse()).  We now depend on repoze.obob >=0.3 to get
    extended exception handling behavior, and we implement a
    'handle_exception' method on our z2bob helper which will turn
    Zope2 Unauthorized and Redirect exceptions into their
    "httpexception" equivalents for consumption by upstream
    middleware.  This was prompted by code found in the wild in
    Plone's OpenId implementation which raises a redirect during
    traversal.

0.3.5 (2008-04-16)

  - "Legacy" virtual hosting (via Virtual Host Monster) did not work
    properly.  Symptom: if you set up proxy-rewrite rules in Apache
    pointing at the Zope root a repoze.zope2 server running under a
    separate paster server, and tried to visit the ZMI via the Apache
    virtualhost's /manage URL, you'd be presented with the
    VirtualHostMonster ZMI configuration page instead of the ZMI's
    framed root UI.  Reason: the PARENTS[0] item was not set up early
    enough (it was set up in traverse rather than
    before_traverse). Since it was depended on by Zope API's which VHM
    called out to to set the virtual root, this didn't work, and the
    resulting traversal name stack was incorrect.

0.3.4 (2008-03-24)

  - Bump ez_setup.py version.

  - When Zope 2 starts, it potentially writes data to the database
    during product initialization.  When multiple clients talk to the
    same ZEO storage at startup, they often simultaneously try to
    write (the same) information to the database concurrently.  This
    causes startup failure due to conflict errors.  We now retry
    product initialization up to five times to work around this issue.

0.3.3 (2008-03-10)

  - repoze.zope2 now properly respects virtual host directives
    provided to it by repoze.vhm xheaders middleware >= 0.4.  Zope's
    VHM can still be used as necessary, but is no longer required.

0.3.2 (2008-03-03)

  - Fix bug reported by Martin Aspeli: repoze.zope2 would choke on
    large images and files (symptom: broken images when images were
    large).  This was due to the fact that the Zope File- and
    Image-rendering machinery used HTTPResponse.write, which
    repoze.zope2's response handling didn't handle properly.  We now
    subclass HTTPResponse (as RepozeHTTPResponse) to solve the issue.

0.3.1 (2008-03-02)

  - mkzope2instance now:

    o takes no arguments, only options.  '-d' replaces the single
      argument path.

    o creates a "log" directory

    o writes out a zeo.conf into "etc" (unconditionally); you can
      start a ZEO instance after installation now via 'bin/runzeo -C
      etc/zeo.conf', after ensuring that the 'address' in the ZEO
      section is correct.

    o allows the specification of 'sandbox' (-s) (replaces
      single-argument instancedir), 'zeo-port' (-z) , 'zope-port'
      (-p), and 'use-zeo' (-z) options.  If 'use-zeo' is specified,
      the zope.conf that's written will use a ClientStorage by
      default.

    o writes a zope.conf with a zodb cache-size of 50000 rather than
      10000.

  - addzope2user, runzope2script, and debugzope2 now respect a
    "ZOPE_CONF" environment variable, which can be used to specify the
    zope.conf configuration file to use.

  - Add a sample  section to the generated zope.conf
    that can be uncommented if the installer wants to use ZEO instead
    of FileStorage or vice versa.

  - Added an (experimental) 'zope2testrunner' script that sets up
    stuff in the environment before running
    'zope.testing.testrunner.run'.  It accepts the same arguments as
    the Zope 3 testrunner.  E.g. 'bin/zope2testrunner -m
    Products.faster'.  This also respects the ZOPE_CONF envvar.

  - Depend on Zope 2.10.5 (zopelib-2.10.5.0) instead of 2.10.4.2 and
    various other updated repoze libraries.

-Chris

What have we done?

• We've reimplemented the Zope 2 publisher, so Zope 2 applications such as Plone can be run without modification within a WSGI environment
• We've successfully externalized features Zope such as transaction management, conflict error retry, virtual hosting services, and identification and authentication, error logging, and profiling into WSGI middleware, so they can be reused outside the Zope stack in a way that's largely familiar to Zope and non-Zope users alike.
• We've packaged up Plone, Grok, Trac, and Django so they can be used without much effort in a Repoze WSGI environment.
• We've created bridges to traditionally-CGI applications like Mailman so they can be served up via WSGI servers.
• We've created applications like repoze.kiss, which are "Zope-like", but do not use any ZODB.
• We've evangelized Deliverance as a theming mechanism.
• We've reached out to other Python web projects like TurboGears, Pylons, and Django, attempting to reuse some of their components, and we've made them aware of ours.
• We've given many, many talks at Zope/Plone/Python user groups and conferences, attempting to bring more folks into the fold
• We've created buildouts using zc.buildout for Plone and Trac.

What are we doing?

• We're playing around with exernalizing declarative security, so we can better integrate dissimilar WSGI applications using the same security model.
• We're attempting to create alternate sessioning components for use under WSGI stacks.
• We're working Repoze into our (Agendaless') own customer deployments

What we intend to do

• We intend to do more evangelism via presenting to Python/Zope/Plone user groups and conferences.
• We intend to reach out more to other web projects by helping them integrate Repoze code into their own projects.
• We intend to push for Plone (and possibly Zope 2) to base a release on Repoze technologies.

Where we need help

We need help a) evangelizing, b) bringing things to ground and c) maintaining our various web presences. On the evangelism front, we need Python folks to understand that Repoze is not just about Zope, instead it's a generically useful set of WSGI components inspired by features of Zope. We need to get Zope people excited about a future that allows them to make use of best-of-breed components created by Python web developers that are not Zope developers. We need to get Python folks excited about the fact that the Zope community is reaching out to the larger Python web development community by making it possible for them to use components that Zope folks have been using for years without knowing anything whatsoever about Zope itself. We need to improve Deliverance's conditional theming support. We need Zope and Plone folks to start using repoze.zope2 and Deliverance "in anger" to start to bring things to ground. We need help maintaining repoze.org, mainly to keep content fresh. We need more developers to start contributing code to the Repoze repository.

If you want to help doing any of these things, please join the repoze-dev maillist and the #repoze IRC channel on freenode.net, and yell!

-Chris

For several years now, we've been making and releasing podcasts that we've referred to as Zopecasts when we felt like it. The last few we've made have been more about Repoze than they've been about Zope, so we've decided to retitle the every-so-often podcast we do as "Repozecast".

The inaugural episode (mp3) is now up. It contains ramblings about Repoze and how it relates to:

• The past Plone Strategic Planning Summit
• The upcoming Plone Symposium East
• The upcoming Python conference
• Recent Zope/Plone/Python user group talks
• New Repoze committers
• New repoze.* packages
• Buildout for repoze.plone
• Deliverance

Enjoy!

-Chris

• It didn't work properly under Python 2.5 due to a difference in the way the Python 2.5 pstats module works
• It didn't explicitly depend on ElementTree, which its templates require (but the templating package is not a setuptools package, so cannot name its own dependencies).

If you tried it and it didn't work, try it again.

easy_install -i http://dist.repoze.org/simple repoze.profile

Or if you've already got it installed..

easy_install -i http://dist.repoze.org/simple -U repoze.profile

- Chris

Version 0.2 adds a (butt-ugly-and-utterly-ripped-off-from-Zope) browser UI that allows you to sort and filter profiling results. It looks like the image below.

To get it, do easy_install -i http://dist.repoze.org/simple repoze.profile, then wire it up in your WSGI pipeline. For example:

 [pipeline:main]
 pipeline = egg:repoze.profile#profile
            egg:Paste#cgitb
            egg:Paste#httpexceptions
            myapp

You should then be able to visit /__profile__ in your browser while talking to your WSGI application to see the profile UI. Note that the middleware (by default) discards the first request info, so you will need to hit your application a few times to see meaningful profile info.

See the README for more detailed configuration information.

- Chris

Motivations

Why pursue this facility? Who is it aimed at and what issues do they have with the status quo?

In a nutshell, this proposal wants to make it radically simpler to brand Plone. The designer's "sitetheming" job will focus on saving HTML to disk. No templating, with weird macro decomposition and API calls. No resource registries. No buildout. No restart.

1. People that produce high-quality designs won't want to learn Plone templating. Imagine if their HTML (and skills therein) was used, untouched.
2. Reducing the amount of "unique-to-Plone" stuff that is important to getting started can help reduce fear-of-the-unknown and learning curve.
3. Sometimes this design already exists and doesn't need to be "compiled" into ZPT/packages.
4. Web designers probably don't want to run a Plone install to maintain the the branding.
5. Theme resources, such as CSS and JS, will be served from the HTTP server directly.
6. Focusing Plone on the semantic information unique to a single resource/URL will promote other goals such as integration.
7. It is possible that performance improves, if less is done in ZPT and Zope.
8. If Plone moves to a more Python-friendly (WSGI) philosophy, then theming non-Plone sites with the same artifacts is a benefit.

Again, just to be clear: the single biggest goal is to be kind to civilians.

Sitetheming in a nutshell

First, a point about nomenclature. Since theming was introduced as a Plone word that covers skinning, templating, and (to a degree) programming, this proposal temporarily chooses a new word. We will call this new facility "sitetheming" to disambiguate jargon skew. Perhaps at the end, we'll just call this "theming".

Most pages in a site share lots of common elements. Logo, site menu links, footer, links to CSS and JS, etc. This is usually referred to "branding", "look-and-feel", or "corporate identity". On larger projects, the people managing the branding have little desire to learn a new system. On smaller projects, with people just learning, they haven't mastered enough yet to make the pages "their own".

Sitetheming is a facility for imposing a common look-and-feel on most site pages using no templating. Instead, the designer or customizer maintains a pile of HTML/CSS/JS/PNG artifacts at some reachable location. The HTML is then used as an "overlay" on the content coming out of the site.

More specifically:

1. Save some HTML and other stuff.
2. Make a rule file that defines spots in the theme that need to be replaced. For example, the big block in the middle column.
3. Later, as content is being served, blocks are pulled out of the content and merged into the theme.

This diagram shows the basic idea:

In this approach, HTML is the API. As long as the theme continues providing elements with the correct ID or class names, and as long as the content coming in has a stable "REST API" of markup structure, the rule file can do its job.

Next

That covers that background and basic information. Next posts will dive into how it will actually work, plus what is needed for Plone to evaluate it as a way of doing branding.

- Paul

This ticket is about putting a site's common look-and-feel on each page. For the moment, I'll call this step "sitetheming". Should we have a special, simple facility for just this audience (integrators, entry-level Plone customizers) and activity?

However, not all will agree on the basic premise, so I'd like to anticipate some of these concerns up-front. If any of this sounds like you, please speak up during this process. The goal is to make something uniquely useful and simple for the non-core-developer audience. We certainly don't want to do something that you won't like!

(If you simply must read more about site themes, here is a writeup I made long ago on the Zope wiki and here is the introduction to Deliverance. Alternatively, wait until my next few posts.)

Why Not

1. Plone 2.5 and/or Plone 3.0 is easy enough to theme. Falls into the "ain't broke, don't fix it" category. My sense is that "theming is hard" is a common complaint by the target audience, but perhaps I'm mis-reading the tea leaves.
2. It's hard, but worth it. Currently, theming/skinning/templating/programming are all part of the same facility. This lets you do just about anything. Creating an isolated, stripped-down "theme engine" thus breaks that proposition.
3. I'm both the developer and the UI person. If you're mostly a developer who also does the UI, you might not want to learn a new facility, as you're still going to be the person doing the page content generation.
4. People will invariably stray over the line. We always encourage a clear line between presentation and content. However, the moment you have something straddling the line (e.g. the contents of a calenar portlet), that discipline feels restraining.
5. I'll never have non-Plone. As part of this proposal, we'd like to encourage moving stuff out the Zope stack, allowing it to be applied across other Python stuff. However, that might not be your itch. Having a corporate ID that could be used with other Plone apps, or potentitally non-Plone apps, might not be a meaningful upside for you.
6. Plone needs to stop changing. Integrators, and even some core Plone folks, are feeling bumfuzzled by all the machinery changes. They've reached their absorption limit. Thus, it wouldn't matter if there was a problem, nor if this was a good solution. It's a new way to do old stuff, and that's bad on the face of it.

That's some of what I'm hearing so far. Before we get into the details of what's on the table, who should care, and even how we can minimize these points, I want to make sure I'm listening to the integrators and entry folks. Any other concerns about this proposal in particular, or the strategic planning summit in general?

- Paul

The "old" installation procedure (using repozeproject) still works for the new packages, which is fortunate, as the documented install procedures on Repoze.org haven't yet been updated to describe the new installation process. I'm getting around to it, I swear... ;-)

- Chris

In a fascinating bit of science fiction, Martin Aspeli explores the possible user experience of Pete and Andy, the next generation of Plone newbies. His tale features both Repoze and Deliverance prominently, as the favored means for skinning and deploying Plone.

Martin then follows up by examining his flight of fancy as a roadmap forward for Plone, in particular highlighting the need for better "getting started" documentation, as well as careful attention to the usability of Plone for the "average" (non-core) developer.

Recommended reading.

Rationale

After running across a portability / build problem with a distutils package recently, I realized that the testing we are doing for Repoze may not have adequately covered our needs to keep the various Repoze components running against all supported platforms (basically, anything where packages with "C" extensions can be built easily).

After casting around a bit, I decided to bite the bullet and try out Buildbot, the same tool used to run the Zope unit tests.

Setting Up the Master

First, I decided to set up the buildbot "master", running on the repoze.org server. This process tracks all pending changes which require builds, and doles the builds out to one or more designated "buildslaves".

Because buildbot runs as one or more long-running daemon processes, I decided to create a new account for it, to enforce a bit of privilege isolation:

  $ sudo /usr/sbin/groupadd buildbot
  $ sudo /usr/sbin/useradd -g buildbot -c "Buildbot" -m -s /bin/bash \
    buildbot

  $ sudo su - buildbot
  $ /usr/bin/virtualenv ~

I then installed buildbot and its setuptools-compatible dependencies (note that those dependencies aren't spelled out in its setup.py, but are document in its INSTALL.txt.

  $ bin/easy_install buildbot zope.interface

The remaining dependency is Twisted, which can't be installed via easy_install (its setup is complex, and hasn't yet adapted to the setuptools regime). So, I installed it the old-fashined distutils way:

  $ mkdir src
  $ cd src
  $ wget http://tmrc.mit.edu/mirror/twisted/Twisted/2.5/Twisted-2.5.0.tar.bz2
  $ tar xjf Twisted-2.5.0.tar.bz2
  $ cd Twisted-2.5.0
  $ ../../bin/python setup.py install
  ...

Now, I could verify that the buildbot application was working:

  $ bin/buildbot --version
  Buildbot version: 0.7.6
  Twisted version: 2.5.0

I then created and initiailzed the buildbot "master" work area, and set up a configuration (lots of iterative tweakage elided here):

  $ mkdir -p var/masters/repoze
  $ bin/buildbot create-master var/masters/repoze
  $ cp var/masters/repoze/master.cfg{.sample,}
  $ vim var/masters/repoze/master.cfg
  $ bin/buildbot start var/masters/repoze

Setting up a "buildslave"

Next, I could work on setting up the first "buildslave":

  $ mkdir -p var/slaves/repoze
  $ bin/buildbot create-slave var/slaves/repoze repoze.org:9980 \
    laguna 

I used the slavename / password which I had created in the master.cfg file. Then, I edited the two "info" files identifying me and my host, and then started the buildbot "slave" daemon:

  $ vim slave/repoze/info/{admin,host}
  $ bin/buildbot start slave/repoze

It connected, and I could see it on the status page.

Now, I tried a "forced" build, which worked (after more iterative tweakage). The "builder recipe" I'm using does the following whenever it is triggered, either manually or when its "change source" notices a change:

1. checks out the branch containing the change (e.g., 'repoze.kiss/trunk');
2. runs 'bin/python setup.py test' in the checkout.

The "waterfall" page shows the build status.

I then set up another pair of buildslaves, one on my Ubuntu Gutsy laptop and one on a Centos4 server, and ensured that the builds worked with them. Carlos de la Guardia has since volunteered an Ubunty Gutsy server, which is great, as my laptop may or may not be online when a change is triggered.

Conclusions

First, the product does what it is designed for: it handles the complex process of dispatching asynchronous builds to the various platforms based on changes to the repository.

Second, the configuration format is an execfiletoo powerful: I would prefer a purely declarative configuration language. Many users, however, won't need to write any really imperative code in this file, which mitigates the risk somewhat. I chose to add some code here, rather than move it out to an importable module, largely to deal with having a single "scheduler" handle changes to any of the supported branches, and to arrange that each of those branches build in a separate subdirectory.