[Testtools] Accumulating a configuration (nightlyBuild main loop)

[Stefan Vigerske]

Hi,

> 	JP and I have been exchanging emails about BuildDirSuffix, and he's been
> persistent enough to convince me that there's a possibility of accidental
> persistence from one build to the next.  I'm testing out a fix (pop the value
> from configuration if it's not specified for a given build).  But it seems to me
> the problem really lies in the way that the main loop of nightlyBuild assembles
> a configuration. We're in nightlyBuild.py.
> 
> 	Where it says `Define loop invariant configuration values', we
> initialise a map (configuration) and a set of sets (configurations).  Prior to
> entering the loop on projects, some persistent keys (rootDir,
> buildMethod) are set in configuration, to be used by all projects.
> 
> 	Now we enter the project loop (`Loop once for each project ...').  We
> set two more keys, project and `clear previous build', to be used for all builds
> of a project. (Arguably, `clear previous build' should be handled as a 
> persistent key, since it's initialised from a global variable.)
> 
> 	Now we enter the build loop, processing the builds specified for the
> project. We fill in a whole bunch more keys in configuration. Eventually, we
> check to see if configuration is already in configurations. If not, we process 
> it (NBbuildConfig.run) and record it in configurations.
> 
> 	Now, it seems to me that at this point the logical thing to do is clear
> configuration back to an empty map and start over to fill in configuration for
> the next build, but that doesn't happen.  The consequence is that once a
> key:value pair is added to configuration, it persists until explicitly removed
> or changed.  This is not safe.

Correct, that is not good.


>  I'd like to propose something a bit different,
> but fairly close to what now exists.
> 
> 	There are three classes of keys in a configuration:  top-level (for all
> projects), project-level (for all builds of a project), and build-level (for an
> individual build).  Right now, rootDir and buildMethod are handled as top-level,
> project and `clear previous build' are handled as project-level, and all the
> rest are (or should be) handled as build-level.
> 
> 	Why not make this explicit, by creating two new variables,
> top_configuration and project_configuration.  These get loaded at the places
> where top-level and project-level keys are processed now.
> 
> 	Then, in the inner loop (`for bc in buildConfigs'), the first action
> is to clear configuration and load it with the values from top_configuration and 
> project_configuration. This should prevent accidental carry-over.

Sounds good to me :-).

> 	Documentation and code comments should be updated to reflect the three 
> classes of parameters. As an added benefit, the user could override top-level 
> and project-level keys in a build configuration, should s/he so desire.

Interesting, I'll see how this will look like.

> 	You guys will regret dragging me into this ...   :-)

Not so far ;-).

Thanks,
Stefan

-- 
Stefan Vigerske
Humboldt University Berlin, Numerical Mathematics
http://www.math.hu-berlin.de/~stefan