The Ever Evolving Sqitch Plan
I’ve been working on the parser for the proposed new deployment plan format, and spent a day thinking it wasn’t going to work at all. But by the end of the day yesterday, I was back on board with it. I think I’ve also figured out how to eliminate the VCS dependency (and thus a whole level of complication). So first, the plan format:
- Names of things (steps and tags) cannot start or end in punctuation characters
@
indicates a tag+
indicates a step to be deployed-
indicates a step to be reverted#
starts a comment:
indicates a required step!
indicates a conflicting step%
is for Sqitch directives
So, here’s an example derived from a previous example:
%plan-syntax-v1
+roles
+users_table
+dr_evil
@alpha
# Some procedures.
+add_user :roles :users_table
+del_user :@alpha
-dr_evil
+upd_user :add_user !dr_evil
@beta # woo!
So we start with a directive for the version of the plan file (thanks for the
suggestion, Tom Davis!). Then we have deployments of the roles
,
users_table
, and dr_evil
steps. After that, it’s tagged as alpha
.
Next, we have a comment, then the deployment of the add_user
step. It requires
that the roles
and users_table
steps be deployed. Then we deploy del_user
.
It requires all steps as of the $alpha
tag. Next we revert the dr_evil
step.
Why? Because the next line deploys upd_user
, which conflicts with dr_evil
(and requires add_user
). And finally, we tag it as beta
.
There are a number of things I like about these changes:
-
Dependencies are spelled out in the plan file, rather than the deploy scripts. This allows the deploy scripts to have nothing special about them at all.
-
We now have a way to explicitly revert a step as part of the overall plan. This is useful for ensuring that conflicts can be dealt with.
-
We can deploy to any point in the plan by specifying a step:
sqitch deploy add_user
Or a tag:
sqitch deploy @alpha
For steps that are duplicated, we can disambiguate by specifying a tag:
sqitch deploy dir_evil --as-of @alpha
Naturally, this requires that a step not be repeated within the scope of a single tag.
Now, as for the VCS dependency, my impetus for this was to allow Sqitch
to get
earlier versions of a particular deploy script, so that it could be modified in
place and redeployed to make changes inline, as described in an earlier post.
However I’ve been troubled as to how to indicate in the plan where to look in
the VCS history for a particular copy of a file. Yesterday, I had an insight:
why do I need the earlier version of a particular deploy script at all? There
are two situations where it would be used, assuming a plan that mentions the
same step at two different points:
- To run it as it existed at the first point, and to run it the second time as it exists at that time.
- To run it in order to revert from the second point to the first.
As to the first, I could not think of a reason why that would be necessary. If I’m bootstrapping a new database, and the changes in that file are idempotent, is it really necessary to run the earlier version of the file at all? Maybe it is, but I could not think of one.
The second item is the bit I wanted, and I realized (thanks in part to prompt from Peter van Hardenberg while at PGCon) that I don’t need a VCS to get the script as it was at the time it was deployed. Instead, all I have to do is store the script in the database as it was at the time it was run. Boom, reversion time travel without a VCS.
As an example, take the plan above. Say we have a database that has been
deployed all the way to @beta
. Let’s add the add_user
step again:
%plan-syntax-v1
+roles
+users_table
+dr_evil
@alpha
# Some procedures.
+add_user :roles :users_table
+del_user :@alpha
-dr_evil
+upd_user :add_user !dr_evil
@beta # woo!
+crypto
+add_user :roles :users_table :crypto
@gamma
The last two lines are the new ones. At this point, the
sql/deploy/add_user.sql
script has been modified to fix a bug that now
requires the crypto
step. If we deploy to a new database, Sqitch will notice
that the same step is listed twice and apply it only once. This works because,
even though add_user
is listed before pg_crypto
, it is actually applied as
described in its second declaration. So the run order would be:
crypto
add_user
del_user
upd_user
Note that this works because crypto
declares no dependencies itself. If it
did, it would be shuffled as appropriate. It would not work if it required, say,
upd_user
, as that would create a circular dependency (add_user
— crypto
—
upd_user
— add_user
).
Now say we want to deploy the changes to the production database, which is
currently on @beta
. That simply runs:
crypto
add_user
If something went wrong, and we needed to revert, all Sqitch has to do is to
read add_user
from the database, as it was deployed previously, and run
that. This will return the add_user
function to its previous state. So, no
duplication and no need for a VCS.
The one thing that scares me a bit is being able to properly detect circular dependencies in the plan parser. I think it will be pretty straight-forward for steps that require other steps. Less so for steps that require tags. Perhaps it will just have to convert a tag into an explicit dependence on all steps prior to that tag.
So, I think this will work. But I’m sure I must have missed something. If you notice it please enlighten me in the comments. And thanks for reading this far!
Looking for the comments? Try the old layout.