Just a Theory

Black lives matter

MySQL’s REPLACE Considered Harmful

So we’ve set up a client with an online poll application using MySQL. Polls are created in Bricolage, and when they’re published, rather than writing data to files, the template writes data to the MySQL database. PHP code on the front-end server then uses the database records to manage the polls.

On the recommendation of one of my colleagues, I was using the MySQL REPLACE statement to insert and update poll answers in the database. At first, this seemed like a cool idea. All I had to do was create a unique index on the story_id and ord (for answer order) columns and I was set. Any time someone reordered the answers or changed their wording in Bricolage, the REPLACE statement would change the appropriate records and just do the right thing.

Or so I thought.

Come the day after the launch of the new site, I get a complaint from the customer that the percentage spread between the answers doesn’t add up to 100%. After some investigation, I realized that the poll_results table is using the ID of each question to identify the votes submitted by readers. This makes sense, of course, and is excellent relational practice, but I have overlooked the fact that REPLACE essentially replaces rows every time it is used. This means that even when a poll answer hasn’t changed, it gets a new ID. Yes, that’s right, its primary key value was changing. Yow!

Now we might have caught this earlier, but the database was developed on MySQL 3.23.58 and, as is conventional among MySQL developers, there were no foreign key constraints. So the poll results were still happily pointing to non-existent records. So a poll might appear to have 800 votes, but the percentages might be counted for only 50 votes. Hence the problem with the percentages not adding up to 100% (nowhere near it, in fact).

Fortunately, the production application is on a MySQL 4.1 server, so I made a number of changes to correct this issue:

  • Added foreign key constraints
  • Exploited a little-known (mis)feature of Bricolage to store primary keys for all poll answers (and questions, for that matter)
  • Switched from REPLACE to INSERT, UPDATE, and DELETE statements using the primary keys

I also started using transactions when making all these updates when a poll is published so that changes are always atomic. Now it works beautifully.

But the lesson learned is that REPLACE is a harmful construct. Yes, it was my responsibility to recognize that it would create new rows and therefore new primary keys. But any construct that changes primary keys should be stricken from any database developer’s toolbox. The fact that MySQL convention omits the use of foreign key constraints makes this a particularly serious issue that can appear to have mysterious consequences.

So my advice to you, gentle reader, is don’t use it.

Looking for the comments? Try the old layout.

The End of Civilization

It’s the end of civilization as we know it.

Looking for the comments? Try the old layout.

What JavaScript Book Should I Buy?

What with all the interest in Ajax, not to mention an increasing reliance on JavaScript in Bricolage (and a need to fix its bugs), I think it’s high time I really learned more JavaScript. So, what JavaScript book should I buy? Bear in mind that I’m an experienced object-oriented application developer who wants to pick up a new language, not a script kiddie or CGI newbie.

Thanks!

Looking for the comments? Try the old layout.

How I Increment Module Version Numbers

Here’s how I quickly increment version numbers in my modules. I call this script reversion:

#!/usr/bin/perl -w

use strict;

unless (@ARGV) {
    print "  Usage: $0 version\n\n";
    exit;
}

my $old = shift;
my $new = $old + .01;
my $dir = shift || '.';

system qq{grep -lr '\Q$old\E' $dir }
  . '| grep -v \\.svn '
  . '| grep -v Changes '
  . '| grep -v META\\.yml '
  . "| xargs $^X -i -pe \""
  . qq{print STDERR \\\$ARGV[0], \\\$/ unless \\\$::seen{\\\$ARGV[0]}++;}
  . qq{s/(\\\$VERSION\\s*=?\\s*'?)\Q$old\E('?)/\\\${1}$new\\\$2/g"};

__END__

Enjoy!

Looking for the comments? Try the old layout.

New LDAP Auth Module for RT

I grabbed the LdapOverlay solution for using an LDAP server to authenticate against Request Tracker today in my continuing efforts to use LDAP for single sign-on for all Kineticode resources. It worked great, but I wanted a couple more things out of it, namely TLS communications with the LDAP server (so that all communications are encrypted), and authentication only for members of a certain LDAP group.

So I refactored LdapOverlay and added these features. You can download it from here. Just set the $LdapTLS variable in your RT_SiteConfig module to a true value to use TLS (but be sure that you also have Net::SSLeay installed!). If you want to allow only members of a certain LDAP group to authenticate to RT, set the DN of the group in the $LdapGroup variable, and set the name of the member attribute (usually “uniqueMember”) in the $LdapGroupAttribute variable.

Enjoy!

Looking for the comments? Try the old layout.

Enforce Foreign Key Integrity in SQLite with Triggers

After some some Googling and experimentation, I’ve figured out how to enforce foreign key constraints in SQLite. I got most of the code from Cody Pisto’s sqlite_fk utility. I couldn’t get it to work, but the essential code for the triggers was in its fk.c file, so I just borrowed from that (public domain) code to figure it out.

Since I couldn’t find documentation for this elsewhere on the Net (though I’m sure it exists somewhere), I decided to just put an example here. Interested? Read on!

Say you have these two table declarations:

create table foo (
    id INTEGER NOT NULL PRIMARY KEY
);

CREATE TABLE bar (
    id INTEGER NOT NULL PRIMARY KEY,
    foo_id INTEGER NOT NULL
            CONSTRAINT fk_foo_id REFERENCES a(id) ON DELETE CASCADE
);

Table bar has a foreign key reference to the primary key column in the foo table. Although SQLite supports this syntax (as well as named foreign key constraints), it ignores them. So if you want the references enforced, you need to create triggers to do the job. Triggers were added to SQLite version 2.5, so most users can take advantage of this feature. Each constraint must have three triggers: one for INSERTs, one for UPDATESs, and one for DELETESs. The INSERT trigger looks like this:

CREATE TRIGGER fki_bar_foo_id
BEFORE INSERT ON bar
FOR EACH ROW BEGIN 
    SELECT CASE
        WHEN ((SELECT id FROM foo WHERE id = NEW.foo_id) IS NULL)
        THEN RAISE(ABORT, 'insert on table "bar" violates foreign key '
                || 'constraint "fk_foo_id"')
    END;
END;

(You can put the RAISE error string all on one line; I’ve concatenated two lines to keep line lengths reasonable here.) If your foreign key column is not NOT NULL, the trigger’s SELECT CASE clause needs to an extra case:

CREATE TRIGGER fki_bar_foo_id
BEFORE INSERT ON bar
FOR EACH ROW BEGIN 
    SELECT CASE
        WHEN ((new.foo_id IS NOT NULL)
            AND ((SELECT id FROM foo WHERE id = new.foo_id) IS NULL))
        THEN RAISE(ABORT, 'insert on table "bar" violates foreign key '
                || 'constraint "fk_foo_id"')
    END;
END;

The UPDATE statements are almost identical; if your foreign key column is NOT NULL, then do this:

CREATE TRIGGER fku_bar_foo_id
BEFORE UPDATE ON bar
FOR EACH ROW BEGIN 
    SELECT CASE
        WHEN ((SELECT id FROM foo WHERE id = new.foo_id) IS NULL))
        THEN RAISE(ABORT, 'update on table "bar" violates foreign key '
                || 'constraint "fk_foo_id"')
    END;
END;

And if NULLs are allowed, do this:

CREATE TRIGGER fku_bar_foo_id
BEFORE UPDATE ON bar
FOR EACH ROW BEGIN 
    SELECT CASE
        WHEN ((new.foo_id IS NOT NULL)
            AND ((SELECT id FROM foo WHERE id = new.foo_id) IS NULL))
        THEN RAISE(ABORT, 'update on table "bar" violates foreign key '
                || 'constraint "fk_foo_id"')
    END;
END;

The DELETE trigger is, of course, the reverse of the INSERT and UPDATE triggers, in that it applies to the primary key table, rather than the foreign key table. To whit, in our example, it watches for DELETEs on the foo table:

CREATE TRIGGER fkd_bar_foo_id
BEFORE DELETE ON foo
FOR EACH ROW BEGIN 
    SELECT CASE
    WHEN ((SELECT foo_id FROM bar WHERE foo_id = OLD.id) IS NOT NULL)
    THEN RAISE(ABORT, 'delete on table "foo" violates foreign key '
                || ' constraint "fk_foo_id"')
    END;
END;

This trigger will prevent DELETEs on the foo table when there are existing foreign key references in the bar table. This is generally the default behavior in databases with referential integrity enforcement, sometimes specified explicitly as ON DELETE RESTRICT. But sometimes you want the deletes in the primary key table to “cascade” to the foreign key tables. Such is what our example declaration above specifies, and this is the trigger to to the job:

CREATE TRIGGER fkd_bar_foo_id
BEFORE DELETE ON foo
FOR EACH ROW BEGIN 
    DELETE from bar WHERE foo_id = OLD.id;
END;

Pretty simple, eh? The trigger support in SQLite is great for building your own referential integrity checks. Hopefully, these examples will get you started down the path of creating your own.

Looking for the comments? Try the old layout.

Bricolage 1.8.3 Released

The Bricolage development team is pleased to announce the release of Bricolage 1.8.3. This maintenance release addresses a number of issues in Bricolage 1.8.2. The most important changes eliminate or greatly reduce the number of deadlocks caused during bulk publishes of many documents. Other changes include new contributed scripts for importing contributors and for generating thumbnail images, Russian localization, and various fixes for database transaction, template formatting, and various user interface fixes. Here are the other highlights of this release:

Improvements

  • Added contrib/thumbnails/precreate-thumbs.pl script to pre-create thumbnails from images. Useful for upgraders. [Scott]
  • Added contrib/bric_import_contribs to import contributors from a tab-delimited file. Development by Kineticode, sponsored by the RAND Corporation. [David]
  • Added the published_version parameter to the list() methods of the story, media, and template classes. This parameter forces the search to return the versions of the assets as they were last published, rather than the most recent version. This will be most useful to those looking up other documents in templates and publishing them, as a way of avoiding pulling documents out from other anyone who might have them checked out! [David]
  • All publishing and distribution jobs are now executed in their own transactions when they are triggered by the user interface. This is to reduce the chances of a deadlock between long-running publishing transactions. [David]
  • Optimized SQL queries for key names or that order by string values to use indexes in the list() and list_ids() methods of the story, media, and template classes. [David]
  • Added Russian localization. [Sergey Samoilenko].
  • Changed the foreign keys in the story, media, and formatting (template) tables so that DELETEs do not cascade, but are restricted. This means that before deleting any source, element, site, workflow, or other related object that has a foreign key reference in an asset table, those rows must be deleted. Otherwise, PostgreSQL will throw an exception. Hopefully, this will put a stop to the mysterious but very rare disappearance of stories from Bricolage. [David]
  • A call to $burner->burn_another in a template that passes in a date/time string in the future now causes a publish job to be scheduled for that time, rather than immediate burning the document and then scheduling the distribution to take place in the future. Reported by Ashlee Caul. [David]
  • Changing the sort order of a list of items in a search interface now properly reverses the entire collection of object over the pages, rather than just the objects for the current page. Thanks to Marshall for the spot! [David]

Bug Fixes

  • Publishing stories not in workflow via the SOAP server works again. [David]
  • The Burner object’s encoding attribute is now setable as well as readable. [David]
  • The category browser works again. [David]
  • Fixed Media Upload bug where the full local path was being used, by adding a “winxp” key to Bric::Util::Trans::FS to account for an update to HTTP::BrowserDetect. [Mark Kennedy]
  • Instances of a required custom field in story elements is no longer required once it has been deleted from the element definition in the element manager. Reported by Rod Taylor. [David]
  • A false value passed to the checked_out parameter of the list() and list_ids() methods of the story, media, and template (formatting) classes now properly returns only objects or IDs for assets that are not checked out. [David]
  • The cover date select widget now works properly in the clone interface when a non-ISO style date preference is selected. Thanks to Susan G. for the spot! [David]
  • Sorting templates based on Asset Type (Element) no longer causes an error. [David]
  • Fixed a number of the callbacks in the story, media, and template profiles so that they didn’t clear out the session before other callbacks were done with it. Most often seen as the error “Can’t call method ‘get_tiles’ on an undefined value” in the media profile, especially with IE/Windows (for some unknown reason). Reported by Ed Stevenson. [David]
  • Fixed typo in clone page that caused all output channels to be listed rather than only those associated with the element itself. [Scott]
  • Fixed double listing of the “All” group in the group membership double list manager. [Christian Hauser]
  • Image buttons now correctly execute the onsubmit() method for forms that define an onsubmit attribute. This means that, among other things, changes to a group profile will persist when you click the “Permissions” button. [David]
  • Simple search now works when it is selected when the “Default Search” preference is set to “Advanced”. Reported by Marshall Roch. [David]
  • Multiple alert types set up to trigger alerts for the same event will now all properly execute. Thanks to Christian Hauser for the spot! [David]
  • Publishing stories or media via SOAP with the published_only parameter (--published-only for bric_republish) now correctly republishes the published versions of documents even if the current version is in workflow. Reported by Adam Rinehart. [David]
  • Users granted a permission greater than READ to the members of the “All Users” group no longer get such permission to any members of the “Global Admins” group unless they have specifically been granted such permission to the members of the “Global Admins” group. Thanks to Marshall Roch for the spot! [David]

For a complete list of the changes, see the changes. For the complete history of ongoing changes in Bricolage, see Bric::Changes.

Download Bricolage 1.8.3 now from the Bricolage Website Downloads page, from the SourceForge download page, and from the Kineticode download page.

About Bricolage

Bricolage is a full-featured, enterprise-class content management and publishing system. It offers a browser-based interface for ease-of use, a full-fledged templating system with complete HTML::Mason, HTML::Template, and Template Toolkit support for flexibility, and many other features. It operates in an Apache/mod_perl environment and uses the PostgreSQL RDBMS for its repository. A comprehensive, actively-developed open source CMS, Bricolage was hailed as “quite possibly the most capable enterprise-class open-source application available” by eWEEK.

Enjoy!

–The Bricolage Team

Looking for the comments? Try the old layout.

Bricolage 1.8.3 Released

The Bricolage development team is pleased to announce the release of Bricolage 1.8.3. This maintenance release addresses a number of issues in Bricolage 1.8.2. The most important changes eliminate or greatly reduce the number of deadlocks caused during bulk publishes of many documents. Other changes include new contributed scripts for importing contributors and for generating thumbnail images, Russian localization, and various fixes for database transaction, template formatting, and various user interface fixes. Here are the other highlights of this release:

Improvements

  • Added contrib/thumbnails/precreate-thumbs.pl script to pre-create thumbnails from images. Useful for upgraders. [Scott]

  • Added contrib/bric_import_contribs to import contributors from a tab-delimited file. Development by Kineticode, sponsored by the RAND Corporation. [David]

  • Added the published_version parameter to the list() methods of the story, media, and template classes. This parameter forces the search to return the versions of the assets as they were last published, rather than the most recent version. This will be most useful to those looking up other documents in templates and publishing them, as a way of avoiding pulling documents out from other anyone who might have them checked out! [David]

  • All publishing and distribution jobs are now executed in their own transactions when they are triggered by the user interface. This is to reduce the chances of a deadlock between long-running publishing transactions. [David]

  • Optimized SQL queries for key names or that order by string values to use indexes in the list() and list_ids() methods of the story, media, and template classes. [David]

  • Added Russian localization. [Sergey Samoilenko].

  • Changed the foreign keys in the story, media, and formatting (template) tables so that DELETEs do not cascade, but are restricted. This means that before deleting any source, element, site, workflow, or other related object that has a foreign key reference in an asset table, those rows must be deleted. Otherwise, PostgreSQL will throw an exception. Hopefully, this will put a stop to the mysterious but very rare disappearance of stories from Bricolage. [David]

  • A call to $burner->burn_another in a template that passes in a date/time string in the future now causes a publish job to be scheduled for that time, rather than immediate burning the document and then scheduling the distribution to take place in the future. Reported by Ashlee Caul. [David]

  • Changing the sort order of a list of items in a search interface now properly reverses the entire collection of object over the pages, rather than just the objects for the current page. Thanks to Marshall for the spot! [David]

Bug Fixes

  • Publishing stories not in workflow via the SOAP server works again. [David]

  • The Burner object’s encoding attribute is now settable as well as readable. [David]

  • The category browser works again. [David]

  • Fixed Media Upload bug where the full local path was being used, by adding a “winxp” key to Bric::Util::Trans::FS to account for an update to HTTP::BrowserDetect. [Mark Kennedy]

  • Instances of a required custom field in story elements is no longer required once it has been deleted from the element definition in the element manager. Reported by Rod Taylor. [David]

  • A false value passed to the checked_out parameter of the list() and list_ids() methods of the story, media, and template (formatting) classes now properly returns only objects or IDs for assets that are not checked out. [David]

  • The cover date select widget now works properly in the clone interface when a non-ISO style date preference is selected. Thanks to Susan G. for the spot! [David]

  • Sorting templates based on Asset Type (Element) no longer causes an error. [David]

  • Fixed a number of the callbacks in the story, media, and template profiles so that they didn’t clear out the session before other callbacks were done with it. Most often seen as the error “Can’t call method “get_tiles” on an undefined value” in the media profile, especially with IE/Windows (for some unknown reason). Reported by Ed Stevenson. [David]

  • Fixed typo in clone page that caused all output channels to be listed rather than only those associated with the element itself. [Scott]

  • Fixed double listing of the “All” group in the group membership double list manager. [Christian Hauser]

  • Image buttons now correctly execute the onsubmit() method for forms that define an onsubmit attribute. This means that, among other things, changes to a group profile will persist when you click the “Permissions” button. [David]

  • Simple search now works when it is selected when the “Default Search” preference is set to “Advanced”. Reported by Marshall Roch. [David]

  • Multiple alert types set up to trigger alerts for the same event will now all properly execute. Thanks to Christian Hauser for the spot! [David]

  • Publishing stories or media via SOAP with the published_only parameter (--published-only for bric_republish) now correctly republishes the published versions of documents even if the current version is in workflow. Reported by Adam Rinehart. [David]

  • Users granted a permission greater than READ to the members of the “All Users” group no longer get such permission to any members of the “Global Admins” group unless they have specifically been granted such permission to the members of the “Global Admins” group. Thanks to Marshall Roch for the spot! [David]

For a complete list of the changes, see the changes. For the complete history of ongoing changes in Bricolage, see Bric::Changes.

Download Bricolage 1.8.3 now from the Bricolage Website Downloads page, from the SourceForge download page, and from the Kineticode download page.

About Bricolage

Bricolage is a full-featured, enterprise-class content management and publishing system. It offers a browser-based interface for ease-of use, a full-fledged templating system with complete HTML::Mason, HTML::Template, and Template Toolkit support for flexibility, and many other features. It operates in an Apache/mod_perl environment and uses the PostgreSQL RDBMS for its repository. A comprehensive, actively-developed open source CMS, Bricolage was hailed as “quite possibly the most capable enterprise-class open-source application available by eWEEK.”

Enjoy!

–The Bricolage Team

Originally published on use Perl;

Reframing the Debate

In light of the election results, I’ve been reflecting on what needs to happen next. I started by thinking about how I’d like to reclaim the word “liberal” for use by the left in general, and Democrats in particular. Hendrik Hertzberg said something interesting about this on “Fresh Air” back in July:

Terry Gross: How do you think the word “liberal” has changed in meaning?

Hertzberg: It has been diabolized by the right, unfortunately. It’s a wonderful, perfectly good word. It’s rooted in “liberty.” It has changed so much that if you go back to 1952, when Richard Nixon gave his acceptance speech at the Republican convention, when he was nominated for Vice President, the whole beginning of his speech was an argument that the Republicans–that he, Nixon, and Eisenhower, the Republicans–were the real liberals. In those days, “liberal” was a word that everybody wanted to have attached to him, and even Robert Taft would pay homage to that word. I think it was a mistake for liberals to let go of that word. I think it’s a much better word than, say, the word “progressive”, because liberty is a better value than progress in my book.

Remembering this discussion, I started thinking about how we might reclaim the word, and this led me to ponder on how the right, starting with Reagan, turned it into a bad word. How did they do it? Well, they changed it from an adjective (“the liberal candidate”) to a noun (“he’s a liberal”). They managed this by applying adjectives to the adjective (but not in the adverbial sense), so we got “card-carrying liberal”, “tax and spend liberal,” and so on. Perhaps, I thought, we just need to apply different adjectives to it in our day-to-day usage, such as “freedom-loving liberal.”

But that’s not too strong, is it? Perhaps we could instead borrow the right’s idea of recasting a common word with a negative connotation. Maybe we could start applying adjectives to other adjectives, such as “conservative.”

I started pondering on what it means to be conservative. It used to be that conservatives had a core set of values that were primarily focused on reducing the size of government. To do so, they wanted to lower taxes in order force lower spending on programs. But it was also important to them to keep the budget balanced. Tax cuts had to come hand-in-hand with reduced spending, because otherwise budget shortfalls would just pass debt on to our children, who would then have to pay more taxes. And in the meantime, the size of government would not actually be reduced.

This changed, as near as I can tell, during the Reagan administration. Here was a conservative president who, for the first time, cut taxes but increased spending. Spending on defense and weapons research shot way up in order to beat the Soviet Union in the so-called “arms race.” Whether or not the arms race was a justifiable policy in our attempt to “win” the “cold war,” when it was accompanied by tax cuts, it gave rise to a new type of conservative, the slash and burn conservative.

And George W. Bush is the biggest slash and burn conservative of them all. Even as he slashed taxes on the richest 1% of the population, he burned through the $237 billion budget surplus (projected by the Congressional Budget Office to accumulate to $5.6 trillion between 2002 and 2011) in near record time, and added to it the current $477 billion budget deficit (expected to accumulate to $5.2 trillion over the next ten years). Ten trillion dollar is a lot, but a slash and burn conservative always believes that there’s money to burn.

Where did this budget shortfall come from? After all, tax breaks alone won’t lead to a deficit when you have the kind of surplus we were expecting in 2000. How has President Bush burned through so much cash in such a short time? Well, it seems that he’s a slash and spend conservative. He found new programs to spend the (nonexistent) money on, such as an incredibly expensive war in Iraq. Indeed, if Bush is the first “CEO President,” as he likes to style himself, then he must be of the Ken Lay Ilk: he’s the Enron president!

I mentioned some of these ideas to Nat yesterday, and before I could get them out, fully formed, he pointed me to a new book from George Lakoff entitled, Don’t think of an Elephant. Salon has a good article describing the book, and it does seem that Lakoff (whose work, I was amused to realize, I read in college when I was getting my MA in Anthropology) is arguing the same thing as I, only with robust tie-in to how people think and process information.

It’s all about reframing the debate, to borrow Lakoff’s terminology. The right has been so good at making a self-contained argument into a single catch-phrase–such as “Clear Skies,” “Healthy Trees,” “partial-birth abortion,” and “death tax”–that makes it easy for people to digest, so that they feel that they understand an issue just from the turn of phrase. We in the left need to follow suit, to find the self contained arguments in the clever turn of phrase. I’m going to do some more thinking on this, to find other ways to reframe the debate.

For too long, the right has defined the terms of the debate, and as the left has responded to them, they have lost, because you can’t refute them. We need to reframe the debate by defining the terms and forcing the right to respond to those terms. The only way this can be done is if we use a few really good ones for core issues, and get them into widespread use, repeated over and over, for the next four years. We need a daily talking points memo to outline the discourse and organize the left, to get us all “on message,” so that people get the message.

Think you’re tired of the phrase “flip-flopper”? Get ready to hear more about our slash and spend president. Ad nauseam.

Looking for the comments? Try the old layout.

enblogment: For Kerry

Following Lawrence Lessig’s lead, I am pleased to endorse John Kerry for President of the United States of America. In fact, like most Oregonians, I’ve already voted by mail. Have you?<

I could say a lot of things about why I support Kerry. But what it really all comes down to is how incredibly well-qualified he is for the job. Looking back over the last few presidents, I think that he is perhaps the best qualified since Richard Nixon–and he comes without all of Nixon’s problems. Ultimately, however, I couldn’t make the argument better than it has already been made, ad nauseam. So I’ll just point to the endorsements that make the case so powerfully that I feel that little more needs be said.

Looking for the comments? Try the old layout.

SVN::Notify 2.41 Adds Plain Text Issue Tracking Links

I expect that this will be my last release of SVN::Notify for a while. I’ve already spent more time on it than I had anticipated. But anyway, this is a pretty solid release. It doesn’t change the API or anything, but I feel that the jump from 2.30 to 2.40 is justified because of the sheer number of changes. From now on, I expect that it will mostly be maintenance, like 2.41, which fixes a minor formatting bug. Grab it now from CPAN.

First, I’ve added a new, complex example of the SVN::Notify::HTML::ColorDiff output that I will keep up-to-date with all future changes. This will allow people to get a better idea of what it’s capable of than my previous contrived examples allowed.

The biggest change is that I’ve moved the Request Tracker, Bugzilla, and JIRA support from SVN::Notify::HTML to SVN::Notify. I realized, after the release of 2.30, that it might be cool to add links to the text-only email message generated by SVN::Notify, too. So I’ve done that, including for ViewCVS links. Unlike in SVN::Notify::HTML, the links won’t be inline in the message (that doesn’t work too well in plain text, IMO), but will come in their own sections after the message. So you’ll get something like this (extreme example):

Log Message:
-----------
Let's try a few links to other applications. First, we have
A Bugzilla Bug # 709. Then we have a JIRA key, TST-1608. And
finally, we have an RT link to Ticket # 4321.

Hey, we could add one to ViewCVS for a Subversion Revision
#606, too!

ViewCVS Links:
-------------
    http://viewsvn.bricolage.cc/?rev=606&view=rev

Bugzilla Links:
--------------
    http://bugzilla.mozilla.org/show_bug.cgi?id=709

RT Links:
--------
    http://rt.cpan.org/NoAuth/Bugs.html?id=4321

JIRA Links:
----------
    http://jira.atlassian.com/secure/ViewIssue.jspa?key=TST-1608

The nice thing is that, for many mail clients, these will be turned into clickable links. You’ll also notice that the text that creates the ViewCVS link is split over two lines. This is new in this release, and works for SVN::Notify::HTML, too. I made a few other tweaks to the regular expressions, as well. Here’s a complete list of changes:

  • Fixed accessor generation so that accessors created for the attributes passed to register_attributes() but a subclass are created in the subclass’ package instead of in SVN::Notify.
  • Changed parsing for JIRA keys to use any set of capital letters followed by a dash and then a number, rather than the literal string “JIRA-” followed by a number. Reported by Garrett Rooney.
  • Modified the regular expression patterns for the RT, Bugzilla, RT, and ViewCVS links to properly match on word boundaries, so that strings like “humbug 12” don’t match.
  • Modified the ViewCVS link regular expression pattern so that it matches strings like “rev 12” as well as “revision 12”.
  • Modified the RT link regular expression pattern so that it matches strings like “RT-Ticket: 23” as well as “Ticket 1234”. Suggested by Jesse Vincent.
  • Added complicated example to try to show off all of the major features. I will keep this up-to-date going forward in order to post sample output on the Web.
  • Fixed the parsing of log messages so that empty lines are no longer eliminated.
  • HTML::ColorDiff now properly handles the listing of binary files in the diff, marking them with a new class, “binary”, and using the same CSS as is used for the “propset” class.
  • In HTML::ColorDiff, Fixed CSS for the “delfile” class to properly wrap it in a border like the other files in the diff.
  • Added labels to the HTML::ColorDiff diff file sections to indicate the type of change (“Modified”, “Added”, “Deleted”, or “Property changes”).
  • Moved the rt_url, bugzilla_url, and jira_url parameters from SVN::Notify::HTML to SVN::Notify, where they are used to add URLs to the text version of log messages.

Enjoy!

Looking for the comments? Try the old layout.

SVN::Notify 2.30 Adds Issue Tracking Links

I released a new version of SVN::Notify last night, 2.30. This new version has a few things going for it.

First, and most obviously from the point of view of users of the HTML subclass, I’ve added new options for specifying Request Tracker, Bugzilla, and JIRA URLs. The --rt-url, --bugzilla-url, and --jira-url options have an effect much like the parallel feature in CVSspam: pass in a string with the spot for the ID represented by %s, such as http://rt.cpan.org/NoAuth/Bugs.html?id=%s for RT or http://bugzilla.mozilla.org/show_bug.cgi?id=%s for Bugzilla. SVN::Notify::HTML will then look for the appropriate strings (such as “Ticket # 1234” for RT or “Bug # 4321” for Bugzilla) and turn them into URLs.

This functionality has been extended to the old --viewcvs-url option, to. For the sake of consistency, it now also requires a URL of the same form (although if SVN::Notify doesn’t see %s in the string, it will append a default and emit a warning), and will be used to create links for strings like “Revision 654” in the log message.

SVN::Notify::HTML has an additional new option, --linkize, that will force any email addresses or URLs it finds in the log message to be turned into links. Again, this works like it does for CVSspam; I’m grateful to Jeffrey Friedl’s Mastering Regular Expressions, Second Edition for the excellent regular expressions for matching URLs and email addresses.

All of this was made possible by moving the processing of options from svnnotify to SVN::Notify->get_options and adding a new class method, SVN::Notify->register_attributes. This second method allows Bricolage subclasses to easily add new attributes; register_attributes() will create accessor methods and add command-line option processing for each new attribute required by a subclass. Then, when you execute svnnotify --handler HTML, SVN::Notify->get_options processes the default options, loads the SVN::Notify::Handler subclass, and then processes any options specified by the subclass. The short story is that all of this is the detail-oriented way of saying that it is easier to subclass SVN::Notify and be able to automatically load the necessary options and attributes via the same executable, svnnotify.

This change was motivated not only by my desire to add the new features to SVN::Notify::HTML, but also by Autrijus’ new modules, SVN::Notify::Snapshot and SVN::Notify::Config. Thanks Autrijus!

I’ll try to get a nice example of all this functionality up in the next few days; if anyone else creates one first, send it to me! But in the meantime, enjoy!

Looking for the comments? Try the old layout.

The Teenage Brain

Overheard:

Steven: Did you see that article in the Sunday Times about the teenage brain?

Julie: What, you mean they found one?

Looking for the comments? Try the old layout.

SVN::Notify 2.22 Improves Diff Parsing

I released SVN::Notify 2.22 last night. The new version fixes a few issues in the parsing of diffs in the HTML subclasses. SVN::Notify::HTML now properly identifies added, deleted, and property setting sections of an included diff file when creating IDs. The lists of the affected files near the top of the email links down into the diff, and now also includes links to the locations in the diff for files that have had only their properties changed.

SVN::Notify::HTML::ColorDiff had similar updates. It now properly outputs added and deleted files in the diff in separate sections, instead of grouping them under the last modified file listed. It also creates separate sections for files that have only had their properties changed. I’ve put an example here.

Grab the new version from CPAN now

Looking for the comments? Try the old layout.

How to Determine Your Transaction ID

Today I had reason to find out what PostgreSQL transaction I was in the middle of at any given moment in Bricolage. Why? I wanted to make sure that a single request was generating multiple transactions, instead of the normal one. It’s a long story, but suffice it to say that lengthy transactions were causing deadlocks. Read this if you’re really interested.

Anyway, here’s how to determine your current transaction using DBI. The query will be the same for any client, of course.

my $sth = $dbh->prepare(qq{
    SELECT transaction
    FROM   pg_locks
    WHERE  pid = pg_backend_pid()
            AND transaction IS NOT NULL
    LIMIT  1
});

$sth->execute;
$sth->bind_columns(\my $txid);
$sth->fetch;
print "Transaction: $txid\n";

Looking for the comments? Try the old layout.

SVN::Notify 2.20 Adds Colorized Diffs

After getting prodded by Erik Hatcher, I went ahead and added another subclass to SVN::Notify. This one adds a pretty colorized diff to the message, instead of just the plain text one. See an example here. I’ve also added links from the lists of affected files into the diffs in the HTML and new HTML::ColorDiff layouts.

Enjoy!

Update: And now I’ve released SVN::Notify 2.21 with a few minor fixes, including XHTML 1.1 compliance.

Looking for the comments? Try the old layout.