Just a Theory

By David E. Wheeler

Posts about PHP

Doomed To Reinvent

There’s an old saying, “Whoever doesn’t understand X is doomed to reinvent it.”X can stand for any number of things. The other day, I was pointing out that such is the case for ORM developers. Take ActiveRecord, for example. As I demonstrated in a 2007 Presentation, because ActiveRecord doesn’t support simple things like aggregates or querying against functions or changing how objects are identified, you have to fall back on using its find_by_sql() method to actually run the SQL, or using fuck typing to force ActiveRecord to do what you want. There are only two ways to get around this: Abandon the ORM and just use SQL, or keep improving the ORM until it has, in effect, reinvented SQL. Which would you choose?

I was thinking about this as I was hacking on a Drupal installation for a client. The design spec called for the comment form to be styled in a very specific way, with image submit buttons. Drupal has this baroque interface for building forms: essentially an array of arrays. Each element of the array is a form element, unless it’s markup. Or something. I can’t really make heads or tails of it. What’s important is that there are a limited number of form elements you can create, and as of Drupal 5, *image* isn’t fucking one of them!.

Now, as a software developer, I can understand this. I sometimes overlook a feature when implementing some code. But the trouble is: why have some bizarre data structure to represent a subset of HTML when you have something that already works: it’s called HTML. Drupal, it seems, is doomed to reinvent HTML.

So just as I have often had to use find_by_sql() as the fallback to get ActiveRecord to fetch the data I want, as opposed to what it thinks I want, I had to fallback on the Drupal form data structure’s ability to accept embedded HTML like so:

$form['submit_stuff'] = array(
  '#weight' => 20,
  '#type'   => 'markup',
  '#value'  => '<div class="form-submits">'
              . '<label></label><p class="message">(Maximum 3000 characters)</p>'
              . '<div class="btns">'
              . '<input type="image" value="Preview comment" name="op" src="preview.png" />'
              . '<img width="1" height="23" src="divider.png" />'
              . '<input type="image" value="Post comment" name="op" src="post.png" />'
              . '</div></div>',
);

Dear god, why? I understand that you can create images using an array in Drupal 6, but I fail to understand why it was ever a problem. Just give me a templating environment where I can write the fucking HTML myself. Actually, Drupal already has one, it’s called PHP!. Please don’t make me deal with this weird hierarchy of arrays, it’s just a bad reimplementation of a subset of HTML.

I expect that there actually is some way to get what I want, even in Drupal 5, as I’m doing some templating for comments and pages and whatnot. But that should be the default IMHO. The weird combining of code and markup into this hydra-headed data structure (and don’t even get me started on the need for the #weight key to get things where I want them) is just so unnecessary.

In short, if it ain’t broke, don’t reinvent it!

</rant>

Looking for the comments? Try the old layout.

pgTAP 0.10 Released, Web Site Launched

Two major announcements this week with regard to pgTAP:

First, I’ve release pgTAP 0.10. The two major categories of changes are compatibility as far back as PostgreSQL 8.0 and new functions for testing database schemas. Here’s a quick example:

BEGIN;
SELECT plan(7);

SELECT has_table( 'users' );
SELECT has_pk('users');
SELECT col_is_fk( 'users', ARRAY[ 'family_name', 'given_name' ]);

SELECT has_table( 'widgets' );
SELECT has_pk( 'widgets' );
SLEECT col_is_pk( 'widgets', 'id' );
SELECT fk_ok(
    'widgets',
    ARRAY[ 'user_family_name', 'user_given_name' ],
    'users',
    ARRAY[ 'family_name', 'given_name' ],
);

SELECT * FROM finish();
ROLLBACK;

Pretty cool, right? Check the documentation for all the details.

Speaking of the documentation, that link goes to the new pgTAP Web site. Not only does it include the complete documentation for pgTAP, but also instructions for integrating pgTAP into your application’s preferred test environment. Right now it includes detailed instructions for Perl + Module::Build and for PostgreSQL, but has only placeholders for PHP and Python. Send me the details on those languages or any others into which you integrate pgTAP tests and I’ll update the page.

Oh, and it has a beer. Enjoy.

I think I’ll take a little time off from pgTAP next week to give Bricolage some much-needed love. But as I’ll be given another talk on pgTAP at PostgreSQL Conference West next month, worry not! I’ll be doing a lot more with pgTAP in the coming weeks.

Oh, and one more thing: I’m looking for consulting work. Give me a shout (david - at - justatheory.com) if you have some PostgreSQL, Perl, Ruby, MySQL, or JavaScript hacking you’d like me to do. I’m free through November.

That is all.

Looking for the comments? Try the old layout.

Windows Virus Hell

So to finish up development and testing of Test.Harness.Browser in IE 6 last week, I rebooted my Linux server (the one running justatheory.com) into Windows 98, got everything working, and rebooted back into Linux. I felt that the hour or two’s worth of downtime for my site was worth it to get the new version of Test.Simple out, and although I had ordered a new Dell, didn’t want to wait for it. And it worked great; I’m very pleased with Test.Simple 0.20.

But then, in unrelated news, I released Bricolage 1.9.0, the first development release towards Bricolage 1.10, which I expect to ship next month. One of the things I’m most excited about in this release is the new PHP templating support. So on George Schlossnagle’s advice, I sent an email to webmaster@php.net. It bounced. It was late on Friday, and I’m so used to bounces being problems on the receiving end, that I simply forwarded it to George with the comment, “What the?” and went to fix dinner for company.

Then this morning I asked George, via IM, if he’d received my email. He hadn’t. I sent it again; no dice. So he asked me to paste the bounce, and as I did so, looked at it more carefully. It had this important tidbit that I’d failed to notice before:

140.211.166.39 failed after I sent the message.
Remote host said: 550-5.7.1 reject content [xbl]
550 See http://master.php.net/mail/why.php?why=SURBL

“That’s curious,” I thought, and went to read the page in question. It said I likely had a domain name in my email associated with a blacklisted IP address. Well, there were only two domain names in that email, bricolage.cc and justatheory.com, and I couldn’t see how either one of them could have been identified as a virus host. But sure enough, a quick search of the CBL database revealed that the IP address for justatheory.com—and therefore my entire home LAN— had been blacklisted. I couldn’t imagine why; at first I thought maybe it was because of past instances of blog spam appearing here, but then George pointed out that the listing had been added on August 18. So I thought back…and realized that was just when I was engaging in my JavaScript debugging exercise.

Bloody Windows!

So I took steps to correct the problem:

  1. Update my router’s firmware. I’ve been meaning to do that for a while, anyway, and was hoping to get some new firewall features. Alas, no, but maybe I’ll be able to connect to a virtual PPTP network the next time I need to.

  2. Blocked all outgoing traffic from any computer on my LAN on port 25. I send email through my ISP, but use port 587 because I found in the last year that I couldn’t send mail on port 25 on some networks I’ve visited (such as in hotels). Now I know why: so that no network users inadvertently send out viruses from their Windows boxes! I’d rather just prevent certain hosts (my Windows boxen) from sending on port 25, but the router’s NAT is not that sophisticated. So I have to block them all.

  3. Rebooted the server back into Windows 98 and installed and ran Norton AntiVirus. This took forever, but found and fixed two instances of WIN32Mimail.l@mm and removed a spyware package.

  4. Rebooted back into Linux and cleared my IP address from the blacklist databases. I don’t expect to ever use that box for Windows again, now that I have the new Dimension.

The new box comes with Windows XP SP 2 and the Symantec tools, so I don’t expect it to be a problem, especially since it can’t use port 25. But this is a PITA, and I really feel for the IT departments that have to deal with this shit day in and day out.

What I don’t understand is how I got this virus, since I haven’t used Windows 98 in this computer in a long time. How long? Here’s a clue: When I clicked the link in Norton AntiVirus to see more information on WIN32Mimail.l@mm, Windows launched my default browser: Netscape Communicator! In addition, I don’t think I’ve used this box to check email since around 2000, and I never click on attachments from unknown senders, and never .exe or .scr files at all (my mail server automatically rejects incoming mail with such attachments, and has for at least a year).

But anyway, it’s all cleaned up now, and I’ve un-blacklisted my IP, so my emails should be deliverable again. But I’m left wondering what can be done about this problem. It’s easy for me to feel safe using my Mac, Linux, and FreeBSD boxes, but, really, what keeps the Virus and worm writers from targeting them? Nothing, right? Furthermore, what’s to stop the virus and worm writers from using port 587 to send their emails? Nothing, right? Once they do start using 587—and I’m sure they will—how will anyone be able to send mail to an SMTP server on one network from another network? Because you know that once 587 becomes a problem, network admins will shut down that port, too.

So what’s to be done about this? How can one successfully send mail to a server not on your local network? How will business people be able to send email through their corporate servers from hotel networks? I can see only a few options:

  • Require them to use a mail server on the local network. They’ll have to reconfigure their mail client to use it, and then change it back when they get back to the office. What a PITA. This might work out all right if there was some sort of DNS-like service for SMTP servers, but then there would then be nothing to prevent the virus software from using it, either.
  • You can’t. You have to authenticate onto the other network using a VPN. Lots of companies rely on this approach already, but smaller companies that don’t have the IT resources to set up a VPN are SOL. And folks just using their ISPs are screwed, too.
  • Create a new email protocol that’s inherently secure. This would require a different port, some sort of negotiation and authentication process, and a way for the hosting network to know that it’s cool to use. But this probably wouldn’t work, either, because then the virus software can also connect via such a protocol to a server that’s friendly to it, right?

None of these answers is satisfactory. I guess I’ll have to set up an authenticating SMTP server and a VPN for Kineticode once port 587 starts getting blocked. Anyone else got any brilliant solutions to this problem?

Looking for the comments? Try the old layout.

Bricolage Now has PHP 5 Templating

Well, I’ve finally gone and done it. I’ve released Bricolage 1.9.0, the first development release towards 1.10.0, which I plan to get out sometime next month. Among the new features in this release that I’m most excited about are a revamped UI, LDAP authentication, and PHP 5 templating support.

The new UI is really nice. Marshall Roch ported it all to XHTML 1.0 strict plus CSS. All the layout is done with CSS now, instead of the old 1999-era table layouts we had before. The result is much smaller page loads, sometimes up to 70% smaller, Marshall tells me, and therefore much faster loads. Playing around with the new version (now powering the Kineticode and Bricolage Web sites, I do indeed find it to be a lot zippier. Couple the new UI with the new mod_gzip support and static Mason components, and things just perk right along! Oh, and the expandable interfaces for both lists of objects and for editing interfaces is very nice, indeed. Thank you Marshall for the great work!

Kineticode added the LDAP authentication support. We’ve been using a patch for it against 1.8.x for our internal version of Bricolage and it works great. Now we authenticate off of an OpenLDAP server, using the same usernames and passwords we use for our email, Subversion, and RT servers. The configuration is simple, via bricolage.conf directives, and you can even limit users who authenticate to members of a particular LDAP group. Users must exist in Bricolage, and if LDAP authentication fails, you can fall back on Bricolage’s internal authentication. I’m hoping that the LDAP support goes a long way towards attracting more enterprise customers with single sign-on requirements.

And finally—yes, you heard right—Bricolage now supports PHP 5 templating in addition to the existing Perl-based templating architectures (Mason, Template Toolkit, and HTML::Template). So how did we add PHP 5 templating to a mod_perl application? Easy: we hired George Schlossnagle of Omni TI to write PHP::Interpreter, an embedded PHP 5 interpreter. Now anyone can natively execute PHP 5 code from a Perl application. Not only that, but the PHP 5 code can reach back into the Perl interpreter to use Perl modules and objects! Here’s an example that I like to show off to the PHP crowd:

<?php
    $perl = Perl::getInstance();
    $perl->eval("use DBI");
    $perl->eval("use DateTime");
    $dbh = $perl->call("DBI::connect", "DBI", "dbi:SQLite:dbname=dbfile");
    $dbh->do("CREATE TABLE foo (bar TEXT, time DATETIME)");
    $now = $perl->call("DateTime::now", "DateTime");
    $ins = $dbh->prepare("INSERT INTO foo VALUES (?, ?)");
    $ins->execute("This is a test", $now);
    $sel = $dbh->prepare("SELECT bar, time FROM foo");
    $sel->execute();
    $a = array("foo", "bar");
    foreach ($sel->fetch() as $val) {
        echo "$val\n";
    }
    $sel->finish();
    $dbh->do("DROP TABLE foo");
    $dbh->disconnect();
?>

Note that George plans to add convenience methods to load Perl modules and call Perl class methods. Now, to execute this code from Perl, all you have to do is write a little script, call it pphp, like so:

use strict;
use PHP::Interpreter;
my $php = PHP::Interpreter->new;
$php->include(shift);

Then just execute your PHP code with the script: pphp try.php. Yes, this does work! For years, when I’ve run across a PHP coder who wanted to try to tell me that PHP was better than Perl, I always had a one-word reply that left him cursing and conceding defeat: “CPAN.” Well no more. Now PHP hackers can use any module on CPAN, too!

And as for Bricolage, the integration of PHP 5 templating is completely transparent. Users just write PHP 5 templates instead of Mason templates and that’s it! For example, this is a fairly common style Bricolage Mason template:

<%perl>;
for my $e ($element->get_elements(qw(header para _pull_quote_))) {
    my $kn = $e->get_key_name;
    if ($kn eq "para") {
        $m->print("<p>", $e->get_data, "</p>\n");
    } elsif ($kn eq "header") {
        # Test sdisplay_element() on a field.
        $m->print("<h3>", $burner->sdisplay_element($e), "</h3>\n");
    } elsif ($kn eq "_pull_quote_" && $e->get_object_order > 1) {
        # Test sdisplay_element() on a container.
        $m->print($burner->sdisplay_element($e));
    } else {
        # Test display_element().
        $burner->display_element($e);
    }
}
$burner->display_pages("_page_");
</%perl>

The same template in PHP 5 looks like this:

<?php
    # Convenience variables.
    $story   = $BRIC["story"];
    $element = $BRIC["element"];
    $burner  = $BRIC["burner"];
    foreach ($element->get_elements("header", "para", "_pull_quote_") as $e) {
        $kn = $e->get_key_name();
        if ($kn == "para") {
            echo "<p>", $e->get_data(), "</p>\n";
        } else if ($kn == "header") {
            # Test sdisplay_element() on a field.
            echo "<h3>", $burner->sdisplay_element($e), "</h3>\n";
        } else if ($kn == "_pull_quote_" && $e->get_object_order() > 1) {
            # Test sdisplay_element() on a container.
            echo $burner->sdisplay_element($e);
        } else {
            # Test display_element().
            $burner->display_element($e);
        }
    }
    $burner->display_pages("_page_");
?>

Yes, you are seeing virtually the same thing. But this is just a simple template from Bricolage’s test suite. The advantage is that PHP 5 coders who are familiar with all the ins and outs of PHP 5 can just jump in an get started writing Bricolage templates without having to learn any Perl! The Bricolage objects have exactly the same API as they do in Perl, because they are exactly the same objects! So everyone uses the same API documentation for the same tasks. The only issue I’ve noticed so far is that PHP 5 does not yet have proper Unicode support. Since all content in Bricolage is stored as UTF-8, this means that the PHP 5 templates must treat it as binary data. But this is okay as long as templaters use the mb_* PHP 5 functions to parse text.

Overall I’m very excited about this, and hope that it helps Bricolage to reach a whole new community of users. I’d like to thank Portugal Telecom—SAPO.pt for sponsoring the development of PHP::Interpreter and its integration into Bricolage. I believe that they’ve really done the Bricolage community a great service, and I hope that the Perl and PHP communities likewise benefit from the integration possible with PHP::Interpreter.

And just so that the other templating architectures don’t feel left out, here is how the above template looks in Template Toolkit:

[% FOREACH e = element.get_elements("header", "para", "_pull_quote_") %]
    [% kn = e.get_key_name %]
    [% IF kn == "para" %]
<p>[% e.get_data %]</p>
    [% ELSIF kn == "header" %]
        [% # display_element() should just return a value. %]
<h3>[% burner.display_element(e) %]</h3>
    [% ELSIF kn == "_pull_quote_" && e.get_object_order > 1 %]
        [% PERL %]
          # There is no sdisplay_element() in the TT burner, but we"ll just
          # Play with it, anyway.
          print $stash->get("burner")->display_element($stash->get("e"));
        [% END %]
    [% ELSE %]
        [% # Test display_element(). %]
        [% burner.display_element(e) %]
    [% END %]
[% END %]
[% burner.display_pages("_page_") %]

And here it is in HTML::Template:

<tmpl_if _page__loop>
<tmpl_loop _page__loop>
<tmpl_include name="testing/sub/util.tmpl">
<tmpl_var _page_>
<tmpl_var page_break>
</tmpl_loop>
<tmpl_else>
<tmpl_include name="testing/sub/util.tmpl">
</tmpl_if>

I had to use the utility template with HTML::Template to get it to work right, don’t ask why. It looks like this:

<tmpl_loop element_loop>
<tmpl_if is_para>
<p><tmpl_var para></p>
</tmpl_if>
<tmpl_if is_header>
<h3><tmpl_var header></h3>
</tmpl_if>
<tmpl_if is__pull_quote_>
<tmpl_var _pull_quote_>
</tmpl_if>
</tmpl_loop>

These templates come from the Bricolage test suite, where until this release, there were never any template tests before. So you can see that the PHP 5 templating initiative has had major benefits for the stability of Bricolage, too. Now that I’ve really worked with all four templating architectures in Bricolage, I can now say that my preference for which to do goes in this order:

  1. Mason, because of its killer autohandler and inheritance architecture
  2. PHP 5 or Template Toolkit are tied for second place
  3. HTML::Template

In truth, all four are capable and have access to the entire Bricolage API so that they can output anything. So what are you waiting for? Download Bricolage and give it a try!

Looking for the comments? Try the old layout.