Just a Theory

By David E. Wheeler

Posts about Windows

Testing Perl Projects on Travis Windows

A few months ago, Travis CI announced early access for a Windows build environment. In the last couple weeks, I spent some time to figure out how to test Perl projects there by installing Strawberry Perl from Chocolatey.

The result is the the sample project winperl-travis. It demonstrates three .travis.yml configurations to test Perl projects on Windows:

  1. Use Windows instead of Linux to test multiple versions of Perl. This is the simplest configuration, but useful only for projects that never expect to run on a Unix-style OS.
  2. Add a Windows build stage that runs the tests against the latest version of Strawberry Perl. This pattern is ideal for projects that already test against multiple versions of Perl on Linux, and just want to make sure things work on windows.
  3. Add a build stage that tests against multiple versions of Strawberry Perl in separate jobs.

See the results of each of the three approaches in the CI build. A peek:

winperl-travis CI build results

The Travis CI-default “Test” stage is the default, and runs tests on two versions of Perl on Windows. The “Windows” stage tests on a single version of Windows Perl, independent of the “Test” stage. And the “Strawberry” stage tests on multiple versions of Windows Perl independent of the “Test” stage.

If, like me, you just want to validate that your Perl project builds and its tests pass on Windows (option 2), I adopted the formula in text-markup project. The complete .travis.yml:

language: perl
perl:
  - "5.28"
  - "5.26"
  - "5.24"
  - "5.22"
  - "5.20"
  - "5.18"
  - "5.16"
  - "5.14"
  - "5.12"
  - "5.10"
  - "5.8"

before_install:
  - sudo pip install docutils
  - sudo apt-get install asciidoc
  - eval $(curl https://travis-perl.github.io/init) --auto

jobs:
  include:
    - stage: Windows
      os: windows
      language: shell
      before_install:
        - cinst -y strawberryperl
        - export "PATH=/c/Strawberry/perl/site/bin:/c/Strawberry/perl/bin:/c/Strawberry/c/bin:$PATH"
      install:
        - cpanm --notest --installdeps .
      script:
        - cpanm -v --test-only .

The files starts with the typical Travis Perl configuration: select the language (Perl) and the versions to test. The before_install block installs a couple of dependencies and executes the travis-perl helper for more flexible Perl testing. This pattern practically serves as boilerplate for new Perl projects.

The new bit is the jobs.include section, which declares a new build stage named “Windows”. This stage runs independent of the default phase, which runs on Linux, and declares os: windows to run on Windows.

The before_install step uses the pre-installed Chocolatey package manager to install the latest version of Strawberry Perl and update the $PATH environment variable to include the paths to Perl and build tools. Note that the Travis CI Window environment runs inside the Git Bash shell environment; hence the Unix-style path configuration.

The install phase installs all dependencies for the project via cpanminus, then the script phase runs the tests, again using cpanminus.

And with the stage set, the text-markup build has a nice new stage that ensures all tests pass on Windows.

The use of cpanminus, which ships with Strawberry Perl, keeps things simple, and is essential for installing dependencies. But projects can also perform the usual gmake test1 or perl Build.PL && ./Build test dance. Install [Dist::Zilla] via cpanminus to manage dzil-based projects. Sadly, prove currently does not work under Git Bash2.

Perhaps Travis will add full Perl support and things will become even easier. In the meantime, I’m pleased that I no longer have to guess about Windows compatibility. The new Travis Windows environment enables a welcome increase in cross-platform confidence.


  1. Although versions of Strawberry Perl prior to 5.26 have trouble installing Makefile.PL-based modules, including dependencies. I spent a fair bit of time trying to work out how to make it work, but ran out of steam. See issue #1 for details.

  2. I worked around this issue for Sqitch by simply adding a copy of prove to the repository.

Sqitch on Windows (and Linux, Solaris, and OS X)

Thanks to the hard-working hamsters at the ActiveState PPM Index, Sqitch is available for installation on Windows. According to the Sqitch PPM Build Status, the latest version is now available for installation. All you have to do is:

  1. Download and install ActivePerl
  2. Open the Command Prompt
  3. Type ppm install App-Sqitch

As of this writing, only PostgreSQL is supported, so you will need to install PostgreSQL.

But otherwise, that’s it. In fact, this incantation works for any OS that ActivePerl supports. Here’s where you can find the sqitch executable on each:

  • Windows: C:\perl\site\bin\sqitch.bat
  • Mac OS X: ~/Library/ActivePerl-5.16/site/bin/sqitch (Or /usr/local/ActivePerl-5.16/site/bin if you run sudo ppm)
  • Linux: /opt/ActivePerl-5.16/site/bin/sqitch
  • Solaris/SPARC (Business edition-only): /opt/ActivePerl-5.16/site/bin/sqitch

This makes it easy to get started with Sqitch on any of those platforms without having to become a Perl expert. So go for it, and then get started with the tutorial!

Looking for the comments? Try the old layout.

SVN::Notify 2.57 Supports Windows

So I finally got ‘round to porting SVN::Notify to Windows. Version 2.57 is making is way to CPAN right now. The solution turned out to be dead simple: I just had to use a different form of piping open() on Windows, i.e., open FH, "$cmd|" instead of open FH, "-|"; exec($cmd);. It’s silly, really, but it works. It really makes me wonder why -| and |- haven’t been emulated on Windows. Whatever.

‘Course the other thing I realized, after I made this change and all the tests pass, was that there is no equivalent of sendmail on Windows. So I added the --smtp option, so that now email can be sent to an SMTP server rather than to a local sendmail. I tested it out, and it seems to work, but I’d be especially interested to hear from folks using wide characters in their repositories: do they get printed properly to Net::SMTP’s connection?

The whole list of changes in 2.57 (the output remains the same as in 2.56):

  • Finally ported to Win32. It was actually a simple matter of changing how command pipes are created.
  • Added --smtp option to enable sending messages to an SMTP server rather than to the local sendmail application. This is essential for Windows support.
  • Added --io-layer to the usage statement in svnnotify.
  • Fixed single-dash arguments in documentation so that they’re all documented with a single dash in SVN::Notify.

Enjoy!

Looking for the comments? Try the old layout.

Port SVN::Notify to Windows

So SVN::Notify doesn’t currently run on Windows. Why not? Well, because I wanted to do things as “rightly” as possible. In terms of efficiency, what that meant was, rather than slurping in whole chunks of data, such as diffs, from svnlook, I instead follows the guidance in perlipc to open a file handle pipe to svnlook and then read from it line-by-line. The method I wrote to create the pipe looks like this:

sub _pipe {
    my ($self, $mode) = (shift, shift);
    # Safer version of backtick (see perlipc(1)).
    local *PIPE;
    my $pid = open(PIPE, $mode);
    die "Cannot fork: $!\n" unless defined $pid;

    if ($pid) {
        # Parent process. Return the file handle.
        return *PIPE;
    } else {
        # Child process. Execute the commands.
        exec(@_) or die "Cannot exec $_[0]: $!\n";
        # Not reached.
    }
}

The problem is that it doesn’t work on Windows. perlipc says:

Note that these operations are full Unix forks, which means they may not be correctly implemented on alien systems. Additionally, these are not true multithreading. If you’d like to learn more about threading, see the modules file mentioned below in the SEE ALSO section.

‘Course, the SEE ALSO section doesn’t have much of for “alien systems,” but I have a comment in my code that suggests that Win32::Process might do for Windows compatibility. But I honestly don’t know.

So what’s the best approach for me to port SVN::Notify to Windows while keeping file handle pipes around for efficiency? Anyone care to take a stab at it, with tests for Winows, and send me a patch?

Looking for the comments? Try the old layout.

No UTF-8 Support on Windows?

This just blows. It will be a while before Bricolage runs on Windows, then. The PostgreSQL team is understandably reluctant to simply include the whole ICU library in PostgreSQL. Maybe it could be compiled into the binaries, though?

Looking for the comments? Try the old layout.

More about…

Windows Virus Hell

So to finish up development and testing of Test.Harness.Browser in IE 6 last week, I rebooted my Linux server (the one running justatheory.com) into Windows 98, got everything working, and rebooted back into Linux. I felt that the hour or two’s worth of downtime for my site was worth it to get the new version of Test.Simple out, and although I had ordered a new Dell, didn’t want to wait for it. And it worked great; I’m very pleased with Test.Simple 0.20.

But then, in unrelated news, I released Bricolage 1.9.0, the first development release towards Bricolage 1.10, which I expect to ship next month. One of the things I’m most excited about in this release is the new PHP templating support. So on George Schlossnagle’s advice, I sent an email to webmaster@php.net. It bounced. It was late on Friday, and I’m so used to bounces being problems on the receiving end, that I simply forwarded it to George with the comment, “What the?” and went to fix dinner for company.

Then this morning I asked George, via IM, if he’d received my email. He hadn’t. I sent it again; no dice. So he asked me to paste the bounce, and as I did so, looked at it more carefully. It had this important tidbit that I’d failed to notice before:

140.211.166.39 failed after I sent the message.
Remote host said: 550-5.7.1 reject content [xbl]
550 See http://master.php.net/mail/why.php?why=SURBL

“That’s curious,” I thought, and went to read the page in question. It said I likely had a domain name in my email associated with a blacklisted IP address. Well, there were only two domain names in that email, bricolage.cc and justatheory.com, and I couldn’t see how either one of them could have been identified as a virus host. But sure enough, a quick search of the CBL database revealed that the IP address for justatheory.com—and therefore my entire home LAN— had been blacklisted. I couldn’t imagine why; at first I thought maybe it was because of past instances of blog spam appearing here, but then George pointed out that the listing had been added on August 18. So I thought back…and realized that was just when I was engaging in my JavaScript debugging exercise.

Bloody Windows!

So I took steps to correct the problem:

  1. Update my router’s firmware. I’ve been meaning to do that for a while, anyway, and was hoping to get some new firewall features. Alas, no, but maybe I’ll be able to connect to a virtual PPTP network the next time I need to.

  2. Blocked all outgoing traffic from any computer on my LAN on port 25. I send email through my ISP, but use port 587 because I found in the last year that I couldn’t send mail on port 25 on some networks I’ve visited (such as in hotels). Now I know why: so that no network users inadvertently send out viruses from their Windows boxes! I’d rather just prevent certain hosts (my Windows boxen) from sending on port 25, but the router’s NAT is not that sophisticated. So I have to block them all.

  3. Rebooted the server back into Windows 98 and installed and ran Norton AntiVirus. This took forever, but found and fixed two instances of WIN32Mimail.l@mm and removed a spyware package.

  4. Rebooted back into Linux and cleared my IP address from the blacklist databases. I don’t expect to ever use that box for Windows again, now that I have the new Dimension.

The new box comes with Windows XP SP 2 and the Symantec tools, so I don’t expect it to be a problem, especially since it can’t use port 25. But this is a PITA, and I really feel for the IT departments that have to deal with this shit day in and day out.

What I don’t understand is how I got this virus, since I haven’t used Windows 98 in this computer in a long time. How long? Here’s a clue: When I clicked the link in Norton AntiVirus to see more information on WIN32Mimail.l@mm, Windows launched my default browser: Netscape Communicator! In addition, I don’t think I’ve used this box to check email since around 2000, and I never click on attachments from unknown senders, and never .exe or .scr files at all (my mail server automatically rejects incoming mail with such attachments, and has for at least a year).

But anyway, it’s all cleaned up now, and I’ve un-blacklisted my IP, so my emails should be deliverable again. But I’m left wondering what can be done about this problem. It’s easy for me to feel safe using my Mac, Linux, and FreeBSD boxes, but, really, what keeps the Virus and worm writers from targeting them? Nothing, right? Furthermore, what’s to stop the virus and worm writers from using port 587 to send their emails? Nothing, right? Once they do start using 587—and I’m sure they will—how will anyone be able to send mail to an SMTP server on one network from another network? Because you know that once 587 becomes a problem, network admins will shut down that port, too.

So what’s to be done about this? How can one successfully send mail to a server not on your local network? How will business people be able to send email through their corporate servers from hotel networks? I can see only a few options:

  • Require them to use a mail server on the local network. They’ll have to reconfigure their mail client to use it, and then change it back when they get back to the office. What a PITA. This might work out all right if there was some sort of DNS-like service for SMTP servers, but then there would then be nothing to prevent the virus software from using it, either.
  • You can’t. You have to authenticate onto the other network using a VPN. Lots of companies rely on this approach already, but smaller companies that don’t have the IT resources to set up a VPN are SOL. And folks just using their ISPs are screwed, too.
  • Create a new email protocol that’s inherently secure. This would require a different port, some sort of negotiation and authentication process, and a way for the hosting network to know that it’s cool to use. But this probably wouldn’t work, either, because then the virus software can also connect via such a protocol to a server that’s friendly to it, right?

None of these answers is satisfactory. I guess I’ll have to set up an authenticating SMTP server and a VPN for Kineticode once port 587 starts getting blocked. Anyone else got any brilliant solutions to this problem?

Looking for the comments? Try the old layout.

iPod Threatens UK Military Security

20 GB iPod

Following up on my screed against the idea of the “iPod security threat”, James Duncan Davidson sent me a link to this story about how the UK military has decided that the iPod is a security threat.

“With USB devices, if you plug it straight into the computer you can bypass passwords and get right on the system,” RAF Wing Commander Peter D’Ardenne told Reuters.

“That’s why we had to plug that gap,” he said, adding that the policy was put into effect when the MoD switched to the USB-friendly Microsoft XP operating system over the past year.

Huh. Do you mean to tell me that if you plug into the USB port of a PC that no one is logged in to, you can get access to the contents of the PC without logging in? You know, that sounds more like a Windows security flaw than an iPod problem. I mean, it’s reasonable for the military to ban external media in order to prevent their personnel and contractors from copying sensitive data onto personal devices for unknown purposes. But this Windows security hole seems, well, huge.

And the truth is that these articles that single out the iPod as a security threat are being disingenuous, in that it’s much easier and much cheaper to use a USB Flash Drive. Furthermore, this banning of storage devices really only keeps honest people honest; those who really want to copy sensitive information to take home will figure out a way to do it if they’re motivated enough.

So yeah, highly sensitive security establishments should ban personal external storage devices to keep honest people honest, but really, they should also fix the real security problem with their operating system of choice.

Looking for the comments? Try the old layout.