testing

testing

Dynamic languages and the .NET VM

There's been a lot of stuff lately about how the .NET VM isn't good for dynamic languages. Via LtU, John Udell wrote an article for InfoWorld about the issue:

The Java virtual machine wasn't designed for dynamically typed languages. Neither was the .Net virtual machine, although at first it seemed so. When Microsoft began talking about the CLR (Common Language Runtime), Perl and Python were often mentioned. The .Net Framework would be available to scripting languages as well as to the core .Net languages. This was a great idea...

All this presumes, of course, that we can get robust support for dynamic languages in the Java and .Net run times. So far, the results are not encouraging. Jython, as we've seen, succeeds by the extraordinary measure of rewriting the Python interpreter in Java. In the .Net world, early efforts to make Perl and Python into first-class .Net languages ran aground. Both projects have now retreated to a dual-run-time strategy. Scripts can produce and consume .Net interfaces but rely on their own virtual machines, not the CLR.

Is there a better solution for these environments? The designers of Parrot, the Perl 6 virtual machine, think not and have struck out on their own. "Why not compile to JVM/.Net?" the Parrot FAQ asks itself. And it answers: "Those VMs are designed for statically typed languages." That's true, sadly. It's a question of priorities, and support for dynamic languages isn't high on the list for Java or .Net.

On his weblog, John writes the following:

... the Parrot VM is not intended to support Perl only, but any dynamic language. ... It would be best, in my view, if the JVM and CLR could serve this purpose. My gut tells me the obstacles are cultural, not technical.

and

I'm no VM guru. Maybe there really is no way for the JVM and CLR virtual machines to properly support dynamic languages. But somehow, I doubt that.

Well, yesterday on Erik's weblog I noticed a link to this great post by Dan Sugalski (he actually wrote that part of the Parrot FAQ), who responds to Udell's article and explains that the reasons clearly are technical, and not cultural.

... both the JVM and .NET are perfectly capable of being target machines. They're fully turing complete, so it's not an issue of capability. But, like the Infocom Z machine, which is also turing complete, the issue is one of speed.

To do closures means capturing and maintaining persistent lexical state. Neither .NET nor the JVM have support for this.... To handle lexicals the way perl needs them means we'd have to basically ignore the system variable allocation system and do it ourselves.

The same goes for the polymorphic ... scalar that perl has. ... To do that with .NET or the JVM would require a custom type capable of doing what perl needs.

So, to make perl work means completely rewriting the system allocation scheme, and using our own custom polymorphic object type. In JVM/.NET bytecode. Doable? Sure. Fast? No way in hell.

To do continuations is non-trivial, and I don't think it's possible to do in the JVM or .NET without treating them as glorified CPUs and use none of their control and stack features. We'd essentially write all the functionality of the interpreter and target the JVM the way we do now with C and hardware CPUs, including complete stack management, completely ignoring any features at all of the VMs. I don't want to think about how slow that would go. It's bad enough doing it all in twisted, insane C targeting real hardware. Another layer of indirection would kill us dead.

He goes on to say that even if the .NET CLR and the JVM could change to support the needs of Perl and other dynamic languages that Parrot targets, "they'd be stupid to do so", because:

All features have costs associated with them, and nothing is free. You design your feature set, then the software to run it, and it's all a huge mass of tradeoffs. This feature lets you do something, but has that cost. Wanting something to be fast means something else is very slow, or effectively impossible, and sometimes two features are mostly incompatible. You make your list, make your choices, and do what you can.

So there you have it. Thanks Dan.

The .NET and Java VMs are simply not meant to support dynamic languages, and that's ok. However, all of this shows why I'm not so hot and bothered about the multi-language features of .NET. The class libraries are very cool (even if I don't like their naming conventions...), and it seems to be good technology. But the multi-language-ness of .NET just doesn't turn me on. Most development in .NET will be done in C# anyway.

Update: Check out the second part to Dan's article: The reason for Parrot, part 2. Man, if he explains continuations as clearly as he did closures, I'll finally be able to understand how they work!

Strategies for learning new languages

Via LtU, Strategies for learning new languages

When you start to learn any new language, you need to first look at the language from almost a linguistic perspective. Is it strongly typed or weakly typed? Is it statically typed or dynamically typed? Does it do garbage collection? What kind? Does it have class-based or prototype-based OO? What type of "higher order" features does it have? (Can I pass functions around as values easily?) What are its value/reference semantics?

Here's one little example of the type of hints you get from looking at a language. You look at Perl, and notice it needs to have different operators for string concatenation and numeric addition. It shows you it's has weak dynamic typing. On the other hand, you look at Ruby, and you have to explicitly convert a numeric string into a number if you want to use it as such. It has a little different mindset, even though things might look similar on the face of it. Do variables auto-vivify? Perl, yes, Python, no. Etc. etc.

So, when you know enough languages, you start to look for little things that key you off to how they work. When I started with Python, I was confused by its scoping, and then I noticed they had a "global" keyword, and I was able to immediately understand that its scoping model was similar to PHP. They've since added lexical scoping.

I've always thought C's approach to reference/value semantics is interesting to look at. Everything is passed by value, but then, some values can be pointers. Also, C doesn't have closures, but then again, all functions are statically declared in the global scope.

The problem arises when a language uses features you're not used to. While I think I understand what's going on, I'm not totally comfortable yet with Python's generators. I've only recently got my brain wrapped around multi-methods, or multiple dispatch, and with either of these features I'd really have to write some code that uses them before I could claim I really understand what is going on. I might understand conceptually how Lisp's macros work, but I've never used them. Right now I'd probably be a terrible Lisp programmer. Don't even ask me about continuations - I wish I grokked them.

One of the messages in the thread had this:

Until you've actually written at least several hundred lines of code in a language you effectively know nothing about it.

That's very true. And moreover, if you don't use a language for awhile, things don't stick. It'd take me a little while to get back up to speed with Python, and even Perl again, since I haven't used them in awhile. Our brain seems to treat programming languages very similarly to how it treats natural languages. You have to use a language for a long time to really become fluent, learn the culture, learn the idioms, for computer languages, to learn the class library, etc. etc.

410: Gone

Mark Pilgrim: "Embracing HTTP error code 410 means embracing the impermanence of all things."

Awesome.

I'm going to put support for the 410 error code in my CMS, once I get error handlers in there.

Why XML doesn't suck

Via Simon, Tim Bray posts a follow up to his XML Is Too Hard For Programmers article.

Syntax can be, and has been, interoperable. The definitions of the telephone network, the Internet, email, and the Web are all bits-on-the-wire definitions of what you send back and forth, and they've all worked well enough to change the world. XML provides a nice set of syntax rules that you can stick in the face of a recalcitrant vendor and say "you claim to be interoperable? Well, ship me some XML then." And these days, they can't say no, and this is good for everyone.

XML Confers Longevity When I'm doing a standup speech, I often ask: "Everyone in the audience who thinks they're going to be using the same word processor in ten years, raise your hand." No hands go up. "Everyone who has data around that's going to have value in ten years?" After a minute's thought, every hand goes up. The lesson is clear: information outlives technology.

And yet, as of today, too much of our intellectual heritage is tied up in fragile, proprietary, binary word processor files. This sucks. XML is the solution.

But let's face it, when you parse XML, you get a data structure that is kind of an ordered sequence and kind of a tree and kind of a hypertext. This maps well onto no known programming paradigm. If you're an object-oriented person, you can pretend that XML elements are serialized objects, and that works sometimes, and if you're a Perl hack, you can pretend that XML is an unusually-well-decorated text stream, and that works sometimes. But the impedence mismatch, I suggest, is just a fact of life, and the benefits we get (i18n, interop, and so on) make it worthwhile.

A couple of people suggested that the natural way forward for XML stream parsing is well-done iterators, which seems like a really great idea to me and I'm surprised that I haven't run across any yet.

That seems to be the approach that BEA is taking (this makes me wish I had written a better post for it).

Last, this is really funny:

The Python people also piped to say "everything's just fine here" but then they always do, I really must learn that language.

Palestinian democracy

Via LGF, a great article from Barbara Lerner at the National Review Online: There Will Never be a Palestinian Democracy.

Also see The Problem With Land For Peace at Right Wing News.

Weblog archives and search results

Via John, Kasia is redesigning her weblog archives, and she's totally right:

We all know how much google loves weblogs.. well that love extends to the archive pages and unfortunately that creates a massive misinformation base of searches.. Many of the searches that bring people to my weblog are a combination of words from several, unrelated entries combined together on one, long, lengthy archive page. The change to my archive pages ought to put a stop to that...