Tuesday, July 17, 2012

A closer language: Julia

Well, I can't seem to get language design out of my mind, but someone got really close to what I've been planning: a human-friendly, math-friendly, high speed, scripting-feel, typing-allowed programming language with multiple dispatch and whatnot. The one I found today is called Julia.

It has been well known for the past few months, but I managed to miss it until today.

My (mostly vapor) language is called Rio, and here's some sloppy "brainstorm about the language" demo code.

Some similarities between Rio and Julia are almost uncanny, but there are strong differences, too. Julia has no namespacing (but maybe they'll fix that), emphasizes fully dynamic typing, has too many keywords, doesn't let you really be a peer to core control constructs, has syntactic macros (which I dislike -- but they at least need special syntax and maybe are hygienic?), has batteries included (vs. immediate emphasis on package management), and so on.

But given they already have the buzz, are way ahead of me, and have people paid to work on it, maybe I should throw in the towel and get my own work done, knowing that Julia does a lot of what I wanted.

Some additional differences arise between Julia and Rio, by the way, like how they've gone reference-based and I was leaning value-based, but there are so many pros and cons on such things. That I can follow them down to various subtleties and think about how they've chosen where to go vs. where I have been headed with Rio just lets me know that we've thought about a lot of similar things. Julia just might be my language in the future. Argh. If only for infinite time and money, eh?


  1. Saw your Github issue on JuliaLang/julia which directed me here, so I figured I'd offer a response to some of your questions:

    Macros are almost hygienic (meaning there are a few corner cases, but mostly we've got stuff covered through `gensym`)

    Namespaces are on their way: https://github.com/JuliaLang/julia/compare/master...jb/exports

    For packages it was just easier to include everything as we didn't have as much, but we have support for packages as well, though it isn't used as much and still takes a little bit of work: https://github.com/JuliaLang/julia/blob/master/base/pkg.jl

    I'm also curious why you think we have to many keywords and what exactly you mean to being a peer to core control structures.


  2. Thanks for info. Hope I wasn't too annoying with my comment there. Depending on how exports work, that might get the job done. Do non-exported types and functions get priority for the current and subloaded modules? I think that's really important for _my_ code to work the way I expect.

    Also glad to see the package stuff in the works. There might be value in breaking down the current Julia core into separate packages in the future, so folks without need for everything can easily grab the core system and folks with the need to easily get more. A large "batteries included" core tends to create a bunch of cruft that no one wants after a few years.

    I found the `gensym` stuff after my blog post. Seems a bit hackish, but I guess it's workable.

    By too many keywords and peer-to-core, I mean `for`, `if`, `begin`, and all that. In Rio plans, most of those are just function calls with `do` being the keyword block starter, similar to how `do` is used in Julia. However, in Rio, `do` can also take params `do(x) ...` such that it is also the anonymous function system and can receive, say, the value that the `for` function is iterating on. I also wouldn't require params for single top-level function calls, so `if something do ...` is a syntax available to any function. And I also had a mechanism for allowing automatic conversion into functions for lazy evaluation (similar effect to call-by-name in Scala but with a different kind of style). This is also important for doing things like `if` and short-circuiting and such.

    I also have an `of` keyword for blocks that become a list of parameters. This is used for things like (sorry no indentation here):

    switch x of
    case 1 do
    case 2 do
    else do

    And all that is done as functions. Only `of`, `do`, and `end` would be keywords there, and no macros either.

  3. To clarify, on exports, whether something is exported or not, my versions (including my types) should have priority in my code and anything I load beneath me. If that means I hide conflicting stuff above me, so be it. I wasn't expecting it anyway.

    Some way to manually reach out to outer definitions could be nice, though, for the rare case that I want that, and that's where some kind of namespace-like qualifier might have value. Or even if I load two conflicting things, I'd say the most recent should have priority, but if I use a namespace qualifier to choose access to an overridden definition, that should be okay, too.

  4. In my original reply "wouldn't require params for single top-level function calls" should be "wouldn't require parentheses for a single param for statement-level function calls".

    1. As far as I'm aware that was a conscious design decision to differentiate function calls from macro calls. Regarding your peer-to-core ideas, the idea is to provide a common syntax and where something more special is needed, on can use macros (see for example the @parallel macro that automatically parallelises a for loop). That way one has a simple base language that can easily be turned into any kind of DSL as it gives you access to the parse tree via the macro system. Hope that makes sense.

    2. Thanks for the reply.

      I don't usually like syntactic macros, so I'm very glad they look @different. I'd have already run away from Julia for sure, otherwise. I'm still debating my opinion on Julia, as it is. I like a _small_ and flexible core, but I don't like _too_ much flexibility in syntax. I've been thinking today that perhaps I understand why syntactic macros are especially popular in system with purely dynamic semantics. My planned tricks for Rio depend on semantics imposed by static analysis.

    3. Actually, I'll take it back on needing static analysis. Autolazifying expressions (for example) could work dynamically, but you'd have to leave everything temporarily lazy until the dispatch was complete. I'm not sure what implications that has for performance.

    4. You might be interested in looking at https://github.com/kk49/julia-delayed-matrix which does lazy evaluation in Julia (though IIRC the autolazifying macro isn't yet part of that repo).

    5. Thanks for the pointer again. I'm not so much interested in lazy generally for the current discussion, just that some common constructs (such as elseif) expect lazy evaluation. Anyway, maybe I'll get on the discussion group sometime to ask other questions (such as if there's a plan to reduce the extreme memory use for all the jitted code and so on).