Thursday, June 25, 2009

Fan vs. Scala: Type System

It's been a while, but here's my next installment of detailed discussion of my personal reasons for preferring Fan to Scala. Here is my next claimed advantage:

2. In Fan, you don't have to figure out existential types or other complex typing. Fan will even do your casting for you in many cases.

There's a huge debate about dynamic vs. static typing. I like static typing (because of easier toolability, performance, and hints for the programmer but yes I know there are pros and cons). I don't like typing to get in my way. That means I only want to say as much as I want to say. That is, infer where possible, and sometimes I just don't care what's co or contravariant. (Oh, the blasphemy!) I also don't want the type system to tie me in knots. Sometimes static enforcement blocks me from doing something correct the way I want to do it, even if other times it catches my mistakes.

Both Fan and Scala (and C# and others) do limited type inference. Usually, you have to specify types for public methods and such. Sure, they could be inferred in some cases. Some languages do that, but I like it to be explicit. The exact limits of type inference differ among these languages, but most of the focus is on local vars. For example (in Fan):

greeting := "hello"
size := greeting.size


In Fan, ':=' is used for var initialization as opposed to the 'var' keyword in Scala or C#. You even use it when specifying an explicit type for the var, just for consistency. I like either style better than the "guess where the var is defined" behavior you get in Python or Ruby. (Actually, Python has fairly simple rules, but they can get annoying sometimes. I've heard Ruby is cleaning up their ambiguity in this arena, too.)

But, anyway, my discussion had more to do with this list of features from Scala.

1. Abstract Types
2. Existential Types (officially not recommended except for interoperability with Java, is my understanding)
3. Generics
4. Type Bounds
5. Variances
6. Views

And probably things I'm forgetting. Yes, the type system is serious in Scala. They want to you get it right. Personally, I see the need for awesome static typing in a language like OCaml that has no dynamic type safety (no casting exceptions) but still wants to be type safe and avoid security risks.

But in a language with dynamic type safety (like Java or ducky languages like Python, JavaScript, or Ruby), how often do you really see casting exceptions? If I make a mistake, it will be found (rather than opening security holes), and I don't usually make mistakes.

So, to make life easy, Fan will do your casting (up or down a hierarchy) automatically. For example (in Fan):

Obj[] items := ["something", "and", "more"]
Str message := items[0]


In Java, you'd have to make that second cast explicit, being a down cast. Welcome to "I don't need generics to have my casting done for me" land.

And while I know that Scala folks will be all over me about this feature of Fan, think first about Scala's automatic type conversion. Fan only casts for you (yay! in my book). Scala, for all its type safety will magically convert integers into strings, if you set up the right magic. I'm personally really scared of automatic variable swapping going on behind my back. It's also something I dislike about C#.

One thing that I don't like about Fan is its lack of generics (except for Lists, Maps, and Funcs). Yep, I like generics (despite my list above on Scala). I just want autocasting and arbitrary support for co and contravariance. I'm a hippie that way. I think Scala's generics go too far. So really, I'd prefer some land in between Scala and Fan on the generics front. But between the two, I'll take Fan.

Side note, Fan also supports dynamic duck typing. If you say, "myVar->something", that does a dynamic lookup rather than the static form "myVar.something". Note that this is different from duck typing support in languages such as C#, Boo, and haXe, where method calls look the same, but the variable type is declared to be dynamic. The two styles have different effects on your code. Scala shuns dynamic typing in favor of static structural typing. I'm not sure either take is fully necessary, but between the two (duck typing vs. structural), I'd prefer structural. I just don't want the rest of Scala typing with it. Sorry for being too quick to find good links, but Google can probably help out.

I'll give at least one link, though, to the official description of Fan's system.

19 comments:

  1. That's a nice run-through Tom. The generics issue is easily the most controversial thing about Fan and likely to prevent certain people from adopting it or even giving it a chance. That, and the lack of IDE support.

    ReplyDelete
  2. Good stuff, Tom, sending you some link love from Twitter.

    --
    Cedric
    closet Fan fan

    ReplyDelete
  3. I can't even imagine trying to work in a language with automatic casting like what you showed. That basically negates almost any benefit that you might reap from a static type system.

    What you're effectively saying when you talk about not caring about co- and contra-variance is that you're willing to trade compile-time safety for slightly more ease-of-use. Personally, I think this is a trade-off which will come back to haunt you in the end. I'm a huge fan of catching errors early, but wanton casting and dynamic dispatch operators only serve to work against such ambitions.

    Coming back to your main point... On paper, Scala's type system does look scary. However, in practice, you don't need to worry about any of that stuff. All you need to know is that List collections really behave *exactly* how you would expect. List[Any] is a supertype of List[String], and adding an Int to a List[String] will produce a List[Any], since Any is the least-common supertype of Int and String. All of this is completely intuitive, and none of it requires any effort on your part. There have been very few times where I have found Scala's type system to restrict expressiveness, and even fewer times where I have been forced to deal with any of its complexities.

    In short, I think you're overreacting. Maybe Fan really is a better language for you, but I think that blaming that decision on Scala's type system is misguided and even somewhat uninformed.

    ReplyDelete
  4. Hey Dan,

    Can you elaborate on "That basically negates almost any benefit that you might reap from a static type system."?

    I'm really not seeing it.

    All Fan does is remove the need for tedious casts that the compiler could have inserted for you. For example, this has always bugged me in Java:

    Object o = ...;
    String s = (String) o;

    Why do I have to cast? Can't the compiler do it for me?

    ReplyDelete
  5. Auto casting sounds a bit dangerous when its a property of the language, you may loose type safety and get into some dark corners with it.
    Scala's implicit is a powerful tool that indeed you can shoot your lag with. But on the upside, the definition of an implicit is explicit (funny game of words). There is no automatic conversion the language will do behind your back unless you explicitly ask it to do so.
    I think its perfectly fine to use implicits like Clazz to RichClazz (e.g. Int to RichInt) when:
    1. RichClazz has essentially the same state and functionality of Clazz with some additional value
    2. Both RichClazz and Clazz are immutable
    Otherwise I would also avoid using implicits, especially when converting between totally different objects with mutable states.

    ReplyDelete
  6. Hi Tom,

    I've been curious about how Fan's approach to "cast inference" would work out in practice. I haven't had time to try it out, so I don't know. But I don't see why it would be much different than working in a dynamic language in that you'd find most problems at runtime right away when you're writing something new. The main downside of dynamic typing I think is later, when refactoring existing code. If you don't have a test that will illuminate every thing that breaks, then you can miss spots that break during a refactor. So my main curiosity is to what extent does that pain go away when using a cast inference approach a la Fan. My guess is that much of it might go away. In other words, that static typing would help me almost as much in Fan as in Scala when doing refactoring later. But I don't know because I haven't tried it yet.

    Have you done much Scala programming? When I talk to Scala programmers, they all have complaints. But the complaints about Scala from people who actually program in it are different than those of people who just read some docs. And one of the main differences is complaints about the complexity of Scala's type system. It is complex for sure, but the main thing that complexity makes hard is writing a Scala compiler, which most people don't need to do. (Although this also affects tool creators, like IDE plug-in makers, so there is a cost there.)

    The main place the complexity of the type system will show up when you're doing normal Scala programming is when you design libraries. Most programmers people will do that from time to time, and they will on occasion get a head scratcher to think about. But it isn't that often even when designing libraries that things are challenging to figure out. In all of ScalaTest I only had to think about variance once, when designing trait Matcher. I went jogging and visualized an inheritance hierarchy, and while jogging up a hill realized Matcher was contravariant. I went back, put in the minus sign, it compiled, and I haven't had to think about it since.

    And neither have the users of ScalaTest. When people use Scala libraries, there's usually very little type system complexity in their faces. Things for the most part just work like you'd expect. Most people who use ScalaTest matchers probably have no idea it is declared contravariant, nor do they need to care. It just works like they would expect it to work. I have had some compiler errors complaining that my existential type did not match the Java wildcard type of the Java method I was trying to override, but that was pretty rare and easy to solve. And I've only used an existential type once for a non-Java interop reason. They may look scary in a doc, but they are actually a lot easier to grasp I think than Java's wildcard types, and very rarely show up in real Scala code anyway.

    Anyway, your post reminded me of a few others I've seen complaining about Scala's type system, which didn't come from actual experience with Scala. If that's not the case, I'd be interested in hearing specific incidents of where Scala's type system was too painful in practice for you.

    ReplyDelete
  7. Thanks much to everyone for the comments so far, from any perspective. On the complexity question, I think every programmer should be empowered to write libraries, and it ought to be easy to do so. I think that's a fundamental aspect of almost all programming. Maybe you could still argue the "get it right once and make usage easy after" perspective. But really, I think libraries ought to be easy to write (and use). And I think intuition really gets a lot of the way there without super strict typing. Just my own perspective.

    ReplyDelete
  8. Bill, I have written Scala programs no larger than a few files, and never really in anger, so you got me there.

    Refactoring with automated casting is no big deal, I don't think. The tricky parts for refactoring come from duck calls ("someObj->doSomething"), and I suspect this could even be an issue with Scala's structural types in some cases. Summary is that if you always have to know a class before you can call operations on it, then refactoring is straightforward. And Fan without using duck calls (or reflection) is fine for that (just like Java without reflection, even without generics).

    ReplyDelete
  9. Hi Tom,

    In response to this:

    Refactoring with automated casting is no big deal, I don't think. The tricky parts for refactoring come from duck calls ("someObj->doSomething"), and I suspect this could even be an issue with Scala's structural types in some cases. Summary is that if you always have to know a class before you can call operations on it, then refactoring is straightforward. And Fan without using duck calls (or reflection) is fine for that (just like Java without reflection, even without generics).

    Actually structural types in Scala give you duck typing and deterministic refactoring. I show an example of this in my Feel of Scala talk, which is on Parleys:

    http://tinyurl.com/dcfm4c

    It is near the end when I change the name of a method in the GoldenGateBridge class. One other thing I show in that video is invokePrivate, which actually uses reflection behind the scenes to invoke a private method. That's duck typing like in dynamic languages, or the -> operator in Fan. The deterministic refactoring is gone, but I show that you can get it back in Scala with a compiler plugin.

    One thing I mention in that talk is that invokePrivate is a bit verbose, because I wanted it to be very obvious to readers of the code that private methods are being invoked. I could have used an operator there, and one of the operators that occurred to me when I was designing the invokePrivate stuff was ->, because it mimics Fan's syntax. But it just doesn't look like dynamic invocation to most people I figured, especially not private methods, and in Scala -> is already used to map keys to values for Maps. But if you really wanted Fan-like dynamic invocation syntax in Scala, you could quite easily make a library similar to invokePrivate in ScalaTest, but with "->" instead of "invokePrivate". Then the syntax for dynamic invocation in your Scala programs would be:

    someObj->'doSomething

    The only difference is the extra tick mark before the method name. You could also pass parameters:

    someObj->'doSomething(some, params)

    And with a compiler plugin, you could get deterministic refactoring. When someone changes the name of the doSomething method on someObj but doesn't change this invocation, your program would give a compile error pointing to this line of code.

    Bill

    ReplyDelete
  10. Hi Tom,

    In response to this:

    Refactoring with automated casting is no big deal, I don't think. The tricky parts for refactoring come from duck calls ("someObj->doSomething"), and I suspect this could even be an issue with Scala's structural types in some cases. Summary is that if you always have to know a class before you can call operations on it, then refactoring is straightforward. And Fan without using duck calls (or reflection) is fine for that (just like Java without reflection, even without generics).

    Actually structural types in Scala give you duck typing and deterministic refactoring. I show an example of this in my Feel of Scala talk, which is on Parleys:

    http://tinyurl.com/dcfm4c

    It is near the end when I change the name of a method in the GoldenGateBridge class. One other thing I show in that video is invokePrivate, which actually uses reflection behind the scenes to invoke a private method. That's duck typing like in dynamic languages, or the -> operator in Fan. The deterministic refactoring is gone, but I show that you can get it back in Scala with a compiler plugin.

    One thing I mention in that talk is that invokePrivate is a bit verbose, because I wanted it to be very obvious to readers of the code that private methods are being invoked. I could have used an operator there, and one of the operators that occurred to me when I was designing the invokePrivate stuff was ->, because it mimics Fan's syntax. But it just doesn't look like dynamic invocation to most people I figured, especially not private methods, and in Scala -> is already used to map keys to values for Maps. But if you really wanted Fan-like dynamic invocation syntax in Scala, you could quite easily make a library similar to invokePrivate in ScalaTest, but with "->" instead of "invokePrivate". Then the syntax for dynamic invocation in your Scala programs would be:

    someObj->'doSomething

    The only difference is the extra tick mark before the method name. You could also pass parameters:

    someObj->'doSomething(some, params)

    Bill

    ReplyDelete
  11. Hi Tom,

    Sorry for the double post. This blogger UI is a bit non-intuitive. One more comment about the compiler plugin for ->. The reason the compiler plugin works for invokePrivate is that usually the type of the reference would be the type of the object with the private method, and if it isn't, it is not unreasonable to ask the programmer to make it so to get the program to compile. But if you really want -> to be for duck typing like in dynamic languages, the type of the reference could be higher up the stack, even all the way up to Object. In that case, the compiler wouldn't be able to see the method, and would give an error. So even though it would work, it wouldn't make sense. If the type of the variable is already the type that has the method, just invoke it statically.

    I'm curious what people use -> for in practice in Fan. Probably for things like I used view bounds for in ScalaTest matchers, which I also showed in the Parleys video. Sometimes people want to do dynamic things in static languages, and -> is a more concise form for basically doing what you can already do with reflection in Java. Is that an accurate assessment?

    ReplyDelete
  12. Firstly, in Scala you don't have to use all those features. I don't use existentials, abstract types or view bounds.

    Secondly, the autocasting you mention sounds like a negative feature. Casting is something to be removed from programs, not added automatically.

    If I have the Java code:

    List[String] list = ...;
    list.add("Bob Smith");

    elsewhere

    String x = list.get(0);

    Then when I refactor list to be a List[Person], I'm quite glad that String x = list.get(0); will no longer compile. It sounds like Fan is a step backwards from Java in this respect (you'll get a ClassCastException instead of a compile error). And you don't want to step backwards from Java. Do you?

    ReplyDelete
  13. Hi Bill,

    I implemented duck call operator as a library awhile ago. It uses exactly the syntax you described.

    someObj->'doSomething(some, params)

    Haven't used it for anything serious, just as a proof that it is doable.

    http://github.com/jonifreeman/dynscala/tree/master

    ReplyDelete
  14. Oops, sorry. Not exactly the same syntax since -> would conflict with Tuple2 construction function from Predef.

    someObj-->'doSomething(some, params)

    ReplyDelete
  15. My claim is that casting isn't bad. However, I don't usually like duck calls. I'll save my opinions on operators for another day.

    ReplyDelete
  16. In response to Joni's:

    Oops, sorry. Not exactly the same syntax since -> would conflict with Tuple2 construction function from Predef.

    someObj-->'doSomething(some, params)

    I considered that operator for invokePrivate for the same reason: because -> really looks like a map operator to Scala people. But it still looked a bit odd to me (whereas I felt -> in Fan looked better). The one that was the best operator I thought was ~~>, because it looked like a "lightening bolt." But that sure didn't look like a private method invocation either, so I settled for the verbose but clear "invokePrivate".

    I think using such an operator and syntax might be OK in Scala for those occasional times when you want to do reflection. It makes reflection code more concise and readable. I suspect that's probably what -> is good for in Fan, but I haven't used Fan so I couldn't say for sure. The one difference is that Fan's -> is standard, so Fan programmers will know what it means. --> or ~~> are not "standard" in Scala so therefore a lot more obscure.

    ReplyDelete
  17. In response to Ricky's:

    Secondly, the autocasting you mention sounds like a negative feature. Casting is something to be removed from programs, not added automatically.

    I think Fan takes an interesting middle approach to the dynamic/static dichotomy. It is static much of the time, but inserts casts here and there, which creates potential runtime errors as you point out, just like a dynamic. But I'd point out it's no more than you'd have in a dynamic language. In fact, probably far less because many times there would be no casts inserted. And as in a dynamic language, you can with testing get most of the bugs out as you write code initially, so you'd minimize those runtime errors. But you get all the performance benefits of static languages on the JVM, and a simpler type system than Scala. So those are the tradeofs. The downside in my mind is when refactoring existing code.

    As I said I think in my initial post, I wonder to what extent things break when I refactor Fan code in ways that would not be caught by the compiler because of autocasting. What percentage of deterministic refactoring do I get in Fan compared to Scala? If it is 90%, then that's a tradeoff some people might want to make in practice sometimes. This is what I'm curious about Fan. How does refactoring larger Fan programs work out in practice.

    ReplyDelete
  18. Bill,

    The number of casts inserted isn't actually relevant. Java inserts quite a lot of casts, if you examine the bytecode that uses generic types.

    The problem is that Fan removes a guarantee that Java makes.

    Fan has a simpler type system than Scala, true enough, but far less useful. If you only aim for simple type systems, ignoring how they can help you, you'll go for an untyped language like Groovy, Scheme or Python.

    Scala tries to give a useful type system, and gets bashed in blogs like these for it. I've never heard of anyone who actually uses Scala getting overwhelmed by the type system's complexity (those who I've noticed dropping Scala have done so because of tool support, or changing jobs, etc.).

    Perhaps I'm guilty of something similar regarding Fan, as I've never used it, but I hope I'm making factual statements with a hint of opinion rather than the other way around. :)

    ReplyDelete
  19. Cedric Beust has done some nice discussion of structural typing vs. duck typing with some nice emphasis on refactoring concerns, by the way. Side note, I tend to look at hierarchies as automatically namespacing the methods in that class. Raw structural typing (and duck typing) removes those namespaces and presumes they mean the same thing. Sometimes it's true, and sometimes it's not.

    ReplyDelete