akrish

I just finished taking a compilers class (well except for the final), and it’s been the single hardest class I’ve taken in college. Homeworks were non-trivial, exams were very difficult, and the class project was to write a compiler for a subset of python. As far as class projects go, this one has been completely ridiculous, my project partners and I have devoted weeks to building this compiler and fortunately we managed to finish it, but many groups I believe did not.

Anyway, in designing the compiler, I developed a new understanding of python features, despite having programmed in python for over a year.  My group and I found a lot of really crazy features of python that lead to problems when designing our compiler. I’m not going to talk about all the cool things that everyone knows about python, like lambdas, function values, nested functions and so on. I thought it would be fun to mention some of the really cool stuff here.

Dynamically Adding to Classes

Python basically implements classes as dictionaries. Instances are simply copies of the dictionary (well technically they are copied lazily, using copy on write), but instances naturally contain all of the mappings that the class contains (as they should). However, you can dynamically add mappings to classes so I can do something like this:

class A(object):
    x = 3
a = A()
print a.x ==> 3
print a.y ==> AttributeError
A.y = 7
print a.y ==>7

And I can even do this for methods:

class A(object):
    x = 3
A.foo = (lambda self,y: (self.x, y))
a = A()
print a.foo(6) ==> (3, 6)

When we discovered this, my project partners and I were amazed. It’s a really cool feature but I can hardly see a use for it (if you do see one, please let me know). Further, it makes implementing classes pretty inefficient since you can’t use a virtual table approach that C++ and Java can use.

Differentiating functions and methods within classes

We submitted our project after thinking we were done and “thoroughly” testing everything. A couple of hours later, we got feedback from the auto-grading system saying we failed (among a couple others) a test case that did something like this:

class A(object):
    x = 4
def g(n):
    print n
a = A()
a.x = g
a.x(5) ==> 5

We failed this test case because we assumed that any call to an attribute reference was a method call, and so we implicitly pushed on a as the first argument. This isn’t the correct behavior as we then pushed on one more argument than we should have. When we saw this, we were again amazed. Python somehow knows that a.x is a function and not a method, so it knows when it should implicitly push on self.

We hacked together a fix that I thought was pretty cool. We were representing all functions as boxed 12 byte function values, where the first byte pointed to a dummy virtual table, the second byte pointed to the code for the function and the 3rd byte was the static link for that function. Since from this perspective, functions and methods were exactly the same thing, we had no way to tell whether the object we had was a function value or a method value, so we were pretty stuck. Instead, we added a byte to all function values that kept track of whether they were functions or methods (yeah this is a huge waste of space, but performance wasn’t our priority at this point). Then at runtime, we are forced to check whether the value we’re trying to call is a function or a method, and behave appropriately in each case.

Unfortunately there’s a huge increase in code size. Now every call in python requires the check mentioned above, and two distinct call instructions, of which only one would get executed (because we were required to push on the number of arguments as a parameter to the callee function). We discovered this bug like 2 hours before the deadline, and we weren’t being graded on performance, so we didn’t really care about this issue. I am interested to know how python does it. My guess is that in the class dictionary, methods aren’t stored as “function values” so you would be able to tell right away, but I haven’t looked it up. If you know how python does it, let me know.

If-Expressions

I didn’t really use If Expressions before this class, and now that I’m more familiar with them, I think they are actually quite useful in writing concise code. I’m not talking about the standard if-elif-else clauses, but more of the analogue to x ? y : z syntax in C. And this feature isn’t that crazy, in fact it’s what I expect python would do. So an if expression looks like x if y else z and the semantics are that this statement returns x if y evaluates to True and otherwise it returns z. So it’s just like the ?: construct in many other languages. It’s cool when you use it with function values. For example:

def foo(x): print x
def bar(x): print -x
y = True
(foo if y else bar)(4) ==> 4
z = False
(foo if y else bar)(4) ==> -4

And that’s really cool. I can really concisely choose what function I want to call at runtime. I haven’t thought of a lot of uses for this exactly, but I can definitely see myself taking advantage of it’s presence.

The global keyword
There are some really cool features with global that I think are pretty standard in most languages but that I’m not used to seeing. For example:

def g():
    global a
    a = 3
print a ==> NameError
g()
print a ==> 3

This means that the when you declare a as global, if it’s not already bound in the global environment, you make a binding for it. I’m not exactly sure how this works in other languages, but I can totally see a language throwing an error if a is not bound in the global environment. Here though, python is smart enough to make the binding for you, and then you can use it after the call to g. This means that there are some really interesting ways that you can dynamically add names to the global environment.

So I’m sure there are more really interesting features of python that I haven’t discovered, and I’m pretty sure there are some that I have found that I’m just no remembering right now but these are some of the coolest language features I’ve seen. As I mentioned throughout, some are definitely more useful than others and I’m sure that in some cases these may be undesirable. If you know of any crazy python features that I didn’t highlight, I’d be interested in hearing about them.

So haven’t written a techy/nerdy post in awhile and I presume that’s because my life outside of work hasn’t been that nerdy during the summer. But I thought it’s time to return to my normal self and so… lets talk about python.

This article is titled the way it is because I’ve heard that phrase uttered so many times in the past year or two that it’s insanse. So I spent a little bit of time last summer learning python and the interns and I worked on a project in python, but it wasn’t too involved and I promptly forgot a lot of it in the months to come. I still used php or perl for a lot of my scripting purposes because I was familiar with them and my friends would continuously nag me to make the move over to python, but I never really got around to it.

So this summer, I was initially given the option to write in whatever language I wanted and thought that it would be a good time to learn python, especially because it’s used in a class I’m taking next semester anyway. I took about a week in the beginning of my internship to learn about python and ported some of my old code over to python. I already knew that python was pretty cool, but it seriously makes programming much much faster. And even though I ended up using perl for my intern project, I’m pretty happy that I learned python and I think I’ll be using it more in the years to come.

Ok so why is python so cool. First there are the reasons that everyone says, it’s like writing pseudo-code, but then it actually works. That’s pretty cool because programmers learn to think in pseudo-code but then have to translate that to whatever language. The closer the language is to pseudo-code the better.

It’s interpreted, which has pros and cons. Interpreted languages are generally slower than compiled languages, and yes I think python is slower than C, but for internal scripts where performance isn’t that critical, python being interpreted is pretty helpful.

Python is a lot like scheme/lisp with lambda expressions and functions as first-class data. I was disappointed that you can only have 1 line lambdas, but they’re still pretty useful. And passing functions around is amazing and extremely useful. Also has built in functions like map() and filter() which just make list manipulation so much easier.

Consider this, lets say I have a list of numbers and I want to make a sorted list of all the even numbers. In python…
sorted ( filter (lambda x: return x % 2 == 0, myList))
In most other languages (except lisp)… I’m pretty sure it would be a lot less concise. And yes the same sort of thing can be done in scheme, but I like scheme a lot too.

I like the way that python allows me to include my test cases of a module within that actual module. The if __name__='__main__': feature is pretty cool. You can’t really do that in perl and it’s annoying because I need to have a testModule.pl script for every Module.pm. With python I pretty much reduce the number of files I have to worry about by a factor of 2.

Also python is pretty easy to read and that makes your programs easier to maintain. I’ve been experiencing this problem where perl isn’t that easy to read (especially if you use a lot of the uncommon features) and now that I have to go back and make changes to my code it’s pretty hard to figure out where I’m supposed to look and how I’m supposed to make the change. And not that I’ve done this, but I think it’d be a lot easier if I wrote in python.

Anyway, I’m not really an expert on python or programming languages in general. In fact I haven’t even used python for a substantial project or anything, but I have been reading about it and playing with it so I found some things that I like. And I think I’ll be using python more in the future so I’ll become more and more familiar with it and probably find more things I like and also some things that I can’t stand.

Disclaimer: If my facts are wrong here please feel free to correct me because these were the impressions that I got and it would be good if I got them corrected.