I just finished taking a compilers class (well except for the final), and it’s been the single hardest class I’ve taken in college. Homeworks were non-trivial, exams were very difficult, and the class project was to write a compiler for a subset of python. As far as class projects go, this one has been completely ridiculous, my project partners and I have devoted weeks to building this compiler and fortunately we managed to finish it, but many groups I believe did not.
Anyway, in designing the compiler, I developed a new understanding of python features, despite having programmed in python for over a year. My group and I found a lot of really crazy features of python that lead to problems when designing our compiler. I’m not going to talk about all the cool things that everyone knows about python, like lambdas, function values, nested functions and so on. I thought it would be fun to mention some of the really cool stuff here.
Dynamically Adding to Classes
Python basically implements classes as dictionaries. Instances are simply copies of the dictionary (well technically they are copied lazily, using copy on write), but instances naturally contain all of the mappings that the class contains (as they should). However, you can dynamically add mappings to classes so I can do something like this:
class A(object):
x = 3
a = A()
print a.x ==> 3
print a.y ==> AttributeError
A.y = 7
print a.y ==>7
And I can even do this for methods:
class A(object):
x = 3
A.foo = (lambda self,y: (self.x, y))
a = A()
print a.foo(6) ==> (3, 6)
When we discovered this, my project partners and I were amazed. It’s a really cool feature but I can hardly see a use for it (if you do see one, please let me know). Further, it makes implementing classes pretty inefficient since you can’t use a virtual table approach that C++ and Java can use.
Differentiating functions and methods within classes
We submitted our project after thinking we were done and “thoroughly” testing everything. A couple of hours later, we got feedback from the auto-grading system saying we failed (among a couple others) a test case that did something like this:
class A(object):
x = 4
def g(n):
print n
a = A()
a.x = g
a.x(5) ==> 5
We failed this test case because we assumed that any call to an attribute reference was a method call, and so we implicitly pushed on a as the first argument. This isn’t the correct behavior as we then pushed on one more argument than we should have. When we saw this, we were again amazed. Python somehow knows that a.x is a function and not a method, so it knows when it should implicitly push on self.
We hacked together a fix that I thought was pretty cool. We were representing all functions as boxed 12 byte function values, where the first byte pointed to a dummy virtual table, the second byte pointed to the code for the function and the 3rd byte was the static link for that function. Since from this perspective, functions and methods were exactly the same thing, we had no way to tell whether the object we had was a function value or a method value, so we were pretty stuck. Instead, we added a byte to all function values that kept track of whether they were functions or methods (yeah this is a huge waste of space, but performance wasn’t our priority at this point). Then at runtime, we are forced to check whether the value we’re trying to call is a function or a method, and behave appropriately in each case.
Unfortunately there’s a huge increase in code size. Now every call in python requires the check mentioned above, and two distinct call instructions, of which only one would get executed (because we were required to push on the number of arguments as a parameter to the callee function). We discovered this bug like 2 hours before the deadline, and we weren’t being graded on performance, so we didn’t really care about this issue. I am interested to know how python does it. My guess is that in the class dictionary, methods aren’t stored as “function values” so you would be able to tell right away, but I haven’t looked it up. If you know how python does it, let me know.
If-Expressions
I didn’t really use If Expressions before this class, and now that I’m more familiar with them, I think they are actually quite useful in writing concise code. I’m not talking about the standard if-elif-else clauses, but more of the analogue to x ? y : z syntax in C. And this feature isn’t that crazy, in fact it’s what I expect python would do. So an if expression looks like x if y else z and the semantics are that this statement returns x if y evaluates to True and otherwise it returns z. So it’s just like the ?: construct in many other languages. It’s cool when you use it with function values. For example:
def foo(x): print x
def bar(x): print -x
y = True
(foo if y else bar)(4) ==> 4
z = False
(foo if y else bar)(4) ==> -4
And that’s really cool. I can really concisely choose what function I want to call at runtime. I haven’t thought of a lot of uses for this exactly, but I can definitely see myself taking advantage of it’s presence.
The global keyword
There are some really cool features with global that I think are pretty standard in most languages but that I’m not used to seeing. For example:
def g():
global a
a = 3
print a ==> NameError
g()
print a ==> 3
This means that the when you declare a as global, if it’s not already bound in the global environment, you make a binding for it. I’m not exactly sure how this works in other languages, but I can totally see a language throwing an error if a is not bound in the global environment. Here though, python is smart enough to make the binding for you, and then you can use it after the call to g. This means that there are some really interesting ways that you can dynamically add names to the global environment.
So I’m sure there are more really interesting features of python that I haven’t discovered, and I’m pretty sure there are some that I have found that I’m just no remembering right now but these are some of the coolest language features I’ve seen. As I mentioned throughout, some are definitely more useful than others and I’m sure that in some cases these may be undesirable. If you know of any crazy python features that I didn’t highlight, I’d be interested in hearing about them.
