Pythonic surprises for a Java programmer - Part 1

I had been a hard core Java programmer through out my career and had moved onto Python a while ago. The transition to python was a roller coaster ride of moments of unbridled exultation, deja vu and mirth followed by those of horror, surprise and deep disgust. Just to crib a bit, Python does not provide encapsulation...arrrghhhhh!!! and the best way to prevent people from calling an internal method seems to be to request them not to do so.

"""
     This method is meant to be consumed locally. 
     Don't call it from outside.... please please please :( 
"""
def _do_something_internal():
    -----------------
    -----------------

This blog (and a few subsequent ones, hopefully) will capture my observations on some gotchas that "pleasantly" surprise people coming from the Java domain (may be relevant to other languages as well).

Variable Scoping

Scenario 1 :

def method1():
    try:
          x = 10
    except:
           pass
    print x

Expected Output : Error. As the Variable x should not be visible out of the try/except scope.
Actual Output : 10

Surprised right. In Python, the scope of a variable is at an enclosing inner method/module level. Conditions in between like if, while, try e.t.c are ignored.



Scenario 2 :

x = 10 
def m1():
   x = 20
   print x 
print x
m1()
print x 

Expected Output : 10, 20, 20
Actual Output : 10, 20, 10

In Python, the variable X inside m1() shadows the module variable X. So effectively the local variable only got modified instead of the global variable.

After looking at the above scenario, what do you think will happen in the below case?

x = 10 
def m1():
   print x 
   x = 20
   print x 
m1()

Output: The interpreter figures out that the variable X shadows the global variable and throws an error : "UnboundLocalError: local variable 'x' referenced before assignment"

Just in case you are wondering how to access and modify the variable in global scope, you can do this:

x = 10
def m1():
   global x
   x = 20
m1()       #Sets the value of x to 20
print x     #Outputs 20 

Scenario 3:
Functions are objects. Ahem... I will repeat... Functions are objects in Python.

 >>> def m1():
...    x = 10
...
>>> m1
<function m1 at 0x0201f2b0>

>>> dir(m1)
['__call__', '__class__', '__closure__', '__code__', '__defaults__', '__delattr__', '__dict__', '__doc__', 
'__format__','__get__', '__getattribute__', '__globals__', '__hash__', '__init__', '__module__', '__name__', 
'__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 
'func_closure', 'func_code','func_defaults', 'func_dict', 'func_doc', 'func_globals', 'func_name']

>>> m1.func_name
'm1'

 >>> m1.func_code.co_varnames
('x',)

So, what this means is that, just like objects you can create and return functions from other functions

Function returning a function

def outer():

   def inner(y):
       print y
   return inner

x = outer()
x(100)            #Prints 100

Scenario 4: Now that we have accepted that functions are objects and can be returned from other functions, Python goes a step further and surprises us with Function Closures. Lets see what it means with an example:

def outer(x):
   y = 20
   def inner():
      print x * y
   inner()         #1
   return inner

func = outer(10)
func()            #2

#1 -- prints 200 as inner function is able to access the variable present in its outer function scope.
#2 -- print 200.  This is a bit surprising as we expect an error. But when the inner method is returned, Python has packed up all the variables needed by the inner function into its scope. To check the values you can execute:

func.__closure__[0].cell_contents  #prints 10
func.__closure__[1].cell_contents  #prints 20

Lets end this article, with one of my favorite gotcha
Scenario 5 : Parameter life time

 def appendToList(x, list=[]):
   list.append(x)
   return list
appendToList(10)   #Prints [10]
appendToList(20)   #Prints [10, 20]

The reason for this behavior is linked to Scenario 3. Functions are objects, remember? So, when the function object is created the list object is created add added to it. The same list object is accessed for all subsequent calls leading to the output as shown above.