Skip to main content

Pythonic surprises for a Java programmer - Part 1



I had been a hard core Java programmer through out my career and had moved onto Python a while ago. The transition to python was a roller coaster ride of moments of unbridled exultation, deja vu and mirth followed by those of horror, surprise and deep disgust. Just to crib a bit, Python does not provide encapsulation...arrrghhhhh!!! and the best way to prevent people from calling an internal method seems to be to request them not to do so.
"""
     This method is meant to be consumed locally. 
     Don't call it from outside.... please please please :( 
"""
def _do_something_internal():
    -----------------
    -----------------
This blog (and a few subsequent ones, hopefully) will capture my observations on some gotchas that "pleasantly" surprise people coming from the Java domain (may be relevant to other languages as well).

Variable Scoping

Scenario 1 :
def method1():
    try:
          x = 10
    except:
           pass
    print x
Expected Output : Error. As the Variable x should not be visible out of the try/except scope.
Actual Output : 10

Surprised right. In Python, the scope of a variable is at an enclosing inner method/module level. Conditions in between like if, while, try e.t.c are ignored.



Scenario 2 :
x = 10 
def m1():
   x = 20
   print x 
print x
m1()
print x 

Expected Output : 10, 20, 20
Actual Output : 10, 20, 10

In Python, the variable X inside m1() shadows the module variable X. So effectively the local variable only got modified instead of the global variable.

After looking at the above scenario, what do you think will happen in the below case?
x = 10 
def m1():
   print x 
   x = 20
   print x 
m1()
Output: The interpreter figures out that the variable X shadows the global variable and throws an error : "UnboundLocalError: local variable 'x' referenced before assignment"

Just in case you are wondering how to access and modify the variable in global scope, you can do this:
x = 10
def m1():
   global x
   x = 20
m1()       #Sets the value of x to 20
print x     #Outputs 20 
Scenario 3:
Functions are objects. Ahem... I will repeat... Functions are objects in Python.
 >>> def m1():
...    x = 10
...
>>> m1
<function m1 at 0x0201f2b0>

>>> dir(m1)
['__call__', '__class__', '__closure__', '__code__', '__defaults__', '__delattr__', '__dict__', '__doc__', 
'__format__','__get__', '__getattribute__', '__globals__', '__hash__', '__init__', '__module__', '__name__', 
'__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 
'func_closure', 'func_code','func_defaults', 'func_dict', 'func_doc', 'func_globals', 'func_name']

>>> m1.func_name
'm1'

 >>> m1.func_code.co_varnames
('x',)

So, what this means is that, just like objects you can create and return functions from other functions

Function returning a function
def outer():

   def inner(y):
       print y
   return inner

x = outer()
x(100)            #Prints 100
Scenario 4: Now that we have accepted that functions are objects and can be returned from other functions, Python goes a step further and surprises us with Function Closures. Lets see what it means with an example:
def outer(x):
   y = 20
   def inner():
      print x * y
   inner()         #1
   return inner

func = outer(10)
func()            #2
#1 -- prints 200 as inner function is able to access the variable present in its outer function scope.
#2 -- print 200.  This is a bit surprising as we expect an error. But when the inner method is returned, Python has packed up all the variables needed by the inner function into its scope. To check the values you can execute:
func.__closure__[0].cell_contents  #prints 10
func.__closure__[1].cell_contents  #prints 20

Lets end this article, with one of my favorite gotcha
Scenario 5 : Parameter life time
 def appendToList(x, list=[]):
   list.append(x)
   return list
appendToList(10)   #Prints [10]
appendToList(20)   #Prints [10, 20]
The reason for this behavior is linked to Scenario 3. Functions are objects, remember? So, when the function object is created the list object is created add added to it. The same list object is accessed for all subsequent calls leading to the output as shown above.

Comments

Popular posts from this blog

Openstack : Fixing Failed to create network. No tenant network is available for allocation issue.

Assumptions : You are using ML2 plugin configured to use Vlans If you try to create a network for a tenant and it fails with the following error: Error: Failed to create network "Test": 503-{u'NeutronError': {u'message': u'Unable to create the network. No tenant network is available for allocation.', u'type': u'NoNetworkAvailable', u'detail': u''}} The problem can be due to missing configuration in the below files: In /etc/neutron/plugins/ml2/ml2_conf.ini network_vlan_ranges =physnet1:1000:2999 (1000:2999 is the Vlan range allocation) In /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini bridge_mappings = physnet1:br-eth1 (in OVS we map the physical network to the OVS bridge) Note You should have created a bridge br-eth1 manually and mapped it to a port ovs-vsctl add-br br-eth1 ovs-vsctl add-port br-eth1 eth1 Once configuration is done, restart the neutron ovs agent on the compute node(s):

Solved: Fix for Git clone failure due to GnuTLS recv error (-9)

My devstack installation was failing with an error reported by the GnuTLS module as shown below: $ git clone https://github.com/openstack/horizon.git /opt/stack/horizon --branch master Cloning into '/opt/stack/horizon'... remote: Counting objects: 154213, done. remote: Compressing objects: 100% (11/11), done. error: RPC failed; curl 56 GnuTLS recv error (-9): A TLS packet with unexpected length was received. fatal: The remote end hung up unexpectedly fatal: early EOF fatal: index-pack failed The following Git config changes fixed the issue for me. Am hoping it will be useful for someone out there: $ git config http.sslVerify false $ git config --global http.postBuffer 1048576000

QuickBite: Tap Vs Veth

Linux supports virtual networking via various artifacts such as: Soft Switches (Linux Bridge, OpenVSwitch) Virtual Network Adapters (tun, tap, veth and a few more) In this blog, we will look at the virtual network adapters tap and veth. From a practical view point, both seem to be having the same functionality and its a bit confusing as to where to use what. A quick definition of tap/veth is as follows: TAP A TAP is a simulated interface which exists only in the kernel and has no physical component associated with it. It can be viewed as a simple Point-to-Point or Ethernet device, which instead of receiving packets from a physical media, receives them from user space program and instead of sending packets via physical media writes them to the user space program. When a user space program (in our case the VM) gets attached to the tap interface it gets hold of a file descriptor, reading from which gives it the data being sent on the tap interface. Writing to the file descri