Clouds, Networking and Technology

Scrapy in the words of its creators: "Scrapy is an application framework for crawling web sites and extracting structured data which can be used for a wide range of useful applications, like data mining, information processing or historical archival."  A screenshot grabbed from the site shows how concise the working code can be: Scrapy works only with Python 2.7. The objective of this blog is to get you started with Scrapy and provide you with enough information to carry further on your own. In this blog, I will setup a scrapy project and retrieve some data off my blog site. Pre-requsites Python 2.7 pip and setuptools Python packages. lxml . Most Linux distributions ships prepackaged versions of lxml. Otherwise refer to http://lxml.de/installation.html OpenSSL . This comes preinstalled in all operating systems, except Windows where the Python installer ships it bundled. Setup pip install scrapy Create a Project scrapy startproject blog This comman...

In this blog, I will walk you through the steps that need to performed to create a MultiNode OpenStack Kilo setup running on Centos 7 VMs. The resultant network topology will be as follows: Pre-requisites VirtualBox A Laptop or Server with 16 GB RAM and around 20 GB free hard disk space CentOS 7 Minimal ISO Internet Connectivity

I have written a blog on Juniper's solution for FWaaS plugin which integrates SRX/vSRX with OpenStack Firewall flow. It has got published on Juniper's website. You can read it here

When inter VLAN routing needs to be configured on Juniper devices the first thing that comes to mind is to use RVI (SVI in cisco land) and be done with it. But, there are certain situations where this approach may not work and this article explores the alternative ways of configuring inter VLAN routing on Juniper devices. Lets say we have a router on a stick topology where an MX/SRX is acting as the router. Depending upon whether its MX or SRX the approach to configure inter VLAN routing varies. The below picture acts as our reference topology for this article: In this topology, we have a switch which has two VLANs 100 & 200 and the tagged packets are sent across to MX/SRX on a trunk port ge-0/0/1. VLAN 100 is assigned to a subnet 10.1.0.0/24 having a gateway ip set to 10.1.0.1. Similarly, VLAN 200 is assigned to a subnet 10.2.0.0/24 having a gateway ip set to 10.2.0.1 Note : In this article I will use an RI instead of the global routing table. Scenario 1 (RVI) ...

I had a quick dab at Django & Openstack Horizon frameworks in order to build a dashboard for one of our Openstack projects. Our requirement was to show a table on the UI with the corresponding CRUD functions. I had used DataTable for this purpose, where in you define the table structure in tables.py and the view rendering is defined in views.py If a column is defined to be a link, DataTable by default uses the object id of each row to generate the corresponding link. In my scenario, I needed the link to point to a different object. Openstack's documentation was not very helpful and a quick grep through the openstack's code gave me the idea. The approach to generate a custom link is as follows inside a DataTable: from django.core.urlresolvers import reverse def get_custom_link(datum):     return reverse('horizon:myproject:mydashboard:detail', kwargs={'key': datum.value}) The value of the key will be used to generate the URL. ...

Linux supports virtual networking via various artifacts such as: Soft Switches (Linux Bridge, OpenVSwitch) Virtual Network Adapters (tun, tap, veth and a few more) In this blog, we will look at the virtual network adapters tap and veth. From a practical view point, both seem to be having the same functionality and its a bit confusing as to where to use what. A quick definition of tap/veth is as follows: TAP A TAP is a simulated interface which exists only in the kernel and has no physical component associated with it. It can be viewed as a simple Point-to-Point or Ethernet device, which instead of receiving packets from a physical media, receives them from user space program and instead of sending packets via physical media writes them to the user space program. When a user space program (in our case the VM) gets attached to the tap interface it gets hold of a file descriptor, reading from which gives it the data being sent on the tap interface. Writing to the file descri...

I have a VM on virtualbox which acts as an all-in-one Openstack setup. When I spawn a VM on it the VM boots up fine but from the browser I am not able to access its console. There are various ways to solve this issue: 1. In the latest version of devstack (as on February 2015) n-novnc is no longer a default service and needs to be added to the local.conf to enable it. enabled_services=n-novnc  ( https://ask.openstack.org/en/question/57993/dashboard-vnc-console-doesnt-work-on-devstack/ ) 2. See if this helps you in tweaking manually : http://docs.openstack.org/admin-guide-cloud/content/nova-vncproxy-replaced-with-nova-novncproxy.html 3.Another geeky solution is that, you can grep the KVM process to figure out the port on which the VNC is getting channeled to and access the console. openstack@Openstack-Server:~/devstack$ ps aux|grep qemu|grep vnc libvirt+  9167  0.6  2.4 1504044 97488 ?       Sl   14:25   ...