A compact introduction to the P4 programming language


alt text

Hey there! I finally got time to write about P4 and what it seeks to accomplish for the networking field. I hope you are as excited as I am!

Goal for today

I am going to first tell you exactly what P4 is, the advantages that it brings to the networking hardware of the future and then move on to the programming to be done for today. The programming involves solving and running one of the SIGCOMM 2017 workshop exercises. Since P4 is written on C/C++ backends, one would have to compile a lot of libraries from source and that would be unnecessarily convoluted for a beginner. Hence, we will use the p4app tool to circumvent this problem (It uses Docker!). It is also important to understand that this is a super-fast introduction in relative terms: It might still be considered lengthy but will probably be the fastest introduction anyone has ever given to P4lang, including the programming aspect of it.

What is P4 and what does it do?

In very simple words, P4 allows a network engineer to program how switches process packets. Currently, if you want some special features for your router, you would talk to your vendor to be able to get them in the next version. Hence, every time you require a new feature the hardware needs to be changed to support it. P4 wishes to eliminate all of that. It is a Domain Specific Language that seeks to incorporate general programmability into routers and switches so that if you want new functionality in your device you can just program it. Most hardware doesn’t support P4 yet, but Barefoot Networks and a lot of major players in the networking field are working towards making P4 supported hardware a reality. Barefoot Networks recently came out with Tofino, which is the world’s first end-user programmable Ethernet switch. I am optimistic that very soon the entire industry would be using hardware that can fully utilize the power of a programmable forwarding dataplane.

alt text

Installation and set-up

p4app

We are just required to install and configure p4app. The installation is a very simple process and can be undertaken by following the steps below:

  • Make sure you have docker on your machine
  • Clone the p4app github repository by typing git clone https://github.com/p4lang/p4app
  • cd into the repo by typing cd p4app
  • To get everything up and running simply run one of their examples. This can be done by the following command: sudo ./p4app examples/simple_router.p4app

For the rest of the set-up we need to take care of a few more things. Also running P4 apps and everything else on how and what we need to run p4app, will be covered in the next section. For now, we are concerned with how to just set everything up.

So the SIGCOMM exercise requires the use of certain scripts by the Mininet hosts called send.py and receive.py. Both these scripts are present here and are referenced for usage later on in the Post Set-up section. For that we will require a script of some sort that can transfer the files into the Docker container on it’s own every time to be available to the p4app program. I have written a script that does this called script.py. You could write your own interpretation for doing this or just use my code. Here is script.py:

import re
import os

output = os.popen('sudo docker ps -a| grep "p4app_"').read()

re1='.*?'
re2='(?:[a-z][a-z0-9_]*)'
re3='.*?'
re4='(?:[a-z][a-z0-9_]*)'
re5='.*?'
re6='(?:[a-z][a-z0-9_]*)'
re7='.*?'
re8='(?:[a-z][a-z0-9_]*)'
re9='.*?'
re10='(?:[a-z][a-z0-9_]*)'
re11='.*?'
re12='(?:[a-z][a-z0-9_]*)'
re13='.*?'
re14='(?:[a-z][a-z0-9_]*)'
re15='.*?'
re16='(?:[a-z][a-z0-9_]*)'
re17='.*?'
re18='(?:[a-z][a-z0-9_]*)'
re19='.*?'
re20='(?:[a-z][a-z0-9_]*)'
re21='.*?'
re22='(?:[a-z][a-z0-9_]*)'
re23='.*?'
re24='((?:[a-z][a-z0-9_]*))'


container_name = ''

rg = re.compile(re1+re2+re3+re4+re5+re6+re7+re8+re9+re10+re11+re12+re13+re14+re15+re16+re17+re18+re19+re20+re21+re22+re23+re24,re.IGNORECASE|re.DOTALL)
m = rg.search(output)
if m:
    container_name = str(m.group(1))

else:
    print "[ERROR] Run this script only after you have run the p4app for scrambler.p4app"

os.system('sudo docker cp send.py ' + container_name + '://scripts/') 
os.system('sudo docker cp receive.py ' + container_name + '://scripts/')

print "[INFO] Operation successful"

The code is fairly straightforward. I am first issuing the command sudo docker ps -a | grep "p4app_". The output of this command would look something like this:

77a6c735246a        p4lang/p4app:latest   "./p4apprunner.py"   8 days ago          Exited (0) 8 days ago       p4app_123456451

The last column here refers to the container name. We need this to be able to perform any actions on it. So I use simple regular expression matching (again, you could create something that suits you better) and find this out. Then I finally execute the cp (copy) command on the docker container and ensure that the scripts are copied in the scripts directory in the container.

Now that we have this ready, just place script.py in the p4app directory.

After this is done create a scrambler.p4app empty directory in the p4app directory. Then we need to create a custom controller for Mininet BMV2 (P4 Behavioural Model 2) that can allow us to have custom tables and a custom control plane (required for the SIGCOMM exercise). Currently p4app forces these things to certain defaults. To do this, go to p4app/docker/scripts/mininet/ in the p4app directory and copy over the appcontroller.py and shortest_path.py files over to the newly created scrambler.p4app directory.

We will only be changing the appcontroller.py file. The following steps need to be performed:

  • Rename appcontroller.py to mycontroller.py
  • Open mycontroller.py and in there change the name of the class from AppController to CustomAppController
  • Then finally find the generateDefaultCommands() function and comment out everything present in it. Do not comment out the function, but just it’s content. It should look like this:

alt text 2

Post Set-up

We are done with all the set-up. Now what is remaining is to solve the Scrambler exercise and then run it as a p4app. To solve the exercise, first go to this relevant starting code here and clone it/download it. Move send.py and receive.py to the p4app directory. Go through the README once as well. The problem is fairly simple– we need to “flip” the Ipv4 and Ethernet addresses as they have been scrambled/flipped by the control plane. Flipping essentially translates to taking the one’s complement. Therefore we can just use the one’s complement bitwise operator of C/C++ ~ to flip the addresses. I am first going to post the entire scrambler.p4 file with the solution here and then briefly explain parts of the code. Here is the solved scrambler.p4:

/* -*- P4_16 -*- */
#include <core.p4>
#include <v1model.p4>

const bit<16> TYPE_IPV4 = 0x800;

/*************************************************************************
*********************** H E A D E R S  ***********************************
*************************************************************************/

typedef bit<9>  egressSpec_t;
typedef bit<48> macAddr_t;
typedef bit<32> ip4Addr_t;

header ethernet_t {
    macAddr_t dstAddr;
    macAddr_t srcAddr;
    bit<16>   etherType;
}

header ipv4_t {
    bit<4>    version;
    bit<4>    ihl;
    bit<8>    diffserv;
    bit<16>   totalLen;
    bit<16>   identification;
    bit<3>    flags;
    bit<13>   fragOffset;
    bit<8>    ttl;
    bit<8>    protocol;
    bit<16>   hdrChecksum;
    ip4Addr_t srcAddr;
    ip4Addr_t dstAddr;
}

struct metadata {
    /* empty */
}

struct headers {
    ethernet_t   ethernet;
    ipv4_t       ipv4;
}

/*************************************************************************
*********************** P A R S E R  ***********************************
*************************************************************************/

parser MyParser(packet_in packet,
                out headers hdr,
                inout metadata meta,
                inout standard_metadata_t standard_metadata) {

    state start {
        transition parse_ethernet;
    }

    state parse_ethernet {
        packet.extract(hdr.ethernet);
        transition select(hdr.ethernet.etherType) {
            TYPE_IPV4: parse_ipv4;
            default: accept;
        }
    }

    state parse_ipv4 {
        packet.extract(hdr.ipv4);
        transition accept;
    }
}


/*************************************************************************
************   C H E C K S U M    V E R I F I C A T I O N   *************
*************************************************************************/

control MyVerifyChecksum(inout headers hdr, inout metadata meta) {
    apply {  }
}


/*************************************************************************
**************  I N G R E S S   P R O C E S S I N G   *******************
*************************************************************************/

control MyIngress(inout headers hdr,
                  inout metadata meta,
                  inout standard_metadata_t standard_metadata) {
    action drop() {
        mark_to_drop();
    }

    action ipv4_forward(macAddr_t dstAddr, egressSpec_t port) {
        standard_metadata.egress_spec = port;
        hdr.ethernet.srcAddr = hdr.ethernet.dstAddr;
        hdr.ethernet.dstAddr = dstAddr;
        hdr.ipv4.ttl = hdr.ipv4.ttl - 1;
	hdr.ethernet.srcAddr = ~hdr.ethernet.srcAddr;
    	hdr.ethernet.dstAddr = ~hdr.ethernet.dstAddr;
    	hdr.ipv4.srcAddr = ~hdr.ipv4.srcAddr;
      	hdr.ipv4.dstAddr = ~hdr.ipv4.dstAddr;
    }
    
    table ipv4_lpm {
        key = {
            hdr.ipv4.dstAddr: lpm;
        }
        actions = {
            ipv4_forward;
            drop;
            NoAction;
        }
        size = 1024;
        default_action = NoAction();
    }
    
    apply {
        if (hdr.ipv4.isValid()) {
            ipv4_lpm.apply();
        }
    }
}

/*************************************************************************
****************  E G R E S S   P R O C E S S I N G   *******************
*************************************************************************/

control MyEgress(inout headers hdr,
                 inout metadata meta,
                 inout standard_metadata_t standard_metadata) {
    apply {  }
}

/*************************************************************************
*************   C H E C K S U M    C O M P U T A T I O N   **************
*************************************************************************/

control MyComputeChecksum(inout headers hdr, inout metadata meta) {
     apply {
	update_checksum(
	    hdr.ipv4.isValid(),
            { hdr.ipv4.version,
	      hdr.ipv4.ihl,
              hdr.ipv4.diffserv,
              hdr.ipv4.totalLen,
              hdr.ipv4.identification,
              hdr.ipv4.flags,
              hdr.ipv4.fragOffset,
              hdr.ipv4.ttl,
              hdr.ipv4.protocol,
              hdr.ipv4.srcAddr,
              hdr.ipv4.dstAddr },
            hdr.ipv4.hdrChecksum,
            HashAlgorithm.csum16);
    }
}

/*************************************************************************
***********************  D E P A R S E R  *******************************
*************************************************************************/

control MyDeparser(packet_out packet, in headers hdr) {
    apply {
        packet.emit(hdr.ethernet);
        packet.emit(hdr.ipv4);
    }
}

/*************************************************************************
***********************  S W I T C H  *******************************
*************************************************************************/

V1Switch(
MyParser(),
MyVerifyChecksum(),
MyIngress(),
MyEgress(),
MyComputeChecksum(),
MyDeparser()
) main;

The first section of the code labeled **** H E A D E R S **** is just including all the header files and defining macros and structs that will be utilized in the code later on. The inclusion of code.p4 is standard to every program and v1model.p4 defines the ingress/egress pipelines, parsing and deparsing, etc.

The second section called **** P A R S E R **** is defining the parser for the ingress pipeline. The third section **** C H E C K S U M - V E R I F I C A T I O N **** could have incorporated custom actions as part of checksum verification for the packet header but instead apply is just left empty because we do not require it for this application. **** I N G R E S S - P R O C E S S I N G **** contains the code for the ingress match-action pipeline with just one table ipv4_lpm with the name signifying the longest prefix matching being done on the IP addresses by the router. The table has two actions- packet dropping (drop) and forwarding (ipv4_forward). In the ipv4_forward we add the code to unscramble our addresses as can be seen in the following lines of code:

hdr.ethernet.srcAddr = ~hdr.ethernet.srcAddr;
hdr.ethernet.dstAddr = ~hdr.ethernet.dstAddr;
hdr.ipv4.srcAddr = ~hdr.ipv4.srcAddr;
hdr.ipv4.dstAddr = ~hdr.ipv4.dstAddr;

**** E G R E S S - P R O C E S S I N G ****, **** C H E C K S U M - C O M P U T A T I O N ****, **** D E P A R S E R **** and **** S W I T C H **** are very straightforward and do not involve writing any custom code. Reading through the code once should give you enough of an idea to understand what is happening.

Running everything

To run everything first remember to copy over these files from the scrambler SIGCOMM tutorial exercise directory (and ones created previously in this tutorial) into the scrambler.p4app directory created inside the p4app directory:

  • scrambler.p4 (solved previously)
  • s1-commands.txt
  • s2-commands.txt
  • s3-commands.txt

As mentioned in the SIGCOMM exercise README, the files s1-commands.txt, s2-commands.txt and s3-commands.txt are used for setting the default control plane (controller) rules.

The last step is to include a configuration file called p4app.json inside the scrambler.p4app directory. This is required to run a P4 app with p4app. The contents of this file are shown below and pretty self-explanatory:

{
    "program": "scrambler.p4",
    "language": "p4-16",
    "targets": {
      "multiswitch": {
	  "controller_module": "mycontroller",
          "auto-control-plane": true,
          "cli": true,
          "links": [["h1", "s1"], ["s1", "s2"], ["s1", "s3"], ["s3", "s2"], ["s2", "h2"], ["s3", "h3"]],
          "hosts": {
            "h1": {
            },
            "h2": {
            },
            "h3": {
            }

          },
          "switches": {
              "s1": {
                  "commands": "s1-commands.txt"
              },
              "s2": {
                  "commands": "s2-commands.txt"
              },
              "s3": {
                  "commands": "s3-commands.txt"
              }
          }
	    }
    }
}

The way this json configuration file is to be written is extensively detailed in the p4app repository’s README. However, for the purposes of this tutorial the one given above will perform the task at hand. As a final sanity check, the directories and file structure containing all the essential files and scripts should look like as given below. If it doesn’t look this way for you, make sure you go through the tutorial again and find out what you missed.

p4app
    |
    |- send.py
    |
    |- receive.py
    |
    |- p4app
    |
    |- scrambler.p4app
    		|
    		|- p4app.json
    		|
    		|- scrambler.p4
    		|
		|- s1.commands.txt
    		|
    		|- s2.commands.txt
    		|
		|- s3.commands.txt		
    		|
		|- mycontroller.py
    		|
		|- shortest_path.py
    		

To finally run the app, type in the following command:

sudo ./p4app run scrambler.p4app

Mininet CLI should open up as soon as you do this without giving any errors. Next, to test whether our implementation works, open up two other terminals in the p4app directory. Then do the following:

  • In one of these, run script.py. If everything worked smoothly, the following message should be seen- [INFO] Operation successful
  • In the same terminal in which you just ran the script, run sudo ./p4app exec m h2 bash and when this executes run python receive.py
  • In the other terminal run sudo ./p4app exec m h1 bash and after this executes run python send.py 10.0.2.2 "Anshuman is awesome"

If everything was done as it was supposed to, you will see the message Anshuman is awesome on the terminal in which you executed receive.py! If you had done this before solving scrambler.p4 the message would never have been delivered. This means we have finally achieved what we set out to do!

Conclusion

Today we talked about the P4 language and saw how powerful and versatile it can make computer networks in the future. Play around with this tutorial and try and make interesting applications. There are also many wonderful tutorials on the tutorials repository maintained by Barefoot Networks on Github. Cheers and a happy new year to you! Stay tuned for more intersting posts.