{ "metadata": { "name": "Python Byte Code Hacking" }, "nbformat": 3, "nbformat_minor": 0, "worksheets": [ { "cells": [ { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Python Byte Code Hacking" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Python Interpreter" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "* Python converts programs into byte code instructions before execution\n", "* Python objects are stored in a heap\n", "* Python interpreter creates, modifies and destroys objects in the heap based on instrunctions in the byte code.\n", "* Python interpreter is modelled after microprocessors\n", "* Just as a microprocessor reads and executes instructions, the Python interpreter also reads and execute byte code instructions\n", "* Instructions act on operands and produce a result\n", "\n", "![Python VM and Heap](files/py-vm-heap.png)\n", "\n", "* The byte code instructions are themselves part of code objects\n", "* They are also stored in the heap\n", "\n", "![Code Objects](files/code-objects.png)" ] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Stack Machine" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "* Interpreters can be classified into types on how the operands / results are stored -- Register Machines and Stack Machines\n", "* Demostrate a hypothetical Stack Machine with an example.\n", "\n", " loadi 1\n", " loadi 2\n", " add\n", " loadi 3\n", " add\n", " \n", "* Executeion of the instructions is shown in the following diagram\n", "\n", "![Stack Machine](files/stack-machine.png)" ] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Python's Stack Machine" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "* Python uses the Stack Machine approach\n", "* Internal stack is a temporary area that contains pointers to Python objects in the heap.\n", "* Instructions push and pop data on to the stack.\n", "\n", "![Python's Stack Machine](files/py-vm-stack.png)" ] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Hello Byte Code" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here is a simple hello world function." ] }, { "cell_type": "code", "collapsed": false, "input": [ "def hello():\n", " print \"Hello World\"" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 3 }, { "cell_type": "markdown", "metadata": {}, "source": [ "The function object has a code object." ] }, { "cell_type": "code", "collapsed": false, "input": [ "print hello.__code__" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "\", line 1>\n" ] } ], "prompt_number": 4 }, { "cell_type": "markdown", "metadata": {}, "source": [ "* The code object has the byte code and data required for the code to execute.\n", "* The byte codes can be accessed through `co_code` attribute." ] }, { "cell_type": "code", "collapsed": false, "input": [ "byte_codes = hello.__code__.co_code\n", "for byte in byte_codes:\n", " print hex(ord(byte)),\n", "print" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "0x64 0x1 0x0 0x47 0x48 0x64 0x0 0x0 0x53\n" ] } ], "prompt_number": 5 }, { "cell_type": "markdown", "metadata": {}, "source": [ "* One data required for the code to execute is a list of constants.\n", "* The byte code refer to these constants using an index into this list.\n", "* The list of constants can be accessed through the `co_consts` attribute." ] }, { "cell_type": "code", "collapsed": false, "input": [ "print hello.__code__.co_consts" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "(None, 'Hello World')\n" ] } ], "prompt_number": 6 }, { "cell_type": "markdown", "metadata": {}, "source": [ "The byte code can be disassembled and be viewed in a more human readable form using the `dis` module." ] }, { "cell_type": "code", "collapsed": false, "input": [ "import dis\n", "dis.disassemble(hello.__code__)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ " 2 0 LOAD_CONST 1 ('Hello World')\n", " 3 PRINT_ITEM \n", " 4 PRINT_NEWLINE \n", " 5 LOAD_CONST 0 (None)\n", " 8 RETURN_VALUE \n" ] } ], "prompt_number": 7 }, { "cell_type": "markdown", "metadata": {}, "source": [ " * `LOAD_CONST` pushes a constant on to the top of the stack. The index of the constant into `co_consts` is specified as operand.\n", " * `PRINT_ITEM` prints TOS to `sys.stdout`.\n", " * `RETURN_VALUE` returns the TOS to the caller." ] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Global Variables" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's look at a more involved function and its byte code." ] }, { "cell_type": "code", "collapsed": false, "input": [ "a = 10\n", "b = 20\n", "def sum_ab():\n", " return a + b" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 8 }, { "cell_type": "markdown", "metadata": {}, "source": [ "The byte code disassembly:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "dis.disassemble(sum_ab.__code__)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ " 4 0 LOAD_GLOBAL 0 (a)\n", " 3 LOAD_GLOBAL 1 (b)\n", " 6 BINARY_ADD \n", " 7 RETURN_VALUE \n" ] } ], "prompt_number": 9 }, { "cell_type": "markdown", "metadata": {}, "source": [ "* `LOAD_GLOBAL` gets the object pointed by the global variable, and pushes a pointer to it on the stack.\n", "* Global variable names used by the function are stored in as list in `co_names` attribute.\n", "* `LOAD_GLOBAL` accepts the index into the list as operand.\n", "* The global variable name is then used to retrieve the object." ] }, { "cell_type": "code", "collapsed": false, "input": [ "print sum_ab.__code__.co_names" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "('a', 'b')\n" ] } ], "prompt_number": 10 }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Attribute Access" ] }, { "cell_type": "code", "collapsed": false, "input": [ "str = \" Hello World \"\n", "def trimmed():\n", " return str.strip()" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 11 }, { "cell_type": "code", "collapsed": false, "input": [ "dis.disassemble(trimmed.__code__)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ " 3 0 LOAD_GLOBAL 0 (str)\n", " 3 LOAD_ATTR 1 (strip)\n", " 6 CALL_FUNCTION 0\n", " 9 RETURN_VALUE \n" ] } ], "prompt_number": 12 }, { "cell_type": "markdown", "metadata": {}, "source": [ "* `LOAD_ATTR` pops the object pointer from the stack.\n", "* `LOAD_ATTR` accepts the index into the `co_names` as operand.\n", "* Finds the object pointed to by the named attribute and pushes it into the stack." ] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Loops" ] }, { "cell_type": "code", "collapsed": false, "input": [ "def hello_loop():\n", " while True:\n", " print \"Hello\"" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 13 }, { "cell_type": "code", "collapsed": false, "input": [ "dis.disassemble(hello_loop.__code__)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ " 2 0 SETUP_LOOP 15 (to 18)\n", " >> 3 LOAD_GLOBAL 0 (True)\n", " 6 POP_JUMP_IF_FALSE 17\n", "\n", " 3 9 LOAD_CONST 1 ('Hello')\n", " 12 PRINT_ITEM \n", " 13 PRINT_NEWLINE \n", " 14 JUMP_ABSOLUTE 3\n", " >> 17 POP_BLOCK \n", " >> 18 LOAD_CONST 0 (None)\n", " 21 RETURN_VALUE \n" ] } ], "prompt_number": 14 }, { "cell_type": "markdown", "metadata": {}, "source": [ "* `POP_JUMP_IF_FALSE` pops the object pointer from the stack.\n", "* Determine the truth value of the object.\n", "* If false, jump to absolute address specified as argument\n", "* `JUMP ABSOLUTE`, jumps to the absolute address specified as argument" ] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Goto Implementation" ] }, { "cell_type": "code", "collapsed": false, "input": [ "def test_goto(n):\n", " s = 0\n", "\n", " label .myLoop\n", "\n", " if n <= 0:\n", " return s\n", " s += n\n", " n -= 1\n", "\n", " goto .myLoop" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 15 }, { "cell_type": "code", "collapsed": false, "input": [ "dis.disassemble(test_goto.__code__)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ " 2 0 LOAD_CONST 1 (0)\n", " 3 STORE_FAST 1 (s)\n", "\n", " 4 6 LOAD_GLOBAL 0 (label)\n", " 9 LOAD_ATTR 1 (myLoop)\n", " 12 POP_TOP \n", "\n", " 6 13 LOAD_FAST 0 (n)\n", " 16 LOAD_CONST 1 (0)\n", " 19 COMPARE_OP 1 (<=)\n", " 22 POP_JUMP_IF_FALSE 29\n", "\n", " 7 25 LOAD_FAST 1 (s)\n", " 28 RETURN_VALUE \n", "\n", " 8 >> 29 LOAD_FAST 1 (s)\n", " 32 LOAD_FAST 0 (n)\n", " 35 INPLACE_ADD \n", " 36 STORE_FAST 1 (s)\n", "\n", " 9 39 LOAD_FAST 0 (n)\n", " 42 LOAD_CONST 2 (1)\n", " 45 INPLACE_SUBTRACT \n", " 46 STORE_FAST 0 (n)\n", "\n", " 11 49 LOAD_GLOBAL 2 (goto)\n", " 52 LOAD_ATTR 1 (myLoop)\n", " 55 POP_TOP \n", " 56 LOAD_CONST 0 (None)\n", " 59 RETURN_VALUE \n" ] } ], "prompt_number": 16 }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Start by Writing a Disassembler" ] }, { "cell_type": "code", "collapsed": false, "input": [ "def ibytecode(byte_code):\n", " byte_len = len(byte_code)\n", " \n", " i = 0\n", " while i < byte_len:\n", " addr = i\n", " opcode = ord(byte_code[i])\n", " if opcode > dis.HAVE_ARGUMENT:\n", " i += 1\n", " operand = ord(byte_code[i])\n", " \n", " i += 1\n", " operand += ord(byte_code[i]) << 8\n", " yield addr, dis.opname[opcode], operand\n", " else:\n", " yield addr, dis.opname[opcode], None\n", " i += 1" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 55 }, { "cell_type": "code", "collapsed": false, "input": [ "def mydisassembler(byte_code):\n", " for addr, opcode, operand in ibytecode(byte_code):\n", " if operand != None:\n", " print \"%d\\t%s(%d)\" % (addr, opcode, operand)\n", " else:\n", " print \"%d\\t%s\" % (addr, opcode)\n", " \n", "byte_code = test_goto.__code__.co_code\n", "mydisassembler(byte_code)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "0\tLOAD_CONST(1)\n", "3\tSTORE_FAST(1)\n", "6\tLOAD_GLOBAL(0)\n", "9\tLOAD_ATTR(1)\n", "12\tPOP_TOP\n", "13\tLOAD_FAST(0)\n", "16\tLOAD_CONST(1)\n", "19\tCOMPARE_OP(1)\n", "22\tPOP_JUMP_IF_FALSE(29)\n", "25\tLOAD_FAST(1)\n", "28\tRETURN_VALUE\n", "29\tLOAD_FAST(1)\n", "32\tLOAD_FAST(0)\n", "35\tINPLACE_ADD\n", "36\tSTORE_FAST(1)\n", "39\tLOAD_FAST(0)\n", "42\tLOAD_CONST(2)\n", "45\tINPLACE_SUBTRACT\n", "46\tSTORE_FAST(0)\n", "49\tLOAD_GLOBAL(2)\n", "52\tLOAD_ATTR(1)\n", "55\tPOP_TOP\n", "56\tLOAD_CONST(0)\n", "59\tRETURN_VALUE\n" ] } ], "prompt_number": 56 }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Extract Label Addresses" ] }, { "cell_type": "code", "collapsed": false, "input": [ "def get_labels(code_obj):\n", " labels = {}\n", " \n", " for addr, opcode, operand in ibytecode(code_obj.co_code):\n", " if opcode == \"LOAD_GLOBAL\":\n", " global_name = code_obj.co_names[operand]\n", " label_addr = addr\n", " \n", " if opcode == \"LOAD_ATTR\":\n", " if global_name == \"label\":\n", " label_name = code_obj.co_names[operand]\n", " labels[label_name] = label_addr\n", " \n", " return labels\n", "\n", "labels = get_labels(test_goto.__code__)\n", "for name, addr in labels.iteritems():\n", " print \"%s: %d\" % (name, addr)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "myLoop: 6\n" ] } ], "prompt_number": 57 }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Extract Goto Locations" ] }, { "cell_type": "code", "collapsed": false, "input": [ "def get_gotos(code_obj):\n", " gotos = {}\n", " \n", " for addr, opcode, operand in ibytecode(code_obj.co_code):\n", " if opcode == \"LOAD_GLOBAL\":\n", " global_name = code_obj.co_names[operand]\n", " goto_addr = addr\n", " \n", " if opcode == \"LOAD_ATTR\":\n", " if global_name == \"goto\":\n", " label_name = code_obj.co_names[operand]\n", " gotos[label_name] = goto_addr\n", " \n", " return gotos\n", " \n", "gotos = get_gotos(test_goto.__code__)\n", "for name, addr in gotos.iteritems():\n", " print \"%d\\tgoto %s\" % (addr, name)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "49\tgoto myLoop\n" ] } ], "prompt_number": 58 }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "NOP the Labels" ] }, { "cell_type": "code", "collapsed": false, "input": [ "def nop_labels(byte_code, labels):\n", " byte_list = list(byte_code)\n", " for label_name, label_addr in labels.iteritems():\n", " label_start = label_addr\n", " label_end = label_addr + 7\n", " \n", " nop = chr(dis.opmap[\"NOP\"])\n", " byte_list[label_start:label_end] = [nop] * 7\n", " \n", " return \"\".join(byte_list)\n", " \n", "byte_code = test_goto.__code__.co_code\n", "labels = get_labels(test_goto.__code__)\n", "noped = nop_labels(byte_code, labels)\n", "\n", "mydisassembler(noped)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "0\tLOAD_CONST(1)\n", "3\tSTORE_FAST(1)\n", "6\tNOP\n", "7\tNOP\n", "8\tNOP\n", "9\tNOP\n", "10\tNOP\n", "11\tNOP\n", "12\tNOP\n", "13\tLOAD_FAST(0)\n", "16\tLOAD_CONST(1)\n", "19\tCOMPARE_OP(1)\n", "22\tPOP_JUMP_IF_FALSE(29)\n", "25\tLOAD_FAST(1)\n", "28\tRETURN_VALUE\n", "29\tLOAD_FAST(1)\n", "32\tLOAD_FAST(0)\n", "35\tINPLACE_ADD\n", "36\tSTORE_FAST(1)\n", "39\tLOAD_FAST(0)\n", "42\tLOAD_CONST(2)\n", "45\tINPLACE_SUBTRACT\n", "46\tSTORE_FAST(0)\n", "49\tLOAD_GLOBAL(2)\n", "52\tLOAD_ATTR(1)\n", "55\tPOP_TOP\n", "56\tLOAD_CONST(0)\n", "59\tRETURN_VALUE\n" ] } ], "prompt_number": 62 }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Replace Gotos by JUMP_ABSOLUTE" ] }, { "cell_type": "code", "collapsed": false, "input": [ "def inject_jump(byte_code, gotos, labels):\n", " byte_list = list(byte_code)\n", " for label_name, goto_addr in gotos.iteritems():\n", " label_addr = labels[label_name]\n", " \n", " jump_abs = chr(dis.opmap[\"JUMP_ABSOLUTE\"])\n", " nop = chr(dis.opmap[\"NOP\"])\n", " \n", " byte_list[goto_addr] = jump_abs\n", " byte_list[goto_addr + 1] = chr(label_addr & 0xFF)\n", " byte_list[goto_addr + 2] = chr(label_addr >> 8)\n", " byte_list[goto_addr + 3] = nop\n", " byte_list[goto_addr + 4] = nop\n", " byte_list[goto_addr + 5] = nop\n", " byte_list[goto_addr + 6] = nop\n", " \n", " return \"\".join(byte_list)\n", " \n", "byte_code = test_goto.__code__.co_code\n", "gotos = get_gotos(test_goto.__code__)\n", "injected = inject_jump(noped, gotos, labels)\n", "\n", "mydisassembler(injected)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "0\tLOAD_CONST(1)\n", "3\tSTORE_FAST(1)\n", "6\tNOP\n", "7\tNOP\n", "8\tNOP\n", "9\tNOP\n", "10\tNOP\n", "11\tNOP\n", "12\tNOP\n", "13\tLOAD_FAST(0)\n", "16\tLOAD_CONST(1)\n", "19\tCOMPARE_OP(1)\n", "22\tPOP_JUMP_IF_FALSE(29)\n", "25\tLOAD_FAST(1)\n", "28\tRETURN_VALUE\n", "29\tLOAD_FAST(1)\n", "32\tLOAD_FAST(0)\n", "35\tINPLACE_ADD\n", "36\tSTORE_FAST(1)\n", "39\tLOAD_FAST(0)\n", "42\tLOAD_CONST(2)\n", "45\tINPLACE_SUBTRACT\n", "46\tSTORE_FAST(0)\n", "49\tJUMP_ABSOLUTE(6)\n", "52\tNOP\n", "53\tNOP\n", "54\tNOP\n", "55\tNOP\n", "56\tLOAD_CONST(0)\n", "59\tRETURN_VALUE\n" ] } ], "prompt_number": 65 }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Putting It All Together" ] }, { "cell_type": "code", "collapsed": false, "input": [ "import new\n", "\n", "def goto_func(func):\n", " code_obj = func.__code__\n", " byte_code = func.__code__.co_code\n", " labels = get_labels(func.__code__)\n", " gotos = get_gotos(func.__code__)\n", " noped = nop_labels(byte_code, labels)\n", " new_byte_code = inject_jump(noped, gotos, labels)\n", " \n", " new_obj = new.code(code_obj.co_argcount,\n", " code_obj.co_nlocals,\n", " code_obj.co_stacksize,\n", " code_obj.co_flags,\n", " new_byte_code,\n", " code_obj.co_consts,\n", " code_obj.co_names,\n", " code_obj.co_varnames,\n", " code_obj.co_filename,\n", " code_obj.co_name,\n", " code_obj.co_firstlineno,\n", " code_obj.co_lnotab)\n", " \n", " return new.function(new_obj, func.func_globals)\n", " \n", "test_goto = goto_func(test_goto)\n", "print test_goto(10)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "55\n" ] } ], "prompt_number": 70 } ], "metadata": {} } ] }