A Technical Deep Dive into Python

December 14 2015
 


Python is a popular high-level programming language used for a wide variety of applications. It emphasizes readability, rapid prototyping, and small, efficient program code bases. In this article, you will gain a better understanding of Python, why it is so popular among programmers, its basic functionality and a few of its drawbacks.

Python has three distinguishing characteristics:

  • Interpreted Language. Python is an interpreted language. It is not compiled in the classic sense of a C program, which is compiled and then run. Instead, you can type in a program, and it will run right away. The disadvantage is that Python is slower than programs like C and C++, which are compiled to machine code and act directly upon the computer’s processor.

  • Dynamically Typed. Python is dynamically typed, which means it has no strict variables. A variable can be almost anything — a string, a numeric value or something else. It is good to have a type for your variables, but you do not have to. The downside with this method is it can lead to confusion. You may think something is an integer, and it is not, and vice versa.

  • Whitespace Language. Many computer languages use curly braces to contain code — Python uses whitespaces, meaning tabs and spaces.

Python is flexible and powerful at the same time. You can make games, tools, script tasks, anything. It uses a flexible approach, with variables as containers for data. You type in what you want to name a variable, followed by the equal sign and then the value you wish to store – perhaps a number or a string. It is that simple.

Different Types of Python Implementations

There are various implementations of Python. These implementations translate the code the programmer writes to bytecode.

Bytecode is different from machine code in that machine code, also known as native code, addresses a computer’s processor directly. Bytecode, sometimes called portable code, is designed to work with a virtual machine instead of the real machine, the computer’s processor. Virtual machines are software programs that emulate various computer hardware systems.

CPython

The most popular form of Python being implemented today, CPython is often thought of as the “default” Python implementation. Written in C, CPython compiles source code to bytecode. After that, it interprets the bytecode, executing it on the fly.

This is different from compiling a language like C into machine code for running directly on a computer. Machine code attains greater speeds than bytecode, but bytecode is portable across different computer platforms.

It is compiled to run on virtual machines, which are software that emulate specific computer hardware. That way, the bytecode does not have to be configured for a myriad of different computer systems.

For example, CPython first compiles the code you write in Python to bytecode. The resulting bytecode then runs on CPython Virtual Machine.

Jython

Another implementation of Python is called Jython. Instead of C, it is written in Java and is interpreted for the Java Virtual Machine (JVM).

The reason there is a variety of Python implementations is each one is better-suited for a unique technology stack. For example, Jython is geared toward a Java stack. It:

Works naturally and effortlessly with Java programs.

Imports a variety of Java classes.

Uses Java classes directly from inside the Jython program.

IronPython

Written in C#, IronPython was created by Jim Hugunin in 2006 and is designed for the .NET stack. The virtual machine it utilizes is CLR (Common Language Runtime) by Microsoft. In practical terms, CLR is the “.NET Virtual Machine.”

Volunteers at the Microsoft open-source center, CodePlex, are currently maintaining the project. IronPython integrates seamlessly with .NET application frameworks, which makes it easy for developers to use IronPython scripts with .NET objects. Another feature of IronPython is that it uses reflection extensively. That means it can evaluate itself and change its behavior as it runs. Meta-data, functions, and values can be modified at runtime.

PyPy

PyPy began life as a Python implementation coded in, that’s right, Python! Currently, it is compiled after being translated into C code. The “Just in Time” compiler turns the Python into machine code at runtime.

Let’s break this down. We said earlier that machine code runs at much greater speeds than bytecode. A method called Just in Time (JIT) compiles portions of the bytecode as it runs. It is a hybrid approach, a mash-up of compilers and interpreters. This technique looks for bytecode that is frequently used, compiles it to native code and puts the result in cache.

It is similar to the RAM on your PC, but in this case, you get significant speed gains because it will grab the popular bytecode from cache. Combining the speed of JIT with the simplicity and flexibility of Python, PyPy is also cross-platform compatible and very light on memory consumption.

How speedy is PyPy? According to the PyPy Speed Center, the average benchmark shows it runs anywhere from 0.14 to 7.0 times quicker than CPython, depending on the task it is performing.

Stackless Python

Stackless Python is a language interpreter that does not use the C stack – instead, it manages programs inside the interpreter itself. The main advantage of Stackless Python is lightweight threading. Threads let you run multiple things at once in the same application. Stackless Python supports micro-threads, which are small threads utilized by multi-core processors. In effect, it offloads some overhead of the operating system and Python onto its own stack.

Python (x,y)

Need to crunch lots of numbers? Python (x,y) (pronounced Python-x-y) may be the answer. It is a specially-designed Python distribution for scientists and engineers to run numerical computations and examine data using these tools:

  • Python programming language.

  • Spyder development environment for scientific applications.

  • Graphical user interfaces from Qt.

Python (x,y) can:

  • Compute on multi-core processors at the same time.

  • Help develop scientific projects from small scripts to massive applications.

  • Do simple programming and object-oriented programming (OOP).

  • Calculate 2D and 3D plotting.

Python (x,y) collects scientific Python libraries in one location, providing a one-stop setup for installing Python, Spyder, and Qt with one click. The goal of Python (x,y) is to make it easier to work with scientific libraries and data by keeping and maintaining all these items in one place. Thus, prototyping is faster, projects are developed quicker, and scientific computing is easier.

The Python Interpreter

The Python interpreter reads the programs you code and processes the instructions. The interpreter and associated libraries are available in binary or source code from the main Python site, Python.org.

The interpreter can process things interactively. It takes the source code and compiles it into bytecode. The Python Virtual Machine incorporates related library modules and interprets the bytecode into a running program.

Code Execution

In many programming languages, variables contain values. Python acts somewhat differently – it uses names and bindings instead of classic programming variables. For example, if you say, “golf = happiness,” you can understand this as, “bind the name golf to the same object happiness is bound to.”

Golf and happiness are bound to the same object. Binding a name to an object does not change the object. However, if you change a property of the object, every other name bound to that object will change in the same fashion.

Deploy Python Applications

Python programs can be deployed in several ways. For bare metal servers, infrastructure-as-a-service, and virtualized servers, you must have access to servers that have a Linux distribution. Then install:

  • A web server.

  • System packages.

  • A database.

  • WSGI (Web Server Gateway Interface) server.

  • Python.

Your application is then installed from source. Another option is to deploy Python as a Platform-as-a-Service (PaaS). There are several Platforms-as-a-Service that have the ability to support Python. They include:

  • OpenShift

  • Google App Engine

  • AWS Elastic Beanstalk

  • Heroku

  • PythonAnywhere

These services have the infrastructure and software layers for deploying web apps. This method takes away much of the drudgery of configuring underlying servers, the web server, the operating system and the WSGI server.

Garbage Collection

Once you’ve deployed Python, it is a good practice to strive to keep memory usage manageable. The system helps you do this with a process called “garbage collection,” which refers to the automatic memory management of Python. It works off a reference counter – any time the system has a reference, the reference count goes up. When a reference is taken away, it counts down. Once the reference counts hit zero, garbage collection is activated, and the object in question is removed from memory.

Memory Leaks

If Python manages memory automatically with garbage collection, how could it possibly have memory leaks? Here is one example of what can happen: Sometimes objects get referenced from global variables. In turn, the object is blocked from being collected. Like an old fishing boat with rotting wood, now you have a leak.

Another problem that can occur is when one object has a reference to a second object, and the second object has a reference back to the first. Becoming a circular loop, this often prevents the reference count from reaching zero, and memory leaks result.

Memory Leak Diagnosis

How are memory leaks diagnosed? Programmers review the source code for common leak sources. You can use the Garbage Collector interface module to debug the source code and identify leaks.

It allows you to disable the Garbage Collector and define debugging criteria. Also, it will let you access objects that the collector can locate, but it cannot free their resources.

Elegance and Simplicity

In this technical deep dive, you learned the reasons behind the development of Python, why it was created to be extremely readable, simple and useful to build fast prototypes. You reviewed different interfaces and considered Python’s three distinguishing characteristics. You also got an idea of how it is implemented, how the code is executed, and how memory is managed.

Why is it so popular? Python focuses on elegance, readability, simplicity, extensibility and compact code. Few languages combine its simple approach with excellent flexibility and power at the same time. For these reasons, while other languages have fallen in popularity over the years, Python gathers more fans with each new program it helps build.

Omed Habib
Omed Habib is a Director of Product Marketing at AppDynamics. He originally joined AppDynamics as a Principal Product Manager to lead the development of their world-class PHP, Node.js and Python APM agents. An engineer at heart, Omed fell in love with web-scale architecture while directing technology throughout his career. He spends his time exploring new ways to help some of the largest software deployments in the world meet their performance needs.

Thank you! Your submission has been received!

Oops! Something went wrong while submitting the form