Understanding different Python Implementations

Recently in one of the posts, I've mentioned,

Python is just awesome! It is the most versatile and readable programming language of all time.

But deep within the core of the statement lies an ambiguity. A dedicated and experienced Python engineer can easily find the flaw. Of course, Python is a great choice for beginners to learn how to program. It is surprisingly the most readable and understandable programming language I've ever encountered.

But what do I mean when I say "Python"? Is it the abstract interface? Or is it the implementation of the abstract interface?

The understanding of an interface and its implementation are one of the concepts that a beginner should be made aware of before heading into the realm of programming. The reason why you should know them is the fact that they usually decide the performance and efficiency of the language in a particular environment. It would be almost impossible for you to deploy a hybrid application, targetting multiple platforms, without getting familiar with different flavours, unless you use some third-party frameworks.

Therefore, in this post, I've brought a brief introduction to what interface and implementation are, and how Python implementations stand when it comes to performance. (Do not confuse Python implementations with Python versions.)

Interface

In layman's terms, an interface can be understood as a set of rules, or say, the grammar of the programming language. It decides what a particular word, or token, would mean and therefore, decides the behaviour of the language in various contexts.

In computer science, it is often termed as an abstract interface of the language. It is neither the data nor the code but just the way how things would happen. It's kind of a blueprint that comprises various syntaxes and their corresponding meanings (a.k.a. semantics).

Implementation

An implementation, on the other hand, is the code that brings those predefined behaviours to life. Technically speaking, it is the realization of the specification or the abstract interface. The way it will be implemented depends solely on the designers and engineers.

Just like developing any other software product, an interface is implemented in one of the well-established programming languages. Therefore, it's obvious to say that the performance of the implementation will depend on the underlying technology used. In addition to that, it also decides whether the language is going to be compiled or interpreted which bring their own set of pros and cons.

So, is Python interpreted or compiled?

Well asking this question is one of the common mistakes that Python beginners commit.

The first thing to realize when making a comparison is that ‘Python’ is an interface. There’s a specification of what Python should do and how it should behave (as with any interface). And there are multiple implementations (as with any interface).

The second thing to realize is that ‘interpreted’ and ‘compiled’ are properties of an implementation, not an interface.

So the question itself isn't really well-formed.

That said, for the most common Python implementation (CPython: written in C, often referred to as simply ‘Python’, and surely what you’re using if you have no idea what I’m talking about), the answer is: interpreted, with some compilation. CPython compiles Python source code to bytecode, and then interprets this bytecode, executing it as it goes.

One thing that I need you to note here that typically 'compilation' is the conversion of high-level language into the machine code. But still, the conversion into bytecode is also called 'compilation' as it sort of compiles into a lower level instruction set. The actual conversion into machine code is done by interpreting the compiled bytecode.

Let’s look at bytecode and machine code more closely, as they will help us understand some of the concepts that come up later in the post.

Bytecode vs. Machine Code

Before moving on to the discussion of Python implementations, it's good to know the difference between bytecode and machine code (a.k.a. native code). Perhaps an example would suffice;

• C compiles to machine code, which is then run directly on your processor. Each instruction instructs your CPU to move stuff around.
• Java compiles to bytecode, which is then run on the Java Virtual Machine (JVM), an abstraction of a computer that executes programs. Each instruction is then handled by the JVM, which interacts with your computer.

This is the reason why machine code looks different depending on the underlying architecture. Bytecode, however, looks the same on all platforms. That is definitely a great feature but the bytecode has to be interpreted before we can make things work. This is where Virtual Machines come into the picture. The idea is to set up a common VM on every single platform, compiled to respective native code, so that it can execute any bytecode program written on any of the systems.

In short: machine code is much faster, but bytecode is more portable and secure.

Returning to CPython implementation, the toolchain process is as follows:

1. CPython compiles your Python source code into bytecode.
2. That bytecode is then executed on the CPython Virtual Machine.

Alternate VMs: Jython, IronPython, and more...

Apart from CPython, there are several other implementations of Python. As I've mentioned earlier, CPython is the most common of all, but there are others that should be mentioned for the sake of this comparison guide.

The topmost in the list is Jython, a Python implementation written in Java that utilizes JVM. While CPython produces bytecode to run on the CPython VM, Jython produces Java bytecode to run on the JVM (this is the same stuff that’s produced when you compile a Java program).

If you're a beginner, you might be thinking, "Why the hell do I need an alternate implementation?". Well, for one, these different Python implementations play nicely with different technology stacks. (For those of you who don't know what a technology stack is, it's a combination of different programming languages, tools and frameworks that developers use to create software applications.)

How?

CPython makes it very easy to write C-extensions for your Python code because in the end it is executed by a C interpreter. Jython, on the other hand, makes it very easy to work with other Java programs, which means you can import any Java classes with no additional effort, summoning up and utilizing your Java classes from within your Jython programs. (If you haven’t thought about it closely, this may sound a bit absurd. We’re at the point where you can mix and mash different languages and compile them all down to the same substance.)

Here's a snapshot of what a Jython programming environment looks like.

[Java HotSpot(TM) 64-bit Server VM (Oracle Corporation)] on 1.8.0_45
>>> from java.util import HashSet
>>> s = HashSet(5)
>>> s.add("Foo")
>>> s.add("Bar")
>>> s
[Foo, Bar]

IronPython is another popular Python implementation, written entirely in C# and targeting the .NET stack. In particular, it runs on what you might call the .NET Virtual Machine, Microsoft’s Common Language Runtime (CLR), comparable to the JVM. Just like Jython, you can import any C# classes to your IronPython code.

IronPython 1.1 (1.1) on .NET 2.0.50727.42
Copyright (c) Microsoft Corporation. All rights reserved.
>>> for i in range(5): print i
0
1
2
3
4
>>>

"But what if I stick only to CPython?"

It's totally fine. It’s possible to survive without ever touching a non-CPython Python implementation. But there are advantages to be had from switching, most of which are dependent on your technology stack. Using a lot of JVM-based languages? Jython might be for you. All about the .NET stack? Maybe you should try IronPython.

To sum up VM based implementations, here is a list for quick reference.

Just in Time Compilation (JIT): PyPy

So till now, we have a Python implementation written in C, one in Java, and one in C#. Next logical step would be a Python implementation written in ... Python.

For beginners here's where things might get confusing. But before doing anything stupid, let's discuss Just in Time (JIT) compilation.

Recall that native machine code is much faster than bytecode. Well, what if we could compile some of our bytecode and then run it as native code? We’d have to pay some price to compile the bytecode (i.e., time), but if the end result was faster, that’d be great! This is the motivation of JIT compilation, a hybrid technique that mixes the benefits of interpreters and compilers. In basic terms, JIT wants to utilize compilation to speed up an interpreted system.

For example, a common approach taken by JITs:

1. Identify bytecode that is executed frequently.
2. Compile it down to native machine code.
3. Cache the result.
4. Whenever the same bytecode is set to be run, instead grab the pre-compiled machine code and reap the benefits (i.e., speed boosts).

Unlike any conventional compilers which convert a program into the native machine code, JIT looks for the optimized way to produce a binary executable.

This is what PyPy implementation is all about: bringing JIT to Python. There are, of course, other goals: PyPy aims to be cross-platform, memory-light, and stackless-supportive. But JIT is really its selling point.

Tests show that PyPy nails it when it comes to speed and performance. This is a brief report from the PyPy Speed Center for reference. Believe me or not, it makes CPython look like a baby who's trying hard not to burst into tears!

(The geometric average of all benchmarks is 0.13 or 7.6 times faster than CPython)

When it comes to understanding PyPy, it's actually a difficult task in itself. There's a lot of confusion around PyPy. In my opinion, that's because PyPy is actually two things:

1. A Python interpreter, written in RPython (not Python (I lied before)). RPython is a subset of Python with static typing. Here's where you lose some flexibility but end up getting more control over memory management and whatnot thus bringing in optimizations.
2. A compiler that compiles RPython code for various targets and adds in JIT. The default platform is C, i.e., an RPython-to-C compiler, but you can also target the JVM and others.

When you write a program in PyPy, the RPython interpreter converts it into bytecode but the interpreter(RPython) itself has to be compiled before it can work on the PyPy program. This compilation could've been done using CPython but that would've resulted in tremendously slow implementation. Instead, the interpreter is compiled down to the code for another platform (e.g., C, JVM, or CLI) to run on our machine, adding in JIT as well. It’s magical: PyPy dynamically adds JIT to an interpreter, generating its own compiler! (Again, this is nuts: we’re compiling an interpreter, adding in another separate, standalone compiler.)

In the end, the result is a standalone executable that interprets Python source code and exploits JIT optimizations. Which is just what we wanted! I know it’s a mouthful, but maybe this diagram can help:

Wrapping Up

After this long discussion, we now know what an interface is and how it is implemented. We went through different implementations of Python and understood them more closely.

Finally, we ended with JIT compiler based PyPy.

At first, it might seem quite confusing to choose what and what not. So here are some suggestions that you may find helpful.

• If you're new to Python and want to just play around it, give a shot to CPython. As you don't have to worry about speed and performance, it's a perfect choice. Here's a guide on how you should start on Windows and Linux.
• If you're a budding developer and looking for integrating Python to your next project, just check your tech stack and make your choice wisely. As mentioned earlier, choosing an implementation based on your tech stack will definitely boost up your application efficiency.
• If you're writing a hybrid application, targeting multiple platforms at once, it would be wise to compile it on different implementations for better performance.
• If your interest is in writing dedicated native application and speed is all that matters to you, then go blindly for PyPy. It's the future of Python!

---

So that's all from my side. I hope you guys would've found this post useful and learned something new. Do mention in comments what your next project is based on. Any queries or suggestions are welcome as well.

Happy Programming!

Python 2 or Python 3: Which should you pick?

Python is surprisingly the most versatile and readable programming language of all time. With a name inspired by the British comedy group Monty Python, the objective of Python Development team was to make it fun and easy to use. Written in a relatively straightforward style with immediate feedback on errors, Python is a great choice for those who are new to Programming.

As python supports multiple programming paradigms, like scripting and object orientation, it is often used in wide range of fields like Scientific Calculations and Emulations to Web and Desktop App Development. And this is why Python offers a lot of potential for those looking to pick up an additional programming language.

General Overview

Python was developed in the late 1980s and first published in 1991, authored by Guido van Rossum, who is still very active in the community.  It was in 1994 when Python's user base grew, and it became a popular choice among amateur programmers for open source development. During this time, Python went through numerous tweaks and bug fixes. Here is a brief walkthrough of its different versions.

Python 2

Published in late 2000, Python 2 signalled a more transparent and inclusive language development process than earlier versions of Python. Additionally, Python 2 included many more programmatic features including a cycle-detecting garbage collector to automate memory management, increased Unicode support to standardise characters, and list comprehensions to create a list based on existing lists. As Python 2 continued to develop, more features were added, including unifying Python’s types and classes into one hierarchy in Python version 2.2.

Python 3

Python 3 is regarded as the future of Python and is the version of the language that is currently in development. Released in 2008, the focus of Python 3 development was to clean up the codebase and remove redundancy, making it clear that there was only one way to perform a given task.

At first, Python 3 was slowly adopted due to the language not being backwards compatible with Python 2, requiring people to make a decision as to which version of the language to use.  Additionally, many package libraries were only available for Python 2, but as the development team behind Python 3 has reiterated that there is an end of life for Python 2 support, more libraries have been ported to Python 3. The increased adoption of Python 3 can be shown by the number of Python packages that now provide Python 3 support, which at the time of writing includes 339 of the 360 most popular Python packages.

Python 2.7

Following the 2008 release of Python 3.0, Python 2.7 was published in July 2010 and planned as the last of the 2.x releases. The intention behind Python 2.7 was to make it easier for Python 2.x users to port features over to Python 3 by providing some measure of compatibility between the two.

Because of Python 2.7’s unique position as a version in between the earlier iterations of Python 2 and Python 3.0, it has persisted as a very popular choice for programmers due to its compatibility with many robust libraries. When we talk about Python 2 today, we are typically referring to the Python 2.7 release as that is the most frequently used version.

Python 2.7, however, is considered to be a legacy language and its continued development, which today mostly consists of bug fixes, will cease completely in 2020.

Key Differences

While Python 2.7 and Python 3 share many similar capabilities, they should not be thought of as entirely interchangeable. Though you can write good code and useful programs in either version, it is worth understanding that there will be some considerable differences in code syntax and handling.

Below are a few examples, but you should keep in mind that you will likely encounter more syntactical differences as you continue to learn Python.

print() function

This is the most well-known change in Python 3. In Python 2, print is treated as a statement instead of a function. So if you want your console to print something on your screen, you will have to do it like this.

>>> print "This is a sample string."
This is a sample string.

With Python 3, print() is now explicitly treated as a function, so to print out the same string above, you can do so simply and easily using the syntax of a function:

>>> print("This is a sample string.")
This is a sample string.

This change made Python’s syntax more consistent and also made it easier to change between different print functions.

Division Operator

In Python 2, any number that you type without decimals is treated as the programming type called integer. While at first glance this seems like an easy way to handle programming types, when you try to divide integers together sometimes you expect to get an answer with decimal places (called a float), as in:

15 / 2 = 7.5

However, in Python 2 integers were strongly typed and would not change to a float with decimal places even in cases when that would make intuitive sense.

When the two numbers on either side of the division ( / ) symbol are integers, Python 2 does floor division so that for the quotient x, the number returned is the largest integer less than or equal to x. This means that when you write 15 / 2 to divide the two numbers, Python 2.7 returns the largest integer less than or equal to 7.5, in this case, 7:

>>> a = 15 / 2
>>> print a
7
>>> 

To override this, you could add decimal places as in 5.0 / 2.0 to get the expected answer 2.5.

In Python 3, integer division became more intuitive, as in:

>>> a = 15 / 2
>>> print(a)
7.5
>>>

You can still use 15.0 / 2.0 to return 7.5, but if you want to do floor division you should use the Python 3 syntax of //, like this:

>>> b = 15 // 2
>>> print(b)
7
>>>

Unicode Support

When programming languages handle the string type — that is, a sequence of characters — they usually convert it into numeric equivalent so that computers can understand it. For this, they use a variety of standards and set of rules like ASCII, UNICODE, UTF-8, etc.

Python 2 uses the ASCII alphabet by default, so when you type "Valar Morghulis!" Python 2 will handle the string as ASCII. Limited to a couple of hundred characters at best in various extended forms, ASCII is not a very flexible method for encoding characters, especially non-English ones.

To use the more versatile and robust Unicode character encoding, which supports over 128,000 characters across contemporary and historic scripts and symbol sets, you would have to type u"Valar Morghulis!", with the 'u' prefix standing for Unicode.

Python 3 uses Unicode by default, which saves programmers extra development time, and you can easily type and display many more characters directly into your program. Because Unicode supports greater linguistic character diversity as well as the display of emojis, using it as the default character encoding ensures that mobile devices around the world are readily supported in your development projects.

Conclusion

As someone starting Python as a new programmer, or an experienced programmer new to the Python language, you will want to consider what you are hoping to achieve in learning the language.

If you are hoping just to learn without a set project in mind, you will likely most want to take into account that Python 3 will continue to be supported and developed, while Python 2.7 will not.

If, however, you are planning to join an existing project, you will likely most want to see what version of Python the team is using, how a different version may interact with the legacy codebase, if the packages the project uses are supported in a different version, and what the implementation details of the project are.

---

I hope you enjoyed this post and learned something new today. If you have any queries or suggestions, please leave a comment below.

Happy Programming!

Install Python3 on CentOS/RHEL and Fedora

Working with Python on Linux isn't as spooky as you think. It's just the same as any other programming language that it supports and it's actually fun. You might be wondering how come we work on Linux with that big black screen in front of us. Well, in that case, you would love to check this post out.

Linux distributions are known for having an enormous collection of software packages, and Python is one of them. A simple application on Linux, say a Calculator, requires the support of multiple programming technologies. And to prepare the system to run those applications flawlessly, most of the distributions are prepackaged with different programming environments; like C, C++, Java, PHP, Python etc. So no typical installation is required.

"Then what the heck is this post all about?"

Well, this post is for those who want to play with the latest version of Python without damaging the previous one. It is usually Python 2.7.x on most of the systems to provide backward compatibilities to the applications because they were written way before Python3 was released. But if you're OKAY with your default Python installation, you can skip this post and start with your first Python script.

Now those of you who are still with me, I would like to mention that these steps for installation are typically for RedHat distributions like Fedora, CentOS etc. But you can try them on others as well. All you need to do is to change the default package manager to yours. Like if you're using Ubuntu, you need to change dnf to apt-get in all your commands. And that's it!

So here we go.

Step 1: Install required packages

Use the following command to install the prerequisites for Python.

$ sudo dnf install gcc

This command will install a package called "gcc" which is acronym GNU C Compiler. Why? Well, I would recommend having a quick tour of various Python implementations.

Step 2: Download Python 3.6

Download the package from the official website for the latest version of Python, or type in the following commands.

$ cd /usr/src
$ wget https://www.python.org/ftp/python/3.6.5/Python-3.6.5.tgz

Now extract the downloaded package.

$ tar xzf Python-3.4.8.tgz

Step 3: Compile the Python source

It is the most crucial part of the installation. Execute these commands to compile the source and install it on your system.

$ cd Python-3.6.5
$ ./configure --enable-optimizations
$ make altinstall

make altinstall is used to prevent replacing the default python binary file in /usr/bin/python

Now after installation, remove the source archive from your system.

$ rm Python-3.6.5.tgz

Step 4: Confirm the installation

And finally, you can check whether the installation was successful or not.

$ python3 --version
Python 3.6.5

Mission accomplished!

---

I hope you would've found this post interesting and worth reading. If you've any queries or suggestions for this post, please mention them in comments below.

Happy Programming!

6 Reasons to Switch from Windows 10 to Linux

For many PC users, the idea of switching away from Windows and onto Linux can be a nerve-wracking one. After all, Linux holds only a minority share of the desktop market, and not all of us know people who are already using it. The idea of making the switch can often feel like taking a blind leap into the unknown. But in recent years, Linux has caught up with Windows for most purposes; in many areas, it has actually overtaken it. And therefore, whether you're completely new to Linux or have dabbled with it once or twice already, I want you to consider running it on your next laptop or desktop—or alongside your existing operating system. Here are some of the reasons for why you should switch from Windows to Linux.

It's free and open

Unlike Microsoft which demands you to upgrade to its latest versions of OS for hundreds of bucks, Linux distributions are completely free to use, tweak and even upgrade! It's also open source which allows you to build a custom-made OS and share with your family and friends. Ain't it cool enough?

Superior security

Windows is notorious for viruses and other malware it tends to pick up. And therefore you have to invest in keeping your system clean and out of infections by means of some premium anti-viruses. Linux, on the other hand, is renowned for its superior security—so much so that security experts recommend it for sensitive applications like online banking, for instance. The reason for being extra-secure lies on the fact that it is an open source entity, which is under the continuous care of engineers, amateurs and computer enthusiasts. Because it's open-source, it's really hard for anyone to target all Linux PCs at once. Unlike Windows, every Linux flavour has a different set of configurations. Most of the users usually compile and make binaries from the source which can be modified in almost indefinite ways. On the other hand, Windows hasn't brought any noticeable change in its core technology, thus making it prone to malware and viruses.

Modest requirements

Windows is often known for the way it forces users to keep upgrading their hardware so as to keep up with the software requirements. Running Windows on minimal configuration will sure be a painful experience as it becomes extremely slow and almost unusable. Whereas Linux is quite the opposite. In fact, it's long been embraced for its ability to run flawlessly even on older or low resource computers. A typical Linux distribution memory requirements may vary from 32MB to 1GB, which is hell lot lesser than that of the Windows 10! Wouldn't it be nice to keep your existing hardware a little longer and put that money towards something else instead?

Flavours for every taste

When it comes to looks, Linux distros rule over Windows. Be it Unity, Cinnamon, Gnome 3, KDE or even low-end desktop environment like Xfce or Lxde, they are much more good looking than Windows desktop. And not only that you're free to choose the Linux flavour, but also the desktop environment. You can experience both Gnome 3 and KDE on the same distribution and customize them to even more astonishing looks as per your requirements.

Community support

Probably the best thing about Linux is its community. You'll never feel alone in the Linux World. Apart from thousands of Linux how-to-do blogs, just drop by any kind of problem you're facing with your system, someone will always try to help out. This is one of the most iconic features of Linux which beats other OS platforms including Windows.

Perfect choice for Developers and Programmers

Last but not the least, Linux distributions are known for providing a perfect development and programming environment for the user. Need to write programs in C, C++, CSS, Java, JavaScript, HTML, PHP, Perl, Python, Ruby, or Vala? Linux supports all of them, and the list goes on. You don't even have to install them on your machine. They're shipped prepackaged with your distribution.

Most of the applications/libraries are designed natively for Linux because of its vast variety of flavours. Just open GitHub, and every other project is supported by Linux distros. All you need is just clone the git package, compile it on your system, and there you go! And when it comes to development, there's something about Linux which is really hard to put in words. The day you get used to Linux, you'll find out that switching to it was one of the best decisions of your life. It's just amazing!

---

I hope you would've found this article useful and worth reading. If you're new to Linux world, please let me know when you're switching to Linux. Believe me or not, you won't regret it, ever! If you use Linux for programming, what are some other reasons why people should consider it? What’s your favourite programming feature or tool on Linux? Let me know them in the comments.

Happy Programming!

Install Python3 on Windows

Now that you've heard of what Python is and made up your mind to give it a try, it's time to get started and prepare your system to execute your programs. Unlike the distros of Linux, Windows doesn't provide a prepackaged version of Python. But that doesn't mean that Windows users should give up the idea of using this flexible programming language or install a standalone Linux environment on their PCs. You can still have Python up and running on your machine.

In case you're looking for installing Python on Linux, you may follow the link. If you're a Windows user, you may want to know why you should switch from Windows to Linux. Though it's not necessary, still highly recommended.

Deciding the version

Unfortunately, there was a significant update to Python several years ago that created a big split between its versions. This can make things a bit confusing to newbies. But don't worry, it's not a rocket science!

You would find out the difference as soon as you visit the Download Python for Windows page. Right at the top, the page asks either to download the latest version of Python 2 or Python 3.

Now comes the million dollar question, which version to download?

Newer is always better. Right?

Well, maybe or maybe not. The right choice of version depends on your goals. There are some applications that are written entirely written in Python 2 like Blender (an animation studio), DropBox etc. which you can't run in pure Python 3 environment unless you rewrite their entire project again in Python 3.

On the other hand, if you're looking for learning Python and making your own projects from scratch, it's good that you use Python 3 as it is more modern and efficient. If you want to know more about Python versions, I would recommend visiting Python's wiki where you can read their well-written overview of the differences.

After deciding the versions you want to select, just click on the desired link and download the installer. For this post, I'll walk you through installing Python 3 for tutorial purposes.

Installing Python

Installing Python is as simple as any other software on Windows. After downloading the installer, just double-click and follow the instructions.

Click on run to provide permission to install Python.

Click on the checkbox to add Python 3.6 to PATH. This is necessary to be able to launch Python from the command line.

After that, click on Install Now to continue installation.

Click on the Close button after the installation completes.

Confirming the installation

Once the installation is complete, you can confirm whether Python is installed properly or not. Just open up the command prompt and enter the following command.

C:/Users/FedUser> python3 -v
Python 3.6.4

And BOOM! Your python installation is successful and your system is ready to run your Python scripts. Type python3 in your command line and write your first Python program.

Troubleshooting: Setting the PATH variable

This section of the post is completely optional and is targeted for those who couldn't confirm their installation from the command line. One of the possible reasons for this issue may be not clicking the "Add Python to PATH" checkbox while installation.

The PATH variable contains all the possible locations where the operating system will search for the command that you've entered on the command prompt. If it's not set to the one where Python is installed, you can't access it from the command line. So you're left with two options to choose,

1. Go to the directory where Python is installed and then run your programs, or
2. Set the PATH variable and access the command anywhere on the system.

Of course, the former one is tedious. So here's the guide to set the PATH for Python.

Hit the 'Start' button and type "Advanced System Settings". Select the "View advanced system settings" option. This will load a window named System Properties. Navigate to the Advanced tab and click on the "Environment Variables" button.

This will open a window named Environment Variables. Search for PATH variable in the list. Click on Edit button and append location of the directory where Python is installed, separating it from previous entries by a semicolon( ; ).

And there it is. Open the command prompt and re-verify the installation.

---

Now that you've successfully installed Python on your system, you're ready to dive into the Python world and tackle with whatever projects you want. In upcoming posts, I'll show you how to write a standalone python script and execute it on your systems. If you've any queries regarding the installation of Python on Windows, do let me know in comments.

Happy Programming!