Python Tutorial

Introduction

2012-07-14T06:50:29+00:00

Many people have a computer but are not getting full value from it because they are unaware of how to write programs. For them I have written this tutorial. Among their number are some of my colleagues in the American Cryptogram Association. Consequently in the teaching I am going to use examples drawn from the interesting world of ciphers, though the programming methods learned will be equally applicable to any application.

Learning to program in python is straightforward. Unfortunately most books complicate things with unnecessary detail, baffling the learner and leading to a premature exit.

I am going to teach you by doing. First we will make a program that enciphers plaintext with a key. Then we shall make a program to decipher with a key. You will learn by doing it, with the minimum amount of extraneous detail but acquiring the skills for writing other programs of your own.

Finally we shall accomplish quite a difficult feat: we will write a program to break a cipher whose key is unknown. The program will use my CHURN algorithm to find the message.

With the confidence you gain from making this start in programming you will probably want to delve into books on python, to widen your knowledge of what can be done. I will give you a few leads.

Why have I chosen the python programming language? Because everyone can download it free, because it is modern and user-friendly, and because it is well supported with Help documents. I will begin by describing what python is and how to get python into your computer.

Downloading Python

2012-07-14T07:58:55+00:00

What is python?

Python is a programming language. The language is made up of instructions that you put in a program in order to tell your computer what to do. For example to get it to print ‘Hello’ on the screen, you just write:

print “Hello”

That’s all there is to programming! You have to learn the available instructions and the required way of typing them into a program. Then you press the ‘run’ button and the computer does as it’s told.

Python is also the software that interprets your instructions into a form that the computer can understand. It is this software that you first have to download and install on your computer.

Downloading python.

At the website http://www.python.org/download/ you will find downloads for both Windows and for Mac. The current version is 3.2.3 (April 2012).

After downloading, an icon will appear on your desktop. Double-click this to instal python, and if you’re operating system is Windows you’ll find a new folder ‘Python 25’ in your root directory and also in your Start|All Programs list. Click this latter folder, and you’ll find an icon called ‘Idle’ inside. Click on ‘Idle’ and python starts by putting a form called Python Shell on your screen.

On the Mac, installation of python puts a folder ‘Python for Mac’ in your Applications folder and this contains ‘Idle’. Click it to get the Python Shell.

If you type 2+2 in the Shell and press Enter, python displays 4. You can play such games. But better is to click File|New Window which will give you a form on which to write your first program. Type the two lines below, making sure you start at the left hand side:

print (‘hello’)
print (2+2)

and press the key F5 to run this little program. After you give a file name (test1.py will do), and press Enter, your program will run, and display the result of your commands in Python Shell, just as shown below. The Shell is where the output of a program is displayed:

The program code

print('hello')

prints the word 'hello' and then a carriage return so that the next item to be printed starts on a new line. If you want to printing something else later on the same line then the command must be altered to

print('hello',end="")

and the printed line will end without a carriage return. This terminology is new to Python v3 and is, in my opinion, much clumsier than previously (which I won't bother you with) but sometimes complexity is the price of progress.

PREVIOUS NEXT

Enter message

2012-07-16T04:39:37+00:00

Entering the message to encipher

The enciphering program takes a message and, with a key, produces a cipher that will look meaningless, effectively hiding the message. Let’s say our message is:

I must go down to the seas again, for the call of the
running tide is a wild call, and a clear call, that
may not be denied.

We have to get this message into the computer. In Idle, we click on File|New Window

to get a blank page on which to write the program. Now we type into our program a name, an equals sign and the message enclosed by triple inverted commas. I've chosen the name 'message' but you can choose almost any name you like, such as 'plaintext' or 'pt' or 'p49'.

message = ‘’’I must go down to the seas again, for the call of the
running tide is a wild call, and a clear call, that
may not be denied.’’’

The message is called a string of letters. Each character is designated by an order number, beginning with zero. So message[0] is ‘I’, message [1] is a space, message[2] is ‘m’ and so on. This ability to designate a letter by its position in a string is an essential part of programming.

Instead of typing the message letter by letter, you can copy and paste it. Whichever way you do it, the message is long enough to cover several lines. We use triple inverted commas around the message to tell python that it’s all one string.

We can measure the length of this string using the python command len() in the following way:

length= len(message)

This line counts the characters in the message, including the spaces, and puts the number in the entity I have called ‘length’. You could have chosen any other name for this entity that appeals to you, like ‘x’ or leng or len1. But you can’t call it ‘len’ because that is reserved as a command.

The name ‘length’ stands for a number and is called a ‘variable’. On my computer, length is 134. It may be different on yours if you’ve typed the message into fewer lines than me, and so have fewer spaces.

We can get the computer to display the message in Python Shell, character by character, with these commands:

for j in range(length):
print (message[j],end="")

You will notice that the first line ends with a colon ‘:’ and the second line is indented four spaces.

The words of the first line mean: make the variable ‘j’ equal to zero, then make it 1, then 2 … all the way up to, but not including, the value of ‘length’ (which you’ll recall is the length of the cipher). The colon and the indentation mean: for every value in the first line carry out the instruction in the second line.

In other words the instruction is print message[0], then print message[1], then message[2]... and so on up to message[length-1]; after that ‘j’ gets to the value of ‘length’ and the routine is finished.

You can indent a whole series of actions to be carried out for every value of ‘j’. So we could write:

for j in range(length):
print (‘The value of j is = ’,end="")
print (j,end="")
print (‘ and the letter is ’,end="")
print (message[j])
print (‘All done’)

The computer would then print out:

The value of j = 0 and the letter is I
The value of j = 1 and the letter is (this is the space)
The value of j = 2 and the letter is m
The value of j = 3 and the letter is u
The value of j = 4 and the letter is s
The value of j = 5 and the letter is t
…all the way up to…
The value of j = 133 and the letter is .

When ‘j’ gets to the value of length the routine is completed. The computer then goes on to the next command, which you can see is not indented, and prints out “All done”.

A final comment – the comma at the end of a print command means ‘don’t end the line’. When the comma is omitted, python will print and then start a new line.

Go ahead and write this program and run it and see what happens.

The next step is to ensure all letters in the message are in lower case and that any punctuation is removed. Converting to lower case is easily done with the special python command:

message = message.lower()

which takes all the steps necessary to convert any upper case letter in the string 'message' to its equivalent lower case letter.

To get rid of punctuation is a bit more complicated. We shall instruct the computer to take each character in the string 'message', then test to see whether it is a letter between 'a' and 'z', and if so to add it to a new string which I’ll call 'plain'. To do this I first have to tell python that 'plain' is a string, otherwise python could equally well think ‘plain’ is a number variable. A string is enclosed by an inverted comma at each end. In this case the string is empty so we write:

plain = ‘’

Here’s the program code to extract the letters from ‘message’ and put them into ‘plain’. Pay particular attention to the indenting which must regularly be 4 spaces, and don’t forget the semi colons! The lines in red beginning with # are comments to help understand the program; the hash tells python they are not part of the program.

#..declare ‘plain’ as an empty string..
plain=’’
#..take each letter of message & if it's between a and z add it to
# the new string 'plain'.
for j in range(length):
if (message[j] >='a' and message[j]<='z'):
plain = plain + message[j]
#...display the list called ‘plain’ in the Shell & its length..
print ("plain= ",end="")
for j in range(length):
print (plain[j],end="")
print ("length=",length)

Add these lines to your program. The complete program is given in Appendix 1 and you can copy it from there if you like. Save the program with File|Save and run it by pressing the F5 key or with File|Run.

Now the punctuation and spaces between words have gone, the letters are all lower case and the length has shrunk appropriately to 93. Here is the output you will get:

plain=imustgodowntotheseasagainforthecalloftherunningtideisawildcallandaclearcallthatmaynotbedenied
length= 93

PREVIOUS NEXT

Enciphering

2012-07-14T08:24:03+00:00

Making a program to encipher by substitution

We have just used two of the most important basic routines in python:

1. repetition:

for x in range(y):
do something

2. conditional testing. We used above

if (message[j] >='a' and message[j]<='z'):

which means “if message[j] is greater than or equal to ‘a’ and if message[j] is less than or equal to ‘z’ “

Here are some other examples:

(a) if x is greater than y if ( x > y):
(b) if x equals y if (x==y):
(c) if x is less than y if (x<y):

x and y stand for numerical variables. If they stand for letters then they must be enclosed in inverted commas, as in the program.

You might like to make a few trial programs of your own with these routines to get the hang of it. For example this will print out 0 to 9:

for k in range(10):
(print k,end="")

The little program below will print the letters of the alphabet. The computer recognizes characters as a numbered list where 'A' is at 65, 'B' at 66, ‘C’ at 67 .. and Z at 90. The line below instructs: "for the variable ‘m’ starting at 65, ending at 90 and moving forward by 1 print out the relevant character".

for m in range(65,91,1):
(print chr(m),end="")

Similarly the lower case letters are numbered from 97 for 'a' to 122 for 'z'

Here's a little conditional program to try. If 'x' is less than 5, it will print 'less', otherwise it will print 'more'.

for x in range (10):
if(x<5):
print ("less ",end="")
else:
print ("more ",end="")

An enciphering program

Enciphering is done with a key which often is an easy-to-remenber word like HYPNOTIZABLE. The rest of the alphabet is then added on, in order: HYPNOTIZABLECDFGJKMQRSUVWX.

This string, that I will call ‘key’, represents the cipher letters. It is then set against a plain alphabet to represent the plain letters, like this:

key: HYPNOTIZABLECDFGJKMQRSUVWX
alphabet: abcdefghijklmnopqrstuvwxyz

So plain 'a' will be enciphered to 'H', plain 'b' to 'Y' and so on.

Following this method for the first 13 letters of ‘plain’

i m u s t g o d o w n t o

we get cipher letters as below:
ACRMQ IFNFU DQF

To mimic this process, the computer must

-start with an empty string called cipher
-take the first plain letter 'i'
-go through the alphabet, letter by letter, until it finds 'i'
-note the position of 'i' and get the cipher letter in the same position, which is 'A'
-add 'A' to the string 'cipher'
-repeat this process for all the other letters in the message.

The string 'cipher' will then hold the encrypted message.

Python provides a command to go through ‘alphabet’ and find the position of a letter. To find the position of ‘i’:

position=alphabet.index(‘i’)

Using this command we write the following programming lines to encipher ‘plain’:

#...declare ‘cipher’ as an empty string…
cipher = ‘’
key = 'HYPNOTIZABLECDFGJKMQRSUVWX'
alphabet='abcdefghijklmnopqrstuvwxyz'

#..for each plain letter in a string ‘length’ long..
for j in range(length):
#....find position of plain letter in alphabet
position=key.index(plain[j])
#...add corresponding key letter to ‘cipher’
cipher = cipher + key[position]

It's usual to print out ciphertext in 5-letter blocks. This is done by adding a few more lines of program code to put a space in the string after every 5 letters:

#...declare the new string ‘cipher’...
cipher = ''
#.. declare the variable ‘n’, to be used as a counter
n = 0
#..for each plain letter up to ‘length’..
for j in range(length):
#...find position of plain letter in alphabet
position=key.index(plain[j])
#...add corresponding key letter to ‘cipher’
cipher = cipher + key[position]
#...increment counter. If counter = 5 add a space
# and reset counter to zero
n = n + 1
if (n==5):
cipher = cipher + ‘ ‘
n=0

Here’s a new, and simpler, way of displaying the string ‘cipher’ in the Shell, and you will notice it is much faster than printing the letters one by one:

print (cipher)

When you run the complete enciphering program (Appendix 2) you will get the ciphertext:

ACRMQ IFNFU DQFQZ OMOHM HIHAD TFKQZ OPHEE FTQZO KRDDA DIQAN OAMHU AENPH EEHDN HPEOH KPHEE QZHQC HWDFQ YONOD AON

There we are, our first program is complete. You now have a simple but effective enciphering tool.

PREVIOUS NEXT

Deciphering

2012-07-14T10:04:47+00:00

Making a program to decipher by substitution.

Deciphering is the reverse of enciphering, so we can use the same programming techniques that we've already learned. The design of the program is called an algorithm, and it's useful to write this out before starting to program.

You begin the program with declarations of the cipher and also ‘plain’, ‘key’ and ‘alphabet’ like this:

cipher=’’’ACRMQ IFNFU DQFQZ OMOHM HIHAD TFKQZ OPHEE FTQZO KRDDA DIQAN OAMHU AENPH EEHDN HPEOH KPHEE QZHQC HWDFQ YONOD AON’’’

key = 'HYPNOTIZABLECDFGJKMQRSUVWX'
alphabet = 'abcdefghijklmnopqrstuvwxyz'
plain = ‘’

Now the algorithm:

remove spaces from ciphertext
take each character in ciphertext:
if it is not a space:
find its position in the key
find letter in same position in alphabet
add alphabet letter to the string ‘plain’
print out ‘plain’ in Python Shell

The full program is shown in Appendix 3. When you run the program of course you get back the plaintext, but without spaces at the end of each word:

imustgodowntotheseasagainforthecalloftherunningtideisawildcallandaclearcallthatmaynotbedenied

PREVIOUS NEXT

Keyboard Input

2012-07-14T10:18:30+00:00

Input data to the program from the keyboard.

Another way of inputting data is straight from the keyboard after starting the program. This is convenient for short strings like the keyword ‘HYPNOTIZABLE’. In this case the program uses the python command 'raw_input()', which stops the program running and asks for the keyword. After you’ve typed it and hit the Return key, the program gets going again.

Here's an example that stops the program, prints in the Shell the message ”Enter the key & press Return”. The program then waits while you type the key, and then prints it out in the Shell.

key = input("Type the key & press Return ");
for j in range(len(key)):
print (key[j],)

It would be a good idea to check for mistakes, such as typing a punctuation mark or a number rather than a key letter, and repeating the input request if a mistake is found.

This step of repeating a section of program is done using the python command ‘while()’ at the outset. So if I have a number ‘x’ and I want to keep adding 1 to it until it reaches 5, my program would be:

x=0;
while(x<5):
x=x+1
print x,
print “all done”

The program will keep doing the indented section, incrementing ‘x’ and printing it, until ‘x’ gets to 5 when it will print “all done”. Try it out for yourself.

Now, coming back to our inputting 'HYPNOTIZABLE', we want to ask for the key, check it’s OK and, if it’s not, repeat the request:

program sets a variable ‘got_key’ = zero
program asks for keyword
human inputs keyword
program puts got_key = 1
program looks at each character of keyword:
if the character is lowercase, make it upper case
if the character is not A to Z make got_key=0
if got_key=0 go back to ‘program asks for keyword’

These program lines carry out the task:

got_key=0
while got_key==0:
key = input("Enter the key & press Return ");
key=key.upper()
got_key=1
for j in range(len(key)):
if(key[j]<'A' or key[j]>'Z'):
got_key=0
print(key)

PREVIOUS NEXT

File Input/Output

2012-07-14T10:10:36+00:00

Input data to the program from file & save data to file.

Most programs need some sort of input. Our enciphering program needed the message and the key. So far I have made these inputs by pasting or typing them into the program.

An alternative is to paste the text into an ordinary text file and make the program read the file. This method avoids a possible corruption of the program when typing in the message.

So, for example, we paste, or type, a message into a file called ‘plaintext.txt’ located in the same directory as our program. Then the following program lines will open the file and read the contents into a string called 'message' and then print out the message.

filename = 'plaintext.txt'
fileid = open(filename,”r”) #...’r’ is for “read”
message=fileid.read()
fileid.close()

The same structure is used for saving the cipher created by the program. Below I've demonstrated this with a pretend piece of ciphertext.

ciphertext ='ABCDE FGHIJ KLMNO PQRST UVWXY Z'
filename = "cipher.txt"
fileid = open(filename,"w") #...”w” is for “write”
fileid.write(ciphertext)
fileid.close()

If you type this little program into Idle|File|New Window and press F5 to run it, you will find a new file 'cipher.txt' has been created in your directory and that it includes the pretend ciphertext.

PREVIOUS NEXT

Key completion

2012-07-14T10:33:57+00:00

The program adds the rest of the alphabet to the key.

After inputting the keyword we need to add on the rest of the alphabet to form the full key (as described in the Enciphering chapter). The algorithm for this task is:

take each letter of the alphabet from the string called alphabet:
convert it to upper case
find out whether this letter is present in ‘key’
if it is not present then add it to ‘key’

Here is the program code that you might like to try:

alphabet = 'abcdefghijklmnopqrstuvwxyz'
key = 'HYPNOTIZABLE'
length_key=len(key)
for j in range(26):
present=0
for k in range(length_key):
if(alphabet[j].upper()==key[k]):
present=1
if(present==0):
key=key+alphabet[j].upper()
print (key)

What is going on here? First of all, each letter of the alphabet is chosen and made into upper case and the variable 'present' is set to zero. Then the alphabet letter is compared with each letter in the keyword. If there is a match the variable 'present' is set to 1. After all keyword letters have been scanned, if 'present' is still zero then the alphabet letter is added to the key.

PREVIOUS NEXT

Functions

2012-07-14T10:44:33+00:00

Functions

The complete program for getting the message from a file ‘message.txt’, inputting the key, extending the key, enciphering and saving the ciphertext to a file ‘cipher.txt’ is given in the Appendix. You will perhaps feel that the program is getting a bit long to follow, and would be quite difficult to understand if you came back to it after a few weeks.
Python has a way of simplifying simplify a program by breaking it up into functional parts, or functions as they are called in programming. The main part of the program is simply the name of these functions like:

get message from file
remove punctuation and spaces
get the key from the keyboard
encipher
save the cipher to file

I think you’d agree that this is very easy to understand.

The functions themselves, each with its program code, are put before the main part of the program. This code is much easier to understand because the function name explains what the code is doing.

It is possible to send data (strings and variables) to a function and to get data back. Data is sent to a function by placing it at the end of the function name, enclosed in brackets. Data is received back from a function into the string or variable that preceded the function call. For example, to get the string called ‘message’ from a file named ‘plaintext.txt’ we could write this line in the main part of the program:

message = get_message(plaintext.txt)

Notice we must join together any words in the name of the function with a ‘_’.

Above the main part of the menu we write the program code for the function. We begin with a definition of its name and, in brackets, the data sent from main and finally a semi-colon. Note that the name of the data can be different in the function and in the main part of the program. We indent the rest of the function’s code by 4 spaces, showing it all belongs to the function. Finally we end with the ‘return’ command, followed with data to be returned to main in brackets:

def get_message(filename):
message=’’
fileid = open(filename,'r') #...’r’ is for “read”
message=fileid.read()
fileid.close()
return(message)

You can send to a function, or receive back, more than one piece of data. So to encipher we can write the command in main:

cipher = encipher(plain,key)

This will send both strings ‘plain’ and ‘key’ to the function. The code for the function, again placed before main, will be:

def encipher(plain,key):
n = 0
length=len(plain)
cipher=’’
for j in range(length):
position=alphabet.index(plain[j])
cipher = cipher + key[position]
n = n + 1
if (n==5):
cipher = cipher + ' '
n=0
return(cipher)

Note that the final line sends back the string ‘cipher’ to main.

Using these functions we can rewrite the program so that it’s easier to follow. The main part is simply:

alphabet = 'abcdefghijklmnopqrstuvwxyz'
message= get_message('plaintext.txt')
plain=remove_spaces(message)
key=get_key()
key=extend_key(key)
cipher=encipher(plain,key)
save_ciphertext(‘cipher.txt’)

and the functions are written above main -- as shown in the Appendix.

PREVIOUS NEXT

python hillclimber

2013-02-07T07:33:24+00:00

A Python program to solve a cipher

This section demonstrates how a Python program can be used to solve a cryptogram, in this case a monoalphabetic substitution cipher. (There is a similar section using javascript here).

The ciphertext to be solved is:
OJNPZ FDWXJ UHDNI JGKFD QDPDQ JODHI XIFJK DSBZW NBQZH HBIZD VIDFF DHQIJ HSBQB JHHJF JHMDN NDLRB NDSUB ODEHJ UPDTD NXQAB HM

The plaintext solution is:

forsalebyownercompletesetofencyclopediabritannica
excellentconditionnolongerrequiredwifeknowseverything

using this Key:
Plaintext : abcdefghijklmnopqrstuvwxyz
Ciphertext: ZWISDOMABCEFGHJKLNPQRTUVXY

If you are not familiar with the Hillclimbing algorithm, you may want to first read this section

Python Program. You can download the Python program here. It runs in the latest version of Python (3.2.3) You will also need to download a table of log tetragram frequencies here to do the scoring. The program and table need to be placed in the same folder.

The program works like this:

1. load the table of log tetragraphs from the file "TetraLog_hank.doc"

2. remove the spaces from the ciphertext

3. make a random mixed alphabet and use it to decipher and score

4. swap every combintion of letters in the mixed alphabet, each time deciphering and scoring. Keep the mixed alphabet only if the score improves.

5. repeat (4) for as long as the score improves. Otherwise return to (3)