This talk explores some of the properties of the columnar transposition cipher, a classical encryption technique that uses a rectangular grid structure to shuffle the characters of the plaintext. This means that the columnar transposition cipher is a permutation, and the group theoretic structure of the cipher admits some interesting features.
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Deconstructing Columnar Transposition Ciphers
1. Deconstructing columnar transposition ciphers
Robert Talbert, PhD.
Department of Mathematics
Grand Valley State University
talbertr@gvsu.edu
Twitter: @RobertTalbert
Google+: google.com/+RobertTalbert
11.07.2013
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
1 / 33
2. Prelude
LOOK IN THE REFRIGERATOR
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
2 / 33
3. Prelude
LOOK IN THE REFRIGERATOR
LOOKINTHEREFRIGERATOR
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
2 / 33
4. Prelude
LOOK IN THE REFRIGERATOR
LOOKINTHEREFRIGERATOR
L
K
T
R
R
E
T
R. Talbert (GVSU)
O
I
H
E
I
R
O
O
N
E
F
G
A
R
Deconstructing CTCs
11.07.2013
2 / 33
5. Prelude
LOOK IN THE REFRIGERATOR
LOOKINTHEREFRIGERATOR
L
K
T
R
R
E
T
O
I
H
E
I
R
O
O
N
E
F
G
A
R
LKTRRETOIHEIROONEFGAR
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
2 / 33
6. Prelude
LOOK IN THE REFRIGERATOR
LOOKINTHEREFRIGERATOR
L
K
T
R
R
E
T
O
I
H
E
I
R
O
O
N
E
F
G
A
R
LKTRRETOIHEIROONEFGAR
A columnar transposition cipher (using three columns)
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
2 / 33
14. Questions
1
Why do columnar transposition ciphers cycle back on themselves, and
what’s the smallest number of encryption steps needed to make this
happen?
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
5 / 33
15. Questions
1
Why do columnar transposition ciphers cycle back on themselves, and
what’s the smallest number of encryption steps needed to make this
happen?
2
What characters in a message are fixed in place by a columnar
transposition cipher, and is there an efficient way to predict where
they will be?
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
5 / 33
16. Questions
1
Why do columnar transposition ciphers cycle back on themselves, and
what’s the smallest number of encryption steps needed to make this
happen?
2
What characters in a message are fixed in place by a columnar
transposition cipher, and is there an efficient way to predict where
they will be?
3
What else can we say about the security of this cipher?
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
5 / 33
17. How encryption and ciphers work in general
Goal: Transform information into format readable only by sender and
chosen recipients.
Readable message
Plaintext
EK(M)
Transformed message
Ciphertext
KEY
DK(EK(M))
Readable message
Plaintext
KEY
Assume that the information is being sent over an open channel.
Ciphertext should yield little/no information about the original
contents of the message.
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
6 / 33
18. Example: Shift cipher
Key: Positive integer s, decided upon in advance by sender and
recipient
Encryption process: Shift every letter in the message forward in the
alphabet by s positions, wrapping around the end of the alphabet if
necessary.
Example: Suppose s = 20 and encrypt MATH RULES.
M
G
A
U
T
N
H
B
R
L
U
O
L
F
E
Y
S
M
Ciphertext: GUNBLOFYM. Decrypt by shifting backwards by 20... or
forwards by
.
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
7 / 33
19. Interlude: Integer congruence modulo n
Definition
Let n be any positive integer and a, b integers. We say that a is
congruent to b modulo n if n divides b − a. Notation: a ≡ b (mod n).
Examples:
12 ≡ 5 (mod 7)
8675309 ≡ 9 (mod 10)
−20 ≡ 6 (mod 26)
780 ≡ 0 (mod 26)
The smallest natural number to which a is congruent modulo n =
Remainder left over when dividing a by n.
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
8 / 33
20. Mathematizing the shift cipher
Number letters 0, 1, . . . , 25
Key: Positive integer s
Es (m) = (m + s) (mod 26)
Ds (m) = (m + (26 − s)) (mod 26)
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
9 / 33
21. Mathematizing the shift cipher
Number letters 0, 1, . . . , 25
Key: Positive integer s
Es (m) = (m + s) (mod 26)
Ds (m) = (m + (26 − s)) (mod 26)
Original
Number-fied
+key
mod 26
Letter-fied
M
12
32
6
G
A
0
20
20
U
Ds (Es (m)) = m + s + 26 − s
R. Talbert (GVSU)
T
19
39
13
N
H
7
27
1
B
R
17
37
11
L
U
20
40
14
O
L
11
31
5
F
(mod 26) = m + 26
Deconstructing CTCs
E
4
24
24
Y
S
18
38
12
M
(mod 26) = m
11.07.2013
9 / 33
23. Columnar transposition ciphers
Message length = L (remove spaces, punctuation, etc.)
Key: Positive integer C
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
10 / 33
24. Columnar transposition ciphers
Message length = L (remove spaces, punctuation, etc.)
Key: Positive integer C
Encryption: Feed characters of message into a rectangular grid, C
columns and L/C rows one row at a time.
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
10 / 33
25. Columnar transposition ciphers
Message length = L (remove spaces, punctuation, etc.)
Key: Positive integer C
Encryption: Feed characters of message into a rectangular grid, C
columns and L/C rows one row at a time.
Decryption: Read off characters from the grid one column at a time.
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
10 / 33
26. CTC Example: C = 3, L = 9
M
T
H
R
U
L
MATH RULES
A
E
S
EK(M)
MHLARETUS
C=3
R. Talbert (GVSU)
DK(EK(M))
MATH RULES
C=3
Deconstructing CTCs
11.07.2013
11 / 33
27. Same message, different C
If C = 2:
M
T
MATHRULES −→ R
L
S
A
H
U −→ MTRLSAHUE
E
The ciphertext depends on both the message length L and the number C
of columns.
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
12 / 33
28. CTCs are functions
The CTC with C = 3, L = 9 is a one-to-one, onto function
{0, 1, 2, . . . , 8} → {0, 1, 2, . . . , 8}
M
T
H
R
U
L
E
S
0
1
2
3
4
5
6
7
8
M
H
L
A
R
E
T
U
S
0
n
f (n)
A
1
2
3
4
5
6
7
8
3
1
4
4
0
0
1
3
2
6
n
R. Talbert (GVSU)
6
3
1 Deconstructing CTCs
8
8
3
2
7
5
0
1
6
2
f(n)
0
5
7
11.07.2013
13 / 33
29. CTCs are functions
The CTC with C = 3, L = 9 is a one-to-one, onto function
{0, 1, 2, . . . , 8} → {0, 1, 2, . . . , 8}
M
T
H
R
U
L
E
S
0
1
2
3
4
5
6
7
8
M
H
L
A
R
E
T
U
S
0
n
f (n)
A
1
2
3
4
5
6
7
8
3
1
4
4
0
0
n
1
3
2
6
5
7
6
2
7
5
8
8
7
8
8
4
f(n)
Let g be the CTC using 02 columns on 9 characters.
0
n
g (n)
R. Talbert (GVSU)
10
1
3
20
5
6
2
1
3
6
4
2
3
1 Deconstructing CTCs
5
7
6
3
11.07.2013
13 / 33
31. Permutations
Definition
A permutation on a finite set X is a bijection X → X .
n
f (n)
0
0
1
3
2
6
3
1
4
4
5
7
6
2
7
5
8
8
Notation: f = (1, 3)(2, 6)(5, 7). Each group = cycle. 0, 4, 8 = fixed
points. So this f is a product of disjoint 2-cycles.
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
14 / 33
32. Permutations
Definition
A permutation on a finite set X is a bijection X → X .
n
f (n)
0
0
1
3
2
6
3
1
4
4
5
7
6
2
7
5
8
8
Notation: f = (1, 3)(2, 6)(5, 7). Each group = cycle. 0, 4, 8 = fixed
points. So this f is a product of disjoint 2-cycles.
n
g (n)
0
0
1
5
2
1
3
6
4
2
5
7
6
3
7
8
8
4
g = (1, 5, 7, 8, 4, 2)(3, 6). Disjoint product of a 6-cycle and a 2-cycle.
Theorem (Cayley)
Every permutation can be written as a product of disjoint cycles.
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
14 / 33
33. CTCs and permutations
Every CTC with C columns enciphering a message of length L is a
permutation on {0, 1, . . . , L − 1}. Notation: πC ,L .
Example
π3,9 = (1, 3)(2, 6)(5, 7)
π2,9 = (1, 5, 7, 8, 4, 2)(3, 6)
π3,21 = (1, 7, 9, 3)(2, 14, 18, 6)(4, 8, 16, 12)(5, 15)(11, 17, 19, 13)
π4,77 = (1, 20, 5, 21, 25, 26, 45, 31, 65, 36, 9, 22, 44, 11, 60, 15, 61, 35, 66, 55,
71, 75, 76, 19, 62, 54, 52, 13, 23, 63, 73, 38, 48, 12, 3, 58, 53, 33, 28, 7, 59,
72, 18, 43, 68, 17, 24, 6, 40, 10, 41, 30, 46, 50, 51, 70, 56, 14, 42, 49, 32, 8,
2, 39, 67, 74, 57, 34, 47, 69, 37, 29, 27, 64, 16, 4)
Demo: Python function to generate the cycle breakdown of πC ,L .
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
15 / 33
34. Will repeated encryption using a CTC always eventually lead back to the
plaintext?
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
16 / 33
35. Will repeated encryption using a CTC always eventually lead back to the
plaintext?
YES because of the cyclical nature of permutations.
Example: π3,9 = (1, 3)(2, 6)(5, 7)
n
0
1
2
3
4
5
6
7
8
R. Talbert (GVSU)
π3,9 (n)
0
3
6
1
4
7
2
5
8
(π3,9 ◦ π3,9 )(n)
0
1
2
3
4
5
6
7
8
Deconstructing CTCs
11.07.2013
16 / 33
36. Will repeated encryption using a CTC always eventually lead back to the
plaintext?
YES because of the cyclical nature of permutations.
Example: π3,9 = (1, 3)(2, 6)(5, 7)
Example: π3,21
n π3,9 (n) (π3,9 ◦ π3,9 )(n)
0
0
0
1
3
1
6
2
2
1
3
3
4
4
4
5
7
5
2
6
6
7
5
7
8
8
8
= (1, 7, 9, 3)(2, 14, 18, 6)(4, 8, 16, 12)(5, 15)(11, 17, 19, 13)
repeats itself after four iterations.
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
16 / 33
37. Definition
The order of the columnar transposition πC ,L is the smallest positive
integer such that
k
πC ,L ◦ πC ,L ◦ · · · ◦ πC ,L = πC ,L = identity function
k times
Fact
Every permutation has a finite order, and that order is the least common
multiple of the lengths of the cycles in its disjoint cycle factorization.
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
17 / 33
38. Definition
The order of the columnar transposition πC ,L is the smallest positive
integer such that
k
πC ,L ◦ πC ,L ◦ · · · ◦ πC ,L = πC ,L = identity function
k times
Fact
Every permutation has a finite order, and that order is the least common
multiple of the lengths of the cycles in its disjoint cycle factorization.
Can we determine the order of πC ,L using only the values of C and L?
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
17 / 33
39. Formula for πC ,L
Results following are from
R. Talbert, “The cycle structure and order of the rail fence cipher”,
Cryptologia, 30(2):159—172, 2006
Theorem (The Big Formula)
Let πC ,L be the permutation underlying a columnar transposition cipher
with C columns and text length L. Let 0 ≤ n < L, n = n mod C , and
L = L mod C . Then:
L
n−n
− (n − L ) if L = 0 and n > L
C +n
C
πC ,L (n) =
n−n
L
+n
otherwise
C
C
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
18 / 33
40. Illustration of The Big Formula: π4,26
0
11
L/C
n = number of columns preceding the
column containing n
n L/C = number of entries counted
in columns preceding n
13
n − n = position of first character in
n’s row
(n − n )/C = number of rows preceding
row containing n
25
C
π4,26 (13) = 7 + 3 = 10
(n − n )/C + n L/C = ending
position of n if no blanks encountered.
L = number of blanks in last row
π4,26 (11) = (3(7) − 1) + 2 = 22
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
19 / 33
41. The rail fence cipher
CTC with 2 columns = π2,L , the rail fence cipher.
Since C = 2, n and L are 0 or 1, so:
Theorem (RFC Formula)
Let π2,L be the permutation for a rail fence cipher on a plaintext of length
L. Let n be an integer with 0 ≤ n L. Then:
n
n even
2
n+L
n odd, L odd
π2,L (n) =
2
n+L−1
n odd, L even
2
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
20 / 33
42. The initial cycle
Definition
The initial cycle of πC ,L is the cycle in the decomposition of πC ,L that
contains the number 1.
Example
π2,15 = (1, 8, 4, 2)(3, 9, 12, 6)(5, 10)(7, 11, 13, 14)
π2,21 = (1, 11, 16, 8, 4, 2)(3, 12, 6)(5, 13, 17, 19, 20, 10)(7, 14)(9, 15, 18)
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
21 / 33
43. Group structure of the initial cycle
Theorem
1
2
If L = 2k − 1 for some k 1, then the initial cycle of
π2,L = (1, 2k−1 , 2k−2 , · · · , 4, 2).
k
For any positive integers k and L, π2,L (1) = 2l1 −k mod L where l1 is
the length of the initial cycle.
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
22 / 33
44. Group structure of the initial cycle
Theorem
1
2
If L = 2k − 1 for some k 1, then the initial cycle of
π2,L = (1, 2k−1 , 2k−2 , · · · , 4, 2).
k
For any positive integers k and L, π2,L (1) = 2l1 −k mod L where l1 is
the length of the initial cycle.
Corollary
The initial cycle of π2,L is the cyclic subgroup generated by 2 in Z∗ .
L
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
22 / 33
46. Connecting initial cycle to other cycles
Example
π2,15 = (1, 8, 4, 2)
(3, 9, 12, 6)
(5, 10, 5, 10)
(7, 11, 13, 14)
π2,21 = (1, 11, 16, 8, 4, 2)
(3, 12, 6, 3, 12, 6)
(5, 13, 17, 19, 20, 10)
(7, 14, 7, 14, 7, 14)
(9, 15, 18, 9, 15, 18)
Theorem
Suppose L is odd. For all x ∈ {0, 1, · · · L − 1},
π2,L (x) = (π2,L (1) · x) mod L.
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
23 / 33
47. Theorem
Suppose L is odd. If γ is a cycle of π2,L , then the length of γ divides the
length of the initial cycle, and hence the order of π2,L is the length of the
initial cycle.
Proof: Let orb(1) denote the initial cycle of π2,L . This is a group with
k
typical element π2,L (1). Define a group action of orb(1) on a cycle γ of
π2,L :
k
k
π2,L (1) · x = π2,L (x)
Exercise: This really is a group action, and if x ∈ γ, then
Fx = {g ∈ orb(1) : g · x = x mod L}
is a subgroup of orb(1). Classical group theory implies |orb(1)/Fx | = |γ|.
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
24 / 33
48. Things fall apart when C 2
Example
π3,19 = (1, 7, 9, 3)(2, 13, 11, 16, 12, 4, 8, 15, 5, 14, 17, 18, 6)
Cycles of lengths 4 and 13 (order = 4 × 13 = 52)
Nontrivial fixed point: 10
Initial cycle does not end in descending powers of 3 mod 19
Initial cycle does not act nicely on the long cycle
It’s not currently known exactly how the cycle structure of πC ,L is
organized if C 2.
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
25 / 33
49. Question 2: Are there characters in the message that are fixed by a CTC?
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
26 / 33
50. Question 2: Are there characters in the message that are fixed by a CTC?
YES:
The first character is always fixed.
The last character is fixed if and only if C divides L.
But what about “nontrivial” fixed points?
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
26 / 33
51. Question 2: Are there characters in the message that are fixed by a CTC?
YES:
The first character is always fixed.
The last character is fixed if and only if C divides L.
But what about “nontrivial” fixed points?
Research with Beth Bjorkman (GVSU mathematics undergrad): “Fixed
points of columnar transposition ciphers”
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
26 / 33
52. Where nontrivial fixed points don’t appear
Each column of the enciphering grid for πC ,L can contain at most one
fixed point.
Corollary: If n is a nonzero character position and C divides n, then n
is not fixed.
Corollary: If n is a character position and n ≡ −1 (mod C ), then n is
not fixed if C divides L; and if C does not divide L, n is fixed if and
only if n = L − 1.
Several other formulas lead to a constant-time algorithm for
locating fixed points. (→ Demo)
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
27 / 33
53. What’s not known (yet)
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
28 / 33
54. The One Big Cycle problem
When does πC ,L consist of just one big cycle?
Example
π2,13 = (1, 7, 10, 5, 9, 11, 12, 6, 3, 8, 4, 2)
π3,29 = (1, 10, 13, 14, 24, 8, 22, 17, 25, 18, 6, 2, 20, 26, 28, 19, 16, 15, 5, 21,
7, 12, 4, 11, 23, 27, 9, 3)
π6,21 = (1, 4, 15, 14, 10, 16, 17, 20, 11, 19, 7, 5, 18, 3, 12, 2, 8, 9, 13, 6)
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
29 / 33
55. Data about OBC
L-values for which πC ,L is one big cycle (10 ≤ L ≤ 500)
C
2
3
4
5
6
L ∈ [10, 500] yielding OBC
11, 13, 19, 29, 37, 53, 59, 61, 67, 83, 101,
131, 139, 149, 163, 173, 179, 181, 197,
227, 269, 293, 317, 347, 349, 373, 379,
419, 421, 443, 461, 467, 491
17, 29, 53, 89, 101, 113, 137, 149, 173,
233, 257, 269, 281, 293, 317, 353, 389,
449, 461
77
None∗
11, 17, 21, 41, 59, 83, 89, 107, 113, 131,
179, 227, 233, 247, 251, 257, 347, 381,
419, 443, 449, 467, 491
∗First value for C = 5 that gives OBC is L = 5287
R. Talbert (GVSU)
Deconstructing CTCs
107,
211,
389,
197,
401,
137,
401,
Frequency
36
21
1
0
25
11.07.2013
30 / 33
56. Maximal order problem
When does πC ,L decompose into cycles, all of whose (distinct)
lengths are mutually coprime?
Example
π3,19 = (1, 7, 9, 3)(2, 13, 11, 16, 12, 4, 8, 15, 5, 14, 17, 18, 6)
π5,27 = (1, 6, 7, 13, 19, 25, 5)(2, 12, 14, 24, 26, 11, 8, 18, 20, 4, 22, 16, 9,
23, 21, 10)(3, 17, 15)
For C = 2, this never happens.
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
31 / 33
57. Extension questions
Both are due to David Austin (GVSU).
Suppose C1 = C2 . Does the composition
πC2 ,L ◦ πC1 ,L
reduce to πC ,L for some C ?
Partial answer: Not always.
π3,9 ◦ π2,9 = (1, 6)(2, 3, 5, 8, 4) = πi,9 ∀i
Extend the CTC idea to a 3-dimensional array. Is this cipher
equivalent to a 2-dimensional columnar transposition? What if we
used higher-dimensional arrays?
R. Talbert (GVSU)
Deconstructing CTCs
11.07.2013
32 / 33