Pycrypto

=========

* (Backwards incompatible) The old Crypto.Hash.HMAC module is
gone, replaced by a copy of hmac.py from Python 2.2's standard
library. It will display a warning on interpreter versions
older than 2.2.

* (Backwards incompatible) Restored the Crypto.Protocol package,
and modernized and tidied up the two modules in it,
AllOrNothing.py and Chaffing.py, renaming various methods
and changing the interface.

* (Backwards incompatible) Changed the function names in
Crypto.Util.RFC1751.

* Restored the Crypto.PublicKey package at user request. I
think I'll leave it in the package and warn about it in the
documentation. I hope that eventually I can point to
someone else's better public-key code, and at that point I
may insert warnings and begin the process of deprecating
this code.

* Fix use of a Python 2.2 C function, replacing it with a
2.1-compatible equivalent. (Bug report and patch by Andrew
Eland.)

* Fix endianness bugs that caused test case failures on Sparc,
PPC, and doubtless other platforms.

* Fixed compilation problem on FreeBSD and MacOS X.

* Expanded the test suite (requires Sancho, from
http://www.mems-exchange.org/software/sancho/)

* Added lots of docstrings, so 'pydoc Crypto' now produces
helpful output. (Open question: maybe *all* of the documentation
should be moved into docstrings?)

* Make test.py automatically add the build/* directory to sys.path.

* Removed 'inline' declaration from C functions. Some compilers
don't support it, and Python's pyconfig.h no longer tells you whether
it's supported or not. After this change, some ciphers got slower,
but others got faster.

* The C-level API has been changed to reduce the amount of
memory-to-memory copying. This makes the code neater, but
had ambiguous performance effects; again, some ciphers got slower
and others became faster. Probably this is due to my compiler
optimizing slightly worse or better as a result.

* Moved C source implementations into src/ from block/, hash/,
and stream/. Having Hash/ and hash/ directories causes problems
on case-insensitive filesystems such as Mac OS.

* Cleaned up the C code for the extensions.

=========

* Added Crypto.Cipher.AES.

* Added the CTR mode and the variable-sized CFB mode from the
NIST standard on feedback modes.

* Removed Diamond, HAVAL, MD5, Sapphire, SHA, and Skipjack. MD5
and SHA are included with Python; the others are all of marginal
usefulness in the real world.

* Renamed the module-level constants ECB, CFB, &c., to MODE_ECB,
MODE_CFB, as part of making the block encryption modules
compliant with PEP 272. (I'm not sure about this change;
if enough users complain about it, I might back it out.)

* Made the hashing modules compliant with PEP 247 (not backward
compatible -- the major changes are that the constructor is now
MD2.new and not MD2.MD2, and the size of the digest is now
given as 'digest_size', not 'digestsize'.

* The Crypto.PublicKey package is no longer installed; the
interfaces are all wrong, and I have no idea what the right
interfaces should be.

=========
* Most importantly, the distribution has been broken into two
parts: exportable, and export-controlled. The exportable part
contains all the hashing algorithms, signature-only public key
algorithms, chaffing & winnowing, random number generation, various
utility modules, and the documentation.

The export-controlled part contains public-key encryption
algorithms such as RSA and ElGamal, and bulk encryption algorithms
like DES, IDEA, or Skipjack. Getting this code still requires that
you go through an access control CGI script, and denies you access if
you're outside the US or Canada.

* Added the RIPEMD hashing algorithm. (Contributed by
Hirendra Hindocha.)

* Implemented the recently declassified Skipjack block
encryption algorithm. My implementation runs at 864 K/sec on a
PII/266, which isn't particularly fast, but you're probably better off
using another algorithm anyway. :)

* A simple XOR cipher has been added, mostly for use by the
chaffing/winnowing code. (Contributed by Barry Warsaw.)

* Added Protocol.Chaffing and Hash.HMAC.py. (Contributed by
Barry Warsaw.)

Protocol.Chaffing implements chaffing and winnowing, recently
proposed by R. Rivest, which hides a message (the wheat) by adding
many noise messages to it (the chaff). The chaff can be discarded by
the receiver through a message authentication code. The neat thing
about this is that it allows secret communication without actually
having an encryption algorithm, and therefore this falls within the
exportable subset.

* Tidied up randpool.py, and removed its use of a block
cipher; this makes it work with only the export-controlled subset
available.

* Various renamings and reorganizations, mostly internal.

=====

* Changed files to work with Python 1.5; everything has been
re-arranged into a hierarchical package. (Not backward compatible.)
The package organization is:
Crypto.
Hash.
MD2, MD4, MD5, SHA, HAVAL
Cipher.
ARC2, ARC4, Blowfish, CAST, DES, DES3, Diamond,
IDEA, RC5, Sapphire
PublicKey.
DSA, ElGamal, qNEW, RSA
Util.
number, randpool, RFC1751

Since this is backward-incompatible anyway, I also changed
module names from all lower-case to mixed-case: diamond -> Diamond,
rc5 -> RC5, etc. That had been an annoying inconsistency for a while.

* Added CAST5 module contributed by <wimlhhhh.org>.

* Added qNEW digital signature algorithm (from the digisign.py
I advertised a while back). (If anyone would like to suggest new
algorithms that should be implemented, please do; I think I've got
everything that's really useful at the moment, but...)

* Support for keyword arguments has been added. This allowed
removing the obnoxious key handling for Diamond and RC5, where the
first few bytes of the key indicated the number of rounds to use, and
various other parameters. Now you need only do something like:

from Crypto.Cipher import RC5
obj = RC5.new(key, RC5.ECB, rounds=8)

(Not backward compatible.)

* Various function names have been changed, and parameter
names altered. None of these were part of the public interface, so it
shouldn't really matter much.

* Various bugs fixed, the test suite has been expanded, and
the build process simplified.

* Updated the documentation accordingly.

=====

* Changed files to work with Python 1.4 .

* The DES and DES3 modules now automatically correct the
parity of their keys.

* Added R. Rivest's DES test (see http://theory.lcs.mit.edu/~rivest/destest.txt)

=====

* REDOC III succumbed to differential cryptanalysis, and has
been removed.

* The crypt and rotor modules have been dropped; they're still
available in the standard Python distribution.

* The Ultra-Fast crypt() module has been placed in a separate
distribution.

* Various bugs fixed.


Py code:

setup.py invokes 2to3 automatically. This handles int/long and print issues,
among others.
setup.py will touch nt.py on win32 after build and build again. This is
necessary so 2to3 can do its magic on that file.

There are still a lot of places in the code that need manual attention even
with 2to3. They mostly have to do with string (2.x) vs. byte/unicode (3.x)
representation

Use "if sys.version_info[0] == 2:" where needed. Ideally, most of the
conditional code can be in py3compat.

Replace str(x) with bstr(x) if bytes were intended. Becomes str(x) in 2.x and
bytes(x) in 3.x through py3compat module.
Replace chr(x) with bchr(x) if bytes were intended. Becomes chr(x) in 2.x and
bytes([x]) in 3.x through py3compat module.
Replace ord(x) with bord(x) if bytes were intended. Becomes ord(x) in 2.x and
x in 3.x through py3compat module.

Comparing a string index to a string literal needs to be changed in 3.x, as
b'string'[0] returns an integer, not b's'.
The comparison can be fixed by indexing the right side, too:
"if s[0]==b('\x30')[0]:" or "if self.typeTag!=self.typeTags['SEQUENCE'][0]:"

String literals need to be bytes if bytes were intended.
Replace "x" with b("x") if bytes were intended. Becomes "x" in 2.x, and
s.encode("x","latin-1") in 3.x through py3compat module.
For example, '"".join' is replaced by 'b("").join', and 's = ""' becomes
's = b("")'.
Search for \x to find literals that may have been intended as byte strings
!! However, where a human-readable ASCII text string output was intended,
such as in AllOrNothing.undigest(), leave as a string literal !!

Only load py21compat.py
"if sys.version_info[0] == 2 and sys.version_info[1] == 1:" .
The assignment to True, False generates syntax errors in 3.x, and >= 2.2 don't
need the compatibility code.

Where print is used with >> to redirect, use a separate function instead.
See setup.py for an example

The string module has been changed in 3.x. It lost join and split, maketrans
now expects bytes, and so on.
Replace string.join(a,b) with b.join(a).
Replace string.split(a) with a.split().
Replace body of white-space-stripping functions with 'return "".join(s.split())'

Integer division via the "/" operator can return a float in 3.x. This causes
issues in Util.number.getStrongPrime. As 2.1 does not support the "//"
operator, divmod(a,b)[0] is used instead, to conform with an existing practice
throughout the rest of the pycrypto code base.

Do not use assert_/failUnless or failIf. These are deprecated and are scheduled
to be removed in Python 3.3 and 3.4.
Use instead assertEqual(expr,True) for assert_ and assertEqual(expr,False) for
failIf

Added unit tests for Crypto.Random.random. Fixed random.shuffle().
random.sample() changed to no longer fail on Python 2.1.

Added unit test for Crypto.Protocol.AllOrNothing.
AllOrNothing changed to no longer fail occasionally.

C code:

Extended "pycrypto_compat.h". It handles define's for Python 3.x forward
compatibility

include "pycrypto_compat.h"
// All other local includes after this, so they can benefit from the
// definitions in pycrypto_compat.h

The compat header defines IS_PY3K if compiling on 3.x
The compat header defines PyBytes_*, PyUnicode_*, PyBytesObject to resolve to
their PyString* counterparts if compiling on 2.x.
PyLong_* can be dangerous depending on code construct (think an if that runs
PyInt_* with else PyLong_*),
therefore it is defined in each individual module if needed and safe to do so.

PyString_* has been replaced with PyBytes_* or PyUnicode_* depending on intent
PyStringObject has been replaced with PyBytesObject or PyUnicodeObject
depending on intent.
PyInt_* has been replaced with PyLong_*, where safe to do (in all cases so far)

The C code uses "ifdef IS_PY3K" liberally.
Code duplication has been avoided.

Module initialization and module structure differ significantly in 3.x.
Conditionals take care of it.

myModuleType.ob_type assignment conditionally becomes PyTypeReady for 3.x.

getattr cannot be used to check against custom attributes in 3.x. For 3.x,
conditional code uses getattro and PyUnicode_CompareWithASCIIString instead
of strcmp

hexdigest() needed to be changed to return a Unicode object with 3.x


TODO for extra credit:
- Check for type of string in functions and throw an error when it's not
correct. While at it, ensure that functions observe the guidelines below
re type.
This is friendlier than just relying on Python's errors.
- Make sure DerSequence slicing is tested, since I took the explicit slice
functions away in 3.x