--- rest25/reference/lexical_analysis.rst	2008-12-19 16:58:05.000000000 +0900
+++ rest262/reference/lexical_analysis.rst	2009-04-05 05:01:44.000000000 +0900
@@ -66,25 +66,25 @@
 constructed from one or more *physical lines* by following the explicit or
 implicit *line joining* rules.
 
 
 .. _physical:
 
 Physical lines
 --------------
 
 A physical line is a sequence of characters terminated by an end-of-line
 sequence.  In source files, any of the standard platform line termination
 sequences can be used - the Unix form using ASCII LF (linefeed), the Windows
-form using the ASCII sequence CR LF (return followed by linefeed), or the
+form using the ASCII sequence CR LF (return followed by linefeed), or the old
 Macintosh form using the ASCII CR (return) character.  All of these forms can be
 used equally, regardless of platform.
 
 When embedding Python, source code strings should be passed to Python APIs using
 the standard C conventions for newline characters (the ``\n`` character,
 representing ASCII LF, is the line terminator).
 
 
 .. _comments:
 
 Comments
 --------
@@ -122,47 +122,43 @@
 which is recognized by Bram Moolenaar's VIM. In addition, if the first bytes of
 the file are the UTF-8 byte-order mark (``'\xef\xbb\xbf'``), the declared file
 encoding is UTF-8 (this is supported, among others, by Microsoft's
 :program:`notepad`).
 
 If an encoding is declared, the encoding name must be recognized by Python. The
 encoding is used for all lexical analysis, in particular to find the end of a
 string, and to interpret the contents of Unicode literals. String literals are
 converted to Unicode for syntactical analysis, then converted back to their
 original encoding before interpretation starts. The encoding declaration must
 appear on a line of its own.
 
-.. % XXX there should be a list of supported encodings.
+.. XXX there should be a list of supported encodings.
 
 
 .. _explicit-joining:
 
 Explicit line joining
 ---------------------
 
 .. index::
    single: physical line
    single: line joining
    single: line continuation
    single: backslash character
 
 Two or more physical lines may be joined into logical lines using backslash
 characters (``\``), as follows: when a physical line ends in a backslash that is
 not part of a string literal or comment, it is joined with the following forming
 a single logical line, deleting the backslash and the following end-of-line
-character.  For example:
-
-.. % 
-
-::
+character.  For example::
 
    if 1900 < year < 2100 and 1 <= month <= 12 \
       and 1 <= day <= 31 and 0 <= hour < 24 \
       and 0 <= minute < 60 and 0 <= second < 60:   # Looks like a valid date
            return 1
 
 A line ending in a backslash cannot carry a comment.  A backslash does not
 continue a comment.  A backslash does not continue a token except for string
 literals (i.e., tokens other than string literals cannot be split across
 physical lines using a backslash).  A backslash is illegal elsewhere on a line
 outside a string literal.
 
@@ -314,60 +310,53 @@
 
 Identifiers and keywords
 ========================
 
 .. index::
    single: identifier
    single: name
 
 Identifiers (also referred to as *names*) are described by the following lexical
 definitions:
 
 .. productionlist::
-   identifier: (`letter`\|"_") (`letter` \| `digit` \| "_")\*
-   letter: `lowercase` \| `uppercase`
+   identifier: (`letter`|"_") (`letter` | `digit` | "_")*
+   letter: `lowercase` | `uppercase`
    lowercase: "a"..."z"
    uppercase: "A"..."Z"
    digit: "0"..."9"
 
 Identifiers are unlimited in length.  Case is significant.
 
 
 .. _keywords:
 
 Keywords
 --------
 
 .. index::
    single: keyword
    single: reserved word
 
 The following identifiers are used as reserved words, or *keywords* of the
 language, and cannot be used as ordinary identifiers.  They must be spelled
-exactly as written here:
-
-.. % 
-.. % 
-
-::
-
-   and       del       from      not       while    
-   as        elif      global    or        with     
-   assert    else      if        pass      yield    
-   break     except    import    print              
-   class     exec      in        raise              
-   continue  finally   is        return             
-   def       for       lambda    try 
+exactly as written here::
 
-.. % When adding keywords, use reswords.py for reformatting
+   and       del       from      not       while
+   as        elif      global    or        with
+   assert    else      if        pass      yield
+   break     except    import    print
+   class     exec      in        raise
+   continue  finally   is        return
+   def       for       lambda    try
 
 .. versionchanged:: 2.4
    :const:`None` became a constant and is now recognized by the compiler as a name
    for the built-in object :const:`None`.  Although it is not a keyword, you cannot
    assign a different object to it.
 
 .. versionchanged:: 2.5
    Both :keyword:`as` and :keyword:`with` are only recognized when the
    ``with_statement`` future feature has been enabled. It will always be enabled in
    Python 2.6.  See section :ref:`with` for details.  Note that using :keyword:`as`
    and :keyword:`with` as identifiers will always issue a warning, even when the
    ``with_statement`` future directive is not in effect.
@@ -377,45 +366,44 @@
 
 Reserved classes of identifiers
 -------------------------------
 
 Certain classes of identifiers (besides keywords) have special meanings.  These
 classes are identified by the patterns of leading and trailing underscore
 characters:
 
 ``_*``
    Not imported by ``from module import *``.  The special identifier ``_`` is used
    in the interactive interpreter to store the result of the last evaluation; it is
    stored in the :mod:`__builtin__` module.  When not in interactive mode, ``_``
-   has no special meaning and is not defined. See section :ref:`import`, "The
-   :keyword:`import` statement."
+   has no special meaning and is not defined. See section :ref:`import`.
 
    .. note::
 
-      The name ``_`` is often used in conjunction with internationalization; refer to
-      the documentation for the :mod:`gettext` module (XXX reference: ../lib/module-
-      gettext.html) for more information on this convention.
+      The name ``_`` is often used in conjunction with internationalization;
+      refer to the documentation for the :mod:`gettext` module for more
+      information on this convention.
 
 ``__*__``
    System-defined names.  These names are defined by the interpreter and its
    implementation (including the standard library); applications should not expect
    to define additional names using this convention.  The set of names of this
    class defined by Python may be extended in future versions. See section
-   :ref:`specialnames`, "Special method names."
+   :ref:`specialnames`.
 
 ``__*``
    Class-private names.  Names in this category, when used within the context of a
    class definition, are re-written to use a mangled form to help avoid name
    clashes between "private" attributes of base and derived classes. See section
-   :ref:`atom-identifiers`, "Identifiers (Names)."
+   :ref:`atom-identifiers`.
 
 
 .. _literals:
 
 Literals
 ========
 
 .. index::
    single: literal
    single: constant
 
 Literals are notations for constant values of some built-in types.
@@ -424,57 +412,57 @@
 .. _strings:
 
 String literals
 ---------------
 
 .. index:: single: string literal
 
 String literals are described by the following lexical definitions:
 
 .. index:: single: ASCII@ASCII
 
 .. productionlist::
-   stringliteral: [`stringprefix`](`shortstring` \| `longstring`)
-   stringprefix: "r" \| "u" \| "ur" \| "R" \| "U" \| "UR" \| "Ur" \| "uR"
-   shortstring: "'" `shortstringitem`\* "'" \| '"' `shortstringitem`\* '"'
-   longstring: ""'" `longstringitem`\* ""'"
-             : \| '"""' `longstringitem`\* '"""'
-   shortstringitem: `shortstringchar` \| `escapeseq`
-   longstringitem: `longstringchar` \| `escapeseq`
+   stringliteral: [`stringprefix`](`shortstring` | `longstring`)
+   stringprefix: "r" | "u" | "ur" | "R" | "U" | "UR" | "Ur" | "uR"
+   shortstring: "'" `shortstringitem`* "'" | '"' `shortstringitem`* '"'
+   longstring: "'''" `longstringitem`* "'''"
+             : | '"""' `longstringitem`* '"""'
+   shortstringitem: `shortstringchar` | `escapeseq`
+   longstringitem: `longstringchar` | `escapeseq`
    shortstringchar: <any source character except "\" or newline or the quote>
    longstringchar: <any source character except "\">
    escapeseq: "\" <any ASCII character>
 
 One syntactic restriction not indicated by these productions is that whitespace
 is not allowed between the :token:`stringprefix` and the rest of the string
 literal. The source character set is defined by the encoding declaration; it is
 ASCII if no encoding declaration is given in the source file; see section
 :ref:`encodings`.
 
 .. index::
    single: triple-quoted string
    single: Unicode Consortium
    single: string; Unicode
    single: raw string
 
 In plain English: String literals can be enclosed in matching single quotes
 (``'``) or double quotes (``"``).  They can also be enclosed in matching groups
-of three single or double quotes (these are generally referred to as *triple-
-quoted strings*).  The backslash (``\``) character is used to escape characters
-that otherwise have a special meaning, such as newline, backslash itself, or the
-quote character.  String literals may optionally be prefixed with a letter
-``'r'`` or ``'R'``; such strings are called :dfn:`raw strings` and use different
-rules for interpreting backslash escape sequences.  A prefix of ``'u'`` or
-``'U'`` makes the string a Unicode string.  Unicode strings use the Unicode
-character set as defined by the Unicode Consortium and ISO 10646.  Some
+of three single or double quotes (these are generally referred to as
+*triple-quoted strings*).  The backslash (``\``) character is used to escape
+characters that otherwise have a special meaning, such as newline, backslash
+itself, or the quote character.  String literals may optionally be prefixed with
+a letter ``'r'`` or ``'R'``; such strings are called :dfn:`raw strings` and use
+different rules for interpreting backslash escape sequences.  A prefix of
+``'u'`` or ``'U'`` makes the string a Unicode string.  Unicode strings use the
+Unicode character set as defined by the Unicode Consortium and ISO 10646.  Some
 additional escape sequences, described below, are available in Unicode strings.
 The two prefix characters may be combined; in this case, ``'u'`` must appear
 before ``'r'``.
 
 In triple-quoted strings, unescaped newlines and quotes are allowed (and are
 retained), except that three unescaped quotes in a row terminate the string.  (A
 "quote" is the character used to open the string, i.e. either ``'`` or ``"``.)
 
 .. index::
    single: physical line
    single: escape sequence
    single: Standard C
@@ -533,25 +521,25 @@
    this escape sequence.
 
 (2)
    Any Unicode character can be encoded this way, but characters outside the Basic
    Multilingual Plane (BMP) will be encoded using a surrogate pair if Python is
    compiled to use 16-bit code units (the default).  Individual code units which
    form parts of a surrogate pair can be encoded using this escape sequence.
 
 (3)
    As in Standard C, up to three octal digits are accepted.
 
 (4)
-   Unlike in Standard C, at most two hex digits are accepted.
+   Unlike in Standard C, exactly two hex digits are required.
 
 (5)
    In a string literal, hexadecimal and octal escapes denote the byte with the
    given value; it is not necessary that the byte encodes a character in the source
    character set. In a Unicode literal, these escapes denote a Unicode character
    with the given value.
 
 .. index:: single: unrecognized escape sequence
 
 Unlike Standard C, all unrecognized escape sequences are left in the string
 unchanged, i.e., *the backslash is left in the string*.  (This behavior is
 useful when debugging: if an escape sequence is mistyped, the resulting output
@@ -608,118 +596,121 @@
 
 Numeric literals
 ----------------
 
 .. index::
    single: number
    single: numeric literal
    single: integer literal
    single: plain integer literal
    single: long integer literal
    single: floating point literal
    single: hexadecimal literal
+   single: binary literal
    single: octal literal
    single: decimal literal
    single: imaginary literal
    single: complex; literal
 
 There are four types of numeric literals: plain integers, long integers,
 floating point numbers, and imaginary numbers.  There are no complex literals
 (complex numbers can be formed by adding a real number and an imaginary number).
 
 Note that numeric literals do not include a sign; a phrase like ``-1`` is
 actually an expression composed of the unary operator '``-``' and the literal
 ``1``.
 
 
 .. _integers:
 
 Integer and long integer literals
 ---------------------------------
 
 Integer and long integer literals are described by the following lexical
 definitions:
 
 .. productionlist::
-   longinteger: `integer` ("l" \| "L")
-   integer: `decimalinteger` \| `octinteger` \| `hexinteger`
-   decimalinteger: `nonzerodigit` `digit`\* \| "0"
-   octinteger: "0" `octdigit`\ +
-   hexinteger: "0" ("x" \| "X") `hexdigit`\ +
+   longinteger: `integer` ("l" | "L")
+   integer: `decimalinteger` | `octinteger` | `hexinteger` | `bininteger`
+   decimalinteger: `nonzerodigit` `digit`* | "0"
+   octinteger: "0" ("o" | "O") `octdigit`+ | "0" `octdigit`+
+   hexinteger: "0" ("x" | "X") `hexdigit`+
+   bininteger: "0" ("b" | "B") `bindigit`+
    nonzerodigit: "1"..."9"
    octdigit: "0"..."7"
-   hexdigit: `digit` \| "a"..."f" \| "A"..."F"
+   bindigit: "0" | "1"
+   hexdigit: `digit` | "a"..."f" | "A"..."F"
 
 Although both lower case ``'l'`` and upper case ``'L'`` are allowed as suffix
 for long integers, it is strongly recommended to always use ``'L'``, since the
 letter ``'l'`` looks too much like the digit ``'1'``.
 
 Plain integer literals that are above the largest representable plain integer
 (e.g., 2147483647 when using 32-bit arithmetic) are accepted as if they were
 long integers instead. [#]_  There is no limit for long integer literals apart
 from what can be stored in available memory.
 
 Some examples of plain integer literals (first row) and long integer literals
 (second and third rows)::
 
    7     2147483647                        0177
    3L    79228162514264337593543950336L    0377L   0x100000000L
-         79228162514264337593543950336             0xdeadbeef						    
+         79228162514264337593543950336             0xdeadbeef
 
 
 .. _floating:
 
 Floating point literals
 -----------------------
 
 Floating point literals are described by the following lexical definitions:
 
 .. productionlist::
-   floatnumber: `pointfloat` \| `exponentfloat`
-   pointfloat: [`intpart`] `fraction` \| `intpart` "."
-   exponentfloat: (`intpart` \| `pointfloat`) `exponent`
-   intpart: `digit`\ +
-   fraction: "." `digit`\ +
-   exponent: ("e" \| "E") ["+" \| "-"] `digit`\ +
+   floatnumber: `pointfloat` | `exponentfloat`
+   pointfloat: [`intpart`] `fraction` | `intpart` "."
+   exponentfloat: (`intpart` | `pointfloat`) `exponent`
+   intpart: `digit`+
+   fraction: "." `digit`+
+   exponent: ("e" | "E") ["+" | "-"] `digit`+
 
 Note that the integer and exponent parts of floating point numbers can look like
 octal integers, but are interpreted using radix 10.  For example, ``077e010`` is
 legal, and denotes the same number as ``77e10``. The allowed range of floating
 point literals is implementation-dependent. Some examples of floating point
 literals::
 
    3.14    10.    .001    1e100    3.14e-10    0e0
 
 Note that numeric literals do not include a sign; a phrase like ``-1`` is
 actually an expression composed of the unary operator ``-`` and the literal
 ``1``.
 
 
 .. _imaginary:
 
 Imaginary literals
 ------------------
 
 Imaginary literals are described by the following lexical definitions:
 
 .. productionlist::
-   imagnumber: (`floatnumber` \| `intpart`) ("j" \| "J")
+   imagnumber: (`floatnumber` | `intpart`) ("j" | "J")
 
 An imaginary literal yields a complex number with a real part of 0.0.  Complex
 numbers are represented as a pair of floating point numbers and have the same
 restrictions on their range.  To create a complex number with a nonzero real
 part, add a floating point number to it, e.g., ``(3+4j)``.  Some examples of
 imaginary literals::
 
-   3.14j   10.j    10j     .001j   1e100j  3.14e-10j 
+   3.14j   10.j    10j     .001j   1e100j  3.14e-10j
 
 
 .. _operators:
 
 Operators
 =========
 
 .. index:: single: operators
 
 The following tokens are operators::
 
    +       -       *       **      /       //      %
