[wwwsearch-commits] r26778 - in
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge:
. examples mechanize test
jjlee at codespeak.net
jjlee at codespeak.net
Fri May 5 02:10:44 CEST 2006
Author: jjlee
Date: Fri May 5 02:10:40 2006
New Revision: 26778
Modified:
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/0.1.0-changes.txt
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/COPYING.txt
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/INSTALL.txt
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/MANIFEST.in
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/README.html.in
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/examples/BSDDBCookieJar.py
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/examples/cookietest.cgi
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/functional_tests.py
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_ClientCookie.py
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_ConnCache.py
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_Debug.py
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_LWPCookieJar.py
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_MSIECookieJar.py
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_MozillaCookieJar.py
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_Opener.py
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_Request.py
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_Util.py
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/__init__.py
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_html.py
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_mechanize.py
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_urllib2_support.py
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_useragent.py
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/setup.py
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/test.py
wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/test/test_mechanize.py
Log:
Finish merge of ClientCookie and pullparser with mechanize :-)
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/0.1.0-changes.txt
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/0.1.0-changes.txt (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/0.1.0-changes.txt Fri May 5 02:10:40 2006
@@ -1,7 +1,20 @@
Recent public API changes:
+- ClientCookie has been moved into the mechanize package and is no
+ longer a separate package. The ClientCookie interface is still
+ supported, but all names must be imported from module mechanize
+ instead of from module ClientCookie. Python 2.3 is now required. (I
+ have no plans to merge ClientForm with mechanize.) Note that the
+ logging work-alike facility is gone. Also, the experimental BSDDB
+ support is now only included as example code, and not installed.
+
+- pullparser has been moved into the mechanize package and is no
+ longer a separate package. Also, the interface of pullparser is no
+ longer supported. Instead, it's just a purely internal
+ implementation detail of mechanize.
+
- ATM, auth support requires Python 2.5 (I hope to fix this soon for
- older Pythons). As a result, some tests fail for 2.2, 2.3, and 2.4.
+ older Pythons). As a result, some tests fail for 2.3 and 2.4.
- Some mechanize.Browser constructor args have been moved to
mechanize.Factory (default_encoding, ...).
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/COPYING.txt
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/COPYING.txt (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/COPYING.txt Fri May 5 02:10:40 2006
@@ -1,4 +1,6 @@
Copyright (c) 2002-2006 John J. Lee <jjl at pobox.com>
+Copyright (c) 1997-1999 Gisle Aas
+Copyright (c) 1997-1999 Johnny Lee
Copyright (c) 2003 Andy Lester
All rights reserved.
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/INSTALL.txt
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/INSTALL.txt (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/INSTALL.txt Fri May 5 02:10:40 2006
@@ -32,8 +32,8 @@
python setup.py easy_install --no-deps .
-Alternatively, just copy the whole mechanize directory into your
-Python path (eg. unix: /usr/local/lib/python2.4/site-packages,
+Alternatively, just copy the whole mechanize directory into a directory
+on your Python path (eg. unix: /usr/local/lib/python2.4/site-packages,
Windows: C:\Python24\Lib\site-packages). Only copy the mechanize
directory that's inside the distributed tarball / zip archive, not the
entire mechanize-x.x.x directory!
@@ -44,13 +44,21 @@
python test.py
-This runs the tests against the source files extracted from the
-package. For help on command line options:
+This runs the tests against the source files extracted from the package.
+For help on command line options:
python test.py --help
+To run the functional tests (which DO access the network), run the
+following command:
-Bugs and comments to jjl at pobox.com.
+ python functional_tests.py
+
+
+Please send bugs and comments to the mailing list (or failing that, to
+jjl at pobox.com):
+
+https://lists.sourceforge.net/lists/listinfo/wwwsearch-general
NO WARRANTY
@@ -62,12 +70,14 @@
Copyright Notices
(C) 2002-2006 John J. Lee. All rights reserved.
- (C) 2003 Andy Lester. All rights reserved. (Perl code from which
- this module is derived)
+ (C) 1995-2001 Gisle Aas. All rights reserved. (Original LWP code)
+ (C) 2002-2003 Johhny Lee. All rights reserved. (MSIE Perl code)
+ (C) 2003 Andy Lester. All rights reserved. (Original WWW::Mechanize
+ Perl code)
This code in this package is free software; you can redistribute it
-and/or modify it under the terms of the BSD or ZPL 2.1 licenses (see
-the file COPYING.txt).
+and/or modify it under the terms of the BSD or ZPL 2.1 licenses (see the
+file COPYING.txt).
John J. Lee <jjl at pobox.com>
May 2006
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/MANIFEST.in
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/MANIFEST.in (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/MANIFEST.in Fri May 5 02:10:40 2006
@@ -5,6 +5,8 @@
include README.html.in
include README.html
include README.txt
+include doc.html.in
+include doc.html
include ChangeLog.txt
include *.py
recursive-include examples *.py
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/README.html.in
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/README.html.in (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/README.html.in Fri May 5 02:10:40 2006
@@ -11,7 +11,8 @@
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<meta name="author" content="John J. Lee <jjl@@pobox.com>">
<meta name="date" content="@(time.strftime("%Y-%m-%d", last_modified))">
- <meta name="keywords" content="Python,HTML,browser,stateful,web,client,client-side,mechanize,form,ClientForm,ClientCookie,pullparser,WWW::Mechanize">
+ <meta name="keywords" content="Python,HTML,HTTP,browser,stateful,web,client,client-side,mechanize,cookie,form,META,HTTP-EQUIV,Refresh,ClientForm,ClientCookie,pullparser,WWW::Mechanize">
+ <meta name="keywords" content="cookie,HTTP,Python,web,client,client-side,HTML,META,HTTP-EQUIV,Refresh">
<title>mechanize</title>
<style type="text/css" media="screen">@@import "../styles/style.css";</style>
<base href="http://wwwsearch.sourceforge.net/mechanize/">
@@ -408,7 +409,7 @@
<h2>FAQs</h2>
<ul>
<li>Which version of Python do I need?
- <p>2.2.1 or above.
+ <p>2.3 or above.
<li>What else do I need?
<p><a href="../ClientCookie/">ClientCookie</a>,
<a href="../ClientForm/">ClientForm</a> and
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/examples/BSDDBCookieJar.py
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/examples/BSDDBCookieJar.py (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/examples/BSDDBCookieJar.py Fri May 5 02:10:40 2006
@@ -7,23 +7,22 @@
COPYING.txt included with the distribution).
**********************************************************************
-THIS IS NOT FULLY TESTED!
+THIS WAS NEVER FULLY TESTED, AND IS NOT MAINTAINED!
**********************************************************************
"""
-from _ClientCookie import CookieJar, MappingIterator
-from _Debug import getLogger
-debug = getLogger("ClientCookie").debug
+from mechanize import CookieJar
+
+# this is importing from a private module: don't do this in your own code!
+from _ClientCookie import MappingIterator
import bsddb
-import cPickle
-pickle = cPickle
-del cPickle
-
-try: StopIteration
-except NameError:
- from _ClientCookie import StopIteration
+import cPickle as pickle
+
+import logging
+debug = getLogger("mechanize").debug
+
def CreateBSDDBCookieJar(filename, policy=None):
"""Return a BSDDBCookieJar given a BSDDB filename.
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/examples/cookietest.cgi
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/examples/cookietest.cgi (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/examples/cookietest.cgi Fri May 5 02:10:40 2006
@@ -2,7 +2,7 @@
# -*-python-*-
# The copy of this script that lives at wwwsearch.sf.net is used by the
-# ClientCookie functional tests.
+# mechanize functional tests.
print "Content-Type: text/html"
print "Set-Cookie: foo=bar\n"
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/functional_tests.py
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/functional_tests.py (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/functional_tests.py Fri May 5 02:10:40 2006
@@ -1,9 +1,27 @@
#!/usr/bin/env python
+# These tests access the network.
+
import os
from unittest import TestCase
import mechanize
+from mechanize import build_opener, install_opener, urlopen, urlretrieve
+from mechanize import CookieJar, HTTPCookieProcessor, \
+ HTTPHandler, HTTPRefreshProcessor, \
+ HTTPEquivProcessor, HTTPRedirectHandler, \
+ HTTPRedirectDebugProcessor, HTTPResponseDebugProcessor
+
+#from cookielib import CookieJar
+#from urllib2 import build_opener, install_opener, urlopen
+#from urllib2 import HTTPCookieProcessor, HTTPHandler
+
+#from ClientCookie import CreateBSDDBCookieJar
+
+## logger = ClientCookie.getLogger("ClientCookie")
+## logger.addHandler(ClientCookie.StreamHandler())
+## logger.setLevel(ClientCookie.DEBUG)
+
def sanepathname2url(path):
import urllib
@@ -60,6 +78,16 @@
r.seek(0)
self.assertEqual(r.read(), html)
+ def test_response_close_and_read(self):
+ opener = ClientCookie.build_opener(ClientCookie.SeekableProcessor)
+ r = opener.open("http://wwwsearch.sf.net/bits/cctest2.txt")
+ # closing response shouldn't stop methods working if we're using
+ # SeekableProcessor (ie. _Util.response_seek_wrapper)
+ r.read()
+ r.close()
+ r.seek(0)
+ self.assertEqual(r.read(), "Hello ClientCookie functional test suite.\n")
+
def test_set_response(self):
from mechanize import Browser
br = Browser()
@@ -104,6 +132,83 @@
"Hello ClientCookie functional test suite.\n")
+class FunctionalTests(TestCase):
+ def test_clientcookie(self):
+ import urllib2
+ # this test page depends on cookies, and an http-equiv refresh
+ #cj = CreateBSDDBCookieJar("/home/john/db.db")
+ cj = CookieJar()
+ handlers = [
+ HTTPCookieProcessor(cj),
+ HTTPRefreshProcessor(max_time=None, honor_time=False),
+ HTTPEquivProcessor(),
+
+ HTTPRedirectHandler(), # needed for Refresh handling in 2.4.0
+# HTTPHandler(True),
+# HTTPRedirectDebugProcessor(),
+# HTTPResponseDebugProcessor(),
+ ]
+
+ o = apply(build_opener, handlers)
+ try:
+ install_opener(o)
+ try:
+ r = urlopen("http://wwwsearch.sf.net/cgi-bin/cookietest.cgi")
+ except urllib2.URLError, e:
+ #print e.read()
+ raise
+ data = r.read()
+ #print data
+ self.assert_(
+ data.find("Your browser supports cookies!") >= 0)
+ self.assert_(len(cj) == 1)
+
+ # test response.seek() (added by HTTPEquivProcessor)
+ r.seek(0)
+ samedata = r.read()
+ r.close()
+ self.assert_(samedata == data)
+ finally:
+ o.close()
+ # uninstall opener (don't try this at home)
+ ClientCookie._urllib2_support._opener = None
+
+ def test_urlretrieve(self):
+ url = "http://www.python.org/"
+ verif = CallbackVerifier(self)
+ fn, hdrs = urlretrieve(url, "python.html", verif.callback)
+ try:
+ f = open(fn)
+ data = f.read()
+ f.close()
+ finally:
+ os.remove(fn)
+ r = urlopen(url)
+ self.assert_(data == r.read())
+ r.close()
+
+## def test_cacheftp(self):
+## from urllib2 import CacheFTPHandler, build_opener
+## o = build_opener(CacheFTPHandler())
+## r = o.open("ftp://ftp.python.org/pub/www.python.org/robots.txt")
+## data1 = r.read()
+## r.close()
+## r = o.open("ftp://ftp.python.org/pub/www.python.org/2.3.2/announce.txt")
+## data2 = r.read()
+## r.close()
+## self.assert_(data1 != data2)
+
+class CallbackVerifier:
+ # for .test_urlretrieve()
+ def __init__(self, testcase):
+ self._count = 0
+ self._testcase = testcase
+ def callback(self, block_nr, block_size, total_size):
+ if block_nr != self._count:
+ self._testcase.fail()
+ self._count = self._count + 1
+
+
if __name__ == "__main__":
import unittest
unittest.main()
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_ClientCookie.py
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_ClientCookie.py (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_ClientCookie.py Fri May 5 02:10:40 2006
@@ -51,46 +51,17 @@
"instance initialised with one)")
DEFAULT_HTTP_PORT = str(httplib.HTTP_PORT)
-try: True
-except NameError:
- True = 1
- False = 0
-
-try:
- from types import UnicodeType
-except ImportError:
- UNICODE = False
-else:
- UNICODE = True
-
-try: StopIteration
-except NameError:
- class StopIteration(Exception): pass
-
-import ClientCookie
from _HeadersUtil import split_header_words, parse_ns_headers
from _Util import startswith, endswith, isstringlike, getheaders
from _Debug import warn, getLogger
-debug = getLogger("ClientCookie.cookies").debug
+debug = getLogger("mechanize.cookies").debug
-try: bool
-except NameError:
- def bool(expr):
- if expr: return True
- else: return False
-
-try: issubclass(Exception, (Exception,))
-except TypeError:
- real_issubclass = issubclass
- from _Util import compat_issubclass
- issubclass = compat_issubclass
- del compat_issubclass
def reraise_unmasked_exceptions(unmasked=()):
# There are a few catch-all except: statements in this module, for
# catching input that's bad in unexpected ways.
# This function re-raises some exceptions we don't want to trap.
- if not ClientCookie.USE_BARE_EXCEPT:
+ if not mechanize.USE_BARE_EXCEPT:
raise
unmasked = unmasked + (KeyboardInterrupt, SystemExit, MemoryError)
etype = sys.exc_info()[0]
@@ -101,7 +72,7 @@
f = StringIO.StringIO()
traceback.print_exc(None, f)
msg = f.getvalue()
- warn("ClientCookie bug!\n%s" % msg)
+ warn("mechanize bug!\n%s" % msg)
IPV4_RE = re.compile(r"\.\d+$")
@@ -254,7 +225,7 @@
# And here, kind of: draft-fielding-uri-rfc2396bis-03
# (And in draft IRI specification: draft-duerst-iri-05)
# (And here, for new URI schemes: RFC 2718)
- if UNICODE and isinstance(path, types.UnicodeType):
+ if isinstance(path, types.UnicodeType):
path = path.encode("utf-8")
path = urllib.quote(path, HTTP_PATH_SAFE)
path = ESCAPED_CHAR_RE.sub(uppercase_escaped_char, path)
@@ -470,7 +441,7 @@
Currently, pre-expired cookies never get this far -- the CookieJar
class deletes such cookies itself.
- cookie: ClientCookie.Cookie object
+ cookie: mechanize.Cookie object
request: object implementing the interface defined by
CookieJar.extract_cookies.__doc__
@@ -480,7 +451,7 @@
def return_ok(self, cookie, request):
"""Return true if (and only if) cookie should be returned to server.
- cookie: ClientCookie.Cookie object
+ cookie: mechanize.Cookie object
request: object implementing the interface defined by
CookieJar.add_cookie_header.__doc__
@@ -530,10 +501,10 @@
call its methods in your overriden implementations before adding your own
additional checks.
- import ClientCookie
- class MyCookiePolicy(ClientCookie.DefaultCookiePolicy):
+ import mechanize
+ class MyCookiePolicy(mechanize.DefaultCookiePolicy):
def set_ok(self, cookie, request):
- if not ClientCookie.DefaultCookiePolicy.set_ok(
+ if not mechanize.DefaultCookiePolicy.set_ok(
self, cookie, request):
return False
if i_dont_want_to_store_this_cookie():
@@ -1006,7 +977,7 @@
class CookieJar:
"""Collection of HTTP cookies.
- You may not need to know about this class: try ClientCookie.urlopen().
+ You may not need to know about this class: try mechanize.urlopen().
The major methods are extract_cookies and add_cookie_header; these are all
you are likely to need.
@@ -1449,7 +1420,7 @@
def set_cookie_if_ok(self, cookie, request):
"""Set a cookie if policy says it's OK to do so.
- cookie: ClientCookie.Cookie instance
+ cookie: mechanize.Cookie instance
request: see extract_cookies.__doc__ for the required interface
"""
@@ -1461,7 +1432,7 @@
def set_cookie(self, cookie):
"""Set a cookie, without checking whether or not it should be set.
- cookie: ClientCookie.Cookie instance
+ cookie: mechanize.Cookie instance
"""
c = self._cookies
if not c.has_key(cookie.domain): c[cookie.domain] = {}
@@ -1479,7 +1450,7 @@
method's approval).
The response object (usually be the result of a call to
- ClientCookie.urlopen, or similar) should support an info method, which
+ mechanize.urlopen, or similar) should support an info method, which
returns a mimetools.Message object (in fact, the 'mimetools.Message
object' may be any object that provides a getallmatchingheaders
method).
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_ConnCache.py
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_ConnCache.py (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_ConnCache.py Fri May 5 02:10:40 2006
@@ -4,7 +4,7 @@
Example:
- from ClientCookie import ConnectionCache
+ from mechanize import ConnectionCache
cache = ConnectionCache()
cache.deposit("http", "example.com", conn)
conn = cache.withdraw("http", "example.com")
@@ -31,9 +31,9 @@
from types import StringType
StringTypes = StringType
-from _Util import compat_isinstance
-from _Debug import getLogger, warn
-debug = getLogger("ClientCookie").debug
+from _Debug import warn
+import logging
+debug = logging.getLogger("mechanize").debug
warn("WARNING: MODULE _ConnCache IS UNUSED AND UNTESTED!")
@@ -105,7 +105,7 @@
if not callable(checker):
if checker is None:
checker = lambda cr: True # drop all of them
- elif compat_isinstance(checker, StringTypes):
+ elif isinstance(checker, StringTypes):
scheme = checker
if reason is None:
reason = "drop %s" % scheme
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_Debug.py
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_Debug.py (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_Debug.py Fri May 5 02:10:40 2006
@@ -1,49 +1,7 @@
import sys
-import ClientCookie
+def warn(text):
+ warnings.warn(text, stacklevel=2)
-try:
- import warnings
-except ImportError:
- def warn(text):
- ClientCookie.WARNINGS_STREAM.write("WARNING: "+text)
-else:
- def warn(text):
- warnings.warn(text, stacklevel=2)
-
-try:
- import logging
-except:
- NOTSET = None
- INFO = 20
- DEBUG = 10
- class NullHandler:
- def write(self, data): pass
- class Logger:
- def __init__(self):
- self.level = NOTSET
- self.handler = NullHandler()
- def log(self, level, text, *args):
- if args:
- text = text % args
- if self.level is not None and level <= self.level:
- self.handler.write(text+"\n")
- def debug(self, text, *args):
- apply(self.log, (DEBUG, text)+args)
- def info(self, text, *args):
- apply(self.log, (INFO, text)+args)
- def setLevel(self, lvl):
- self.level = lvl
- def addHandler(self, handler):
- self.handler = handler
- LOGGER = Logger()
- def getLogger(name): return LOGGER
- class StreamHandler:
- def __init__(self, strm=None):
- if not strm:
- strm = sys.stderr
- self.stream = strm
- def write(self, data):
- self.stream.write(data)
-else:
- from logging import getLogger, StreamHandler, INFO, DEBUG, NOTSET
+import logging
+from logging import getLogger, StreamHandler, INFO, DEBUG, NOTSET
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_LWPCookieJar.py
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_LWPCookieJar.py (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_LWPCookieJar.py Fri May 5 02:10:40 2006
@@ -24,12 +24,8 @@
from _HeadersUtil import join_header_words, split_header_words
from _Util import startswith, iso2time, time2isoz
from _Debug import getLogger
-debug = getLogger("ClientCookie").debug
+debug = getLogger("mechanize").debug
-try: True
-except NameError:
- True = 1
- False = 0
def lwp_cookie_str(cookie):
"""Return string representation of Cookie in an the LWP cookie file format.
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_MSIECookieJar.py
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_MSIECookieJar.py (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_MSIECookieJar.py Fri May 5 02:10:40 2006
@@ -19,7 +19,7 @@
MISSING_FILENAME_TEXT, LoadError
from _Util import startswith
from _Debug import getLogger
-debug = getLogger("ClientCookie").debug
+debug = getLogger("mechanize").debug
try: True
except NameError:
@@ -44,7 +44,7 @@
MSIE stores create and expire times as Win32 FILETIME, which is 64
bits of 100 nanosecond intervals since Jan 01 1601.
- ClientCookie expects time in 32-bit value expressed in seconds since the
+ mechanize expects time in 32-bit value expressed in seconds since the
epoch (Jan 01 1970).
"""
@@ -321,7 +321,7 @@
cj = MSIECookieJar(delayload=1)
# find cookies index file in registry and load cookies from it
cj.load_from_registry()
- opener = ClientCookie.build_opener(ClientCookie.HTTPCookieProcessor(cj))
+ opener = mechanize.build_opener(mechanize.HTTPCookieProcessor(cj))
response = opener.open("http://example.com/")
Iterating over a delayloaded MSIECookieJar instance will not cause any
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_MozillaCookieJar.py
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_MozillaCookieJar.py (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_MozillaCookieJar.py Fri May 5 02:10:40 2006
@@ -17,18 +17,6 @@
from _Debug import getLogger
debug = getLogger("ClientCookie").debug
-try: True
-except NameError:
- True = 1
- False = 0
-
-try: issubclass(Exception(), (Exception,))
-except TypeError:
- real_issubclass = issubclass
- from _Util import compat_issubclass
- issubclass = compat_issubclass
- del compat_issubclass
-
class MozillaCookieJar(FileCookieJar):
"""
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_Opener.py
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_Opener.py (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_Opener.py Fri May 5 02:10:40 2006
@@ -9,11 +9,6 @@
"""
-try: True
-except NameError:
- True = 1
- False = 0
-
import urllib2, string, bisect, urlparse
from _Util import startswith, isstringlike
@@ -52,7 +47,7 @@
if isstringlike(url_or_req):
req = Request(url_or_req, data)
else:
- # already a urllib2.Request or ClientCookie.Request instance
+ # already a urllib2.Request or mechanize.Request instance
req = url_or_req
if data is not None:
req.add_data(data)
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_Request.py
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_Request.py (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_Request.py Fri May 5 02:10:40 2006
@@ -8,11 +8,6 @@
"""
-try: True
-except NameError:
- True = 1
- False = 0
-
import urllib2, string
from _ClientCookie import request_host
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_Util.py
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_Util.py (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_Util.py Fri May 5 02:10:40 2006
@@ -8,20 +8,10 @@
"""
-try: True
-except NameError:
- True = 1
- False = 0
-
import re, string, time, copy, urllib
from types import TupleType
from cStringIO import StringIO
-try:
- from exceptions import StopIteration
-except ImportError:
- from ClientCookie._ClientCookie import StopIteration
-
def startswith(string, initial):
if len(initial) > len(string): return False
return string[:len(initial)] == initial
@@ -30,24 +20,6 @@
if len(final) > len(string): return False
return string[-len(final):] == final
-def compat_issubclass(obj, tuple_or_class):
- # for 2.1 and below
- if type(tuple_or_class) == TupleType:
- for klass in tuple_or_class:
- if issubclass(obj, klass):
- return True
- return False
- return issubclass(obj, tuple_or_class)
-
-def compat_isinstance(obj, tuple_or_class):
- # for 2.1 and below
- if type(tuple_or_class) == TupleType:
- for klass in tuple_or_class:
- if isinstance(obj, klass):
- return True
- return False
- return isinstance(obj, tuple_or_class)
-
def isstringlike(x):
try: x+""
except: return False
@@ -89,39 +61,7 @@
result.append(current)
return result
-try:
- from calendar import timegm
- timegm((2045, 1, 1, 22, 23, 32)) # overflows in 2.1
-except:
- # Number of days per month (except for February in leap years)
- mdays = [0, 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31]
-
- # Return 1 for leap years, 0 for non-leap years
- def isleap(year):
- return year % 4 == 0 and (year % 100 <> 0 or year % 400 == 0)
-
- # Return number of leap years in range [y1, y2)
- # Assume y1 <= y2 and no funny (non-leap century) years
- def leapdays(y1, y2):
- return (y2+3)/4 - (y1+3)/4
-
- EPOCH = 1970
- def timegm(tuple):
- """Unrelated but handy function to calculate Unix timestamp from GMT."""
- year, month, day, hour, minute, second = tuple[:6]
- assert year >= EPOCH
- assert 1 <= month <= 12
- days = 365*(year-EPOCH) + leapdays(EPOCH, year)
- for i in range(1, month):
- days = days + mdays[i]
- if month > 2 and isleap(year):
- days = days + 1
- days = days + day - 1
- hours = days*24 + hour
- minutes = hours*60 + minute
- seconds = minutes*60L + second
- return seconds
-
+from calendar import timegm
# Date/time conversion routines for formats used by the HTTP protocol.
@@ -589,7 +529,7 @@
class closeable_response:
"""Avoids unnecessarily clobbering urllib.addinfourl methods on .close().
- Only supports responses returned by ClientCookie.HTTPHandler.
+ Only supports responses returned by mechanize.HTTPHandler.
After .close(), the following methods are supported:
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/__init__.py
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/__init__.py (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/__init__.py Fri May 5 02:10:40 2006
@@ -1,9 +1,45 @@
-from _useragent import UserAgent, HTTPProxyPasswordMgr
from _mechanize import Browser, \
BrowserStateError, LinkNotFoundError, FormNotFoundError, \
__version__
+from _useragent import UserAgent, HTTPProxyPasswordMgr
+
from _html import Link, \
Factory, DefaultFactory, RobustFactory, \
FormsFactory, LinksFactory, TitleFactory, \
RobustFormsFactory, RobustLinksFactory, RobustTitleFactory
+
+# If you hate the idea of turning bugs into warnings, do:
+# import mechanize; mechanize.USE_BARE_EXCEPT = False
+USE_BARE_EXCEPT = True
+
+from _ClientCookie import Cookie, CookiePolicy, DefaultCookiePolicy, \
+ CookieJar, FileCookieJar, LoadError, request_host
+from _LWPCookieJar import LWPCookieJar, lwp_cookie_str
+from _MozillaCookieJar import MozillaCookieJar
+from _MSIECookieJar import MSIECookieJar
+from _urllib2_support import \
+ Request, \
+ OpenerDirector, build_opener, install_opener, urlopen, \
+ OpenerFactory, urlretrieve, BaseHandler, HeadParser, \
+ RobotExclusionError
+try:
+ from _urllib2_support import XHTMLCompatibleHeadParser
+except ImportError:
+ pass
+from ClientCookie._urllib2_support import \
+ HTTPHandler, HTTPRedirectHandler, \
+ HTTPRequestUpgradeProcessor, \
+ HTTPEquivProcessor, SeekableProcessor, HTTPCookieProcessor, \
+ HTTPRefererProcessor, \
+ HTTPRefreshProcessor, HTTPErrorProcessor, \
+ HTTPResponseDebugProcessor, HTTPRedirectDebugProcessor, \
+ HTTPRobotRulesProcessor
+
+import httplib
+if hasattr(httplib, 'HTTPS'):
+ from ClientCookie._urllib2_support import HTTPSHandler
+del httplib
+
+from _Util import http2time as str2time
+from _Util import response_seek_wrapper
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_html.py
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_html.py (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_html.py Fri May 5 02:10:40 2006
@@ -8,13 +8,11 @@
"""
-from __future__ import generators
-
import re, copy, urllib, htmlentitydefs
from urlparse import urljoin
-import ClientCookie
-from ClientCookie._HeadersUtil import split_header_words, is_html as _is_html
+import _Request
+from _HeadersUtil import split_header_words, is_html as _is_html
## # XXXX miserable hack
## def urljoin(base, url):
@@ -120,9 +118,9 @@
link_class=Link,
urltags=None,
):
- import pullparser
+ import _pullparser
if link_parser_class is None:
- link_parser_class = pullparser.TolerantPullParser
+ link_parser_class = _pullparser.TolerantPullParser
self.link_parser_class = link_parser_class
self.link_class = link_class
if urltags is None:
@@ -143,7 +141,6 @@
def links(self):
"""Return an iterator that provides links of the document."""
- import pullparser
response = self._response
encoding = self._encoding
base_url = self._base_url
@@ -200,7 +197,7 @@
form_parser_class = ClientForm.FormParser
self.form_parser_class = form_parser_class
if request_class is None:
- request_class = ClientCookie.Request
+ request_class = _Request.Request
self.request_class = request_class
self.backwards_compat = backwards_compat
self._response = None
@@ -231,12 +228,12 @@
self._encoding = encoding
def title(self):
- import pullparser
- p = pullparser.TolerantPullParser(
+ import _pullparser
+ p = _pullparser.TolerantPullParser(
self._response, encoding=self._encoding)
try:
p.get_tag("title")
- except pullparser.NoMoreTokensError:
+ except _pullparser.NoMoreTokensError:
return None
else:
return p.get_text()
@@ -384,7 +381,7 @@
url = clean_url(url, encoding)
text = link.firstText(lambda t: True)
if text is BeautifulSoup.Null:
- # follow pullparser's weird behaviour rigidly
+ # follow _pullparser's weird behaviour rigidly
if link.name == "a":
text = ""
else:
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_mechanize.py
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_mechanize.py (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_mechanize.py Fri May 5 02:10:40 2006
@@ -22,10 +22,10 @@
import urllib2, urlparse, sys, copy
-import ClientCookie
-
from _useragent import UserAgent
from _html import DefaultFactory
+from _Util import response_seek_wrapper
+import _Request
__version__ = (0, 1, 0, "a", None) # 0.1.0a
@@ -62,7 +62,7 @@
if sys.version_info[:2] >= (2, 4):
- from ClientCookie._Opener import OpenerMixin
+ from _Opener import OpenerMixin
else:
class OpenerMixin: pass
@@ -76,7 +76,7 @@
Public attributes:
- request: current request (ClientCookie.Request or urllib2.Request)
+ request: current request (mechanize.Request or urllib2.Request)
form: currently selected form (see .select_form())
"""
@@ -93,7 +93,7 @@
factory: object implementing the mechanize.Factory interface.
history: object implementing the mechanize.History interface. Note this
interface is still experimental and may change in future.
- request_class: Request class to use. Defaults to ClientCookie.Request
+ request_class: Request class to use. Defaults to mechanize.Request
by default for Pythons older than 2.4, urllib2.Request otherwise.
The Factory and History objects passed in are 'owned' by the Browser,
@@ -113,7 +113,7 @@
if request_class is None:
if not hasattr(urllib2.Request, "add_unredirected_header"):
- request_class = ClientCookie.Request
+ request_class = _Request.Request
else:
request_class = urllib2.Request # Python >= 2.4
@@ -168,9 +168,8 @@
response = error
## except (IOError, socket.error, OSError), error:
## # Yes, urllib2 really does raise all these :-((
-## # See test_urllib2.py in stdlib and in ClientCookie for examples
-## # of socket.gaierror and OSError, plus note that FTPHandler raises
-## # IOError.
+## # See test_urllib2.py for examples of socket.gaierror and OSError,
+## # plus note that FTPHandler raises IOError.
## # XXX I don't seem to have an example of exactly socket.error being
## # raised, only socket.gaierror...
## # I don't want to start fixing these here, though, since this is a
@@ -194,7 +193,7 @@
def set_response(self, response):
"""Replace current response with (a copy of) response."""
- from ClientCookie._Util import closeable_response
+ from _Util import closeable_response
# sanity check, necessary but far from sufficient
if not (hasattr(response, "info") and hasattr(response, "geturl") and
hasattr(response, "read")):
@@ -207,8 +206,8 @@
if not hasattr(response, 'closeable_response'):
# we expect to get here if a urllib2 handler constructed the
# response, i.e. the response is an urllib.addinfourl, instead of a
- # ClientCookie._Util.closeable_response as returned by
- # e.g. ClientCookie.HTTPHandler
+ # _Util.closeable_response as returned by
+ # e.g. mechanize.HTTPHandler
try:
code = response.code
except AttributeError:
@@ -222,7 +221,7 @@
response = closeable_response(
response.fp, response.info(), response.geturl(), code, msg)
if not hasattr(response, "seek"):
- response = ClientCookie.response_seek_wrapper(response)
+ response = response_seek_wrapper(response)
# 0) don't want to copy here, but
# 1) don't want to copy some of the time and not other times
# 2) need response to be .close()able and .seek()able
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_urllib2_support.py
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_urllib2_support.py (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_urllib2_support.py Fri May 5 02:10:40 2006
@@ -13,17 +13,11 @@
import copy, time, tempfile, htmlentitydefs, re
-import ClientCookie
from _ClientCookie import CookieJar, request_host
from _Util import isstringlike, startswith, getheaders, closeable_response
from _HeadersUtil import is_html
from _Debug import getLogger
-debug = getLogger("ClientCookie.cookies").debug
-
-try: True
-except NameError:
- True = 1
- False = 0
+debug = getLogger("mechanize.cookies").debug
CHUNK = 1024 # size of chunks fed to HTML HEAD parser, in bytes
@@ -181,9 +175,7 @@
# -------------------------------------------------------------------
- # Beware, the following encoding code is cut-and-pasted between
- # ClientCookie, ClientForm, mechanize and pullparser, and they differ
- # subtly :-(((
+ # XXXXXXX fix me!!!
# This particular variant is identical to that in mechanize.
def unescape(data, entities, encoding):
@@ -526,13 +518,14 @@
https_request = http_request
https_response = http_response
+ # XXXXX remove these two
class HTTPResponseDebugProcessor(BaseHandler):
handler_order = 900 # before redirections, after everything else
def http_response(self, request, response):
if not hasattr(response, "seek"):
response = response_seek_wrapper(response)
- info = getLogger("ClientCookie.http_responses").info
+ info = getLogger("mechanize.http_responses").info
try:
info(response.read())
finally:
@@ -545,7 +538,7 @@
class HTTPRedirectDebugProcessor(BaseHandler):
def http_request(self, request):
if hasattr(request, "redirect_dict"):
- info = getLogger("ClientCookie.http_redirects").info
+ info = getLogger("mechanize.http_redirects").info
info("redirecting to %s", request.get_full_url())
return request
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_useragent.py
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_useragent.py (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/mechanize/_useragent.py Fri May 5 02:10:40 2006
@@ -13,11 +13,14 @@
import sys
import urllib2
-import ClientCookie
+
if sys.version_info[:2] >= (2, 4):
from urllib2 import OpenerDirector, BaseHandler, HTTPErrorProcessor
else:
- from ClientCookie import OpenerDirector, BaseHandler, HTTPErrorProcessor
+ from _Opener import OpenerDirector
+ from _urllib2_support import BaseHandler, HTTPErrorProcessor
+
+import _urllib2_support
class HTTPRefererProcessor(BaseHandler):
def http_request(self, request):
@@ -99,7 +102,7 @@
handler_classes = {
# scheme handlers
- "http": ClientCookie.HTTPHandler,
+ "http": _urllib2_support.HTTPHandler,
"ftp": urllib2.FTPHandler, # CacheFTPHandler is buggy in 2.3
"file": urllib2.FileHandler,
"gopher": urllib2.GopherHandler,
@@ -108,25 +111,26 @@
"_unknown": urllib2.UnknownHandler,
# HTTP{S,}Handler depend on HTTPErrorProcessor too
"_http_error": HTTPErrorProcessor,
- "_http_request_upgrade": ClientCookie.HTTPRequestUpgradeProcessor,
+ "_http_request_upgrade": _urllib2_support.HTTPRequestUpgradeProcessor,
"_http_default_error": urllib2.HTTPDefaultErrorHandler,
# feature handlers
"_basicauth": urllib2.HTTPBasicAuthHandler,
"_digestauth": urllib2.HTTPBasicAuthHandler,
- "_redirect": ClientCookie.HTTPRedirectHandler,
- "_cookies": ClientCookie.HTTPCookieProcessor,
- "_refresh": ClientCookie.HTTPRefreshProcessor,
+ "_redirect": _urllib2_support.HTTPRedirectHandler,
+ "_cookies": _urllib2_support.HTTPCookieProcessor,
+ "_refresh": _urllib2_support.HTTPRefreshProcessor,
"_referer": HTTPRefererProcessor, # from this module, note
- "_equiv": ClientCookie.HTTPEquivProcessor,
- "_seek": ClientCookie.SeekableProcessor,
+ "_equiv": _urllib2_support.HTTPEquivProcessor,
+ "_seek": _urllib2_support.SeekableProcessor,
"_proxy": urllib2.ProxyHandler,
"_proxy_basicauth": urllib2.ProxyBasicAuthHandler,
"_proxy_digestauth": urllib2.ProxyDigestAuthHandler,
+ "_robots": _urllib2_support.HTTPRobotRulesProcessor,
# debug handlers
- "_debug_redirect": ClientCookie.HTTPRedirectDebugProcessor,
- "_debug_response_body": ClientCookie.HTTPResponseDebugProcessor,
+ "_debug_redirect": _urllib2_support.HTTPRedirectDebugProcessor,
+ "_debug_response_body": _urllib2_support.HTTPResponseDebugProcessor,
}
default_schemes = ["http", "ftp", "file", "gopher"]
@@ -137,14 +141,11 @@
"_refresh", "_equiv",
"_basicauth", "_digestauth",
"_proxy", "_proxy_basicauth", "_proxy_digestauth",
- "_seek",
+ "_seek", "_robots",
]
- if hasattr(ClientCookie, 'HTTPSHandler'):
- handler_classes["https"] = ClientCookie.HTTPSHandler
+ if hasattr(_urllib2_support, 'HTTPSHandler'):
+ handler_classes["https"] = _urllib2_support.HTTPSHandler
default_schemes.append("https")
- if hasattr(ClientCookie, "HTTPRobotRulesProcessor"):
- handler_classes["_robots"] = ClientCookie.HTTPRobotRulesProcessor
- default_features.append("_robots")
def __init__(self):
OpenerDirector.__init__(self)
@@ -224,7 +225,7 @@
"this class can't do HTTP Referer: use mechanize.Browser instead")
def set_cookiejar(self, cookiejar):
- """Set a ClientCookie.CookieJar, or None."""
+ """Set a mechanize.CookieJar, or None."""
self._set_handler("_cookies", obj=cookiejar)
# XXX could use Greg Stein's httpx for some of this instead?
@@ -289,36 +290,33 @@
"""
self._set_handler("_referer", handle)
self._handle_referer = bool(handle)
+ # XXXXX isn't this always true?
def set_seekable_responses(self, handle):
"""Make response objects .seek()able."""
self._set_handler("_seek", handle)
def set_debug_redirects(self, handle):
- """Log information about HTTP redirects.
-
- This includes refreshes, which show up as faked 302 redirections at the
- moment.
+ """Log information about HTTP redirects (including refreshes).
Logging is performed using module logging. The logger name is
- "ClientCookie.http_redirects". To actually print some debug output,
+ "mechanize.http_redirects". To actually print some debug output,
eg:
- logger = logging.getLogger("ClientCookie.http_redirects")
- logger.addHandler(logging.StreamHandler())
+ import sys, logging
+ logger = logging.getLogger("mechanize.http_redirects")
+ logger.addHandler(logging.StreamHandler(sys.stdout))
logger.setLevel(logging.INFO)
Other logger names relevant to this module:
- "ClientCookie.http_responses"
- "ClientCookie.cookies" (or "cookielib" if running Python 2.4)
+ "mechanize.http_responses"
+ "mechanize.cookies" (or "cookielib" if running Python 2.4)
To turn on everything:
- for logger in [
- logging.getLogger("ClientCookie"),
- logging.getLogger("cookielib"),
- ]:
- logger.addHandler(logging.StreamHandler())
- logger.setLevel(logging.INFO)
+ import sys, logging
+ logger = logging.getLogger("mechanize")
+ logger.addHandler(logging.StreamHandler(sys.stdout))
+ logger.setLevel(logging.INFO)
"""
self._set_handler("_debug_redirect", handle)
@@ -371,6 +369,7 @@
self.add_handler(newhandler)
self._ua_handlers[name] = newhandler
+# XXXXX cruft
def remove(sequence, obj):
# for use when can't use .remove() because of obj.__cmp__ :-(
# (ClientCookie only requires Python 2.0, which doesn't have __lt__)
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/setup.py
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/setup.py (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/setup.py Fri May 5 02:10:40 2006
@@ -3,6 +3,20 @@
Stateful programmatic web browsing, after Andy Lester's Perl module
WWW::Mechanize.
+
+The library is layered: mechanize.Browser (stateful web browser),
+mechanize.UserAgent (configurable URL opener), plus urllib2 handlers.
+
+Features include: ftp:, http: and file: URL schemes, browser history,
+high-level hyperlink and HTML form support, HTTP cookies, HTTP-EQUIV and
+Refresh, Referer [sic] header, robots.txt, redirections, proxies, and
+Basic and Digest HTTP authentication. mechanize's response objects are
+(lazily-) .seek()able and still work after .close().
+
+Much of the code originally derived from Perl code by Gisle Aas
+(libwww-perl), Johnny Lee (MSIE Cookie support) and last but not least
+Andy Lester (WWW::Mechanize). urllib2 was written by Jeremy Hylton.
+
"""
def unparse_version(tup):
@@ -39,10 +53,7 @@
## open("mechanize/_mechanize.py").read())
## VERSION = unparse_version(str_to_tuple(VERSION_MATCH.group(1)))
VERSION = "0.1.0a"
-INSTALL_REQUIRES = [
- "ClientForm>=0.2.2, ==dev",
- "ClientCookie>=1.3.0, ==dev",
- "pullparser>=0.1.0, ==dev"]
+INSTALL_REQUIRES = ["ClientForm>=0.2.2, ==dev"]
NAME = "mechanize"
PACKAGE = True
LICENSE = "BSD" # or ZPL 2.1
@@ -71,6 +82,7 @@
Topic :: System :: Archiving :: Mirroring
Topic :: System :: Networking :: Monitoring
Topic :: System :: Systems Administration
+Topic :: Text Processing
Topic :: Text Processing :: Markup
Topic :: Text Processing :: Markup :: HTML
Topic :: Text Processing :: Markup :: XML
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/test.py
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/test.py (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/test.py Fri May 5 02:10:40 2006
@@ -10,7 +10,9 @@
# Modules containing tests to run -- a test is anything named *Tests, which
# should be classes deriving from unittest.TestCase.
-MODULE_NAMES = ["test_cookielib", "test_urllib2"]
+MODULE_NAMES = ["test_date", "test_mechanize", "test_misc", "test_cookies",
+ "test_headers", "test_urllib2", "test_pullparser",
+ ]
import sys, os, traceback, logging
from unittest import defaultTestLoader, TextTestRunner, TestSuite, TestCase
@@ -18,7 +20,7 @@
level = logging.DEBUG
#level = logging.INFO
#level = logging.NOTSET
-#logging.getLogger("cookielib").setLevel(level)
+#logging.getLogger("mechanize").setLevel(level)
class TestProgram:
"""A command-line program that runs a set of tests; this is primarily
Modified: wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/test/test_mechanize.py
==============================================================================
--- wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/test/test_mechanize.py (original)
+++ wwwsearch/mechanize/branch/mechanize-clientcookie-pullparser-merge/test/test_mechanize.py Fri May 5 02:10:40 2006
@@ -6,8 +6,6 @@
from unittest import TestCase
import StringIO, re, UserDict, urllib2
-import ClientCookie
-
import mechanize
FACTORY_CLASSES = [mechanize.DefaultFactory]
try:
@@ -283,7 +281,6 @@
class BrowserTests(TestCase):
def test_referer(self):
- import ClientCookie
b = TestBrowser()
url = "http://www.example.com/"
r = MockResponse(url,
@@ -303,7 +300,7 @@
b.add_handler(MockHandler([("http_open", r)]))
# Referer not added by .open()...
- req = ClientCookie.Request(url)
+ req = mechanize.Request(url)
b.open(req)
self.assert_(req.get_header("Referer") is None)
# ...even if we're visiting a document
@@ -725,7 +722,7 @@
class ResponseTests(TestCase):
def test_set_response(self):
import copy
- from ClientCookie import response_seek_wrapper
+ from mechanize import response_seek_wrapper
br = TestBrowser()
url = "http://example.com/"
@@ -781,7 +778,7 @@
ua.set_handled_schemes, ["blah", "_blah"])
ua.set_handled_schemes(["blah"])
- req = ClientCookie.Request("blah://example.com/")
+ req = mechanize.Request("blah://example.com/")
r = ua.open(req)
exp_calls = [("blah_open", (req,), {})]
assert len(ua.calls) == len(exp_calls)
@@ -789,7 +786,7 @@
self.assertEqual(expect, got[1:])
ua.calls = []
- req = ClientCookie.Request("blah://example.com/")
+ req = mechanize.Request("blah://example.com/")
ua._set_handler("_blah", True)
r = ua.open(req)
exp_calls = [
More information about the wwwsearch-commits
mailing list