From commits-noreply at bitbucket.org Thu Jul 7 08:30:20 2011 From: commits-noreply at bitbucket.org (Bitbucket) Date: Thu, 07 Jul 2011 06:30:20 -0000 Subject: [execnet-commit] commit/execnet: 4 new changesets Message-ID: <20110707063020.20696.73097@bitbucket01.managed.contegix.com> 4 new changesets in execnet: http://bitbucket.org/hpk42/execnet/changeset/394993213fc1/ changeset: 394993213fc1 user: alex_gaynor date: 2011-07-06 01:32:45 summary: Fix for people using comcast (sigh). affected #: 2 files (250 bytes) --- a/conftest.py Wed Mar 02 17:09:08 2011 +0100 +++ b/conftest.py Tue Jul 05 16:32:45 2011 -0700 @@ -36,6 +36,9 @@ help=("set gateway setup scope, default: session.")) group.addoption('--pypy', action="store_true", dest="pypy", help=("run some tests also against pypy-c")) + group.addoption('--broken-isp', action="store_true", dest="broken_isp", + help=("Skips tests that assume your ISP doesn't put up a landing " + "page on invalid addresses")) def pytest_report_header(config): lines = [] --- a/testing/test_gateway.py Wed Mar 02 17:09:08 2011 +0100 +++ b/testing/test_gateway.py Tue Jul 05 16:32:45 2011 -0700 @@ -241,6 +241,7 @@ """) py.test.raises(channel.RemoteError, channel.receive) + at py.test.mark.skipif("config.option.broken_isp") def test_socket_gw_host_not_found(gw): py.test.raises(execnet.HostNotFound, 'execnet.makegateway("socket=qwepoipqwe:9000")' http://bitbucket.org/hpk42/execnet/changeset/ecabfc1a79aa/ changeset: ecabfc1a79aa user: alex_gaynor date: 2011-07-06 02:08:44 summary: Added dont_write_bytecode flag for Popen gateways. affected #: 5 files (1.0 KB) --- a/CHANGELOG Tue Jul 05 16:32:45 2011 -0700 +++ b/CHANGELOG Tue Jul 05 17:08:44 2011 -0700 @@ -1,3 +1,10 @@ +1.0.10 +-------------------------------- + +- added a ``dont_write_bytecode`` option to Popen gateways, this sets the + ``sys.dont_write_bytecode`` flag on the spawned process, this only works on + CPython 2.6 and higher. + 1.0.9 -------------------------------- @@ -7,7 +14,7 @@ - channel.makefile() objects now have a isatty() returning False - group.allocate_id(spec) allows to early-determine an (automatic) id - + - internal refactorings and cleanups (thanks Ronny Pfannschmidt): - refactor message types into received handler functions - refactor b(chr(opcode)) to bchr(opcode) @@ -29,17 +36,17 @@ for writing a "pure-function" checker so that on Python2.6 onwards non-pure functions will be rejected. -- enhance rsyncing to also sync permissions (stat().st_mode) - of directories and files. +- enhance rsyncing to also sync permissions (stat().st_mode) + of directories and files. (should also resolve http://bitbucket.org/hpk42/py-trunk/issue/68/) -- fix rsyncing of symlinks, thanks to Charles Solar +- fix rsyncing of symlinks, thanks to Charles Solar (should also resolve http://bitbucket.org/hpk42/py-trunk/issue/70/) - update internal usage of apipkg to 1.0b6 - remote_exec(module) now makes sure that the linecache is updated - before reading and sending the source. thanks Ronny, Matt. + before reading and sending the source. thanks Ronny, Matt. - removed all trailing whitespace from source files @@ -47,42 +54,42 @@ -------------------------------- - try to avoid a random KeyboardInterrupt Error when threads - are ending. + are ending. -- extend xspec syntax to allow for one or multiple "env:NAME=value" - environment variable settings which will be set on the remote side. +- extend xspec syntax to allow for one or multiple "env:NAME=value" + environment variable settings which will be set on the remote side. (thanks Jakub Gustak) 1.0.6 -------------------------------- -- fix jython/windows interactions +- fix jython/windows interactions - fix waitclose/callback-with-endmarker race condition - fix race condition where multiple threads sending data over channels - would crash the serializer and process + would crash the serializer and process 1.0.5 -------------------------------- - more care during receiver-thread finalization during interp-shutdown, should get rid of annoying and meaningless exceptions -- fix glitch in ssh-fileserver example -- experimentally add "setup.py test" support - will run py.test +- fix glitch in ssh-fileserver example +- experimentally add "setup.py test" support - will run py.test 1.0.4 -------------------------------- - try to deal more cleanly with interpreter shutdown setting globals to - None. this avoids (hopefully) some bogus tracebacks at process exit. + None. this avoids (hopefully) some bogus tracebacks at process exit. 1.0.3 -------------------------------- -- refine termination some more: CTRL-C and gateway.exit will - now try harder to interrupt remote execution. this +- refine termination some more: CTRL-C and gateway.exit will + now try harder to interrupt remote execution. this helps to avoid left-over ssh-processes. -- fix read-on-non-blocking-files issue probably related to jython only: - the low-level read on subprocess pipes may be non-blocking, returning +- fix read-on-non-blocking-files issue probably related to jython only: + the low-level read on subprocess pipes may be non-blocking, returning less bytes than requested - so we now loop. - Windows/python2.4: fix bug that killing subprocesses would fail - make RemoteError and TimeoutError available directly on execnet namespace @@ -91,30 +98,30 @@ - update internal copy of apipkg - always skip remote tests if no ssh specs given -1.0.2 +1.0.2 -------------------------------- -- generalize channel-over-channel sending: you can now have channels +- generalize channel-over-channel sending: you can now have channels anywhere in a data structure (i.e. as an item of a container type). Add according examples. -- automatically close a channel when a remote callback raises - an exception, makes communication more robust because until - now failing callbacks rendered the receiverthread unuseable +- automatically close a channel when a remote callback raises + an exception, makes communication more robust because until + now failing callbacks rendered the receiverthread unuseable leaving the remote side in-accessible. - internally split socket gateways, speeds up popen-gateways - by 10% (now at <50 milliseconds per-gateway on a 1.5 GHZ machine) + by 10% (now at <50 milliseconds per-gateway on a 1.5 GHZ machine) -- fix bug in channel.receive() that would wrongly raise a TimeoutError - after 1000 seconds (thanks Ronny Pfannschmidt) +- fix bug in channel.receive() that would wrongly raise a TimeoutError + after 1000 seconds (thanks Ronny Pfannschmidt) 1.0.1 -------------------------------- - revamp and better structure documentation -- new method: gateway.hasreceiver() returns True +- new method: gateway.hasreceiver() returns True if the gateway is still receive-active. remote_status now only carries information about remote execution status. @@ -123,7 +130,7 @@ - new: execnet.Group can be indexed by integer - new: group.makegateway() uses group.default_spec if no spec is given - and the execnet.default_group uses ``popen`` as a default spec. + and the execnet.default_group uses ``popen`` as a default spec. - have popen-gateways use imports instead of source-strings, also improves debugging/tracebacks, as a side effect @@ -135,26 +142,26 @@ within time. - EOFError on channel.receive/waitclose if the other - side unexpectedly went away. When a gateway exits + side unexpectedly went away. When a gateway exits it now internally sends an explicit termination message - instead of abruptly closing. + instead of abruptly closing. -- introduce a timeout parameter to channel.receive() +- introduce a timeout parameter to channel.receive() and default to periodically internally wake up to let KeyboardInterrupts pass through. - EXECNET_DEBUG=2 will cause tracing to go to stderr, - which with popen slave gateways will relay back + which with popen slave gateways will relay back tracing to the instantiator process. - + 1.0.0 -------------------------------- -* introduce execnet.Group for managing gateway creation - and termination. Introduce execnet.default_group through which +* introduce execnet.Group for managing gateway creation + and termination. Introduce execnet.default_group through which all "global" calls are routed. cleanup gateway termination. - All Gateways get an id through which they can be + All Gateways get an id through which they can be retrieved from a group object. * deprecate execnet.XYZGateway in favour of direct makegateway() calls. @@ -163,14 +170,14 @@ way to indirectly setup a socket server ("installvia") through a gateway url. -* refine and automatically test documentation examples +* refine and automatically test documentation examples 1.0.0b3 -------------------------------- -* fix EXECNET_DEBUG to work with win32 +* fix EXECNET_DEBUG to work with win32 * add support for serializing longs, sets and frozensets (thanks - Benjamin Peterson) + Benjamin Peterson) * introduce remote_status() method which on the low level gives information about the remote side of a gateway * disallow explicit close in remote_exec situation @@ -196,12 +203,12 @@ * integrated new serializer code from Benjamin Peterson * improved support for Jython-2.5.1 -1.0.0alpha2 +1.0.0alpha2 ---------------------------- * improve documentation, new website -* use sphinx for documentation, added boilerplate files and setup.py +* use sphinx for documentation, added boilerplate files and setup.py * fixes for standalone usage, adding boilerplate files --- a/execnet/gateway.py Tue Jul 05 16:32:45 2011 -0700 +++ b/execnet/gateway.py Tue Jul 05 17:08:44 2011 -0700 @@ -235,13 +235,18 @@ """ This Gateway provides interaction with a newly started python subprocess. """ - def __init__(self, id, python=None): + def __init__(self, id, python=None, spec=None): """ instantiate a gateway to a subprocess started with the given 'python' executable. """ if not python: python = sys.executable - args = [str(python), '-u', '-c', popen_bootstrapline] + args = [str(python), '-u'] + if spec is not None and spec.dont_write_bytecode: + args.append("-B") + # Slight gymnastics in ordering these arguments because CPython (as of + # 2.7.1) ignores -B if you provide `python -c "something" -B` + args.extend(['-c', popen_bootstrapline]) super(PopenGateway, self).__init__(args, id=id) def _remote_bootstrap_gateway(self, io): --- a/execnet/multi.py Tue Jul 05 16:32:45 2011 -0700 +++ b/execnet/multi.py Tue Jul 05 17:08:44 2011 -0700 @@ -75,7 +75,7 @@ spec = XSpec(spec) self.allocate_id(spec) if spec.popen: - gw = gateway.PopenGateway(python=spec.python, id=spec.id) + gw = gateway.PopenGateway(python=spec.python, id=spec.id, spec=spec) elif spec.ssh: gw = gateway.SshGateway(spec.ssh, remotepython=spec.python, ssh_config=spec.ssh_config, id=spec.id) --- a/execnet/xspec.py Tue Jul 05 16:32:45 2011 -0700 +++ b/execnet/xspec.py Tue Jul 05 17:08:44 2011 -0700 @@ -12,7 +12,7 @@ * if no "=value" is given, assume a boolean True value """ # XXX allow customization, for only allow specific key names - popen = ssh = socket = python = chdir = nice = None + popen = ssh = socket = python = chdir = nice = dont_write_bytecode = None def __init__(self, string): self._spec = string --- a/testing/test_gateway.py Tue Jul 05 16:32:45 2011 -0700 +++ b/testing/test_gateway.py Tue Jul 05 17:08:44 2011 -0700 @@ -241,6 +241,22 @@ """) py.test.raises(channel.RemoteError, channel.receive) + @py.test.mark.skipif('sys.version_info < (2, 6)') + def test_dont_write_bytecode(self): + check_sys_dont_write_bytecode = """ + import sys + channel.send(sys.dont_write_bytecode) + """ + + gw = execnet.makegateway('popen') + channel = gw.remote_exec(check_sys_dont_write_bytecode) + ret = channel.receive() + assert not ret + gw = execnet.makegateway('popen//dont_write_bytecode') + channel = gw.remote_exec(check_sys_dont_write_bytecode) + ret = channel.receive() + assert ret + @py.test.mark.skipif("config.option.broken_isp") def test_socket_gw_host_not_found(gw): py.test.raises(execnet.HostNotFound, http://bitbucket.org/hpk42/execnet/changeset/02fa48b373ae/ changeset: 02fa48b373ae user: alex_gaynor date: 2011-07-06 23:44:53 summary: Added a bit to the docs aout the dont_write_bytecode flag. affected #: 1 file (244 bytes) --- a/doc/basics.txt Tue Jul 05 17:08:44 2011 -0700 +++ b/doc/basics.txt Wed Jul 06 14:44:53 2011 -0700 @@ -40,6 +40,11 @@ subprocess; running with the lowest CPU priority ("nice" level). By default current dir will be the current dir of the instantiator. +* ``popen//dont_write_bytecode`` uses the same executable as the current + Python, and also passes the ``-B`` flag on startup, which tells Python not + write ``.pyc`` or ``.pyo`` files. Note that this only works under CPython + 2.6 and newer. + * ``popen//env:NAME=value`` specifies a subprocess that uses the same interpreter as the one it is initiated from and additionally remotely sets an environment variable ``NAME`` to ``value``. http://bitbucket.org/hpk42/execnet/changeset/87b1ef746e15/ changeset: 87b1ef746e15 user: hpk42 date: 2011-07-07 08:29:29 summary: bumping version 1.0.10.dev1 - adding alex to authors file affected #: 4 files (48 bytes) --- a/CHANGELOG Wed Jul 06 14:44:53 2011 -0700 +++ b/CHANGELOG Thu Jul 07 08:29:29 2011 +0200 @@ -1,9 +1,9 @@ -1.0.10 +1.0.10.dev -------------------------------- - added a ``dont_write_bytecode`` option to Popen gateways, this sets the ``sys.dont_write_bytecode`` flag on the spawned process, this only works on - CPython 2.6 and higher. + CPython 2.6 and higher. Thanks to Alex Gaynor. 1.0.9 -------------------------------- --- a/LICENSE Wed Jul 06 14:44:53 2011 -0700 +++ b/LICENSE Thu Jul 07 08:29:29 2011 +0200 @@ -6,7 +6,8 @@ This package contains some code and contributions from Armin Rigo, Benjamin Peterson, Carl Friedrich Bolz, - Maciej Fijalkowski, Ronny Pfannschmidt and others + Maciej Fijalkowski, Ronny Pfannschmidt, + Alex Gaynor and others which are useable under the MIT license. @@ -14,4 +15,4 @@ the code under a different license than the GPL please contact me. -holger krekel, November 2009, holger at merlinux eu +holger krekel, 2011, holger at merlinux eu --- a/execnet/__init__.py Wed Jul 06 14:44:53 2011 -0700 +++ b/execnet/__init__.py Thu Jul 07 08:29:29 2011 +0200 @@ -3,7 +3,7 @@ (c) 2010, Holger Krekel and others """ -__version__ = '1.0.9' +__version__ = '1.0.10.dev1' import execnet.apipkg --- a/setup.py Wed Jul 06 14:44:53 2011 -0700 +++ b/setup.py Thu Jul 07 08:29:29 2011 +0200 @@ -40,7 +40,7 @@ name='execnet', description='execnet: rapid multi-Python deployment', long_description = __doc__, - version='1.0.9', + version='1.0.10.dev1', url='http://codespeak.net/execnet', license='GPL V2 or later', platforms=['unix', 'linux', 'osx', 'cygwin', 'win32'], Repository URL: https://bitbucket.org/hpk42/execnet/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email. From commits-noreply at bitbucket.org Thu Jul 7 08:42:01 2011 From: commits-noreply at bitbucket.org (Bitbucket) Date: Thu, 07 Jul 2011 06:42:01 -0000 Subject: [execnet-commit] commit/execnet: hpk42: mention --broken-isp Message-ID: <20110707064201.20696.27448@bitbucket01.managed.contegix.com> 1 new changeset in execnet: http://bitbucket.org/hpk42/execnet/changeset/b6e1dfaa89d1/ changeset: b6e1dfaa89d1 user: hpk42 date: 2011-07-07 08:41:55 summary: mention --broken-isp affected #: 1 file (149 bytes) --- a/CHANGELOG Thu Jul 07 08:29:29 2011 +0200 +++ b/CHANGELOG Thu Jul 07 08:41:55 2011 +0200 @@ -5,6 +5,10 @@ ``sys.dont_write_bytecode`` flag on the spawned process, this only works on CPython 2.6 and higher. Thanks to Alex Gaynor. +- added a pytest --broken-isp option to skip tests that assume + DNS queries for unknown hosts actually are resolved as such (Thanks + Alex Gaynor) + 1.0.9 -------------------------------- Repository URL: https://bitbucket.org/hpk42/execnet/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.