Connecting different Python interpreters¶
Dumping and loading values across interpreter versions¶
New in version 1.1.
Execnet offers a new safe and fast cross-interpreter serialization of python objects which you can use to dump builtin python data structures and load them later with the same or a different python interpreter (including between Python2 and Python3). The standard library offers the pickle and marshal modules but they do not work safely between different interpreter versions. Using xml/json requires a mapping of Python objects and is not easy to get right. Moreover, execnet allows to control handling of bytecode/strings/unicode types. Here is an example:
# using python2 import execnet with open("data.py23", "wb") as f: f.write(execnet.dumps(["hello", "world"])) # using Python3 import execnet with open("data.py23", "rb") as f: val = execnet.loads(f.read(), py2str_as_py3str=True) assert val == ["hello", "world"]
See the cross-interpreter serialization of python objects for more details on string conversion options. Please note, that you can not dump user-level instances, only builtin python types.
Connect to Python2/Numpy from Python3¶
Here we run a Python3 interpreter to connect to a Python2.6 interpreter that has numpy installed. We send items to be added to an array and receive back the remote “repr” of the array:
import execnet gw = execnet.makegateway("popen//python=python2.6") channel = gw.remote_exec(""" import numpy array = numpy.array([1,2,3]) while 1: x = channel.receive() if x is None: break array = numpy.append(array, x) channel.send(repr(array)) """) for x in range(10): channel.send(x) channel.send(None) print (channel.receive())
will print on the CPython3.1 side:
array([1, 2, 3, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
A more refined real-life example of python3/python2 interaction is the anyvc project which uses version-control bindings in a Python2 subprocess in order to offer Python3-based library functionality.
Reconfiguring the string coercion between python2 and python3¶
Sometimes the default configuration of string coercion (2str to 3str, 3str to 2unicode) is inconvient, thus it can be reconfigured via gw.reconfigure and channel.reconfigure. Here is an example session on a Python2 interpreter:
>>> import execnet >>> execnet.makegateway("popen//python=python3.2") <Gateway id='gw0' receive-live, 0 active channels> >>> gw=execnet.makegateway("popen//python=python3.2") >>> gw.remote_exec("channel.send('hello')").receive() u'hello' >>> gw.reconfigure(py3str_as_py2str=True) >>> gw.remote_exec("channel.send('hello')").receive() 'hello' >>> ch = gw.remote_exec('channel.send(type(channel.receive()).__name__)') >>> ch.send('a') >>> ch.receive() 'str' >>> ch = gw.remote_exec('channel.send(type(channel.receive()).__name__)') >>> ch.reconfigure(py2str_as_py3str=False) >>> ch.send('a') >>> ch.receive() u'bytes'
Work with Java objects from CPython¶
Use your CPython interpreter to connect to a Jython 2.5.1 interpreter and work with Java types:
import execnet gw = execnet.makegateway("popen//python=jython") channel = gw.remote_exec(""" from java.util import Vector v = Vector() v.add('aaa') v.add('bbb') for val in v: channel.send(val) """) for item in channel: print (item)
will print on the CPython side:
Work with C# objects from CPython¶
(Experimental) use your CPython interpreter to connect to a IronPython interpreter which can work with C# classes. Here is an example for instantiating a CLR Array instance and sending back its representation:
import execnet gw = execnet.makegateway("popen//python=ipy") channel = gw.remote_exec(""" import clr clr.AddReference("System") from System import Array array = Array[float]([1,2]) channel.send(str(array)) """) print (channel.receive())
using Mono 2.0 and IronPython-1.1 this will print on the CPython side:
Using IronPython needs more testing, likely newer versions will work better. please feedback if you have information.