Changelogs » Execnet

PyUp Safety actively tracks 382,903 Python packages for vulnerabilities and notifies you when to upgrade.



  * Removed the ``apipkg`` dependency.


  * Update calls of `threading.Event.isSet()` to `threading.Event.is_set()`, which avoids a deprecation warning with Python 3.10.


  * Dropped support for Python 3.4.
  * `118 <>`__: Fixed internal leak that should make
  ``execnet`` execute remote code in the main thread more often; previously it would sometimes
  spawn a thread to execute a ``remote_exec`` call, even when the caller
  didn't issue multiple ``remote_exec`` calls at the same time. Some frameworks require code
  to execute in the main thread, so the previous behavior would break them on occasion (see
  `pytest-dev/pytest-xdist620 <>`__
  for an example).
  * `115 <>`__: Current working directory is now
  restored when calling ``script/``. The script now also loops by default
  when called from the command-line.


  * `108 <>`__: Revert ``linecache`` optimization introduced in ``1.7.0`` which
  broke remote execution.


  * `102 <>`__: Show paths in stack traces
  generated by ``remote_exec()``.
  * `100 <>`__: Fix flaky hangs in ``workerpool.waitall``.


  * `98 <>`__: Internal change to avoid
  using deprecated ``funcargs`` name in pytest 5+.


  * ``execnet`` no longer supports Python 2.6 and 3.3 (85). Users of those Python versions
  using a recent enough ``pip`` should not be affected, as ``pip`` will only install
  ``1.5.0`` for them.
  * Update test suite to support ``pytest>4``.


  - support shell escaping in python pathnames of popen.
  Eugene Ciurana discovered that execnet breaks if you use
  pathnames with spaces in a "python=" part of a spec.
  We now use shlex.split to split the string.  There is a
  potential for regressions if you used quote or escape
  sequences as part of your python command.
  - Only insert importdir into sys.path if it is not already in the path.
  This prevents a bug when using enum34 with python 3.6 and
  The issue is that enum34 installs an 'enum' module in site-packages
  which is normally shadowed by the stdlib version of enum, however in site-packages is added at the front the the
  search path. This means on the workers enum34 is hit for import enum
  which in turn causes import re to fail (as it makes use of the new
  enum features in 3.6).
  - fix 49 - use inspect.getfullargspec if possible to avoid deprecationwarnings
  - fix 56 - use partials in safe_terminate to avoid a bad carried binding
  - fix spec parsing on Windows due to path containing '\' characters.


  - fix a regression of the Serializer created by the implied opcode ordering
  which resulted in a incompatible opcode mapping
  *warning* stored serialized objects created with 1.4.0 are incompatible
  with previous versions and future versions
  additionally stored serialized objects containing complex objects will
  have a incompatible opcode when read with execnet < 1.4.0
  and wont be loadable with execnet 1.4.0 either
  its strongly suggested to avoid using the Serializer of execnet 1.4.0
  this affects devpi and the external pytest-cache plugin


  - de-vendor apipkg and use the pypi dependency instead
  (this also fixes the bpython interaction issues)
  - Fix issue38: provide ability to connect to Vagrant VMs easily
  using :code:`vagrant_ssh=defaut` or :code:`vagrant_ssh=machinename`
  this feature is experimental and will be refined in future releases.
  Thanks Christian Theune for the discussion and the initial pull request.
  - add support for serializing the "complex" type. Thanks Sebastian


  - fix issue33: index.txt to correctly mention MIT instead of GPL.
  - fix issue35: adapt some doctests, fix some channel tests for py3.
  - use subprocess32 when available for python < 3.
  - try to be a bit more careful when interpreter is shutting down
  to avoid random exceptions, thanks Alfredo Deza.
  - ignore errors on orphan file removal when rsyncing
  - fix issue34: limit use of import based bootstrap


  - fix issue22 -- during interpreter shutdown don't throw
  an exception when we can't send a termination sequence
  anymore as we are about to die anyway.
  - fix issue24 -- allow concurrent creation of gateways
  by guarding automatic id creation by a look.
  Thanks tlecomte.
  - majorly refactor internal thread and IO handling.
  execnet can now operate on different thread models,
  defaults to "thread" but allows for eventlet and
  gevent if it is installed.
  - gateway.remote_exec() will now execute in multiple
  threads on the other side by default.  The previous
  neccessity of running "gateway.remote_init_threads()"
  to allow for such concurrency is gone.  The latter
  method is now a no-op and will be removed in future
  versions of execnet.
  - fix issue20: prevent AttributError at interpreter shutdown
  by not trying to send close/last_message messages if the
  world around is half destroyed.
  - fix issue21: allow to create local gateways with sudo aka
  makegateway("popen//python=sudo python").
  Thanks Alfredo Deza for the PR.
  - streamline gateway termination and simplify proxy
  implementation. add more internal tracing.
  - if execution hangs in computation, we now try to
  send a SIGINT to ourselves on Unix platforms
  instead of just calling thread.interrupt_main()
  - change license from GPL to MIT
  - introduce execnet.dump/load variants of dumps/loads
  serializing/unserializing mechanism.
  - improve channel.receive() communication latency on python2
  by changing the default timeout of the underlying Queue.get
  to a regular None instead of the previous default -1
  which caused an internal positive timeout value
  (a hack probably introduced to allow CTRL-C to pass
  through for <python2.5 versions).
  - extended ssh-syntax to allow passing of command line args,
  e.g.  "ssh= -p 50 hostname". The options are passed to
  the underlying ssh client binary.  Thanks tundish.
  - fix issue15: interoperability with inspect.getstack().
  Thanks Peter Feiner.
  - fix issue10 : skip PYTHONDONTWRITEBYTECODE test if
  it we are running with PYTHONDONTWRITEBYTECODE set.
  - dont try the jython pid fixup on a RemoteIO
  - avoid accidentally setting exc_info() in


  - introduce execnet.dumps/loads providing serialization between
  python interpreters.
  - group.remote_exec now supports kwargs as well
  - support per channel string coercion configuration
  - now reads correct amounts of bytes from nonblocking fd's
  - added a ``dont_write_bytecode`` option to Popen gateways, this sets the
  ``sys.dont_write_bytecode`` flag on the spawned process, this only works on
  CPython 2.6 and higher.  Thanks to Alex Gaynor.
  - added a pytest --broken-isp option to skip tests that assume
  DNS queries for unknown hosts actually are resolved as such (Thanks
  Alex Gaynor)
  - fix issue 1 - decouple string coercion of channels and gateway
  - fix issue 2 - properly reconfigure the channels string coercion for rsync,
  so it can send from python2 to python3
  - fix issue 9 - propperly terminate the worker threadpools in safe_terminate
  - fix issue 8 - no longer kill remote pids locally on jython ssh gateways
  - refactor socketserver, so it can be directly remote_exec'd for starting a socket gateway on a remote


  - add gw.reconfigure() to configure per gateway options.  Currently supported:
  py2str_as_py3str and py3str_as_py2str to configure string deserialization
  - channel.makefile() objects now have a isatty() returning False
  - group.allocate_id(spec) allows to early-determine an (automatic) id
  - internal refactorings and cleanups (thanks Ronny Pfannschmidt):
  - refactor message types into received handler functions
  - refactor b(chr(opcode)) to bchr(opcode)
  - reorder Message ctor args, rename msgtype to msgcode
  - refactor gateway.send to take message's init args instead of a message
  - inline and remove Message.writeto/readfrom
  - refactor collection loading to avoid the indirection over tuple
  - remove the unused NamedThreadPool


  - new ``gateway.remote_exec(func, **kwargs)`` style fo executing
  a pure function with parameters.  The function on the remote
  side also needs to accept a ``channel`` which allows it to
  communicate back and forth.  Thanks to Ronny Pfannschmidt
  for implementing it with special kudos to Maciej Fijalkowski
  for writing a "pure-function" checker so that on Python2.6
  onwards non-pure functions will be rejected.
  - enhance rsyncing to also sync permissions (stat().st_mode)
  of directories and files.
  (should also resolve
  - fix rsyncing of symlinks, thanks to Charles Solar
  (should also resolve
  - update internal usage of apipkg to 1.0b6
  - remote_exec(module) now makes sure that the linecache is updated
  before reading and sending the source.  thanks Ronny, Matt.
  - removed all trailing whitespace from source files


  - try to avoid a random KeyboardInterrupt Error when threads
  are ending.
  - extend xspec syntax to allow for one or multiple "env:NAME=value"
  environment variable settings which will be set on the remote side.
  (thanks Jakub Gustak)


  - fix jython/windows interactions
  - fix waitclose/callback-with-endmarker race condition
  - fix race condition where multiple threads sending data over channels
  would crash the serializer and process


  - more care during receiver-thread finalization during interp-shutdown,
  should get rid of annoying and meaningless exceptions
  - fix glitch in ssh-fileserver example
  - experimentally add " test" support - will run py.test


  - try to deal more cleanly with interpreter shutdown setting globals to
  None. this avoids (hopefully) some bogus tracebacks at process exit.


  - refine termination some more: CTRL-C and gateway.exit will
  now try harder to interrupt remote execution.  this
  helps to avoid left-over ssh-processes.
  - fix read-on-non-blocking-files issue probably related to jython only:
  the low-level read on subprocess pipes may be non-blocking, returning
  less bytes than requested - so we now loop.
  - Windows/python2.4: fix bug that killing subprocesses would fail
  - make RemoteError and TimeoutError available directly on execnet namespace
  - fix some doc and test issues (thanks thm and ronny), add ssh_fileserver example
  - update internal copy of apipkg
  - always skip remote tests if no ssh specs given


  - generalize channel-over-channel sending: you can now have channels
  anywhere in a data structure (i.e. as an item of a container type).
  Add according examples.
  - automatically close a channel when a remote callback raises
  an exception, makes communication more robust because until
  now failing callbacks rendered the receiverthread unuseable
  leaving the remote side in-accessible.
  - internally split socket gateways, speeds up popen-gateways
  by 10% (now at <50 milliseconds per-gateway on a 1.5 GHZ machine)
  - fix bug in channel.receive() that would wrongly raise a TimeoutError
  after 1000 seconds (thanks Ronny Pfannschmidt)


  - revamp and better structure documentation
  - new method: gateway.hasreceiver() returns True
  if the gateway is still receive-active. remote_status
  now only carries information about remote execution status.
  - new: execnet.MultiChannel provides basic iteration/contain interface
  - new: execnet.Group can be indexed by integer
  - new: group.makegateway() uses group.default_spec if no spec is given
  and the execnet.default_group uses ``popen`` as a default spec.
  - have popen-gateways use imports instead of source-strings,
  also improves debugging/tracebacks, as a side effect
  popen-gateway startup can be substantially faster (>30%)
  - refine internal gateway exit/termination procedure
  and introduce group.terminate(timeout) which will
  attempt to kill all subprocesses that did not terminate
  within time.
  - EOFError on channel.receive/waitclose if the other
  side unexpectedly went away.  When a gateway exits
  it now internally sends an explicit termination message
  instead of abruptly closing.
  - introduce a timeout parameter to channel.receive()
  and default to periodically internally wake up
  to let KeyboardInterrupts pass through.
  - EXECNET_DEBUG=2 will cause tracing to go to stderr,
  which with popen worker gateways will relay back
  tracing to the instantiator process.


  * introduce execnet.Group for managing gateway creation
  and termination.  Introduce execnet.default_group through which
  all "global" calls are routed.  cleanup gateway termination.
  All Gateways get an id through which they can be
  retrieved from a group object.
  * deprecate execnet.XYZGateway in favour of direct makegateway() calls.
  * refine socketserver-examples, experimentally introduce a
  way to indirectly setup a socket server ("installvia")
  through a gateway url.
  * refine and automatically test documentation examples


  * fix EXECNET_DEBUG to work with win32
  * add support for serializing longs, sets and frozensets  (thanks
  Benjamin Peterson)
  * introduce remote_status() method which on the low level gives
  information about the remote side of a gateway
  * disallow explicit close in remote_exec situation
  * perform some more detailed tracing with EXECNET_DEBUG


  * make internal protocols more robust against serialization failures
  * fix a seralization bug with nested tuples containing empty tuples
  (thanks to ronny for discovering it)
  * setting the environment variable EXECNET_DEBUG will generate per
  process trace-files for debugging


  * added new examples for NumPy, Jython, IronPython
  * improved documentation
  * include for lazy-importing
  * integrated new serializer code from Benjamin Peterson
  * improved support for Jython-2.5.1


  * improve documentation, new website
  * use sphinx for documentation, added boilerplate files and
  * fixes for standalone usage, adding boilerplate files
  * imported py/execnet and made it work standalone