1

[If this were Gnome glib, I would create a GSource object that would be called each time before gnome glib's event loop blocked on select.]

Here is what I am trying to accomplish with asyncio that as an asyncio noob I don't yet know how to do:

I need to integrate with a transport object over which I have no source code influence. The transport maintains a write buffer from which it sends data via TCP/IP whenever its socket is writable. The transport is integrated with asyncio, so it only makes progress in the context of asyncio's event loop. The transport exposes the method get_write_buffer_size().

I need to create either an async future or coroutine (or something else that will efficiently accomplish the task) on which I can call loop.run_until() that would cause the loop.run_until() call to return as soon as possible after the transport's write buffer empties out (i.e., transport.get_write_buffer_size() returns to 0).

How can the above be accomplished efficiently and "asyncionically"?

Thank you, Vitaly

vitaly-krugl
  • 31
  • 4
  • Do you have access to the file descriptor used by `transport`? – user4815162342 Mar 19 '18 at 21:54
  • the socket used by the transport is in a private member of the transport instance, so my protocol technically shouldn't be accessing it. user4815162342, what did you have in mind? – vitaly-krugl Mar 20 '18 at 23:39
  • I had in mind using `loop.add_writer` to register a callback that will check whether `transport.get_write_buffer_size()` has reached 0. – user4815162342 Mar 21 '18 at 08:57
  • I see. Unless I am missing something, using `loop.add_writer()` is not an option because the transport registers `loop.add_writer()` on the socket when it has something to write. So, my call to `loop.add_writer()` would replace the transport's callback. I know, the "add" part of `add_writer()` sounds like you could have multiple writer callbacks, but you can't. It just replaces the prior one, if any. I could probably clone the file descriptor and use `add_writer()` on that, but I think we're going down a slipper slope with this idea. – vitaly-krugl Mar 22 '18 at 05:54
  • It should have been called `set_writer()`. – vitaly-krugl Mar 22 '18 at 05:55
  • Hmm, you're right, `add_writer` and friends will not be of use here. Maybe you could create a dummy file descriptor to detect when the event loop is idling, as described in [this answer](https://stackoverflow.com/a/48491563/1600898)? – user4815162342 Mar 22 '18 at 07:33
  • Thanks again. It looks like the `detect_iowait()` example at that link is intended to flush out tasks that could complete within "one iteration" of the event loop. The problem there is that the reset socket connection makes that socket always readable, so the event loop will always terminate the I/O poll ASAP, thus turning the system into a polling one with a very high CPU utilization. – vitaly-krugl Mar 23 '18 at 19:15
  • 1
    For my blocking wrapper, I will hijack the transport's `Transport.set_write_buffer_limits(0, 0)` feature to get callbacks whenever the transport's write buffer becomes empty. I opened a [ticket][1] with bugs.python.org about this. They might add a `write_buffer_drained` callback in a future python release. [1]: https://bugs.python.org/issue33118 – vitaly-krugl Mar 23 '18 at 19:32

0 Answers0