Why Would A Timeout Avoid A Tornado Hang?

Waiting on a concurrent.futures.Future from a ThreadPoolExecutor in a Tornado coroutine sometimes hangs for me: def slowinc(x): time.sleep(0.1) return x + 1 yield tp_execut

Solution 1:

I don't have a complete solution but I think I can offer a simpler workaround: start up a background PeriodicCallback that does nothing in a short interval: PeriodicCallback(lambda: None, 500).start(). This will make sure the IOLoop wakes up periodically without intruding into all your yield executor.submit() calls.

The symptom suggests that the problem lies in the "waker" behavior of add_callback: https://github.com/tornadoweb/tornado/blob/d9c5bc8fb6530a03ebbb6da667e26685b8eee0ea/tornado/ioloop.py#L929-L944

This code was changed in Tornado 4.3 (https://github.com/tornadoweb/tornado/pull/1511/files). If you're on 4.3, see if the problem still exists in 4.2. Could anything in your "unpleasant" environment be causing thread.get_ident() to behave differently than tornado expects?

There are reports of (rare) problems with the waker "pipe" on windows: https://github.com/tornadoweb/tornado/pull/1364