Skip to content

Enhance asyncio.staggered.staggered_race as promised #145963

@jonathandung

Description

@jonathandung

Feature or enhancement

Proposal:

In Lib/asyncio/staggered.py, lines 14-65:

async def staggered_race(coro_fns, delay, *, loop=None):
    """Run coroutines with staggered start times and take the first to finish.

    This method takes an iterable of coroutine functions. The first one is
    started immediately
    ... more documentation ...
    """
    # TODO: when we have aiter() and anext(), allow async iterables in coro_fns.

The todo comment indicates the potential enhancement of allowing PEP 492 async iterables to be passed as the coro_fns argument. However, this comment never seemed to have been followed up.

It is currently highly non-trivial to convert an async iterable into a regular iterable in a non-async context, and annoying even in an async context. Extra work will have to be done by the user, whereas just some simple code changes in this source would fix this problem entirely.

Most likely, the change would involve implementing a simple async version of enumerate (it doesn't even have to be a class, nor does it require implementation in C) that accepts iterables, async or not. Implementation (has room for improvement):

import collections.abc
async def aenumerate(ait, start=0):
    if isinstance(ait, collections.abc.AsyncIterable):
        async for j in ait:
            yield start, j
            start += 1
    elif isinstance(ait, collections.abc.Iterable):
        for j in ait:
            yield start, j
            start += 1
    else:
        raise TypeError('ait is not iterable')

Then the changes would be as follows:

  • Line 68:
aenum_coro_fns = aenumerate(coro_fns)
# original: enum_coro_fns = enumerate(coro_fns)
  • Line 109, in run_one_coro, under try block:
this_index, coro_fn = anext(aenum_coro_fns)
# original: this_index, coro_fn = next(enum_coro_fns)

These are the only occurrences of coro_fns and enum_coro_fns; the main logic is unaffected.

This is a simple change with essentially no drawbacks, and I hope to get it across before the feature freeze.

I would open a pull request, but I am not sure about the performance of aenumerate and where to put it, since it appears to be an overall useful function. This may require further discussion.

See the source code of asyncio.staggered here.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    stdlibStandard Library Python modules in the Lib/ directorytopic-asynciotype-featureA feature request or enhancement

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions