目录

Python协程Asyncio进阶学习笔记

一、前言:

  • 我们在上一章节中学习了通过Asyncio编写协成程序的方法。通过几个简单的demo,初步学习了协程函数的编写及调用。
  • 如果在实际开发中使用协程来处理异步问题,肯定会复杂许多。
  • 固本章将针对Python中的Asyncio做进一步的学习。

二、进阶学习

  • 我们先来看下上一个章节的例子,回忆回忆:

      import asyncio
    
      async def do_some_work():
        print('start working....');
        r = await asyncio.sleep(1);
        print('working ended.');
    
      loop = asyncio.get_event_loop();
      loop.run_until_complete(do_some_work())
      print('call close')
      loop.close()
    

进阶学习点一:

  • 细心的小伙伴可能注意到了,我们在最后面都会调用loop.close语句来关闭我们的loop

  • 如果不关闭,会如何呢?

  • 不关闭的话,我们将可以继续使用我们的loop来调用执行编写的协程代码。

  • 代码修改如下:

      import asyncio
    
      async def do_some_work():
      	print('start working....');
      	r = await asyncio.sleep(1);
      	print('working ended.');
    
      loop = asyncio.get_event_loop();
      m1_coroutine = do_some_work();
      m2_coroutine = do_some_work();
    
      loop.run_until_complete(m1_coroutine)
      loop.run_until_complete(m2_coroutine)
      print('call close')
      loop.close()
    
  • 结果输出为:

      start working....
      working ended.
      start working....
      working ended.
      call close
    
  • 可以看到,在调用close之前,我们将可以继续使用loop

  • 有些小伙伴可能会问了,如果不关闭会怎样呢?这个我也不好说,不过建议用完及调用close清理对象,一来防止对象误用,二来减少资源开销。

进阶学习点二:让协程对象运行的两种方式

  • 1、在另一个已经运行的协程中用 await 等待它

  • 2、通过 ensure_future 函数触发它的执行

  • 看示例代码:

      import asyncio
    
      async def do_some_work(name):
      	print('%s start working....'%name);
      	r = await asyncio.sleep(1);
      	print('%s working ended.'%name);
    
      loop = asyncio.get_event_loop();
      m1_coroutine = do_some_work('m1');
      m2_coroutine = do_some_work('m2');
    
      print(asyncio.iscoroutine(m1_coroutine))
      loop.run_until_complete(m1_coroutine)
      loop.run_until_complete(asyncio.ensure_future(m2_coroutine))
      print('call close')
      loop.close()
    
  • 结果输出为:

      True
      m1 start working....
      m1 working ended.
      m2 start working....
      m2 working ended.
      call close
    
  • 其实我们的run_until_complete接收的是一个future类型的参数,而我们的do_some_work返回的是Coroutine类型的。

  • 但是干好,我们的ensure_future接收一个Coroutine并返回future的数据

  • 然而,我门发现两个都可以正常运行,这是为什么呢?

  • 其实如果我们不做ensure_future操作的话,run_until_complete函数内部会自动将我们的做一次wrapper,所以,你可以很明确的调用ensure_future,也可以省略都可以哦。

进阶学习点三:程序不退出的调用

  • run_until_complete 和 run_forever

  • 我们之前一直都是在用run_until_complete

  • run_until_complete 是一个阻塞(blocking)调用,直到协程运行结束,它才返回。

  • 而run_forever可以让我们的协程函数调用完后程序不退出,这有什么运行场景呢?如搭建Web 服务。

  • 代码修改如下

      import asyncio
    
      async def do_some_work(name):
      	print('%s start working....'%name);
      	r = await asyncio.sleep(1);
      	print('%s working ended.'%name);
    
      loop = asyncio.get_event_loop();
      m1_coroutine = do_some_work('m1');
      m2_coroutine = do_some_work('m2');
    
      asyncio.ensure_future(m1_coroutine)
      asyncio.ensure_future(m2_coroutine)
      loop.run_forever();
      print('call close')
      loop.close()
    
  • 结果输出如下:

      m1 start working....
      m2 start working....
      m1 working ended.
      m2 working ended.
    
  • 我们发现,call close没有输出。

  • 如果使用run_forever,我们需要调用loop.stop,其才会停下来哦

  • 代码修改如下:

      import asyncio
    
      async def do_some_work(name,loop,sleep_time):
      	print('%s start working....'%name);
      	r = await asyncio.sleep(sleep_time);
      	print('%s working ended.'%name);
      	print('%s stop() be called.'%name);
      	loop.stop();
    
      loop = asyncio.get_event_loop();
      m1_coroutine = do_some_work('m1',loop,1);
      m2_coroutine = do_some_work('m2',loop,3);
    
      asyncio.ensure_future(m1_coroutine)
      asyncio.ensure_future(m2_coroutine)
      loop.run_forever();
      print('call close')
      loop.close()
    
  • 结果输出如下:

      m1 start working....
      m2 start working....
      m1 working ended.
      m1 stop() be called.
      call close
      Task was destroyed but it is pending!
      task: <Task pending coro=<do_some_work() done, defined at /Users/xxxx/PycharmProjects/LearnDemo/com/xxxx/hello.py:23> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x1033312e8>()]>>
    
  • 可以看到,在m1调用了stop之后,loop.close就会被调用了,这验证了我们可以通过stop来控制loop的停止

  • 但是我们发现,我们的m2还没执行完,这可怎么办呢?看我们的进阶学习点四、五;

进阶学习点四、五:gather 和 add_done_callback的使用

  • gather:可以让多个协程函数同时运行

  • add_done_callback:在协程完成了触发回调函数

  • 代码修改如下:

      import asyncio
      import functools
    
      async def do_some_work(name,sleep_time):
      	print('%s start working....'%name);
      	await asyncio.sleep(sleep_time);
      	print('%s working ended.'%name);
    
      def done_call_back(result):
      	print(result)
      	print('is Done.')
      	loop.stop();
    
      loop = asyncio.get_event_loop();
      m1_coroutine = do_some_work('m1',1);
      m2_coroutine = do_some_work('m2',3);
    
      f = asyncio.gather(m1_coroutine,m2_coroutine)
      f.add_done_callback(done_call_back)
      loop.run_forever();
      print('call close')
      loop.close()
    
  • 结果输出如下:

      m2 start working....
      m1 start working....
      m1 working ended.
      m2 working ended.
      <_GatheringFuture finished result=[None, None]>
      is Done.
      call close
    
  • OK,这样便解决了我们的需求。

  • 但是,解决的不够优雅,我们是直接在done_call_back函数里调用里外面的loop

  • 有小伙伴说,你传loop进去不就可以了么,似乎是的。

  • 但是我们不能这样写f.add_done_callback(done_call_back(loop)),如果这样写,在调用add_done_callback时我们的done_call_back就被调用了。

  • 那么我们该如何呢?此时我们可以使用functools.partial,其可进行方法与参数的绑定,但是不会立即调用

  • 代码修改如下

      import asyncio
      import functools
    
      async def do_some_work(name,sleep_time):
      	print('%s start working....'%name);
      	await asyncio.sleep(sleep_time);
      	print('%s working ended.'%name);
    
      def done_call_back(loop,result):
      	print(result)
      	print('is Done.')
      	loop.stop();
    
      loop = asyncio.get_event_loop();
      m1_coroutine = do_some_work('m1',1);
      m2_coroutine = do_some_work('m2',3);
      f = asyncio.gather(m1_coroutine,m2_coroutine)
      f.add_done_callback(functools.partial(done_call_back,loop))
      loop.run_forever();
      print('call close')
      loop.close()
    
  • 其结果是一样的:

      m2 start working....
      m1 start working....
      m1 working ended.
      m2 working ended.
      <_GatheringFuture finished result=[None, None]>
      is Done.
      call close
    
  • ok,学到这里,相信有关协程方面的问题,基本都能解决了。