Shulou(Shulou.com)06/02 Report--

This article introduces the relevant knowledge of "how to understand yield from grammar in Python". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

. Why use a cooperative program?

In the previous article, we successfully transitioned from the basic understanding and use of generators to collaborative programs.

But there must be a lot of people who only know what Xiecheng is, but don't know why they use it. In other words, you don't know under what circumstances to use a cooperative program? What are the advantages of it compared to multithreading?

Before I start talking about yield from, I'd like to solve this problem that has puzzled many people.

For instance. Let's say we're a reptile. We have to crawl multiple web pages, here is a simple example of two web pages (two spider functions), get HTML (time-consuming IO), and then HTML row parsing to get the data we are interested in.

Our code structure is simplified as follows:

Def spider_01 (url):

Html = get_html (url)

...

Data = parse_html (html)

Def spider_02 (url):

Html = get_html (url)

...

Data = parse_html (html)

As we all know, get_html () waiting to return to the page is very IO-consuming, a page is fine, if we crawl the page data is extremely large, the waiting time is very amazing, is a great waste.

Smart programmers, of course, would think that if they could pause here in get_html (), instead of foolishly waiting for the page to return, they would do something else. After a while, go back to the place where you just paused, receive the returned html content, and then you can parse parse_html (html).

Using conventional methods, it is almost impossible to achieve the effect we want above. So Python is very thoughtful and gives us such a function from the language itself, which is the yield syntax. You can achieve the effect of pausing in a function.

Try to think about it. If there is no cooperative program, we will write a concurrent program. There may be the following problems

1) using the most conventional synchronous programming to achieve asynchronous concurrency is not ideal, or extremely difficult. 2) due to the existence of GIL locks, multithreading needs to lock and unlock frequently and switch threads, which greatly reduces the concurrency performance.

The emergence of the cooperative process can just solve the above problems. It is characterized by

00001. The co-program switches tasks in a single thread.

00002. Use synchronous way to realize async

00003. Locks are no longer needed, improving concurrency performance

. Detailed explanation of the usage of yield from

Yield from is a syntax that appears only in Python3.3. So this feature is not available in Python2.

Yield from needs to be followed by an iterable object, which can be a normal iterable object, an iterator, or even a generator.

Simple application: stitching iterable objects

We can compare an example of using yield with an example of using yield from.

Use yield

# string astr='ABC'# list alist= [1Magazine 3] # Dictionary adict= {"name": "wangbm", "age": 18} # Generator agen= (i for i in range (4jue 8))

Def gen (* args, * * kw):

For item in args:

For i in item:

Yield i

New_list=gen (astr, alist, adict, agen) print (list (new_list)) # ['Agar,' Bread, 'Che, 1,2,3,' name', 'age', 4, 5, 6, 7]

Use yield from

# string astr='ABC'# list alist= [1Magazine 3] # Dictionary adict= {"name": "wangbm", "age": 18} # Generator agen= (i for i in range (4jue 8))

Def gen (* args, * * kw):

For item in args:

Yield from item

New_list=gen (astr, alist, adict, agen) print (list (new_list)) # ['Agar,' Bread, 'Che, 1,2,3,' name', 'age', 4, 5, 6, 7]

From the comparison of the above two ways, we can see that by adding an iterable object after yield from, he can yield each element of the iterable object one by one, and the code is more concise and the structure is clearer than yield.

Complex applications: nesting of generators

If you think that yield from has only the above features, then you underestimate it, and its more powerful features are yet to come.

When a generator is added to the yield from, the nesting of the generation is implemented.

Of course, to achieve the nesting of generators, it is not necessary to use yield from, but the use of yield from allows us to avoid having to handle unexpected exceptions ourselves, and allows us to focus on the implementation of business code. If you use yield to implement it, it will only increase the difficulty of writing the code, reduce the development efficiency, and reduce the readability of the code. Now that Python is so thoughtful, of course we should make good use of it.

Before you explain it, you need to know these concepts first.

1. Caller: client (caller) code that calls delegate generator 2, delegate generator: generator function containing yield from expression 3, child generator: generator function added after yield from

You may not know what they all mean, it doesn't matter, let's take a look at this example.

This example is to achieve real-time calculation of the average. For example, if 10 is passed in for the first time, the return average is 10. 5%. If the second pass in 20, the return average is (10: 20) / 2: 15. The third pass in 30, and the return average is (10: 20) / 3: 20.

# Sub-generator def average_gen ():

Total = 0

Count = 0

Average = 0

While True:

New_num = yield average

Count + = 1

Total + = new_num

Average = total/count

# delegation generator def proxy_gen ():

While True:

Yield from average_gen ()

# caller def main ():

Calc_average = proxy_gen ()

Next (calc_average) # pre-excitation generator

Print (calc_average.send (10)) # print: 10.0

Print (calc_average.send (20)) # print: 15.0

Print (calc_average.send (30)) # print: 20.0

If _ _ name__ = ='_ _ main__':

Main ()

If you read the above code carefully, you should be able to understand the relationship between the caller, the delegate generator, and the child generator. I won't say any more.

The role of the delegate generator is to establish a two-way channel between the caller and the child generator.

What does the so-called two-way channel mean? The caller can send a message directly to the sub-generator through send (), and the value of the sub-generator yield is also returned directly to the caller.

You may often see some code that can be assigned in front of the yield from. What kind of usage is this?

You might think that the value returned by the child generator yield was intercepted by the delegate generator. You can write a demo to run the experiment yourself, it's not what you think. Because as we said before, the delegated generator only acts as a bridge, it establishes a two-way channel, and it has no right and no way to intercept the content returned by the sub-generator yield.

In order to explain this usage, I still use the above example and make some modifications to it. Added some notes, I hope you can understand.

According to convention, let's give an example.

# Sub-generator def average_gen ():

Total = 0

Count = 0

Average = 0

While True:

New_num = yield average

If new_num is None:

Break

Count + = 1

Total + = new_num

Average = total/count

# each return means the end of the current collaboration process.

Return total,count,average

# delegation generator def proxy_gen ():

While True:

# only when the child generator is about to return, the variable to the left of the yield from will be assigned, and the following code will be executed.

Total, count, average = yield from average_gen ()

Print ("calculation completed!\ nTotal {} values: {}, average: {}" .format (count, total, average))

# caller def main ():

Calc_average = proxy_gen ()

Next (calc_average) # preexcitation cooperation program

Print (calc_average.send (10)) # print: 10.0

Print (calc_average.send (20)) # print: 15.0

Print (calc_average.send (30)) # print: 20.0

Calc_average.send (None) # ends the cooperation process

# if calc_average.send (10) is called again here, a new one will be reopened because the previous cooperation program has already ended

If _ _ name__ = ='_ _ main__':

Main ()

After running, output

10.015.020.0 calculation complete! A total of 3 values are passed in, summing up: 60, and average: 20.0

. Why use yield from

At this point, I'm sure you have to ask, since the delegated generator only acts as a two-way channel, what else do I need to delegate the generator to do? How about I just call the sub-generator directly from the caller?

High energy early warning

Let's talk about what's so special about yield from that we have to use it.

Because it can help us handle exceptions.

If we remove the delegate generator and directly invoke the subgenerator. Then we need to change the code to look like this, we need to catch exceptions and handle them ourselves. It's not as easy as it makes yield from.

# Sub-generator # sub-generator def average_gen ():

Total = 0

Count = 0

Average = 0

While True:

New_num = yield average

If new_num is None:

Break

Count + = 1

Total + = new_num

Average = total/count

Return total,count,average

# caller def main ():

Calc_average = average_gen ()

Next (calc_average) # preexcitation cooperation program

Print (calc_average.send (10)) # print: 10.0

Print (calc_average.send (20)) # print: 15.0

Print (calc_average.send (30)) # print: 20.0

#-Note-

Try:

Calc_average.send (None)

Except StopIteration as e:

Total, count, average = e.value

Print ("calculation completed!\ nTotal {} values: {}, average: {}" .format (count, total, average))

#-Note-

If _ _ name__ = ='_ _ main__':

Main ()

At this time, you may say, isn't it just an exception of StopIteration? It's no big deal to catch yourself.

If you knew what yield from did for us in obscurity behind our backs, you wouldn't say that.

For details of what yield from has done for us, you can refer to the following code.

# some instructions "" _ I: subgenerator, but also an iterator _ y: the value produced by the subgenerator _ r:yield from expression _ s: the value sent by the caller through send () _ e: exception object ""

_ I = iter (EXPR)

Try:

_ y = next (_ I) except StopIteration as _ e:

_ r = _ e.value

Else:

While 1:

Try:

_ s = yield _ y

Except GeneratorExit as _ e:

Try:

_ m = _ i.close

Except AttributeError:

Pass

Else:

_ m ()

Raise _ e

Except BaseException as _ e:

_ x = sys.exc_info ()

Try:

_ m = _ i.throw

Except AttributeError:

Raise _ e

Else:

Try:

_ y = _ m (* _ x)

Except StopIteration as _ e:

_ r = _ e.value

Break

Else:

Try:

If _ s is None:

_ y = next (_ I)

Else:

_ y = _ i.send (_ s)

Except StopIteration as _ e:

_ r = _ e.value

BreakRESULT = _ r

The above code is a little more complicated. Interested students can combine the following instructions to study it.

00001. The value generated by the iterator (that is, the sub-generator) is returned directly to the caller

00002. Any value sent to the delegate producer (that is, the external producer) using the send () method is passed directly to the iterator. If the end value is None, the iterator next () method is called; if not None, the iterator's send () method is called. If a call to the iterator produces a StopIteration exception, the delegate producer resumes execution of statements that follow the yield from; if the iterator generates any other exceptions, it is passed to the delegate producer.

00003. A sub-generator may be just an iterator, not a generator as a co-program, so it does not support the .close () and .close () methods, that is, an AttributeError exception may be generated.

00004. Exceptions thrown to the delegate producer other than GeneratorExit exceptions will be passed to the iterator's throw () method. If the iterator throw () call produces a StopIteration exception, the delegate producer resumes and continues execution, and the other exceptions are passed to the delegate producer.

00005. If the GeneratorExit exception is thrown to the delegate producer, or if the delegate producer's close () method is called, it will also be called if the iterator has close (). If the close () call produces an exception, the exception is passed to the delegate producer. Otherwise, the delegate producer throws a GeneratorExit exception.

00006. When the iterator ends and throws an exception, the value of the yield from expression is the first parameter in its StopIteration exception.

00007. The return expr statement in a generator will exit from the generator and throw a StopIteration (expr) exception.

This is the end of the content of "how to understand yield from grammar in Python". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.