Special thanks to INADA Naoki for pushing the patch through
the last mile, Serhiy Storchaka for reviewing the code, and to
Victor Stinner for suggesting the idea (originally implemented
in the PyPy project).
Issue #27213: Rework CALL_FUNCTION* opcodes to produce shorter and more
efficient bytecode:
* CALL_FUNCTION now only accepts position arguments
* CALL_FUNCTION_KW accepts position arguments and keyword arguments, but keys
of keyword arguments are packed into a constant tuple.
* CALL_FUNCTION_EX is the most generic, it expects a tuple and a dict for
positional and keyword arguments.
CALL_FUNCTION_VAR and CALL_FUNCTION_VAR_KW opcodes have been removed.
2 tests of test_traceback are currently broken: skip test, the issue #28050 was
created to track the issue.
Patch by Demur Rumed, design by Serhiy Storchaka, reviewed by Serhiy Storchaka
and Victor Stinner.
Summary of changes:
1. Coroutines now have a distinct, separate from generators
type at the C level: PyGen_Type, and a new typedef PyCoroObject.
PyCoroObject shares the initial segment of struct layout with
PyGenObject, making it possible to reuse existing generators
machinery. The new type is exposed as 'types.CoroutineType'.
As a consequence of having a new type, CO_GENERATOR flag is
no longer applied to coroutines.
2. Having a separate type for coroutines made it possible to add
an __await__ method to the type. Although it is not used by the
interpreter (see details on that below), it makes coroutines
naturally (without using __instancecheck__) conform to
collections.abc.Coroutine and collections.abc.Awaitable ABCs.
[The __instancecheck__ is still used for generator-based
coroutines, as we don't want to add __await__ for generators.]
3. Add new opcode: GET_YIELD_FROM_ITER. The opcode is needed to
allow passing native coroutines to the YIELD_FROM opcode.
Before this change, 'yield from o' expression was compiled to:
(o)
GET_ITER
LOAD_CONST
YIELD_FROM
Now, we use GET_YIELD_FROM_ITER instead of GET_ITER.
The reason for adding a new opcode is that GET_ITER is used
in some contexts (such as 'for .. in' loops) where passing
a coroutine object is invalid.
4. Add two new introspection functions to the inspec module:
getcoroutinestate(c) and getcoroutinelocals(c).
5. inspect.iscoroutine(o) is updated to test if 'o' is a native
coroutine object. Before this commit it used abc.Coroutine,
and it was requested to update inspect.isgenerator(o) to use
abc.Generator; it was decided, however, that inspect functions
should really be tailored for checking for native types.
6. sys.set_coroutine_wrapper(w) API is updated to work with only
native coroutines. Since types.coroutine decorator supports
any type of callables now, it would be confusing that it does
not work for all types of coroutines.
7. Exceptions logic in generators C implementation was updated
to raise clearer messages for coroutines:
Before: TypeError("generator raised StopIteration")
After: TypeError("coroutine raised StopIteration")
namespace if it occurs as a free variable in a nested block. This limitation
of the compiler has been lifted, and a new opcode introduced (DELETE_DEREF).
This sample was valid in 2.6, but fails to compile in 3.x without this change::
>>> def f():
... def print_error():
... print(e)
... try:
... something
... except Exception as e:
... print_error()
... # implicit "del e" here
This sample has always been invalid in Python, and now works::
>>> def outer(x):
... def inner():
... return x
... inner()
... del x
There is no need to bump the PYC magic number: the new opcode is used
for code that did not compile before.
svn+ssh://pythondev@svn.python.org/python/trunk
........
r72912 | benjamin.peterson | 2009-05-25 08:13:44 -0500 (Mon, 25 May 2009) | 5 lines
add a SETUP_WITH opcode
It speeds up the with statement and correctly looks up the special
methods involved.
........
r72920 | benjamin.peterson | 2009-05-25 15:12:57 -0500 (Mon, 25 May 2009) | 1 line
take into account the fact that SETUP_WITH pushes a finally block
........
r72940 | benjamin.peterson | 2009-05-26 07:49:59 -0500 (Tue, 26 May 2009) | 1 line
teach the peepholer about SETUP_WITH
........
POP_JUMP_IF_{TRUE,FALSE} and JUMP_IF_{TRUE,FALSE}_OR_POP. This avoids executing
a POP_TOP on each conditional and sometimes allows the peephole optimizer to
skip a JUMP_ABSOLUTE entirely. It speeds up list comprehensions significantly.
This patch by Antoine Pitrou optimizes the bytecode for conditional branches by
merging the following "POP_TOP" instruction into the conditional jump. For
example, the list comprehension "[x for x in l if not x]" produced the
following bytecode:
1 0 BUILD_LIST 0
3 LOAD_FAST 0 (.0)
>> 6 FOR_ITER 23 (to 32)
9 STORE_FAST 1 (x)
12 LOAD_FAST 1 (x)
15 JUMP_IF_TRUE 10 (to 28)
18 POP_TOP
19 LOAD_FAST 1 (x)
22 LIST_APPEND 2
25 JUMP_ABSOLUTE 6
>> 28 POP_TOP
29 JUMP_ABSOLUTE 6
>> 32 RETURN_VALUE
but after the patch it produces the following bytecode:
1 0 BUILD_LIST 0
3 LOAD_FAST 0 (.0)
>> 6 FOR_ITER 18 (to 27)
9 STORE_FAST 1 (x)
12 LOAD_FAST 1 (x)
15 POP_JUMP_IF_TRUE 6
18 LOAD_FAST 1 (x)
21 LIST_APPEND 2
24 JUMP_ABSOLUTE 6
>> 27 RETURN_VALUE
Notice that not only the code is shorter, but the conditional jump
(POP_JUMP_IF_TRUE) jumps right to the start of the loop instead of going through
the JUMP_ABSOLUTE at the end. "continue" statements are helped
similarly.
Furthermore, the old jump opcodes (JUMP_IF_FALSE, JUMP_IF_TRUE) have been
replaced by two new opcodes:
- JUMP_IF_TRUE_OR_POP, which jumps if true and pops otherwise
- JUMP_IF_FALSE_OR_POP, which jumps if false and pops otherwise
svn+ssh://pythondev@svn.python.org/python/trunk
........
r59544 | raymond.hettinger | 2007-12-18 01:13:45 +0100 (Tue, 18 Dec 2007) | 1 line
Add more namedtuple() test cases. Neaten the code and comments.
........
r59545 | christian.heimes | 2007-12-18 04:38:03 +0100 (Tue, 18 Dec 2007) | 3 lines
Fixed for #1601: IDLE not working correctly on Windows (Py30a2/IDLE30a1)
Amaury's ideas works great. Should we build the Python core with WINVER=0x0500 and _WIN32_WINNT=0x0500, too?
........
r59546 | christian.heimes | 2007-12-18 10:00:13 +0100 (Tue, 18 Dec 2007) | 1 line
Make it a bit easier to test Tcl/Tk and idle from a build dir.
........
r59547 | christian.heimes | 2007-12-18 10:12:10 +0100 (Tue, 18 Dec 2007) | 1 line
Removed several unused files from the PCbuild9 directory. They are relics from the past.
........
r59548 | raymond.hettinger | 2007-12-18 19:26:18 +0100 (Tue, 18 Dec 2007) | 29 lines
Speed-up dictionary constructor by about 10%.
New opcode, STORE_MAP saves the compiler from awkward stack manipulations
and specializes for dicts using PyDict_SetItem instead of PyObject_SetItem.
Old disassembly:
0 BUILD_MAP 0
3 DUP_TOP
4 LOAD_CONST 1 (1)
7 ROT_TWO
8 LOAD_CONST 2 ('x')
11 STORE_SUBSCR
12 DUP_TOP
13 LOAD_CONST 3 (2)
16 ROT_TWO
17 LOAD_CONST 4 ('y')
20 STORE_SUBSCR
New disassembly:
0 BUILD_MAP 0
3 LOAD_CONST 1 (1)
6 LOAD_CONST 2 ('x')
9 STORE_MAP
10 LOAD_CONST 3 (2)
13 LOAD_CONST 4 ('y')
16 STORE_MAP
........
r59549 | thomas.heller | 2007-12-18 20:00:34 +0100 (Tue, 18 Dec 2007) | 2 lines
Issue #1642: Fix segfault in ctypes when trying to delete attributes.
........
r59551 | guido.van.rossum | 2007-12-18 21:10:42 +0100 (Tue, 18 Dec 2007) | 2 lines
Issue #1645 by Alberto Bertogli. Fix a comment.
........
r59553 | raymond.hettinger | 2007-12-18 22:24:09 +0100 (Tue, 18 Dec 2007) | 12 lines
Give meaning to the oparg for BUILD_MAP: estimated size of the dictionary.
Allows dictionaries to be pre-sized (upto 255 elements) saving time lost
to re-sizes with their attendant mallocs and re-insertions.
Has zero effect on small dictionaries (5 elements or fewer), a slight
benefit for dicts upto 22 elements (because they had to resize once
anyway), and more benefit for dicts upto 255 elements (saving multiple
resizes during the build-up and reducing the number of collisions on
the first insertions). Beyond 255 elements, there is no addional benefit.
........
r59554 | christian.heimes | 2007-12-18 22:56:09 +0100 (Tue, 18 Dec 2007) | 1 line
Fixed#1649: IDLE error: dictionary changed size during iteration
........
r59557 | raymond.hettinger | 2007-12-18 23:21:27 +0100 (Tue, 18 Dec 2007) | 1 line
Simplify and speedup _asdict() for named tuples.
........
r59558 | christian.heimes | 2007-12-19 00:22:54 +0100 (Wed, 19 Dec 2007) | 3 lines
Applied patch #1635: Float patch for inf and nan on Windows (and other platforms).
The patch unifies float("inf") and repr(float("inf")) on all platforms.
........
r59559 | raymond.hettinger | 2007-12-19 00:51:15 +0100 (Wed, 19 Dec 2007) | 1 line
Users demand iterable input for named tuples. The author capitulates.
........
r59560 | raymond.hettinger | 2007-12-19 01:21:06 +0100 (Wed, 19 Dec 2007) | 1 line
Beef-up tests for dict literals
........
r59561 | raymond.hettinger | 2007-12-19 01:27:21 +0100 (Wed, 19 Dec 2007) | 1 line
Zap a duplicate line
........
Allows dictionaries to be pre-sized (upto 255 elements) saving time lost
to re-sizes with their attendant mallocs and re-insertions.
Has zero effect on small dictionaries (5 elements or fewer), a slight
benefit for dicts upto 22 elements (because they had to resize once
anyway), and more benefit for dicts upto 255 elements (saving multiple
resizes during the build-up and reducing the number of collisions on
the first insertions). Beyond 255 elements, there is no addional benefit.
The compiler package hasn't been updated yet; test_compiler.py fails.
Otherwise all tests seem to be passing now. There are no occurrences
of __metaclass__ left in the standard library.
Docs have not been updated.
* Fix some docstrings and one Print -> print.
* Fix test_{class,code,descrtut,dis,extcall,parser,popen,pkg,subprocess,syntax,traceback}.
These were the ones that generated code with a print statement.
In most remaining failing tests there's an issue with the soft space.
This was started by Mike Bland and completed by Guido
(with help from Neal).
This still needs a __future__ statement added;
Thomas is working on Michael's patch for that aspect.
There's a small amount of code cleanup and refactoring
in ast.c, compile.c and ceval.c (I fixed the lltrace
behavior when EXT_POP is used -- however I had to make
lltrace a static global).
Yury Selivanov
Serhiy Storchaka
Victor Stinner
Eric V. Smith
Yury Selivanov
Benjamin Peterson
Larry Hastings
Alexander Belopolsky
Nick Coghlan
Amaury Forgeot d'Arc
Antoine Pitrou
Alexandre Vassalotti
Jeffrey Yasskin
Georg Brandl
Thomas Wouters
Christian Heimes
Raymond Hettinger
Guido van Rossum
Brett Cannon
Neal Norwitz
Walter Dörwald