A bunch of code currently uses PyInterpreterState.modules directly instead of PyImport_GetModuleDict(). This complicates efforts to make changes relative to sys.modules. This patch switches to using PyImport_GetModuleDict() uniformly. Also, a number of related uses of sys.modules are updated for uniformity for the same reason.
Note that this code was already reviewed and merged as part of #1638. I reverted that and am now splitting it up into more focused parts.
PR #1638, for bpo-28411, causes problems in some (very) edge cases. Until that gets sorted out, we're reverting the merge. PR #3506, a fix on top of #1638, is also getting reverted.
Known limitations of the current implementation:
- documentation changes are incomplete
- there's a reference leak I haven't tracked down yet
The leak is most visible by running:
./python -m test -R3:3 test_importlib
However, you can also see it by running:
./python -X showrefcount
Importing the array or _testmultiphase modules, and
then deleting them from both sys.modules and the local
namespace shows significant increases in the total
number of active references each cycle. By contrast,
with _testcapi (which continues to use single-phase
initialisation) the global refcounts stabilise after
a couple of cycles.
importlib._bootstrap is now frozen into Python/importlib.h and stored
as _frozen_importlib in sys.modules. Py_Initialize() loads the frozen
code along with sys and imp and then uses _frozen_importlib._install()
to set builtins.__import__() w/ _frozen_importlib.__import__().
The name must be encodable to ASCII because dynamic module must have a function
called "PyInit_NAME", they are written in C, and the C language doesn't accept
non-ASCII identifiers.
- is_builtin(), init_builtin(), load_builtin() and other builtin related
functions use Unicode strings, instead of byte strings
- Rename _PyImport_FixupExtensionUnicode() to _PyImport_FixupExtensionObject()
- Rename _PyImport_FindExtensionUnicode() to _PyImport_FindExtensionObject()
* Rename _PyImport_FindExtension() to _PyImport_FindExtensionUnicode():
the filename becomes a Unicode object instead of byte string
* Rename _PyImport_FixupExtension() to _PyImport_FixupExtensionUnicode():
the filename becomes a Unicode object instead of byte string
Changes to make __file__ a proper Unicode object, using the default
filesystem encoding.
This is a bit tricky because the default filesystem encoding isn't
set by the time we import the first modules; at that point we fudge
things a bit. This is okay since __file__ isn't really used much
except for error reporting.
Tested on OSX and Linux only so far.
When an extension imports another extension in its
initXXX() function, the variable _Py_PackageContext is
prematurely reset to NULL. If the outer extension then
calls Py_InitModule(), the extension is installed in
sys.modules without its package name. The
manifestation of this bug is a "SystemError:
_PyImport_FixupExtension: module <package>.<extension>
not loaded".
To fix this, importdl.c just needs to retain the old
value of _Py_PackageContext and restore it after the
initXXX() method is called. The attached patch does this.
This patch applies to Python 2.1.1 and the current CVS.
* in import.c, #ifdef out references to dynamic loading based on
HAVE_DYNAMIC_LOADING
* clean out the platform-specific crud from importdl.c.
[ maybe fold this function into import.c and drop the importdl.c file? Greg.]
* change GetDynLoadFunc's "funcname" parameter to "shortname". change
"name" to "fqname" for clarification.
* each GetDynLoadFunc now creates its own funcname value.
WARNING: as I mentioned previously, we may run into an issue with a
missing "_" on some platforms. Testing will show this pretty quickly,
however.
* move pathname munging into dynload_shlib.c
Duzan, for AIX, to support C++ objects with static initializers, when
using the genuine IBM C++ compiler (namely xlC/xlC_r).
See accompanying patches to configure.in and acconfig.h.
"""
Following up Robin Dunn's troubles with freeze, here's a patch that
fixes an oddity regarding the import logic of shared modules on AIX.
Symbol resolution of shared modules is now handled properly for the cases
when the python library is linked to a binary with an arbitrary name.
This includes the standard python[version] executable, but also applications
that are embedding the python core (i.e. linked with libpython[version].a,
the latter being static or shared).
"""
The following patches (relative to 1.5.2b1) enable Python dynamic
loading to work on NetBSD platforms that use ELF (presnetly mips and
alpha systems). They automaticly determine wether the system is ELF or
a.out rather than using astatic list of platforms so that when other
NetBSD platforms move to ELF, python will continue to work without
change.
This is a patch that Bill Bummgarner did for 1.4 that hasn't made its
way into the distribution yet. This is important if you want to use
the ObjC module.
Steve Dower
Oren Milman
Eric Snow
Serhiy Storchaka
Benjamin Peterson
Nick Coghlan
Victor Stinner
Christian Heimes
Brett Cannon
Victor Stinner
Antoine Pitrou
Martin v. Löwis
Neal Norwitz
Guido van Rossum
Fred Drake
Thomas Wouters