Issue #26588: Optimize memory footprint of _tracemalloc before non-zero domain
is used. Start with compact key (Py_uintptr_t) and also switch to pointer_t key
when the first memory block with a non-zero domain is tracked.
Issue #26588:
* Pass the hash table rather than the key size to hash and compare functions
* _Py_HASHTABLE_READ_KEY() and _Py_HASHTABLE_ENTRY_READ_KEY() macros now expect
the hash table as the first parameter, rather than the key size
* tracemalloc_get_traces_fill(): use _Py_HASHTABLE_ENTRY_READ_DATA() rather
than pointer dereference
* Remove the _Py_HASHTABLE_ENTRY_WRITE_PKEY() macro
* Move "PKEY" and "PDATA" macros inside hashtable.c
* Optimize tracemalloc_add_trace(): modify hashtable entry data (trace) if the
memory block is already tracked, rather than trying to remove the old trace
and then add a new trace.
* Add _Py_HASHTABLE_ENTRY_WRITE_DATA() macro
Issue #26588: Fix _tracemalloc start/stop: don't play with the reentrant flag.
set_reentrant(1) fails with an assertion error if tracemalloc_init() is called
first in a thread A and tracemalloc_start() is called second in a thread B. The
tracemalloc is imported in a thread A. Importing the module calls
tracemalloc_init(). tracemalloc.start() is called in a thread B.
Issue #26530:
* Add C functions _PyTraceMalloc_Track() and _PyTraceMalloc_Untrack() to track
memory blocks using the tracemalloc module.
* Add _PyTraceMalloc_GetTraceback() to get the traceback of an object.
Issue #26588:
* The _tracemalloc now supports tracing memory allocations of multiple address
spaces (domains).
* Add domain parameter to tracemalloc_add_trace() and
tracemalloc_remove_trace().
* tracemalloc_add_trace() now starts by removing the previous trace, if any.
* _tracemalloc._get_traces() now returns a list of (domain, size,
traceback_frames): the domain is new.
* Add tracemalloc.DomainFilter
* tracemalloc.Filter: add an optional domain parameter to the constructor and a
domain attribute
* Sublte change: use Py_uintptr_t rather than void* in the traces key.
* Add tracemalloc_config.use_domain, currently hardcoded to 1
Issue #26588: Remove copy_data, free_data and get_data_size callbacks from
hashtable.h. These callbacks are not used in Python and makes the code more
complex.
Remove also the _Py_HASHTABLE_ENTRY_DATA_AS_VOID_P() macro which uses an unsafe
pointer dereference (can cause memory alignment issue). Replace the macro usage
with _Py_HASHTABLE_ENTRY_READ_DATA() which is implemented with the safe
memcpy() function.
Issue #26588: hashtable.h now supports keys of any size, not only
sizeof(void*). It allows to support key larger than sizeof(void*), but also to
use less memory for key smaller than sizeof(void*).
Change pushed by mistake, the patch is still under review :-/
"""
_tracemalloc: add domain to trace keys
* hashtable.h: key has now a variable size
* _tracemalloc uses (pointer: void*, domain: unsigned int) as key for traces
"""
Issue #26564: _PyObject_DebugDumpAddress() now dumps the traceback where a
memory block was allocated on memory block. Use the tracemalloc module to get
the traceback.
PyObject_Calloc(), _PyObject_GC_Calloc(). bytes(int) and bytearray(int) are now
using ``calloc()`` instead of ``malloc()`` for large objects which is faster
and use less memory (until the bytearray buffer is filled with data).
platforms. Bug seen on 64-bit Linux when using "make profile-opt".
Only align the "frame_t" structure on 32-bit when Visual Studio is used. Before
the alignment to 32-bit was applied to the whole file any compiler supporting
"#pragma pack(4)" which includes GCC.
In Python 3.3, PyThread_set_key_value() did nothing if the key already exists
(if the current value is a non-NULL pointer).
When _PyGILState_NoteThreadState() is called twice on the same thread with a
different Python thread state, it still keeps the old Python thread state to
keep the old behaviour. Replacing the Python thread state with the new state
introduces new bugs: see issues #10915 and #15751.
the function did nothing if the key already exists (if the current value is a
non-NULL pointer).
_testcapi.run_in_subinterp() now correctly sets the new Python thread state of
the current thread when a subinterpreter is created.
between tracemalloc_remove_trace() and tracemalloc_add_trace() to reduce the
risk of race condition.
tracemalloc_add_trace() cannot fail anymore in tracemalloc_realloc() when
tracemalloc_realloc() resizes a memory block.
PyGILState_Ensure()/PyGILState_Release() calls to the raw wrappers to simplify
the code.
Rename also tracemalloc_log_alloc/log_free() to
tracemalloc_add_trace/remove_trace().
The limit is now 178,956,969 on 64 bit (it is greater on 32 bit because
structures are smaller).
Use int instead of Py_ssize_t to store the number of frames to have smaller
traceback_t objects.
Martin Panter
Benjamin Peterson
Raymond Hettinger
doko@ubuntu.com
Serhiy Storchaka
Victor Stinner