Close #20105: set __traceback__ when chaining exceptions in C

Doc/c-api/exceptions.rst

@@ -90,6 +90,16 @@ in various ways.  There is a separate error indicator for each thread.
    the class in that case.  If the values are already normalized, nothing happens.
    The delayed normalization is implemented to improve performance.
 
+   .. note::
+
+      This function *does not* implicitly set the ``__traceback__``
+      attribute on the exception value. If setting the traceback
+      appropriately is desired, the following additional snippet is needed::
+
+         if (tb != NULL) {
+           PyException_SetTraceback(val, tb);
+         }
+
 
 .. c:function:: void PyErr_Clear()
 

Doc/whatsnew/3.4.rst

@@ -264,6 +264,9 @@ name of the codec responsible for producing the error::
     >>> import codecs
 
     >>> codecs.decode(b"abcdefgh", "hex")
+    Traceback (most recent call last):
+      File "/usr/lib/python3.4/encodings/hex_codec.py", line 20, in hex_decode
+        return (binascii.a2b_hex(input), len(input))
     binascii.Error: Non-hexadecimal digit found
 
     The above exception was the direct cause of the following exception:
@@ -273,6 +276,11 @@ name of the codec responsible for producing the error::
     binascii.Error: decoding with 'hex' codec failed (Error: Non-hexadecimal digit found)
 
     >>> codecs.encode("hello", "bz2")
+    Traceback (most recent call last):
+      File "/usr/lib/python3.4/encodings/bz2_codec.py", line 17, in bz2_encode
+        return (bz2.compress(input), len(input))
+      File "/usr/lib/python3.4/bz2.py", line 498, in compress
+        return comp.compress(data) + comp.flush()
     TypeError: 'str' does not support the buffer interface
 
     The above exception was the direct cause of the following exception:

Lib/test/test_codecs.py

@@ -2522,6 +2522,7 @@ class ExceptionChainingTest(unittest.TestCase):
         with self.assertRaisesRegex(exc_type, full_msg) as caught:
             yield caught
         self.assertIsInstance(caught.exception.__cause__, exc_type)
+        self.assertIsNotNone(caught.exception.__cause__.__traceback__)
 
     def raise_obj(self, *args, **kwds):
         # Helper to dynamically change the object raised by a test codec

Misc/NEWS

@@ -36,6 +36,9 @@ Core and Builtins
 Library
 -------
 
+- Issue #20105: the codec exception chaining now correctly sets the
+  traceback of the original exception as its __traceback__ attribute.
+
 - asyncio: Various improvements and small changes not all covered by
   issues listed below.  E.g. wait_for() now cancels the inner task if
   the timeout occcurs; tweaked the set of exported symbols; renamed

Objects/exceptions.c

@@ -2689,8 +2689,11 @@ _PyErr_TrySetFromCause(const char *format, ...)
      * types as well, but that's quite a bit trickier due to the extra
      * state potentially stored on OSError instances.
      */
-
-    Py_XDECREF(tb);
+    /* Ensure the traceback is set correctly on the existing exception */
+    if (tb != NULL) {
+        PyException_SetTraceback(val, tb);
+        Py_DECREF(tb);
+    }
 
 #ifdef HAVE_STDARG_PROTOTYPES
     va_start(vargs, format);