[pull] main from python:main

Doc/c-api/init.rst

@@ -2306,6 +2306,12 @@ is resumed, and its locks reacquired.  This means the critical section API
 provides weaker guarantees than traditional locks -- they are useful because
 their behavior is similar to the :term:`GIL`.
 
+Variants that accept :c:type:`PyMutex` pointers rather than Python objects are also
+available. Use these variants to start a critical section in a situation where
+there is no :c:type:`PyObject` -- for example, when working with a C type that
+does not extend or wrap :c:type:`PyObject` but still needs to call into the C
+API in a manner that might lead to deadlocks.
+
 The functions and structs used by the macros are exposed for cases
 where C macros are not available. They should only be used as in the
 given macro expansions. Note that the sizes and contents of the structures may
@@ -2351,6 +2357,23 @@ code triggered by the finalizer blocks and calls :c:func:`PyEval_SaveThread`.
 
    .. versionadded:: 3.13
 
+.. c:macro:: Py_BEGIN_CRITICAL_SECTION_MUTEX(m)
+
+   Locks the mutex *m* and begins a critical section.
+
+   In the free-threaded build, this macro expands to::
+
+     {
+          PyCriticalSection _py_cs;
+          PyCriticalSection_BeginMutex(&_py_cs, m)
+
+   Note that unlike :c:macro:`Py_BEGIN_CRITICAL_SECTION`, there is no cast for
+   the argument of the macro - it must be a :c:type:`PyMutex` pointer.
+
+   On the default build, this macro expands to ``{``.
+
+   .. versionadded:: next
+
 .. c:macro:: Py_END_CRITICAL_SECTION()
 
    Ends the critical section and releases the per-object lock.
@@ -2380,6 +2403,23 @@ code triggered by the finalizer blocks and calls :c:func:`PyEval_SaveThread`.
 
    .. versionadded:: 3.13
 
+.. c:macro:: Py_BEGIN_CRITICAL_SECTION2_MUTEX(m1, m2)
+
+   Locks the mutexes *m1* and *m2* and begins a critical section.
+
+   In the free-threaded build, this macro expands to::
+
+     {
+          PyCriticalSection2 _py_cs2;
+          PyCriticalSection2_BeginMutex(&_py_cs2, m1, m2)
+
+   Note that unlike :c:macro:`Py_BEGIN_CRITICAL_SECTION2`, there is no cast for
+   the arguments of the macro - they must be :c:type:`PyMutex` pointers.
+
+   On the default build, this macro expands to ``{``.
+
+   .. versionadded:: next
+
 .. c:macro:: Py_END_CRITICAL_SECTION2()
 
    Ends the critical section and releases the per-object locks.

Doc/library/os.path.rst

@@ -64,7 +64,7 @@ the :mod:`glob` module.)
       Accepts a :term:`path-like object`.
 
 
-.. function:: basename(path)
+.. function:: basename(path, /)
 
    Return the base name of pathname *path*.  This is the second element of the
    pair returned by passing *path* to the function :func:`split`.  Note that
@@ -118,7 +118,7 @@ the :mod:`glob` module.)
       Accepts a :term:`path-like object`.
 
 
-.. function:: dirname(path)
+.. function:: dirname(path, /)
 
    Return the directory name of pathname *path*.  This is the first element of
    the pair returned by passing *path* to the function :func:`split`.
@@ -237,7 +237,7 @@ the :mod:`glob` module.)
       Accepts a :term:`path-like object`.
 
 
-.. function:: isabs(path)
+.. function:: isabs(path, /)
 
    Return ``True`` if *path* is an absolute pathname.  On Unix, that means it
    begins with a slash, on Windows that it begins with two (back)slashes, or a
@@ -261,7 +261,7 @@ the :mod:`glob` module.)
       Accepts a :term:`path-like object`.
 
 
-.. function:: isdir(path)
+.. function:: isdir(path, /)
 
    Return ``True`` if *path* is an :func:`existing <exists>` directory.  This
    follows symbolic links, so both :func:`islink` and :func:`isdir` can be true
@@ -372,7 +372,7 @@ the :mod:`glob` module.)
       Accepts a :term:`path-like object` for *path* and *paths*.
 
 
-.. function:: normcase(path)
+.. function:: normcase(path, /)
 
    Normalize the case of a pathname.  On Windows, convert all characters in the
    pathname to lowercase, and also convert forward slashes to backward slashes.
@@ -509,7 +509,7 @@ the :mod:`glob` module.)
       Added Windows support.
 
 
-.. function:: split(path)
+.. function:: split(path, /)
 
    Split the pathname *path* into a pair, ``(head, tail)`` where *tail* is the
    last pathname component and *head* is everything leading up to that.  The
@@ -525,7 +525,7 @@ the :mod:`glob` module.)
       Accepts a :term:`path-like object`.
 
 
-.. function:: splitdrive(path)
+.. function:: splitdrive(path, /)
 
    Split the pathname *path* into a pair ``(drive, tail)`` where *drive* is either
    a mount point or the empty string.  On systems which do not use drive
@@ -550,7 +550,7 @@ the :mod:`glob` module.)
       Accepts a :term:`path-like object`.
 
 
-.. function:: splitroot(path)
+.. function:: splitroot(path, /)
 
    Split the pathname *path* into a 3-item tuple ``(drive, root, tail)`` where
    *drive* is a device name or mount point, *root* is a string of separators
@@ -583,7 +583,7 @@ the :mod:`glob` module.)
    .. versionadded:: 3.12
 
 
-.. function:: splitext(path)
+.. function:: splitext(path, /)
 
    Split the pathname *path* into a pair ``(root, ext)``  such that ``root + ext ==
    path``, and the extension, *ext*, is empty or begins with a period and contains at

Doc/library/urllib.request.rst

@@ -210,6 +210,9 @@ The :mod:`urllib.request` module defines the following functions:
       Windows a UNC path is returned (as before), and on other platforms a
       :exc:`~urllib.error.URLError` is raised.
 
+   .. versionchanged:: 3.14
+      The URL query and fragment components are discarded if present.
+
    .. versionchanged:: 3.14
       The *require_scheme* and *resolve_host* parameters were added.
 

Doc/library/zipfile.rst

@@ -558,14 +558,6 @@ The following data attributes are also available:
    it should be no longer than 65535 bytes.  Comments longer than this will be
    truncated.
 
-.. attribute:: ZipFile.data_offset
-
-   The offset to the start of ZIP data from the beginning of the file. When the
-   :class:`ZipFile` is opened in either mode ``'w'`` or ``'x'`` and the
-   underlying file does not support ``tell()``, the value will be ``None``
-   instead.
-
-   .. versionadded:: 3.14
 
 .. _path-objects:
 

Doc/whatsnew/3.14.rst

@@ -2192,6 +2192,7 @@ urllib
   - Discard URL authority if it matches the local hostname.
   - Discard URL authority if it resolves to a local IP address when the new
     *resolve_host* argument is set to true.
+  - Discard URL query and fragment components.
   - Raise :exc:`~urllib.error.URLError` if a URL authority isn't local,
     except on Windows where we return a UNC path as before.
 

Include/cpython/critical_section.h

@@ -73,22 +73,32 @@ typedef struct PyCriticalSection2 PyCriticalSection2;
 PyAPI_FUNC(void)
 PyCriticalSection_Begin(PyCriticalSection *c, PyObject *op);
 
+PyAPI_FUNC(void)
+PyCriticalSection_BeginMutex(PyCriticalSection *c, PyMutex *m);
+
 PyAPI_FUNC(void)
 PyCriticalSection_End(PyCriticalSection *c);
 
 PyAPI_FUNC(void)
 PyCriticalSection2_Begin(PyCriticalSection2 *c, PyObject *a, PyObject *b);
 
+PyAPI_FUNC(void)
+PyCriticalSection2_BeginMutex(PyCriticalSection2 *c, PyMutex *m1, PyMutex *m2);
+
 PyAPI_FUNC(void)
 PyCriticalSection2_End(PyCriticalSection2 *c);
 
 #ifndef Py_GIL_DISABLED
 # define Py_BEGIN_CRITICAL_SECTION(op)      \
     {
+# define Py_BEGIN_CRITICAL_SECTION_MUTEX(mutex)    \
+    {
 # define Py_END_CRITICAL_SECTION()          \
     }
 # define Py_BEGIN_CRITICAL_SECTION2(a, b)   \
     {
+# define Py_BEGIN_CRITICAL_SECTION2_MUTEX(m1, m2)  \
+    {
 # define Py_END_CRITICAL_SECTION2()         \
     }
 #else /* !Py_GIL_DISABLED */
@@ -118,6 +128,11 @@ struct PyCriticalSection2 {
         PyCriticalSection _py_cs;                                       \
         PyCriticalSection_Begin(&_py_cs, _PyObject_CAST(op))
 
+# define Py_BEGIN_CRITICAL_SECTION_MUTEX(mutex)                         \
+    {                                                                   \
+        PyCriticalSection _py_cs;                                       \
+        PyCriticalSection_BeginMutex(&_py_cs, mutex)
+
 # define Py_END_CRITICAL_SECTION()                                      \
         PyCriticalSection_End(&_py_cs);                                 \
     }
@@ -127,6 +142,11 @@ struct PyCriticalSection2 {
         PyCriticalSection2 _py_cs2;                                     \
         PyCriticalSection2_Begin(&_py_cs2, _PyObject_CAST(a), _PyObject_CAST(b))
 
+# define Py_BEGIN_CRITICAL_SECTION2_MUTEX(m1, m2)                       \
+    {                                                                   \
+        PyCriticalSection2 _py_cs2;                                     \
+        PyCriticalSection2_BeginMutex(&_py_cs2, m1, m2)
+
 # define Py_END_CRITICAL_SECTION2()                                     \
         PyCriticalSection2_End(&_py_cs2);                               \
     }

Include/internal/pycore_critical_section.h

@@ -21,16 +21,6 @@ extern "C" {
 #define _Py_CRITICAL_SECTION_MASK           0x3
 
 #ifdef Py_GIL_DISABLED
-# define Py_BEGIN_CRITICAL_SECTION_MUT(mutex)                           \
-    {                                                                   \
-        PyCriticalSection _py_cs;                                       \
-        _PyCriticalSection_BeginMutex(&_py_cs, mutex)
-
-# define Py_BEGIN_CRITICAL_SECTION2_MUT(m1, m2)                         \
-    {                                                                   \
-        PyCriticalSection2 _py_cs2;                                     \
-        _PyCriticalSection2_BeginMutex(&_py_cs2, m1, m2)
-
 // Specialized version of critical section locking to safely use
 // PySequence_Fast APIs without the GIL. For performance, the argument *to*
 // PySequence_Fast() is provided to the macro, not the *result* of
@@ -75,8 +65,6 @@ extern "C" {
 
 #else  /* !Py_GIL_DISABLED */
 // The critical section APIs are no-ops with the GIL.
-# define Py_BEGIN_CRITICAL_SECTION_MUT(mut) {
-# define Py_BEGIN_CRITICAL_SECTION2_MUT(m1, m2) {
 # define Py_BEGIN_CRITICAL_SECTION_SEQUENCE_FAST(original) {
 # define Py_END_CRITICAL_SECTION_SEQUENCE_FAST() }
 # define _Py_CRITICAL_SECTION_ASSERT_MUTEX_LOCKED(mutex)
@@ -119,6 +107,7 @@ _PyCriticalSection_BeginMutex(PyCriticalSection *c, PyMutex *m)
         _PyCriticalSection_BeginSlow(c, m);
     }
 }
+#define PyCriticalSection_BeginMutex _PyCriticalSection_BeginMutex
 
 static inline void
 _PyCriticalSection_Begin(PyCriticalSection *c, PyObject *op)
@@ -194,6 +183,7 @@ _PyCriticalSection2_BeginMutex(PyCriticalSection2 *c, PyMutex *m1, PyMutex *m2)
         _PyCriticalSection2_BeginSlow(c, m1, m2, 0);
     }
 }
+#define PyCriticalSection2_BeginMutex _PyCriticalSection2_BeginMutex
 
 static inline void
 _PyCriticalSection2_Begin(PyCriticalSection2 *c, PyObject *a, PyObject *b)

Include/internal/pycore_pylifecycle.h

@@ -41,6 +41,7 @@ extern PyStatus _Py_HashRandomization_Init(const PyConfig *);
 
 extern PyStatus _PyGC_Init(PyInterpreterState *interp);
 extern PyStatus _PyAtExit_Init(PyInterpreterState *interp);
+extern PyStatus _PyDateTime_InitTypes(PyInterpreterState *interp);
 
 /* Various internal finalizers */
 

Lib/test/datetimetester.py

@@ -7295,6 +7295,34 @@ def test_update_type_cache(self):
             """)
         script_helper.assert_python_ok('-c', script)
 
+    def test_concurrent_initialization_subinterpreter(self):
+        # gh-136421: Concurrent initialization of _datetime across multiple
+        # interpreters wasn't thread-safe due to its static types.
+
+        # Run in a subprocess to ensure we get a clean version of _datetime
+        script = """if True:
+        from concurrent.futures import InterpreterPoolExecutor
+
+        def func():
+            import _datetime
+            print('a', end='')
+
+        with InterpreterPoolExecutor() as executor:
+            for _ in range(8):
+                executor.submit(func)
+        """
+        rc, out, err = script_helper.assert_python_ok("-c", script)
+        self.assertEqual(rc, 0)
+        self.assertEqual(out, b"a" * 8)
+        self.assertEqual(err, b"")
+
+        # Now test against concurrent reinitialization
+        script = "import _datetime\n" + script
+        rc, out, err = script_helper.assert_python_ok("-c", script)
+        self.assertEqual(rc, 0)
+        self.assertEqual(out, b"a" * 8)
+        self.assertEqual(err, b"")
+
 
 def load_tests(loader, standard_tests, pattern):
     standard_tests.addTest(ZoneInfoCompleteTest())

Lib/test/test_urllib.py

@@ -1526,6 +1526,14 @@ def test_url2pathname(self):
         self.assertEqual(fn('////foo/bar'), f'{sep}{sep}foo{sep}bar')
         self.assertEqual(fn('data:blah'), 'data:blah')
         self.assertEqual(fn('data://blah'), f'data:{sep}{sep}blah')
+        self.assertEqual(fn('foo?bar'), 'foo')
+        self.assertEqual(fn('foo#bar'), 'foo')
+        self.assertEqual(fn('foo?bar=baz'), 'foo')
+        self.assertEqual(fn('foo?bar#baz'), 'foo')
+        self.assertEqual(fn('foo%3Fbar'), 'foo?bar')
+        self.assertEqual(fn('foo%23bar'), 'foo#bar')
+        self.assertEqual(fn('foo%3Fbar%3Dbaz'), 'foo?bar=baz')
+        self.assertEqual(fn('foo%3Fbar%23baz'), 'foo?bar#baz')
 
     def test_url2pathname_require_scheme(self):
         sep = os.path.sep

Lib/test/test_zipfile/test_core.py

@@ -3470,60 +3470,6 @@ def test_execute_zip64(self):
         self.assertIn(b'number in executable: 5', output)
 
 
-class TestDataOffsetPrependedZip(unittest.TestCase):
-    """Test .data_offset on reading zip files with an executable prepended."""
-
-    def setUp(self):
-        self.exe_zip = findfile('exe_with_zip', subdir='archivetestdata')
-        self.exe_zip64 = findfile('exe_with_z64', subdir='archivetestdata')
-
-    def _test_data_offset(self, name):
-        with zipfile.ZipFile(name) as zipfp:
-            self.assertEqual(zipfp.data_offset, 713)
-
-    def test_data_offset_with_exe_prepended(self):
-        self._test_data_offset(self.exe_zip)
-
-    def test_data_offset_with_exe_prepended_zip64(self):
-        self._test_data_offset(self.exe_zip64)
-
-class TestDataOffsetZipWrite(unittest.TestCase):
-    """Test .data_offset for ZipFile opened in write mode."""
-
-    def setUp(self):
-        os.mkdir(TESTFNDIR)
-        self.addCleanup(rmtree, TESTFNDIR)
-        self.test_path = os.path.join(TESTFNDIR, 'testoffset.zip')
-
-    def test_data_offset_write_no_prefix(self):
-        with io.BytesIO() as fp:
-            with zipfile.ZipFile(fp, "w") as zipfp:
-                self.assertEqual(zipfp.data_offset, 0)
-
-    def test_data_offset_write_with_prefix(self):
-        with io.BytesIO() as fp:
-            fp.write(b"this is a prefix")
-            with zipfile.ZipFile(fp, "w") as zipfp:
-                self.assertEqual(zipfp.data_offset, 16)
-
-    def test_data_offset_append_with_bad_zip(self):
-        with io.BytesIO() as fp:
-            fp.write(b"this is a prefix")
-            with zipfile.ZipFile(fp, "a") as zipfp:
-                self.assertEqual(zipfp.data_offset, 16)
-
-    def test_data_offset_write_no_tell(self):
-        # The initializer in ZipFile checks if tell raises AttributeError or
-        # OSError when creating a file in write mode when deducing the offset
-        # of the beginning of zip data
-        class NoTellBytesIO(io.BytesIO):
-            def tell(self):
-                raise OSError("Unimplemented!")
-        with NoTellBytesIO() as fp:
-            with zipfile.ZipFile(fp, "w") as zipfp:
-                self.assertIsNone(zipfp.data_offset)
-
-
 class EncodedMetadataTests(unittest.TestCase):
     file_names = ['\u4e00', '\u4e8c', '\u4e09']  # Han 'one', 'two', 'three'
     file_content = [