Skip to content

Commit 3aef494

Browse files
miss-islingtonlysnikolaouencukou
authored
[3.14] gh-142518: Document thread-safety guarantees of dict operations (GH-144184) (#144708)
* Address feedback; move thread safety section below see-also * Address feedback - don't mention equality comparison only * Change admonition to rubric; cross-reference glossary --------- (cherry picked from commit 35dc547) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> Co-authored-by: Petr Viktorin <encukou@gmail.com>
1 parent 8c4ce4b commit 3aef494

File tree

1 file changed

+140
-0
lines changed

1 file changed

+140
-0
lines changed

Doc/library/stdtypes.rst

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5561,6 +5561,146 @@ can be used interchangeably to index the same dictionary entry.
55615561
of a :class:`dict`.
55625562

55635563

5564+
.. _thread-safety-dict:
5565+
5566+
.. rubric:: Thread safety for dict objects
5567+
5568+
Creating a dictionary with the :class:`dict` constructor is atomic when the
5569+
argument to it is a :class:`dict` or a :class:`tuple`. When using the
5570+
:meth:`dict.fromkeys` method, dictionary creation is atomic when the
5571+
argument is a :class:`dict`, :class:`tuple`, :class:`set` or
5572+
:class:`frozenset`.
5573+
5574+
The following operations and functions are :term:`lock-free` and
5575+
:term:`atomic <atomic operation>`.
5576+
5577+
.. code-block::
5578+
:class: good
5579+
5580+
d[key] # dict.__getitem__
5581+
d.get(key) # dict.get
5582+
key in d # dict.__contains__
5583+
len(d) # dict.__len__
5584+
5585+
All other operations from here on hold the :term:`per-object lock`.
5586+
5587+
Writing or removing a single item is safe to call from multiple threads
5588+
and will not corrupt the dictionary:
5589+
5590+
.. code-block::
5591+
:class: good
5592+
5593+
d[key] = value # write
5594+
del d[key] # delete
5595+
d.pop(key) # remove and return
5596+
d.popitem() # remove and return last item
5597+
d.setdefault(key, v) # insert if missing
5598+
5599+
These operations may compare keys using :meth:`~object.__eq__`, which can
5600+
execute arbitrary Python code. During such comparisons, the dictionary may
5601+
be modified by another thread. For built-in types like :class:`str`,
5602+
:class:`int`, and :class:`float`, that implement :meth:`~object.__eq__` in C,
5603+
the underlying lock is not released during comparisons and this is not a
5604+
concern.
5605+
5606+
The following operations return new objects and hold the :term:`per-object lock`
5607+
for the duration of the operation:
5608+
5609+
.. code-block::
5610+
:class: good
5611+
5612+
d.copy() # returns a shallow copy of the dictionary
5613+
d | other # merges two dicts into a new dict
5614+
d.keys() # returns a new dict_keys view object
5615+
d.values() # returns a new dict_values view object
5616+
d.items() # returns a new dict_items view object
5617+
5618+
The :meth:`~dict.clear` method holds the lock for its duration. Other
5619+
threads cannot observe elements being removed.
5620+
5621+
The following operations lock both dictionaries. For :meth:`~dict.update`
5622+
and ``|=``, this applies only when the other operand is a :class:`dict`
5623+
that uses the standard dict iterator (but not subclasses that override
5624+
iteration). For equality comparison, this applies to :class:`dict` and
5625+
its subclasses:
5626+
5627+
.. code-block::
5628+
:class: good
5629+
5630+
d.update(other_dict) # both locked when other_dict is a dict
5631+
d |= other_dict # both locked when other_dict is a dict
5632+
d == other_dict # both locked for dict and subclasses
5633+
5634+
All comparison operations also compare values using :meth:`~object.__eq__`,
5635+
so for non-built-in types the lock may be released during comparison.
5636+
5637+
:meth:`~dict.fromkeys` locks both the new dictionary and the iterable
5638+
when the iterable is exactly a :class:`dict`, :class:`set`, or
5639+
:class:`frozenset` (not subclasses):
5640+
5641+
.. code-block::
5642+
:class: good
5643+
5644+
dict.fromkeys(a_dict) # locks both
5645+
dict.fromkeys(a_set) # locks both
5646+
dict.fromkeys(a_frozenset) # locks both
5647+
5648+
When updating from a non-dict iterable, only the target dictionary is
5649+
locked. The iterable may be concurrently modified by another thread:
5650+
5651+
.. code-block::
5652+
:class: maybe
5653+
5654+
d.update(iterable) # iterable is not a dict: only d locked
5655+
d |= iterable # iterable is not a dict: only d locked
5656+
dict.fromkeys(iterable) # iterable is not a dict/set/frozenset: only result locked
5657+
5658+
Operations that involve multiple accesses, as well as iteration, are never
5659+
atomic:
5660+
5661+
.. code-block::
5662+
:class: bad
5663+
5664+
# NOT atomic: read-modify-write
5665+
d[key] = d[key] + 1
5666+
5667+
# NOT atomic: check-then-act (TOCTOU)
5668+
if key in d:
5669+
del d[key]
5670+
5671+
# NOT thread-safe: iteration while modifying
5672+
for key, value in d.items():
5673+
process(key) # another thread may modify d
5674+
5675+
To avoid time-of-check to time-of-use (TOCTOU) issues, use atomic
5676+
operations or handle exceptions:
5677+
5678+
.. code-block::
5679+
:class: good
5680+
5681+
# Use pop() with default instead of check-then-delete
5682+
d.pop(key, None)
5683+
5684+
# Or handle the exception
5685+
try:
5686+
del d[key]
5687+
except KeyError:
5688+
pass
5689+
5690+
To safely iterate over a dictionary that may be modified by another
5691+
thread, iterate over a copy:
5692+
5693+
.. code-block::
5694+
:class: good
5695+
5696+
# Make a copy to iterate safely
5697+
for key, value in d.copy().items():
5698+
process(key)
5699+
5700+
Consider external synchronization when sharing :class:`dict` instances
5701+
across threads. See :ref:`freethreading-python-howto` for more information.
5702+
5703+
55645704
.. _dict-views:
55655705

55665706
Dictionary view objects

0 commit comments

Comments
 (0)