Repository navigation
Description
Environment
- CPython 3.13.14 (Windows 11),
sqlite3module. The behavior also appears in earlier CPython versions; I only runtime-tested 3.13.
What happens
sqlite3.Connection.backup() can block indefinitely when the source database is locked by another connection. The timeout argument of sqlite3.connect() does not bound this wait — it applies to locked table operations, not to the backup retry loop.
In the repro below, a parent process holds BEGIN EXCLUSIVE on the source database. The child calls src.backup(dst) and never returns while the lock is held (killed after 4 seconds). Once the lock is released, the identical child code finishes in 0.06 s.
Repro
# parent.py: hold an exclusive write lock for 4 seconds, then release
import sqlite3, subprocess, sys, time
CHILD = """
import sqlite3, sys
src = sqlite3.connect(sys.argv[1], timeout=1)
dst = sqlite3.connect(sys.argv[2], timeout=1)
print('child: calling backup()', flush=True)
src.backup(dst)
print('child: backup() returned', flush=True)
"""
conn = sqlite3.connect("locked.db")
conn.execute("CREATE TABLE t(x)")
conn.commit()
conn.execute("BEGIN EXCLUSIVE")
conn.execute("INSERT INTO t VALUES (1)")
p = subprocess.Popen([sys.executable, "-c", CHILD, "locked.db", "backup.db"],
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
try:
p.communicate(timeout=4)
except subprocess.TimeoutExpired:
p.kill()
p.communicate()
print("child was still blocked after 4 s")
conn.rollback()
conn.close()Observed:
child: calling backup()
child was still blocked after 4 s
child: calling backup() # control run, lock released
child: backup() returned
Documentation gap
The current backup() docs say:
Works even if the database is being accessed by other clients or concurrently by the same connection.
and define sleep as:
The number of seconds to sleep between successive attempts to back up remaining pages.
They do not say that backup() retries on SQLITE_BUSY/SQLITE_LOCKED until it succeeds or a real error occurs, nor that connect(timeout=...) has no effect on this wait. A reader can reasonably assume the connection timeout applies, which is not the case.
Suggested documentation change
Add a note to the sqlite3.Connection.backup() section, e.g.:
backup()retries internally when the source database is locked; thetimeoutpassed toconnect()does not limit this wait. To bound the total time, pass aprogresscallback that checks your own deadline and raises, or callbackup()with a bounded number ofpagesper step.
If a code change is considered instead, an optional deadline/timeout parameter on backup() would be useful, but I understand that is an API change and may need broader discussion.
Impact
A background backup thread that relies on connect(timeout=...) can hang forever if another process holds a write lock, with no exception and no timeout to trigger cleanup or retry logic.
No project-specific or private data is included in this report; the repro uses only the standard library.
Metadata
Metadata
Assignees
Labels
Projects
- StatusShow more project fieldsNo status
Activity
daneeller commentedon Nov 22, 2026
A verified workaround that could be worth adding to the
backup()docs, including one non-obvious detail.A bounded wait can be implemented with the
progressparameter ofbackup()itself, using a monotonic clock:Verified on 3.13 with another connection holding
BEGIN EXCLUSIVE: the above aborted in ~1.8 s with a 1 s deadline and completed in 0.01 s once the lock was released, instead of blocking indefinitely.The non-obvious part:
Connection.set_progress_handler()does not fire duringbackup()'s busy-wait. I first tried that API and it hung for over 2 minutes — the backup step loop retries on SQLITE_BUSY without executing VM instructions, so the connection-level progress handler never gets a chance to run. Only theprogress=argument tobackup()is invoked on those retries.Suggested addition to the
Connection.backup()docs:picnixz commentedon Nov 22, 2026
I think this is the expected behavior and that the docs are sufficiently clear. At least, this is wgat I would expect if something is locked.
daneeller commentedon Nov 22, 2026
Thanks for the clarification — that makes sense, I'll keep the progress-callback workaround on my side rather than pushing for a doc change. Appreciated you taking the time.
picnixz commentedon Nov 22, 2026
That being said, we may consider making the timeout handled by the backup retry loop. Or add another timeout parameter to backup().