Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…x-org#16510)

)

This splits thinsg into two queries, but most of the time we won't have
new event backwards extremities so this shouldn't actually add an extra
RTT for the majority of cases.

Note this removes the check for events with no prev events, but that was
part of MSC2716 work that has since been removed.

Bumps [pygithub](https://github.com/pygithub/pygithub) from 1.59.1 to 2.1.1.
- [Release notes](https://github.com/pygithub/pygithub/releases)
- [Changelog](https://github.com/PyGithub/PyGithub/blob/main/doc/changes.rst)
- [Commits](PyGithub/PyGithub@v1.59.1...v2.1.1)

---
updated-dependencies:
- dependency-name: pygithub
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [types-pillow](https://github.com/python/typeshed) from 10.0.0.3 to 10.1.0.0.
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-pillow
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.37 to 3.1.40.
- [Release notes](https://github.com/gitpython-developers/GitPython/releases)
- [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES)
- [Commits](gitpython-developers/GitPython@3.1.37...3.1.40)

---
updated-dependencies:
- dependency-name: gitpython
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [types-requests](https://github.com/python/typeshed) from 2.31.0.2 to 2.31.0.10.
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-requests
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [black](https://github.com/psf/black) from 23.9.1 to 23.10.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](psf/black@23.9.1...23.10.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

This table was no longer used, except for a background process
which purged old entries in it.

…g#16530)

If using the script remotely, there's no particularly convincing reason
to disable certificate verification, as this makes the connection
interceptible.

If on the other hand, the script is used locally (the most common use
case), you can simply target the HTTP listener and avoid TLS altogether.
This is what the script already attempts to do if passed a homeserver
configuration YAML file.

…ce (matrix-org#16531)

…tart (matrix-org#16540)

This could happen if the last rows in the account data stream were inserted into `account_data`. After a restart the max account ID would be calculated without looking at the `account_data` table, and so have an old ID.

…rg#16473)

* Fix bug where a new writer advances their token too quickly

When starting a new writer (for e.g. persisting events), the
`MultiWriterIdGenerator` doesn't have a minimum token for it as there
are no rows matching that new writer in the DB.

This results in the the first stream ID it acquired being announced as
persisted *before* it actually finishes persisting, if another writer
gets and persists a subsequent stream ID. This is due to the logic of
setting the minimum persisted position to the minimum known position of
across all writers, and the new writer starts off not being considered.

* Fix sending out POSITIONs when our token advances without update

Broke in matrix-org#14820

* For replication HTTP requests, only wait for minimal position

…the variable `${DS_PROMETHEUS}` and remove `__inputs` (matrix-org#16471)

…ion sending worker (matrix-org#16515)

This reverts commit 5fe76b9.

I think I had this accidentally commited on my local develop branch, and
so it accidentally got merged into upstream develop.

This should re-land with corrections in matrix-org#16504.

…org#16504)

Twisted trunk makes a change to the `TLSMemoryBIOFactory` where
the underlying protocol is changed from `TLSMemoryBIOProtocol` to
`BufferingTLSTransport` to improve performance of TLS code (see
twisted/twisted#11989).

In order to properly hook this code up in tests we need to pass the test
reactor's clock into `TLSMemoryBIOFactory` to avoid the global (trial)
reactor being used by default.

Twisted does something similar internally for tests:
https://github.com/twisted/twisted/blob/157cd8e659705940e895d321339d467e76ae9d0a/src/twisted/web/test/test_agent.py#L871-L874

Fixes matrix-org#16417

…#16550)

… of tuples (matrix-org#16505)

This should use fewer allocations and improves type hints.

This adds a module API which allows a module to update a user's
presence state/status message. This is useful for controlling presence
from an external system.

To fully control presence from the module the presence.enabled config
parameter gains a new state of "untracked" which disables internal tracking
of presence changes via user actions, etc. Only updates from the module will
be persisted and sent down sync properly).

Mostly to improve type safety.

…ion (matrix-org#16549)

…trix-org#16557)

c.f. matrix-org#16481

Follow on from / actually correctly does matrix-org#16557

…#16559)

This is mostly useful for federated rooms where some users
would get stuck in the invite or knock state when the room
was purged from their homeserver.

* complement: enable dirty runs

* Add changelog

* Set a low connpool limit when running in Complement

Dirty runs can cause many containers to be running concurrently,
which seems to easily exhaust resources on the host. The increased
speedup from dirty runs also seems to use more db connections on
workers, which are misconfigured currently to have
`SUM(workers * cp_max) > max_connections`, causing
```
FATAL:  sorry, too many clients already
```
which results in tests failing.

* Try p=2 concurrency to restrict slowness of servers which causes partial state join tests to flake

* Debug logging

* Only run flakey tests

* Only adjust connection pool limits in worker mode

* Move cp vars to somewhere where they get executed in CI

* Move cp values back to where they actually work

* Debug logging

* Try p=1 to see if this makes worker mode happier

* Remove debug logging

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: David Robertson <[email protected]>

We can bail early if the from token is greater than or equal to the
current token.

Co-authored-by: Patrick Cloke <[email protected]>

This reverts commit 4724a6d.

… background update (matrix-org#16594)

A regression from removing the cursor_to_dict call, adds back
the wrapping into a tuple.

pip was using a vendored setuptools that was incompatible with
Python 3.12. Upgrading cibuildwheels to a version with a newer
version of pip (and thus a newer version of setuptools) fixes
the issue.

Synapse will soon be forked by Element under an AGPLv3.0 licence (with CLA, for
proprietary dual licensing). You can read more about this here:

* https://matrix.org/blog/2023/11/06/future-of-synapse-dendrite/
* https://element.io/blog/element-to-adopt-agplv3/

The Matrix.org Foundation copy of the project will be archived. Any changes needed
by server administrators will be communicated via our usual
[announcements channels](https://matrix.to/#/#homeowners:matrix.org), but we are
striving to make this as seamless as possible.

This minor release was needed only because of CI-related trouble on [v1.96.0](https://github.com/matrix-org/synapse/releases/tag/v1.96.0), which was never released.

- Fix building of wheels in CI. ([\matrix-org#16653](matrix-org#16653))

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAmVXZL4ACgkQM/xY9qcR
# MEg6uhAAj7GPCkqUgD8l1+9adqFip7OdNMMVTBxrIfuTT003nsfNm3rKwBNeB1ok
# ydK5p95Bd3rM7/ezD8GUDzQjf/8aVEfjOCMmMNj+GJK6WDjEu97ANWsJlfaZmgck
# LWmBN0A7dBkvyBBe9vIJHqcNcBjaJyeznrTIx5RmaFO63PuTKg/T1Z1jJ8igrqe6
# qQA/FmnVYV/A6IoPLVg88ITQ7o5GQHkOYHG2NEO4U2mc5h4/kXG4kWYN2cqOU4YG
# 1NkkzsA5s/fUj/Z9vZAYXyyB9c/y6XE/iOxGG/qr8v3BJeSN7iJTCdygbz+UKDZF
# xuAonFnHZe1NO5V4RVf5G2ToyCFpioWadyZvClOWpM7iWBAaTEr1vYLI/foN7ddU
# 00YPXeIK+ot+93JVMq9nEzBxMQYqSZJcAD5m9qSiMWp/TlD/1AoZxhcvZrdYUJMU
# L8U4fX+kWIegM6ZPrBC55FjLxnnsD4NiQ9DFgrNDv84vqKJ2A++d+VzKFnTj8AQa
# BfFOtYQ4Md1vGsRkmaI/sQRo4qpUDTRjlk3aa17VEs/RSQQ1m2nvS/R4GKRYntIL
# gEcmeO9uGrjnq7NFpZ38p1a+pGkBag6br17qqWPfBB3+Br7ocZ5TOS+3391hfw0V
# MPniZM5/ODtIph2XRMjQ7YnXhvcpig+r08AwR7FUZJ4xSm0ksT0=
# =bGtZ
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri Nov 17 13:03:58 2023 GMT
# gpg:                using RSA key 177B595E4DFCB510C556750833FC58F6A7113048
# gpg: Good signature from "Patrick Cloke <[email protected]>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 177B 595E 4DFC B510 C556  7508 33FC 58F6 A711 3048

# Conflicts:
#	.github/workflows/latest_deps.yml
#	.github/workflows/release-artifacts.yml
#	.github/workflows/tests.yml
#	.github/workflows/twisted_trunk.yml
#	synapse/handlers/room.py
#	synapse/storage/databases/main/receipts.py

Since requests proxy via hungry the IP is masked already. Responsibility
for tracking this is no longer with synapse.