https://github.com/libremonde-org/paper-research-privacy-matrix.org/blob/master/part1/README.md
TL;DR
matrix.org and vector.im receive a lot of private, personal and identifiable data on a regular basis, or metadata that can be used to precisely identify and/or track users/server, their social graph, usage pattern and potential location. This is possible both by the default configuration values in synapse/Riot that do not promote privacy, and by specific choices made by their developers to not disclose, inform users or resolve in a timely manner several known behaviours of the software.
Data sent on a potential regular basis based on a common web/desktop+smartphone usage even with a self-hosted client and Homeserver:
The #Matrix ID of users, usually including their username.
Email addresses, phone numbers of the user and their contacts.
Associations of Email, phone numbers with Matrix IDs.
Usage patterns of the user.
IP address of the user, which can give more or less precise geographical location information.
The user's devices and system information.
The other servers that users talks to.
Room IDs, potentially identifying the Direct chat ones and the other user/server.
With default settings, they allow unrestricted, non-obfuscated public access to the following potentially personal data/info:
Matrix IDs mapped to Email addresses/phone numbers added to a user's settings.
Every file, image, video, audio that is uploaded to the Homeserver.
Profile name and avatar of users.
See below for a detailed analysis.