The server software appears to be available and updated now, which they’ve been spotty about in the past. I’ve updated to remove the closed-source part since that is not correct.
As for phone number: Signal still requires me to enter a phone number to create an account as of about 5 minutes ago.
Where the metadata goes I think is important as well.
All Signal metadata necessarily goes through Signal’s servers and is tied to your phone number, but not all Matrix metadata ever gets near the Matrix.org if you are using a different homeserver.
I think both are less than ideal in that regard, and I think Briar (strictly P2P) has a much better model for dealing with this at the expense of generally being a UX disaster.