This afternoon a moderator "DustyArne" on the FD ED forums for Xbox shared a tip that might improve the Starport services menus but also the Galaxy map loading times on Xbox One.
I suffered from extreme loading times when opening the services at a Starport, opening the Galaxy map could take more the 2 minutes sometimes.
I followed Dusty's tip and changed my dns to the free google dns in the Xbox One network settings.
Primary 8.8.8.8
Secondary 8.8.4.4
It might help you to, the changes for me are dramatic, Starport services open instantly, just as the mission board and passenger lounge.
The biggest change for me is the Galaxy map though, 5 to 6 seconds max waiting time, I was stunned.
If you have these problems then maybe this could help.
Fly save o7