Horizons is a paid expansion, the most recent release was free content for both the base game and Horizons.
There are times it will be advantageous to use non-Horizons, for example if your SRV gets stuck on a planet. You can self destruct or load non-Horizons, either way will transport you back into your ship (because there are no SRVs in non-Horizons). If you choose the second way then your SRV will be back in your ship when you reload Horizons.
There are other reasons a person might choose to play non-Horizons but probably not as a beginner.