Psychonauts

I think most of us are psychonauts on here, i.e. explorers of inner space and altered states. I have gone to altered states "naturally" since my childhood, others use drugs to go there.
I recently read some literature by for example Timothy Leary or John Lilly on these things, and I noticed how "innocently-optimistic" they were in those days on these topics. 'Use LSD or Ketamine? Loose contact to your surroundings and "trip" for hours without being responsive? No danger. Just do it.'
While of course today we know of many "casualties" regarding psychedelic drug use.
So, the question for me is; is there a danger or a negative side to being a psychonaut or going to the altered? For example, I always shied away from those altered states where you lose control of yourself, or your body. On the other hand I had experiences where I completely snapped for a second; losing control, self and consciousness ("Ego-Death") and those were the most rewarding experiences I had in my life. Or on a lesser note, that thing at parties, when the music and strobes gets you "high" and you lose control of your body and it starts to move and dance by itself and you are just "watching by" and are enjoying the show - very pleasurable.
The problem is, if there is a danger to being a psychonaut; the danger would be hard to calculate as this a very complex and chaotic thing, unpredictable. For example a minor altered states can turn into a massive one within a moment; and if there would be a danger in a massive one, it might be better to not go to the altered at all, as these things can happen all the time.
"Organized" religions, from Sufism to Thelema, of course always stressed *there* is always danger, so you have rules to follow when doing these things, or even better, put all your faith in and give up your independence to a guru. But the danger here is of course that the altered is very hostile to rules and likes to bend them or turn them on you, and also your state of mind defines how the altered goes: if you go to the altered with the belief that you depend on rules, very likely something bad will happen when you come to a situation where these rules don't work anymore (and this is very likely because of the unpredictable nature of the altered). And blind faith in a guru has its downsides too.
The problem is also that, as we say in Germany, the genie can't get in the bottle anymore once it's out, so you can't really stop being a psychonaut, so it might be better to hope it's good and without danger as you can't really avoid it, at least if you have a natural inclination to be this way. All my attempts to not naturally go into altered states anymore were huge failures and put me in a lot of danger, and this is even an understatement.
So, it might be better to stay with the "innocent-optimistic" approach and enjoy the "trip" while it lasts. And it is a trip!