What to do when switching games

OK, the manual says you don't have to turn off the PlayStation to
switch games, game manuals state the opposite. What is the right way?

Straight from Sony: Switch the PlayStation OFF whenever you switch
games. That is the safest way and causes less strain on the CD drive,
assuming you wait until the CD stops before you take it out.

What about Memory cards? On the package it says: "Data may disappear
if you use the on/off button too many times while the card is still in
the PlayStation."

This only applies when you rapidly turn it on and off repeatedly, which
of course you never do. People have left them in for months and played their
PS every day without any problems. Taking the card out everytime you
switch a game, just to put it back in, will most likely wear it out faster!