Is there a word to refer to a character in a play ... who is talked about but never actually shows up on stage or screen?

Yes, such a character is known as a "stage sprite".

Some stage sprites are too scared to show up in which case they are said to have "stage fright".

But, a stage sprite with stage fright is not the norm. Usually they are just written out of the script.