Yea, that's the only way to shut it down if you're in that situation.
Random trivia:
This only happens with mixed-mode games that use audio tracks. When you close the emulator by clicking on the X, only the data part gets shut down along with epsxe's GUI, but the audio tracks will continue to be played by the (virtual) CDrom drive cause it did not get a proper signal from the emulator to stop playing. The file -> exit method does it correctly.
With single-track games you won't have this issue. You can tell how many tracks a game has by looking inside the cue or ccd file with notepad. Or loading it in CDmage/isobuster.
Give me what I want or face destruction........IN JAPANESE!