I've also encountered that. The node form and the image field's value will be submitted as empty and resulting in as you say that the image get removed from the node.
It is an unfortunate side effect of how the form api works. Someone could try to work around it in Panels, and/or limit panels to prevent you from doing this, but both of these would be hacks on top of form api.
One workaround is adding the pane General form that you will find under Form when adding panes. Then click settings for the pane and under CSS properties set the class to "element-invisible". This way all other form elements that you have not placed as panes will be present but hidden by css.
I know it feels wrong but at least it get's the job done :)