I have D9.5 site with CKEditor 4 and media library module. In CKEditor, when I double click on some existing media I get popup to select new media to replace clicked one. After selecting new visually new one replaces old one. Old is not visible any more, but it's still there. I.e. if I click "Source" button I can see both in source. If I switch back to visual mode both are shown. Is this some kind of bug or configuration failure? I'm not finding any discussion on this issue so I wonder if maybe I didn't configure it correct.
I sounds like CKeditor is not cleaning up after it self. I would not rely on that method for replacing media, but rather delete the old media and then insert the new. That gives you a much better chance of having clean code. I would document the behavior with screen shots and send it it all the Ckeditor team so they can see if they have to update the function that cleans up the generated code. Also, if you can, I would see if you can duplicate the problem in other builds and other places, the more you can duplicated it the stronger case you will have for it being a bug :)