Is it possible to add/remove form elements with clientside js? I am using ajax, but it is a bit too slow for the kind of dynamic elements I need.
You can hide and unhide form elements with js. For example you can prepare a popup and deliver it with the main form and then trigger it in js.
You can't generate new elements, because they will be ignored when submitted.
You could build your own form solution with custom js and php code. But this is probably not the best way on the long term.
The best answer to your question, how to build dynamic form solutions, is to investigate, what you can do with a js framework and Rest API.