The way terms are related is from child towards parent, not parent towards child.
So you need to start with taxonomy term views (Block Views Display) showing all the terms, filter them according to Parent term to only show children terms, this will show all of them, no matter what parent, but will not show parents (because they don't have parents of their own).
You can directly add any custom fields you have on the terms because this is a term type Views.
Then add a contextual filter for the Parent term, (Provide Default Value, from URL context).
When you place the Block on the page of the Parent term (don't forget to set the Block filter option) it will automatically filter it to show only the children terms of that parent.
This may need a bit different solution if Block is to be placed on a node Views type. Node type will already have a contextual filter for terms and the additional contextual filter for child terms will have to work together with it.