How do I change the URL present under Home » Administration » Configuration » System in my Drupal site?
I want to get rid of the stuff circled in red, but it only seems that I can add different pages only in the blank input box next to it.
It is because you run Drupal in local; if you uploaded your site to a valid domain or set a virtual host, you get rid of it.
You can set your virtual Host following How to Add Virtual Directory Alias on Apache Xampp. You could also use Acquia Desktop dev2 which handles virtual hosts by itself.