So I created a custom block and put the block on every page but unfortunately the block causes a white screen... I can't login to change the block because the block is on every page, as I mentioned. Is it possible using an FTP program to access blocks via the server and delete/change them?
First of all, blocks are stored in the Database, not in your web directory. Therefore, FTP is useless here.
If the custom block was created using a custom module, simply disable the module.
If the custom block was created using the Drupal User Interface, you can disable your custom block by changing the status from 1 to 0 or edit the block body by login in into your Database using PHPMyAdmin and finding the correct block table.