I was able to remove the Address field with hook alter when it's single value field but don't know how to do it when it's a multiple value field. Same with renaming the fields. Thanks.
There's a patch for this in the Addressfield issues queue but it's still under discussion and doesn't work with the latest dev. You could look at the code in the patch though; it may provide you with insight on how to do what you want.