It'll be something more along the lines of:

    $form['field_mytax_reference'][LANGUAGE_NONE][0]['#default_value'] = $taxTermId;

Have a look at http://drupal.stackexchange.com/questions/27767/why-is-hook-form-alter-so-messy-in-d7/27784#27784 for a bit more info.