You can coax something out of Views without PHP (see Partial Solution below) but not, I think, quite what you want. A module or - against your team policy Views PHP - may be required.
Part of why some of the usual Views workarounds won't solve this problem is the edge cases. If we knew that ranges of dates would always be within the very same month, then this could be done in Views without PHP using the approach outlined below. But that seems unlikely! You need to consider what you want to do when you get cases such as 31 May 2013 to 2 Jun 2013 or 28 Dec 2012 to 4 Jan 2013. What do you want displayed? If you want '31 May - 2 Jun 2013' and '28 Dec 2012 - 4 Jan 2013' respectively, but in non month-crossing cases '2 - 3 May 2012', then some code is going to need to be involved somewhere.
A partial solution can be massaged out of Views. It will avoid repeating the year but not the month. So you get, e.g. '2 May to 3 May 2012', '28 Dec to 4 Jan 2013', '31 May to 2 Jun 2013' from it. Just possibly this might be close enough for some use cases.
The strategy is to add the date field twice (excluding it from display) and apply different formats / configurations. You can then use
Global: Custom text fields to join the results together.
Create two date formatters. One to give '12 Mar' when given 12th March 2012 (
j M) and one to give '2012' when give 12th March 2012 (
Add the date field (and exclude from display). Set it to show both the START and END dates using the '12 Mar' format. When both START and END are present this will yield '3 Mar to 6 Mar' etc. When only START is present, it will yield '3 Mar'. Recommendation use the 'Administrative title' in the 'More' section to document this, e.g. 'Date or range without year'. This helps at the final step.
Add the date field (and exclude from display). Set it to show just the END date. Use the '2013' format. When an END date isn't present, Views will fall back to using the start date. Recommendation use the 'Administrative title' in the 'More' section to document this, e.g. 'Year'. This helps at the final step.
Global: Custom text field and use 'Replacement Patterns' to output the 'Date or range without year' followed by the 'Year'. (Easy if you've documented with the 'Administrative title' - trial and error if not.)
Note: If you want to detect the absence of an end date at step (3), Views will do this if you make use of the No Results area. It only defaults to the Start Date if the No Results area is not used. That gives some, albeit, limited scope to vary formatting depending on the presence of absence of an end date.