I solved by using the module entity reference multiple display which let you to configure different view modes for your field reference. You can split in groups to achieve that.
You install the module and after it is enabled, you go to your content type and in manage display choose rendered entity with different view modes. I have attached a picture
From then you can choose how many groups you would like to display in my case I just need two, the first one I needed to be displayed as view mode teaser and the second one I need no picture in the teaser and just need to display at most two elements in that group.
If anyone has another way to handle this please post your answer here.