Invasive non-native species are an important cause of biodiversity loss, particularly in fresh waters. The mosquitofish Gambusia holbrooki are among the world’s worst invasive species: they have caused extirpations of native species and are known to sometimes cause trophic cascades and ecosystem effects. This invasive species is also known to impact threatened fishes such as the Spanish toothcarp (Aphanius iberus), which is endemic to Mediterranean Spain. However, it is unclear if the impact of mosquitofish on many fishes is more through resource competition, agonistic interactions or predation, and how often mosquitofish cause trophic cascades. To clarify these questions, we performed a 48-day mesocosm experiment in eutrophic conditions to test for interspecific effects and clarify the impact mechanism using six treatments: the two fish species alone each at two densities, and the two fish species mixed or separated with a net that prevented direct interactions among them. We observed clear fish treatment effects on several variables. At low initial fish densities, the population growth rate of mosquitofish was orders of magnitude greater than that of the Spanish toothcarp, likely contributing to its invasive success and ecological impact. At high fish densities, turbidity, chlorophyll a concentration and daytime dissolved oxygen percentage increased, whereas total phosphorus decreased; crucially, the trophic cascade caused by mosquitofish was stronger than that by toothcarp. The experiment also demonstrated that the interspecific effects of mosquitofish on toothcarp were more important than those of intraspecific competition. The invasive species produced effects on population growth rate, size structure, and fish condition (mass-length relationship) of toothcarp. Effects on population growth rate of toothcarp seem more caused by resource competition, whereas impacts on size structure and condition seem also caused by more direct interactions. The diversity of effects of mosquitofish underscores the difficulty of predicting the impact of invasive species. Our study further provides an approach to differentiate the effects of resource competition from other more direct ecological interactions and so to clarify the impact mechanism of aquatic invasive species.