Actually, a big portion of both Toyotas and Hondas are assembled in the U.S.. I don't think Celicas are though. As far as I know, it hasn't actually affected quality very much though. All the important components come from Japan, as far as I know, so its really not that big of an issue.
A major factor in the lower quality of so-called "American" cars (Chevy, Ford, etc.) is that most of the components (engines, air conditioners, etc) are made in countries like Mexico and Brazil. If you look at what really fails on American cars the most, its the so-called components (air conditioners, alternators, water pumps, stuff like that). On the Saturn I owned before the Celica, the air conditioner had to be replaced SIX times, and the alternator twice.
Japanese cars, even though largely assembled here in the U.S., are still predominantly Japanese. U.S. cars are in reality mostly made in Mexico, Central, and South America.