How are the EU doing looking after Ireland, their member, ensuring there is no hard boarder? A tiny EU nation but they stick up for their own. One of the reasons free trade agreements are so tough is everyone, every industry, every country, every region has to be happy. If we as a power member of the EU said we can't have zero tariffs on Japanese cars it will undermine our industry and threaten thousands of jobs, this Japanese FTA wouldn't have happened exactly like this - that is my guess.
Furthermore, it is more complex than that. Jaguar moved to Slovakia. Sony to Amsterdam - both nothing to do with a Japanese trade agreement. But they also left. There is only so much you can explain away, before you have to hold you hands up and admit Brexit is having an effect, as predicted.