I dont think its as binary as that. But I would wager it is weighted that way, considering the ease/difficulty of non EU/EU entry into the UK.
For example, it is very easy and profitable for an EU resident - particularly from the east - to come to the UK, work for minimum wage, and then go home a (relatively) wealthy man**.
Now try doing that coming from India...
Ill have a look later for numbers, but my bet is that a higher proportion of Non-EU immigrants are of a more skilled/qualified nature.
From what I have seen the majority of Brexiteers want immigration controlled. They want us to have the ability to decide who comes and who doesnt. And so, if need be, say "no" to people as/when we dont have the capacity or requirement for them.
Controlling numbers is a part of that, but it isnt the whole.
Personally I think the EU/Non-EU split is disgusting. On one hand I am against the lack of control we have over the EU side, on the other I am absolutely disgusted by the disparity in process/access based solely upon race.
Ive mentioned it before, Ive had Indian friends go through the process and it is horrific. All the while absolutely anyone from the EU can breeze in without a care. That disparity is awful and I personally am all in favour of making it fair. A more streamlined policy that applies to all would be "right" in my mind.
Roll into that the ability to attract people we need, encourage nurses for example, and even better.
**Its an extreme example, not the norm I know, but 400% true. Had a chap working here as a contractor. He commutes in from Bulgaria, stays in a Travellodge Mon-Thu and then flies home. He was on something like £400 a day, and was very keen to explain just how lavish his life was back home as a result. Lord of the manor type stuff, building an enormous house, pool, tennis courts, staff...