So, if the actions of a devolved but subordinate level of government go against the state’s interests, the leaders of that state should stay mum? That's certainly not the view taken by successive US administrations; they have often condemned state-level actions, even when the federal government has been legally powerless to do anything in practice.
The UK has no written constitution as in the US, but a clear constitutional settlement nonetheless. Now I am not a lawyer, but as far as I understand it, the UK Parliament retains sole authority to legislate over so-called reserved matters. This can only be altered by further primary legislation of the UK Parliament. Within the Scotland Act 1998 reservations to all devolved matters are those concerned with the UK as ‘a state’ and include e.g. the Constitution, foreign affairs and defence.
Let's play a little thought experiment. What if a devolved level of government takes an action that is within its legal competence but which leads to war with the state and a third country? Would the state have the right to curtail the otherwise legal actions of the subordinate level of government to defend the whole country's interests and security? Most people would say so.
That was an extreme example, but the point is serious and recognized in international law. Under the so-called laws of state responsibility, a state is responsible for the actions of its officials and organs, even if the organ or official is formally independent. It is even responsible if the organ – in this case a devolved level of government - is acting ultra vires, that is, “beyond the powers” of the state. Indeed, entities not even classified as organs of the state may still be imputable, when they are otherwise empowered to exercise elements of governmental authority, and act in that capacity in a particular instance. So the UK is legally responsible for Scotland’s actions.
In other words, if the power to conduct foreign relations is truly an exclusive competency of the UK government, with no role for the devolved bodies, a logical consequence is that some devolved actions and indeed laws impinging on foreign relations are invalid, even in the absence of already-established UK government policy. Libyagate is not only about Gordon Brown's politically-calculated absence, or Kenny MacAskill’s misguided notions of compassion, but about a constitutional grey area that should be explored further, at the very least by the Judicial Committee of the Privy Council, which is responsible for legal matters arising from Scottish devolution.