On further investigation of this problem I have found that if I select the Background Layer and then do LayersAdd then the LoadSelection is working just great and as expected, meaning the loaded selection comes in the same size as was saved.
But if the Background layer is very large then the time taken to add a new layer increases substantially.