As a result of his refusal to work for the Occupiers, Gordon was sent to the Channel Island of Alderney in 1943 as a civilian worker for Organisation Todt, the civil and military engineering organisation of Nazi Germany. Gordon was accompanied by another young Jerseyman, Walter Gallichan.

After they arrived, they were sent to the Organisation Todt farm, Mignot Farm, and after a week were moved to the Soldiers Home, where they were told to scrub floors. Later, the young men sneaked into the office of a German officer so that they could hear the English news on the radio. A German nurse caught them, and they then refused an order to return to Mignot farm. They were sent to Alderney's Norderney concentration camp as punishment.

Of the daily regime in Norderney, Gordon later recalled that in the morning the men queued up for a ladle of coffee, then sent to roll call. They were then told what work they would be doing. At midday, they paused work for half an hour, during which they were given a slice of bread and some hot water in which floated a cabbage leaf. They then worked until 18:00, at which point they were given the same meal again, before another roll call. In the months with longer daylight, they were sometimes put to work again in the evening.

After D-Day on 6 June 1944, this work involved digging trenches around Braye Bay, but before that the work included toiling in the stone quarry, where they worked with a sledgehammer. The stone was then taken to the stone crusher to be used on gun emplacements.

Gordon testified that if you dragged your feet the guards would whip them. If they were working too slowly in the stone quarry, they would beat them. One night Gordon was a bit slow getting out of bed for a roll call and a guard hit him and knocked his teeth out.

About three weeks after D-Day, the remaining prisoners were sent to Guernsey, and then on to Jersey. Walter and Gordon were released. Gordon applied for a job as an auxiliary policeman with the States of Jersey.