Ah! That's a bit odd.
Opening the CSV file in notepad is actually showing:
41708,tr_103xxxxxxxxxxxxxx,STRIPE TRANSFER,pending,gbp,4.68,0,,SOME BANK,1234,GB,1,5,0.32,4.68,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,5,0.32,4.68
No sign of any odd characters at all, but still not importing. Opening the sheet in Excel defaults it to currency (actually shows the £ in the cell), but when editing the value it's simply a number (no symbols or other characters at all).