A peculiar story about DHL, UTF-8 and Skype
Recently I needed to send a parcel and I decided to use DHL and pay online. Unfortunately the form always showed me an obscure error. I tried to debug it. In the end I managed to find out the precise cause of the error and it happened to be related to UTF-8 encoding and Skype.
Preamble
I already tried to pay online for sending through DHL long time ago and didn’t manage due to precisely the same error. Still this time I was eager to try again, because it was more convenient than filling everything in a hurry in a store (and 1 Euro cheaper, of course).
Everything starts from DHL online franking form. The form itself looks pretty usual - sender, recipient, customs. I filled everything in. This took quite some time (customs part is the most unpleasant). And then the final part of the form comes - what to do with the package if delivery fails? Destroy or return to the sender? I felt like receiving my stuff back, so please return to the sender. And that’s where the story starts.
The “delivery failed” part of the form shows an error and clearly asks for a valid telephone number. Hm, probably they would like to call me when returning the parcel, seems reasonable, but they did not even ask for my phone above (there is no such field). Moreover the recipient phone number looks ok to me (I’ve checked international phone number format). I was confused.
One obvious workaround is to let them destroy the parcel (presumably they don’t need my phone for that), but the error does not go away. Even if you fill in the form from scratch (yes, I did this), you just get exactly the same error when choosing to destroy the package.
Last time I had exactly the same issue and I just concluded that the form is buggy. I needed to send the parcel soon, so I went the old way - to a shop and that was it. However, this time I was curious to understand what was wrong.
Hypothesis 1: Form field missing
My original hypothesis was that they indeed wanted my phone number, but the form was buggy and the field was not shown. Since the form just sends a POST request, I could modify the request and resend it manually without the form. Indeed the form posts to https://www.dhl.de/int-versenden/gw/rest/api/ShipmentEditor
and the sender part has an empty phone
field. So I copied the request as cURL command, modified the phone number and got precisely the same error back.
However, the error in raw json response had property
field and the value was address.receiver
. So they did not like the recipient phone, just the error message was positioned very poorly (hey, DHL, I hope you are reading this).
Hypothesis 2: Recipient phone number problem
Wrong format
I checked international phone number format again just in case. Everything seemed correct. I tried using my own phone number exactly in the same format and it worked.
Their phone validation does not know the country / area code I am sending too
Since my phone number worked, I assumed that their validation is not very familiar with the destination country. I tried a phone number from the destination country, but from another carrier - error. I tried imitating my phone number (i.e. length and positions of zeros), but with other country area code - same error. Then I decided to go to their website in the destination country and just take their phone number. This worked, but their area code was different.
Their validation does not know area code
They had another phone with a different code and it worked too. This felt weird, but I still needed to send the parcel. Theoretically I could just use their customer support number as the recipient telephone number and provide a proper one in “additional address details” field, but this felt risky. Fortunately, I decided to play a bit more and modified the area code in their number to match the one I needed. It still worked. Then I modified the remainder of the number to fully match my original number and it actually worked. But why?
Something is wrong with my original recipient number string
Now I had two phone numbers looking exactly the same, but one caused an error and another one worked. Here I remembered that I copied the first number from Skype (to avoid making any typos), the second one I copied from DHL website and then edited it in the form itself.
I saved both strings into separate txt files and looked at their byte representations (whitespaces are mine for convenience):
from Skype
e280aa 2b34343031323334353637383930 e280ac 0a
from their website + modification
2b34343031323334353637383930 0a
As you can see from Skype we get the phone number with e280aa
and e280ac
. Since I used web.skype.com
for this, I looked at the code
and there were these characters, just encoded differently.
After some googling, I found out that e280aa
is a UTF8 character called LEFT-TO-RIGHT EMBEDDING
(U+202A), e280ac
- POP DIRECTIONAL FORMATTING
(U+202C).
Victory
Ok, so now I knew what was causing the issue and could bypass it and send my parcel. Did I just do that? Obviously not. I was curious why these characters were in Skype in the first place.
Just looking at the UI did not help:
However, left-to-right and directional formatting ringed a bell - this may have something to do with right-to-left languages. Let’s check the UI in some right-to-left language:
Wow, so everything is mirrored, but not the phone number. Now let’s remove these characters
and we get
+
just jumps behind the number (i.e. gets written from right-to-left due to the language).
This now seems reasonable. The Arabic numerals are written and read from left-to-right (i.e. like in English) even in right-to-left languages, so the phone number itself is just a number and should not change. However, the unary plus +
could move and these 2 UTF characters just do the following:
browser: <prints some text right-to-left and meets e280aa>
e280aa: hey, browser! How-dee-do? Could you do me a
favor and print left-to-right for a bit?
browser: oh, sure, not a big deal. <prints the
phone number with '+' on the left of it and meets e280ac>
e280ac: hey, browsee! How are html standards these days?
e280aa asked me to ask you to go back
to right-to-left, thanks a lot for helping us out!
browser: oh, whatever <continues printing right-to-left>
Then I just sent the parcel and indeed the experience was much better when prepaying online (+1 Euro cheaper as well).