Financial Esperanto: Why Local Language Characters are Causing Payment Problems
Although initiatives such as the International Standards Team for Harmonisation (IST)1 represent a major step forward in streamlining payment processing, some obstacles still remain. These obstacles can be overcome, but this always assumes that all those concerned realise and admit that those obstacles exist – something that is by no means always the case!
An example of one such unappreciated obstacle is the use of local language characters in financial EDI messages – both EDIFACT and XML. While local banks already have to support these characters for domestic payments, the majority of banks only support the Latin alphabet (plus a handful of special characters) in their back office systems.
Why is this a problem?
Where a bank system does not support local characters either the bank or the corporate will typically replace them with characters that it does support. The local characters concerned do not even have to be particularly exotic for this to happen. For example, when transmitting from the United States back to Europe some banks translate the EDIFACT PAYEXT message into X12 format, which does not recognise the German umlaut. This is just the tip of the iceberg – with other languages (especially Asian ones where the Latin character set is of little use) the situation is far more severe.
As a result, it is not uncommon for information, such as payee name and address, to become corrupted. Where paper-based payments/remittance advice notes are involved, this can result in non (or delayed) delivery by the local postal service. (A very approximate idea of the scale of the problem can be obtained by opening a web site for which your browser does not have local language support – much of the content will be gibberish).
At first glance, this may appear a trivial problem that could be addressed by local workarounds, but in practice this is not the case. Furthermore, it is likely to become increasingly problematic for a number of reasons:
Technically, the solution to the issue of local language character representation already exists in UNICODE. This is a global character-encoding standard that can represent all characters for computer usage, including technical symbols and characters used in publishing. In total, version 2.0 of UNICODE can represent 38,885 characters. These characters can be represented using a number of different encoding formats. Perhaps the most useful of these is UTF8, which is a multi-byte variable width format that also includes the ASCII character set using single byte encoding.
This is more than just meaningless technical jargon; it also has a practical value, because a multi-byte character set can represent characters using one or more bytes per character. This is particularly useful when dealing with Asian languages, as accurate representation of their complex language characters then becomes possible.
The UNICODE/UTF8 standard is also seeing increasing support from others involved both directly and indirectly in the payment chain. For example, until recently the SWIFT network was a limiting factor on the character sets possible in inter-bank communication. However, this has started to improve with the introduction of SWIFTNet, which is capable of supporting any character set by transporting information in UTF8 encoded format. (Unfortunately, at present this capability cannot be used cross-border, due to money laundering restrictions.) Support has also come from vendors such as Oracle, which supports UTF8 format as of release 7 of its database.
Unfortunately, the global banks have not as yet followed this lead, though some have instituted partial workarounds instead. A case in point is the sort of local arrangement offered by banks in regions such as Asia, where a double byte solution is needed to cope with the language characters. These solutions typically consist of a local vendor database that is linked to the customer’s payment order by a vendor number. Therefore if the bank has to print a cheque or remittance advice on behalf of its client it takes the vendor name and address details in the local language characters from this database. While this addresses the immediate problem, it also leaves the paying corporation with the not inconsiderable task of having to create and maintain the vendor database.
Another temporary solution adopted by some corporations is to use a translation table, which uses a combination of Latin characters to represent a phonetic equivalent of the local character. For example, in the case of the Philips payment factory the letter “o” with an umlaut has the umlaut replaced by the letter “e”. So an “o” with an umlaut would appear as “oe”. However, this is not a viable long-term solution, and it is also incapable of handling the far more complex problems posed by Asian languages where the character sets are entirely different.
The solution to the local language character conundrum really comes in two parts:
In the first case, EDI standards (both EDIFACT and XML) must be extended to specify which data elements can be specified in local language characters and which cannot:
There has been some progress on this already, in that UNCEFACT PAYEXT v4 already contains
UNICODE capabilities. However, this version is very unlikely to be implemented by banks, as they have already made clear their preference for XML.
Secondly, banks must adapt their payments processing systems to:
Some banks are definitely becoming aware of this issue, but a major effort on the part of the banks will be needed. The primary stumbling block remains bank back office systems. In many cases these are several decades old, which makes the systems changes needed to accommodate local language characters a major task.
It is nevertheless a very necessary task and one that will become ever more necessary as cross border payment traffic continues to rise. Ironically, in some respects banks have already embarked on this journey, as the XML payment message types they are now implementing already support local language characters. Therefore at the customer interface level they have already arrived – the next and final step is to complete the process with their internal systems.
1 The IST, which combines the efforts of IFX2, OAGi3, SWIFT4 and TWIST5, has defined an XML standard for corporate to bank payment initiation and status messages.