Unicode versus legacy Hindi fonts: why your text turns into gibberish

You send a Hindi file to a friend and it opens as a mess of strange symbols. Or you paste text from one program to another and the letters break. The cause is almost always the difference between Unicode and legacy fonts.

Legacy fonts

Before Unicode became common, Hindi was typed using special fonts such as Kruti Dev and others. These fonts do something clever and fragile. When you type a key, the text file stores an ordinary English letter. The font then draws that letter as a Devanagari shape.

So a file might actually contain what looks like English nonsense, and only the presence of the right font makes it appear as Hindi. If you open it on a computer without that font, you see the nonsense.

Problems that follow:

• The file cannot be searched, since the computer does not know it is Hindi.
• Copying text to another app breaks it.
• Spell checkers cannot work on it.
• Converting it to other formats can destroy it.
• Publishing online requires a conversion.

Unicode

Unicode is a universal standard. Every Devanagari letter has its own proper code, whatever font displays it. The text itself is genuinely Hindi, and any Unicode font can show it.

Benefits:

• It works across devices and programs.
• It is searchable and can be spell checked.
• It can be moved between software without trouble.
• Modern typesetting tools expect it.

What to do

For any new work, type in Unicode. On a phone, add a Hindi keyboard. On a computer, you can use the built in Hindi keyboard or a phonetic input tool, which lets you type in Roman letters and converts to Devanagari.

If your manuscript is in a legacy font, convert it to Unicode before it goes to a typesetter or an editor. Free converters exist, but the result needs checking carefully, because conversion can introduce errors, particularly with matras and conjuncts.

How to tell which you have

Take a short passage and paste it into a plain text editor. If it still shows as Hindi, it is Unicode. If it shows as English letters and symbols, it is probably a legacy font.

Check after converting

After conversion, read the whole text, or at least a good sample. Look for:

• Matras attached to the wrong letter.
• Missing or extra halant signs.
• Letters substituted by lookalikes.
• Broken conjuncts.

Ask a Hindi reader to look at it too.

Keep a clean master

Once your text is in Unicode, keep one master file. Make changes in it, and generate everything else, such as the typeset book and any web version, from it. This stops different versions drifting apart.

A note for printers

If a printer asks for your Hindi in a legacy font, ask why. Most can accept Unicode and a properly embedded font in a PDF. Embedding the font means it travels with the file and displays correctly everywhere.

Moving to Unicode takes an afternoon and saves countless problems later.