Support for Malayalam (ml) Language - Hyphenation

[gcafepgdi]

This is to request the Malayalam language hyphenation support in Typst.

"Malayalam is one of the main languages spoken (by 40 million people) in South India, with a rich cultural and literary history. It is one of the oldest languages in India with classical language status."

Issue

Malayalam text output by Typst has too much spacing between some words.

Sample in Typst

Sample Text in Latex with Proper hyphenation and justification

References:

• https://tex.stackexchange.com/questions/668159/typesetting-malayalam-in-xelatex-lualatex-gives-error
• https://thottingal.in/blog/2014/07/20/typesetting-malayalam-using-xetex/
• https://github.com/libindic/hyphenation/blob/master/libindic/hyphenation/rules/hyph_ml_IN.dic

[laurmaedje]

We excluded Indic languages because we couldn't find any evidence that hyphenation was actually used with them in practice. Perhaps that wasn't true. Do you have any sources of non-TeX related places where Malayalam is hyphenated?

[tilman-schieber]

If you don't speak the language It is hard to find evidence of hyphenation as hyphens are not commonly used. The words are just broken at end of line without any symbol.
Firefox and Chromium have added support for hyphenation in indic languages, see for example here.
Here is a blog that documents that indesign, scribus, libreoffice and so on all support hyphenation for indic languages.

The author of this blog also has a repo with all his hyphenation patterns for indic languages:
https://github.com/santhoshtr/hyphenation

[gcafepgdi]

I can confirm that the latest version of Typst (Version 0.12.0) significantly improves the rendering of Malayalam text. While there is still some minor extra spacing between certain words, this issue is negligible. Additionally, the Unicode rendering of Malayalam text is now nearly identical to that of Scribus, and likely comparable to other renderers utilizing the Harfbuzz engine.

For reference, the same sample text shared in the initial post of this thread has been re-rendered using Typst v0.12.0 and is included here to demonstrate the substantial improvements.

Thank you to the Typst team for their efforts.

[santhoshtr]

I have filed this PR #30 to add Malayalam and many other Indic languages