Emoji
Overview of Emoji
Emoji are the fun colorful characters commonly used in internet communications. They are made up of one or more Unicode characters. The Unicode Consortium publishes new major versions of the Unicode Standard once per year. As of September 12, 2023, the latest version is 15.1.
CLDR is a related dataset from the Unicode Consortium. The CLDR data gets updated for each new Unicode release but there are also other releases between Unicode versions. These other releases are less significant for emoji. The version of CLDR that corresponds with Unicode 15.1 is 44, released October 31, 2023.
Ubuntu uses Google's Noto Color Emoji font as the default color emoji font for all Ubuntu desktop flavors. The font is also used by default in Debian's Budgie, Cinnamon, and GNOME desktop environments. The font was updated in November 2023 to support Unicode 15.1.
Ubuntu 18.04 LTS was the first Ubuntu release to support color emoji, while Debian 10 was the first Debian release to support color emoji.
Upstream
Work can be done upstream to add support for new emoji releases.
Update GTK's emoji chooser
1. Read the instructions at https://gitlab.gnome.org/GNOME/gtk/-/blob/main/gtk/emoji/README.md
2. Find the corresponding emojibase-data git tag for the Unicode/CLDR release you want. Then clone the repo and checkout that tag.
git clone https://github.com/milesj/emojibase -b emojibase-data@15.3.0
3. Clone the GTK repo and create a new working branch
git clone https://gitlab.gnome.org/GNOME/gtk
4. Visit https://gitlab.gnome.org/GNOME/gtk and fork the repo if necessary
5.
cd gtk/gtk/emoji git checkout -B emoji-15
6. Open convert-emoji.c to find the instructions to build the convert-emoji script.
gcc -o convert-emoji convert-emoji.c `pkg-config --cflags --libs json-glib-1.0`
7. Run the script for every language code. This is for Bengali (bn):
./convert-emoji ../../../emojibase/packages/data/bn/data.raw.json bn.data
8. Commit your work
git add *.data git commit #Use a commit message formatted like https://gitlab.gnome.org/GNOME/gtk/-/commit/76a1f1714
9.Now push your branch to your fork of the GTK repo. Create a merge request.
10. After your merge request is merged, check out the latest stable branch for GTK4 (as of January 2024, it is gtk-4-12) and propose the cherry-picked change there. Also do this for the gtk-3-24 branch for GTK3.
Update GNOME Characters
1. Clone the repo and create a new branch
git clone https://gitlab.gnome.org/GNOME/gnome-characters cd gnome-characters/lib git checkout -B unicode-15.1
2. Visit https://gitlab.gnome.org/GNOME/gnome-characters and fork the repo if necessary
3. Edit regenerate.sh and update the CLDR, Unicode and Emoji version variables to the correct new versions.
4. Run ./regenerate.sh
5. Commit your work
git add . git commit -m "lib: Update to Emoji 15.1"
6. Now push your branch to your fork of the GNOME Characters repo. Create a merge request.
Distro
The upstream emoji support work should be packaged for Debian and Ubuntu.
fonts-noto-color-emoji packaging
1. Clone the Debian repository:
gbp clone https://salsa.debian.org/fonts-team/fonts-noto-color-emoji
2. Import the new release
gbp import-orig --uscan
3. Run dch -i to add a new debian/changelog entry and set the version number to the new version with a -1 suffix. Then
git add debian/changelog git commit -m "New upstream release
4. Build the package. A full explanation of git-buildpackage, sbuild, etc. is out of scope for this documentation.
gbp buildpackage
5. Install the resulting package. Close any apps that you want to test (including your web browser) then reopen them. Follow the test case.
6. If the package works well and you have upload and push permissions, upload and push the package. Otherwise, submit merge requests for review.
7. Sometimes, it may be necessary to update nototools first.
8. It is important to update fonts-noto-color-emoji before pushing other distro updates for emoji like GTK or GNOME Characters.
GTK packaging
1. When a new GTK3 or GTK4 version is released with updates to the emoji support, it is required to update debian/missing-sources/ or the new emoji support will not be available.
2. Import the new GTK version. This is for 4.12.5.
gbp clone --add-upstream-vcs https://salsa.debian.org/gnome-team/gtk4 cd gtk4 gbp import-orig --uscan --upstream-vcs-tag=4.12.5
3. Once you have the emojibase data from step 2 of Update GTK's emoji chooser, copy the emojibase data to debian/missing-sources
rm -r debian/missing-sources/emojibase/packages/data/* cp -r ../emojibase/packages/data/* debian/missing-sources/emojibase/packages/data git add debian/missing-sources git commit -m "missing-sources: Update emojibase"
4. For GTK4 only, also update the copy of the Unicode data. This data is currently not included in the GTK3 package. Unlike the emojibase data, it is not included in the build. Therefore, it is good practice to update this data, but not required for emoji support to work correctly. Make sure you have the right version of unicode-data installed before running these commands.
cp /usr/share/unicode/emoji/* debian/missing-sources/unicode* # Rename the git folder to match git mv debian/missing-sources/unicode.org-Public-emoji-14.0 debian/missing-sources/unicode.org-Public-emoji-15.1 git commit -m "missing-sources: Copy emoji 15.1 files from Debian's unicode-data package"
5. Build the package and install the packages.
6. Close any apps you wish to test. In participating apps, press Ctrl+. to bring up the emoji chooser. Search for one of the new emoji. Select it and verify that the emoji has been inserted correctly. gnome-text-editor is a good test app for GTK4. gedit is a good test app for GTK3.
7. Once the packages are verified to work, upload the new GTK packages like normal or submit merge requests for review.
8. The same basic steps apply to GTK4 or GTK3 (packaged as gtk+3.0). GTK2 does not include any emoji support.
9. It is also possible to update GTK before a new upstream release is tagged. You can use a git snapshot or cherry-pick git commits. If you cherry-pick git commits, the patches are binary patches and the patch file names will need to be added to a file debian/source/include-binaries for the package to be buildable. A git cherry-pick would be required for most GTK4 stable updates since the GNOME developers only support a GTK stable series for about six months.
unicode-data packaging
1. In Debian, new versions of unicode-data should be uploaded to Experimental and then reverse dependencies should be checked before asking the Debian Release Team to start the unicode-data transition. Documentation about Debian transitions can be found at https://release.debian.org/transitions/ . Several hours after uploading unicode-data to Experimental, an automatic tracker should show on that page showing the affected reverse dependencies.
2. In particular, these packages will need to be updated:
- golang-github-mattn-go-runewidth
- gucharmap
- node-unicode-data
- python-unicodedata2
- utf8proc
- wine
3. It may take several weeks after the initial release of a Unicode version for the upstream projects to be updated.
4. Because of the required package transition, it is likely not worth trying to backport new unicode-data releases to stable Debian or Ubuntu releases.
ibus
ibus has its own emoji chooser. On Ubuntu 23.10, the keyboard shortcut is Super+. Unlike GTK's emoji chooser, this is keyboard navigable only.
To include support for additional emoji, update the unicode-cldr-core package first, then rebuild ibus.
unicode-cldr-core appears safe to update without starting a transition or causing build issues in its reverse dependencies (unlike unicode-data).
fcitx5
TODO: fctix5 might work like ibus but it is unclear how to test how fcitx5 works.
other Unicode packaging
1. glib and harfbuzz are updated for Unicode changes. However, the changes are likely not required for new emoji to display correctly. Therefore, these changes do not need to be cherry-picked to stable glib release branches and likely do not need to be pushed to stable Debian or Ubuntu releases.
2. Years ago, it was sometimes needed to update pango for new emoji to display correctly. This has not seemed to be needed in recent Unicode updates.
SRU
1. It is useful to update emoji support for stable Ubuntu releases using the SRU process.
2. fonts-noto-color-emoji needs to be released to -updates and fully phased before other emoji updates should be allowed into -updates. You can upload other emoji updates to the proposed queue at the same time as fonts-noto-color-emoji but set the appropriate block-proposed tag (block-proposed-jammy for Jammy) in the bug reports for the other emoji packages to alert the SRU Team to not release those packages to -updates. After fonts-noto-color-emoji has fully phased, remove the block-proposed tags. Leave a comment on the bug when adding or removing block-proposed tags to explain why the action is being taken.
3. Examples of previously accepted SRUs for emoji packages:
GTK4 (technically, this was not an emoji update, but it gives an example of how new GTK4 releases are handled. If you do an emoji update, add a bonus test case for testing that the Ctrl+. emoji chooser works for inserting new and older emojis in gnome-text-editor.)
4. It might not be worth trying to backport new emoji beyond the most recent Ubuntu LTS.
5. The GNOME Characters app only supports modern emoji since Ubuntu 22.10.
6. There are almost no GTK4 apps packaged in Ubuntu 22.04 LTS.
Debian Stable Updates
1. Read the documentation
2. Unlike Ubuntu, Debian bugfixes are only released in a large batch on the stable release update day. The exceptions is for security fixes or for certain urgent non-security bugfixes. Neither of those exceptions apply here.
3. It is acceptable to update fonts-noto-color-emoji at the same time as other emoji updates since the updates all land simultaneously. Debian does not use a randomized phased update system.
4. Future scheduled point releases can be found at https://release.debian.org/
5. Accepted updates for the next Debian Stable release can be seen on the queue overview.
6. Examples of previously accepted SRUs for emoji packages:
GTK4 2.042-0+deb12u1 (technically, this was not an emoji update, but it gives an example of how new GTK4 releases are handled. If you do an emoji update, add a bonus test case for testing that the Ctrl+. emoji chooser works for inserting new and older emojis in gnome-text-editor.)
Other References
- https://blog.emojipedia.org/ - https://home.unicode.org/ - https://fonts.google.com/noto/specimen/Noto+Color+Emoji - https://blogs.gnome.org/mclasen/2017/08/18/post-guadec-distractions/
DesktopTeam/Emoji (last edited 2024-01-22 20:32:34 by jbicha)