thumbnailer

Dependents/Clients

Notes

If you want to clear the cache, you can run

  • $ thumbnailer-admin clear

It's a good idea to do this before starting the tests.

To see stats for the cache, run

  • $ thumbnailer-admin stats -v

The stats are for the three internal caches. The image cache stores full-size artwork that is expensive to retrieve (downloaded from the server, or extracted from an mp3 or video). The thumbnail cache contains scaled thumbnails. The failure cache keeps track of authoritative "no artwork available" answers, so the thumbnailer doesn't try to endlessly extract images from media that is known to not yield any artwork.

If you want to stress-test the thumbnailer with larger numbers of thumbnails, you can create hard links to a file (photo, mp3, or mp4). This saves having to copy tons of data around. The thumbnailer uses the file name as part of its key so, as far as it is concerned, all the thumbnails in these hard-linked files are different and cached separately.

To follow the operation of the cache, run

  • $ tail -f ~/.cache/upstart/dbus.log

By default, the log only shows cache misses and errors. If you want to see cache hits as well, run

  • $ gsettings set com.canonical.Unity.Thumbnailer log-level 2

Test Plan

Tests without network connection

  • Disconnect from the Internet (e.g. by switching the phone to flight mode)

  • Untar music.tar into the Music folder. It contains four tracks as follows:

    • Djaa (contains embedded art)
    • Money (contains embedded art)
    • Moribayassa (contains no art)
    • Time (contains no art)

  • Ensure the thumbnail cache is empty by running thumbnailer-admin clear.

  • Run the music app and go to the song list. Check that you see a thumbnail for Djaa and Money, but not for Moribayassa and Time.

  • Go to the artist list and check that there is no artwork for either artist.
  • Go to the album list and check that neither album shows artwork.
  • Try playing all four songs and check that the thumbnails are displayed correctly in their larger size as the song is playing.
  • Check in all sections: artists, tracks, albums... and check that only Djaa and Money have artwork.

  • Run the Music scope and check Tracks, Artists, and Albums sections. Verify that the same thumbnails appear or are missing as for the music app.

  • Drop testsong.mp3 into the Music folder. The album cover for this song is a circle of solid red on a transparent background. Find the song in the Album view and verify that the background around the circle is the same as the album background. Tap on the album to get the song list for the album. Verify that the background around the red circle now matches the background on that page, that is, verify that the area around the circle is indeed transparent.

thumbnailer-service

  • Check that the thumbnailer-service is running after any addition of images and right after opening my music. 
    •       $ adb shell
      $ ps aux | grep thumbnailer-service
      phablet  18586  0.8  0.9  87972  9664 ?        Sl   12:36   0:00 /usr/lib/arm-linux-gnueabihf/thumbnailer/thumbnailer-service
  • Wait 30 seconds without adding any image and check that the thumbnailer-service shuts down

Tests with network connection

  • Connect to the Internet again (e.g. turn off flight mode)

Warning /!\ Until we remove the fallback image support from the QML plugin and update users, you must kill all running apps from the app switcher and re-start the dash with restart unity8-dash. This clears Qt's texture cache and forces a refresh on scope pages that are pre-loaded during start-up.

  • Open the music scope and look at the tracks section. Check that you get thumbnails for all four tracks.
  • Check the albums section. You should see artwork for both albums.

  • Check the artists section. You should see artwork for Pink Floyd, but no artwork for Mamady Keïta.

  • For all thumbnails, tap on them to open the preview screen and check that the image has the right aspect ratio.

  • Open the music app and check that everything matches what you see in the scope. (The only artwork missing should be the artist art for Mamady Keïta.)

  • Display the song list in the music app and go to the Music folder and run 
    for i in $(seq 1 200)
do
    cp artwork-known-artist.mp3 ${i}.mp3
done
    After a few seconds, you should see the songlist refresh with all the new songs. (The artwork will be the same but, as far as the thumbnailer is concerned, these are different images that are cached separately.) Scroll forward a page or two. You should see the thumbnails gradually fill in. Scroll forward as fast as possible, all the way to end of the list. Then scroll backwards again, looking for the point at which the thumbnails start to display. You should be able to scroll down while the thumbnails are being filled in and follow progress that way. Scroll around again as fast as possible. Thumbnails should appear instantly with no noticeable delay. (You can follow progress in dbus.log to verify that the thumbnailer delivers cache hits.)
  • With the music app in the song list, go to the music folder and run 
    rm [0-9]*.mp3
    You should see the songlist update to the original four entries after a few seconds.

  • Untar music-no-artwork.tar.gz into the music folder with the music app in the song list. This drops 200 "songs" that contain a second of silence each into a subdirectory. The songs do not have embedded artwork but contain artist and album ID3 tags that are (mostly) known to 7digital. You should see the song list start to fill in with thumbnails after a few seconds.

    Run tail -f dbus.log in a terminal and scroll down just a page. You should see the thumbnails for the new songs appear within a few seconds. Now scroll down the end of the list as fast as possible. You should see the thumbnails for songs at the end of the list appear within 10 seconds or so. Now scroll back up again, reasonably quickly. As you scroll, you should see a few thumbnails, and sections of songs where no thumbnails are available yet. If you stop scrolling, again, the missing thumbnails should be filled in quickly (on the order of a few seconds, not half a minute). The trace in the log will tell you what happened for each request. You may see timeout errors in the log. If so, it just means that the server was slow. For these requests, you will end up getting the "no artwork" image. However, if you scroll away far enough (such that you see a lot of hits for the section you scrolled into) and then scroll back to the song that had the timeout error, the song should eventually get its image. You may also see messages "unexpected download error, disabling server access for 20 seconds". This happens when the server has a glitch and returns a "surprising" error that we don't expect to see in normal operation. If this happens, and you keep scrolling into unexplored territory, you will see lots of messages about "server access retry time not reached yet". After 20 seconds, normal operation should pick up again (unless the server is really ill). Eventually, you should see artwork for all songs, except song 98, for which 7digital does not have an image.

  • Go to the album list. You should see album art being displayed almost instantly because the thumbnailer scales these thumbnails from previously-cached full-size images.
  • Go to the artist list. You should see most of the images fill in as they arrive from the remote server. (There are a bunch of albums from Russian and Chinese artists towards the end for which 7digital does not have artist images.)

Videos and pictures

  • Copy a video in a supported format into ~/Videos. Take a 5-6 second video.
  • Open my video and check that the videos have a thumbnail
  • Copy an image file to ~/Pictures and take a picture with the camera
  • Open the gallery app and check that all pictures have the expected thumbnail.
    • Verify that thumbnail images have correct aspect ratio.
    • Tap on a picture to bring it full screen, verify that there are no issues

  • Record a very short video file (<1 s) with the camera app and check that thumbnail for it is shown (1391286)

Cache

  • Check the local cache directory, and that its size is modified before and after adding and showing results in my music or my video. 
    • $ ll ~/.cache/unity-thumbnailer/images
$ ll ~/.cache/unity-thumbnailer/thumbnails
    You can check this after any addition and after updating the results in my music or my video.
  • Erase the thumbnailer cache 
    • $ adb shell
$ thumbnailer-admin clear
  • Reboot the device
  • Open my music and my video and check that thumbnails are OK.

Security

The thumbnailer can produce thumbnails of local files on behalf of an application. It should not return thumbnails for content an application can not access.

  • Log into the phone with adb shell

  • Clear the thumbnail cache by running thumbnailer-admin clear

  • Determine the gallery app by running sudo aa-status | grep gallery. It should look something like com.ubuntu.gallery_gallery_2.9.1.1183 (the version number might be different).

  • Copy an image to ~/.local/share/com.ubuntu.music/image.jpg

  • Try to produce a thumbnail running under the gallery's AppArmor profile (substitute in the correct profile name): 

    • $ cd ~/.local/share/com.ubuntu.gallery/
$ aa-exec -p com.ubuntu.gallery_gallery_2.9.1.1183 \
   thumbnailer-admin get ~/.local/share/com.ubuntu.music/image.jpg .
    An error message should be printed indicating that permission is denied.
  • Create a thumbnail for the image while unconfined: 
    • $ thumbnailer-admin get ~/.local/share/com.ubuntu.music/image.jpg .

    This should produce a file image_0x0.jpg.

  • Now that the image is in the cache, try producing a thumbnail again as the gallery profile: 
    • $ aa-exec -p com.ubuntu.gallery_gallery_2.9.1.1183 \
   thumbnailer-admin get ~/.local/share/com.ubuntu.music/image.jpg .
    This should still result in an error.
  • Now try creating a thumbnail using a symlink to the image: 
    • $ ln -sf ../com.ubuntu.music/image.jpg .
$ aa-exec -p com.ubuntu.gallery_gallery_2.9.1.1183 \
   thumbnailer-admin get ~/.local/share/com.ubuntu.gallery/image.jpg .
    This should also result in a permission denied error.

Process/Merges/TestPlan/thumbnailer (last edited 2016-03-02 06:37:16 by localhost)