Creating a Google Docs Lens - Instructors: Neil Patel

   1 === ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - || Support in #ubuntu || Upcoming Schedule: || Questions in #ubuntu-classroom-chat || Event: App Developer Week - Current Session: Creating a Google Docs Lens - Instructors: njpatel
   2 [17:00] <dpm> thanks Satoris for a great session!
   3 [17:00] <ClassBot> Logs for this session will be available at following the conclusion of the session.
   4 [17:01] <njpatel> Hi! My name is Neil Patel, I'm the System Architect in the Canonical Desktop Experience (DX) Team and I am also the Technical Lead for the Unity project. I'd like to talk today about the massive changes we've done to the Lens infrastructure in Unity this cycle, with an introduction to writing a simple Google Docs Lens & Scope!
   5 [17:01] <njpatel> Although I have a rough outline for at least the first half of this session, feel free to ask questions in #ubuntu-classroom-chat as you have them, and I'll answer as soon as it makes sense :)
   6 [17:01] <njpatel> I should add, all the information here, and much more, will be available on later on today and across the weekend.
   7 [17:02] <njpatel> So, a quick history on Lenses: They started off being named "Places", and they were, in Natty, little daemons that would show up on your Launcher and allow you to search through Applications/Files/AskUbuntu/Gwibber etc.
   8 [17:02] <njpatel> Although Lenses in that form were pretty good, we wanted them to be more powerful (as did the Lens authors!) so for Oneiric we decided that we should swallow our pride and re-archtect them to provide more features, and make it easier to add new features later on i.e. make a base API that we can carry forward through 12.04 and beyond.
   9 [17:02] <njpatel> So, people who are currently re-writing their Places as Lenses, sorry :) We promise it won't be this bad again!
  10 [17:02] <njpatel> One of the largest changes we made to the Lenses was to split out the concept of what controls the renderering/filters of a Lens to what provides data. In essence, we wanted to be able to make a Lens be able to just define it's properties, but then let anything be able to provide data to the Lens.
  11 [17:03] <njpatel> So, the job of a Lens is to create an Lens instance, add some categories, add some filters and thats it, it no longer does any searching of it's own. So what does the searching? Scopes!
  12 [17:03] <njpatel> Scopes are the engine of the Dash. They plug into Lenses over D-Bus and then provide data to the Lens.
  13 [17:03] <njpatel> This is best described in an example: We now ship a Music Lens by default in Oneiric. It ships with a Banshee Scope which provides search results for Banshee. However, instead of being stuck with a usesless Lens because you use Amarok or Rhythmbox (or you having to create an alternative Lens for your favourite music player), you can just create the Amarok or Rhythmbox Scope for the Music Lens.
  14 [17:03] <njpatel> The Scope would plug right into the Lens, providing results from Amarock in the main Lens.
  15 [17:04] <njpatel> Also, a Lens is not confined to having only one Scope. The Music Lens could easily have a Banshee Scope, a GrooveShark Scope, a Spotify Scope etc. So you can search multiple data sources from one Lens, just install the ones you like :)
  16 [17:04] <njpatel> Due to not many people likely to want to ship a Lens by itself without a Scope (wouldn't be very useful!) we have made it easy for you to ship a default Scope with your Lens, without having to create an extra daemon, and you'll see this later on.
  17 [17:04] <njpatel>  
  18 [17:05] <njpatel> The largest user-visible change we made for a Lens is it can now create Filters
  19 [17:05] <njpatel> Filters allow the user to easily, er, filter the search results
  20 [17:05] <njpatel> We ship with four filters by default: CheckOption, RadioOption, MultiRange and Ratings
  21 [17:06] <njpatel> You can create as many as you'd like, and show/hide them as needed
  22 [17:07] <njpatel> The filters API is meant to be easy to use and we hope to add more filters in the 12.04 cycle
  23 [17:07] <njpatel> (we'd love to hear ideas!)
  24 [17:08] <njpatel> woah, lag :)
  25 [17:09] <njpatel> Andy80 asked: how do we control how the data found by a scope is displayed? The Lens provide the filters, ok... but can I design the view and incorporate it into the dash?
  26 [17:09] <njpatel> The Lens controls how the results are displayed per-category
  27 [17:09] <njpatel> So, we ship with a vertical or horizontal orientation for the results, but we really would like to see more
  28 [17:10] <njpatel> The complexity comes in that the results are rendered by Unity, and so must be written with Nux
  29 [17:10] <njpatel> We'd happily accept new renderers and are planning to add some whizzbang ones for 12.04!
  30 [17:11] <njpatel>  
  31 [17:11] <njpatel> On that point, the two most important things a Lens will do is add Filters and Categories to itself
  32 [17:12] <njpatel> We spoke about filters (and will come back to that in the example), so i should cover Categories
  33 [17:12] <njpatel> Categories are headings inside the dash when you search for something inside a Lens
  34 [17:13] <njpatel> e.g. the Apps lens has "Most Frequently Used", "Installed" and "Apps Available for Download"
  35 [17:14] <njpatel> When you want to add a result to the results model inside a Scope, you tell Unity which category to put the result in. If a Category has no results, Unity will not show it
  36 [17:14] <njpatel> We normally add three Categories, but there is no limit
  37 [17:14] <njpatel> three just seems to fit nicely in the way Unity renderers the Dash :D
  38 [17:15] <ClassBot> Andy80 asked: how do we control how the data found by a scope is displayed? The Lens provide the filters, ok... but can I design the view and incorporate it into the dash?
  39 [17:15] <njpatel> woops
  40 [17:15] <ClassBot> Andy80 asked: I suppose we will have to write two different Lens (one for Unity and one for Unity-2D) but since the Scope exposes data trough an API/d-bus ecc... is the same Scope usable on both versions of Unity?
  41 [17:16] <njpatel> Any Lens you write works fine with both Unity and Unity-2D, they share the same code for talking to Lenses
  42 [17:17] <njpatel> This is because a Lens does not contain any specific UI code (i.e. gtk or Qt or Nux), it just requests that a category should use the horizontal tile, for instance, and it's up to the Unitys to render it correctly
  43 [17:18] <njpatel> It's actually one of my favourite things about Lenses/Scopes, they are purely data with requests for renderering, the actual renderering is handled elsewhere and therefore can be changed to match it's environment (i.e. searching your lenses from a webpage ;)
  44 [17:18] <njpatel> <Andy80> njpatel: so they can be written in any language? C, C++, Python, Vala ecc...?
  45 [17:18] <njpatel> Yep
  46 [17:19] <njpatel> Due to this boundary between the renderering-side (Unity) and the data side(Lenses or Scopes), you can write them in any language
  47 [17:19] <njpatel> libunity (the library you use to write them in) is a GObject library with full GObject introspection support
  48 [17:20] <njpatel> it also means, C++ Unity can have a Vala Applications Lens which has a Python Freedesktop-menus Scope and a Javascript Web Apps Scope :)
  49 [17:20] <njpatel> we like that :D
  50 [17:21] <njpatel>  
  51 [17:21] <njpatel> Okay, so the other important thing is for Unity to actually find the Lens
  52 [17:22] <njpatel> for this, you need to choose a "id" for your lens i.e. Music lens = "music", Google Docs Lens = gdocs
  53 [17:22] <njpatel> you use that id and install your .lens file in /usr/share/unity/lenses/$id/$id.lens
  54 [17:22] <njpatel> This is an example .lens file:
  55 [17:23] <njpatel> We use DBus activation to start a lens, so you also need to install a .service file into /usr/share/dbus-1/services
  56 [17:24] <njpatel> with the .lens file, Unity can start up, have a look at what lenses are installed, be able to load their icon etc, and then, when the user does  a search, activate the Lens daemon (which will in turn activate it's scopes)
  57 [17:25] <njpatel> if you are writing a Scope, you have a Scope file which only contains the DBusName and DBusPath attributes and you install it in the Lens's folder in /usr/share/unity/lenses
  58 [17:26] <njpatel> you'll also need a dbus .service file as Scopes are also started with dbus activation
  59 [17:26] <njpatel>  
  60 [17:26] <njpatel> As I mentioned earlier, we recognise that there are not many Lenses that people would like to ship which don't have at least one Scope readily available to make it useful
  61 [17:27] <njpatel> What we didn't want was to  have you jump through hoops to create .scope files, .service files and a new binary just for this, so we have a little function that allows you to ship a Lens daemon with an in-built Scope easily
  62 [17:28] <njpatel> this does't stop other Scopes from plugging into your Lens, but just lets you get up-and-running much more easily
  63 [17:28] <njpatel>  
  64 [17:29] <njpatel> So, for this session I had originally wanted to write a  Scope for the Music lens, but ran into some difficulty with getting an API key for an OSS project (urgh), so I decided to be a bit daring and started a Google Documents Lens + Scope using Python (a language which we did not have support for until this morning!)
  65 [17:30] <njpatel> The code is available at lp:unity-lens-gdocs
  66 [17:30] <njpatel> but I have some pastes of the a few stages of the code which I thought would be useful to go over
  67 [17:31] <njpatel> So, the first commit I did looked something like:
  68 [17:31] <njpatel> I should say, excuse any bad python in there, I'm a C/C++ guy :)
  69 [17:32] <njpatel>  
  70 [17:32] <njpatel> So, from line 27
  71 [17:32] <njpatel> You can see we create the Lens object
  72 [17:33] <njpatel> hopefully the properties we set are obvious enough
  73 [17:33] <njpatel> but you have some top-level control here over your Lens, like whether you want it to show in the global search results
  74 [17:33] <njpatel> what the search hint is, etc etc
  75 [17:33] <njpatel> then the first two important things are done
  76 [17:34] <njpatel> populate_filters() does exactly what it says on the tin
  77 [17:34] <njpatel> As in the comments, I love the RadioOption filter as it's one of the easiest to use :)
  78 [17:35] <njpatel> CheckOption, RadioOption and MultiRange filters are all "OptionFilter" sub-classes in libunity. This type of filter is basically one which can have multiple options in it, and an option can be active or inactive
  79 [17:36] <ClassBot> Andy80 asked: you defined both scope + lens in that single python code?
  80 [17:37] <njpatel> Yep, I'm using the lens.add_local_scope() method to have my Lens be automatically useful when installed without having to also ship a separate scope in the package
  81 [17:37] <njpatel> this does mean that my lens and scope are in the same process, and this might be okay for some lenses and not okay for others, you would need to judge that yourself
  82 [17:38] <njpatel> due to this Lens being very much tied to a single service, having the scope ship with the lens and in the same process is fine
  83 [17:38] <njpatel> My goal is to make gdocs lens also ship a scope for the files lens, if you'd just like to search all files in one lens
  84 [17:38] <njpatel> but this is a good enough example for now  :)
  85 [17:38] <njpatel>  
  86 [17:39] <njpatel> So, the filters you add here are synced with Unity (so it knows what to display) and also down to the Scopes (so they can query them during search)
  87 [17:39] <njpatel> I'll show you how the Scope uses the filters during a search later on
  88 [17:40] <njpatel> As mentioned, check, radio and multi-range work very similarly, the Ratings filter is the simplest, providing you with a rating of 0.0 to 1.0 depending on how many stars the user has clicked
  89 [17:41] <njpatel> Using the visible property on a filter, you should be able to show/hide filters depending on which other options are clicked
  90 [17:41] <njpatel> be a bit careful, though, i don't think either Unity has a nice transition for that change yet, so it might surprise the user a bit!
  91 [17:42] <njpatel>  
  92 [17:42] <njpatel> so, in populate_categories() we add the categories we'd like to use
  93 [17:43] <njpatel> for each category, we can choose and icon, set it's name and choose how we'd like it rendered
  94 [17:44] <njpatel> The order in which you add these categories is the numbering you use in the Scope when adding a result. So, for the first category, the integer you'd use when adding the result would be 0
  95 [17:44] <njpatel> usually an enum would be best to make your code readable for others and yourself down the line!
  96 [17:45] <njpatel> coming back up, if you ignore line 46 & 47, calling lens.export() would get you ready for Scopes that want to plug in and ready fro Unity to start showing/search your lens
  97 [17:45] <njpatel> if you plan on making a Lens and separate scopes, this is all you need to do
  98 [17:46] <njpatel> however, for our GDocs lens we want to also provide some results from GDocs (as we're not really expecting anything else to plug into such a specific Lens)
  99 [17:46] <njpatel>  
 100 [17:47] <njpatel> so, looking at lines 46 & 47, we can see we create a class that internally creates a Scope, and then inform our Lens that a "local scope" exists i.e. tell it that there is a scope that doesn't have a .scope file and cannot be activated, but is actually in the same process as itself
 101 [17:48] <njpatel>  
 102 [17:48] <njpatel> Okay, so looking at the UserScope class
 103 [17:48] <njpatel> My idea is to give this Lens the ability to handle multiple google accounts (say work + personal), so i've encapsulated the logic of talking to a single account into one class and one scope
 104 [17:49] <njpatel> therefore, when I do add support for multiple accounts, I'll be creating a UserScope for each, and calling lens.add_local_scope() for each too
 105 [17:49] <njpatel> the start of the __init__ of the class mostly deals with the gdata bits
 106 [17:50] <njpatel> line 129 onwards does the fun stuff, namely it creates the Scope by telling the Unity.Scope class where to export it's internal dbus object
 107 [17:50] <ClassBot> There are 10 minutes remaining in the current session.
 108 [17:50] <njpatel> after that, it connects to two signals to know when the normal search and global search changes
 109 [17:52] <njpatel> we keep them different, including two different results models, as we fully expect the Lens to behave a bit differently with global search versus with normal search. A good example of this is that a normal search will take into account filters but the global one won't
 110 [17:52] <njpatel> moving on, you can see there are some methods to grab the search string and then just one method to actually update the results, either global or normal
 111 [17:53] <njpatel> in update_results_model(), we have a placeholder that just grabs all the documents and throws it in the results model, it doesn't deal with the search string yet
 112 [17:54] <njpatel> so, the next thing would be to get it to do that
 113 [17:54] <njpatel>
 114 [17:54] <njpatel> that is the diff for making the Scope  actually search
 115 [17:54] <njpatel> as you can see, we now take into account the global search (though as we are not doing any filtering we can still keep it simple when searching)
 116 [17:55] <njpatel> the rest of the diff is mostly just constructing a good gdata search URI
 117 [17:55] <njpatel> running out of time!
 118 [17:55] <ClassBot> There are 5 minutes remaining in the current session.
 119 [17:55] <njpatel> moving on, this shows us responding to filters
 120 [17:56] <njpatel> if you look at apply_filters()
 121 [17:57] <njpatel> You can see that, for each filter that we know/care about, we query the scope to get the Filter object, and then we query the filter to see what the current active option is
 122 [17:57] <njpatel> however, we only do this if we are not doing a global search
 123 [17:58] <njpatel> the "id" property of the filter is the same as what the Lens set when creating the filter. Through this mechanism, you can always be sure you are reacting to the correct filter
 124 [17:59] <njpatel> also you can see I changed the filters a bit so I could be a bit lazy with ids :)
 125 [17:59] <njpatel> so, we're at the end, it would have been good to have more time but I'll be updating the wiki page with lots more info
 126 [17:59] <njpatel>
 127 [18:00] <njpatel> also, I'll be continuing to develop this lens if you want to subscribe to the branch on Launchpad to keep up!
 128 [18:00] <dpm> thanks a lot njpatel, that was awesome!

MeetingLogs/appdevweek1109/CreatingGoogleDocsLens (last edited 2011-09-12 15:34:25 by bl18-252-24)