== App Developer Week -- Zeitgeist API & Zeitgeist Application Integration -- m4n1sh & seiflotfy -- Tue, Apr 12th, 2011 == {{{#!irc [18:01] Next up are m4n1sh (or m4n1sh_) and seiflotfy from the Zeitgeist project, who are going to talk about integrating Zeitgeist to your application === ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - https://wiki.ubuntu.com/Classroom || Support in #ubuntu || Upcoming Schedule: http://is.gd/8rtIi || Questions in #ubuntu-classroom-chat || Event: Ubuntu App Developer Week - Current Session: Zeitgeist API & Zeigeist Application Integration - Instructors: m4n1sh, seiflofty [18:01] Logs for this session will be available at http://irclogs.ubuntu.com/2011/04/12/%23ubuntu-classroom.html following the conclusion of the session. [18:02] Hello everyone [18:02] My name is Manish Sinha [18:02] and seiflotfy is Seif Lotfy [18:02] we both work in Zeitgeist team [18:02] I am just giving a small introduction [18:03] Seif is the founder of Zeitgeist [18:03] and I work on integration and some engine work [18:03] Let us start seiflotfy :) [18:03] first a basic introduction [18:03] Zeitgeist is an event logger [18:03] which logs the events which happens on your computer [18:03] events can be opening files [18:03] closing files [18:03] creating files [18:04] connecting to internet [18:04] receiving a call [18:04] all the event logging doesnt happen automagically === ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - https://wiki.ubuntu.com/Classroom || Support in #ubuntu || Upcoming Schedule: http://is.gd/8rtIi || Questions in #ubuntu-classroom-chat || Event: Ubuntu App Developer Week - Current Session: Zeitgeist API & Zeigeist Application Integration - Instructors: m4n1sh, seiflofty, manish [18:04] zeitgeist is a daemon running on your computer [18:04] Logs for this session will be available at http://irclogs.ubuntu.com/2011/04/12/%23ubuntu-classroom.html following the conclusion of the session. [18:04] whose work is to get events [18:04] means log them [18:05] when some application sends the data to it [18:05] and applications can also query events from the daemon [18:05] it is like a event store [18:05] when we talk about integration, we mean creating small plugins/extensions/addons/addins which sends the relevant events to the daemon [18:06] so basically it is the plugins which push data [18:06] all the plugins which push events are called dataproviders [18:06] their work is to only push data [18:07] like whenever you open a note, a tomboy extension tells zeitgeist that the user opened the note [18:07] this extension is called an Event [18:07] the most important part of Zeitgeist is called an event [18:07] event is a packet of information which tells the daemon of "What happened" [18:07] take the literal meaning of event [18:08] each event instance has metadata contained with it [18:08] like timestamp [18:08] what happened [18:08] how it happened [18:08] any payload [18:08] which application was involved in the event etc [18:08] zeitgeist runs as a daemon and exposes an API via DBus [18:09] DBus is an interprocess communication software [18:09] it helps applications talk to each other [18:09] to inspect the zeitgeist API [18:09] you can install d-feet from the repositories [18:09] and open it [18:09] and search for zeitgeist [18:09] you will get something like this [18:09] http://i.imgur.com/DKx0G.png [18:09] the zeitgeist API looks like this [18:09] http://zeitgeist-project.com/docs/0.5.2/dbus_api.html#org-gnome-zeitgeist-log [18:10] let us start with the basic building block of zeitgeist [18:10] an Event [18:10] how event looks like is given here [18:10] http://zeitgeist-project.com/docs/0.6/dbus_api.html#index-0 [18:10] the first thing is an event Id [18:10] which recognizes it uniquely [18:10] 2nd is Timestamp telling when it happened [18:11] 3rd is interpretation [18:11] which means [18:11] “what happened” [18:11] 4th is Manifestation [18:11] which means [18:11] “how did this happen” [18:11] then we have an actor which looks like application://tomboy.desktop [18:11] which tells which application was involved in the event [18:12] this is the metadata for an event [18:12] each event can then contain something called as Subject [18:12] each event can have more than one subject [18:12] now we should go to what is subject contained of [18:12] URI which is something like file:///tmp/my.txt [18:13] Interpretation - the abstract notion of what the subject [18:13] Manifestation - the abstract notion of how the subject is stored or available [18:13] like the subject is stored as a File [18:13] Origin - the URI where the user accessed the subject from [18:13] Mimetype [18:13] which is something like text/plain [18:13] image/png [18:14] Text - a descriptive explanation [18:14] Storage - how it is stored [18:14] I know all these were a bit boring, but it is important for a developer to understand what an event actually is [18:14] to access Zeitgeist API [18:14] you have Python bindings [18:14] C/Vala bindings [18:14] C# bindings [18:15] so you can choose your language [18:15] if you are working on a application which needs zeitgeist integration and it is coded in Python then you can use the python binding of zeitgeist [18:15] you lets move to the API [18:15] this is the API [18:15] http://zeitgeist-project.com/docs/0.5.2/dbus_api.html#org-gnome-zeitgeist-log [18:16] lets not go so deep in the API [18:16] only a few methods [18:16] like InsertEvents [18:16] which inserts events in the daemon [18:17] GetEvents - which gets events from the daemon when you specify the event ids [18:17] And 3rd [18:17] FindEventIds - you can search for events [18:17] and get back the event Ids [18:17] FindEvents - you can search for events, and get the actual events instead of just the ID [18:18] when searching for an event you use the event template [18:18] it is like a pattern [18:18] the best part is that event template are events themselves [18:18] they are represented the same way [18:18] let me take you people to an example which will clear all the doubts [18:18] we will take a working example [18:19] our example uses python API [18:19] git.gnome.org/browse/rhythmbox/tree/plugins/rbzeitgeist/rbzeitgeist/__init__.py [18:19] please open this file [18:20] now I am teaching how to push events [18:20] not pulling [18:20] check line 36 and 37 of the link I have [18:20] *gave [18:20] from zeitgeist.client import ZeitgeistClient [18:20] from zeitgeist.datamodel import Event, Subject, Interpretation, Manifestation [18:21] we imported Evebt, Subject, Manifestation and Interpretation [18:21] now we have to determine the Interpretation for Event and Subject [18:21] similarly Manifestation for Event and Subject [18:22] Event Interpretation is either LEAVE_EVENT or ACCESS_EVENT [18:22] which means "what happened" [18:23] ACCESS_EVENT means that the track started [18:23] we are at the start of the event [18:23] means we started playing the track [18:23] LEAVE_EVENT means we left the track [18:23] means track finished [18:23] Event Manifestation: USER_ACTIVITY/SCHEDULED_ACTIVITY : 120/122 - "how did this happen" [18:23] How did it happen? [18:23] It happened via User Activity or Scheduled Activity [18:24] UserActivity means that the user himself pressed Next track [18:24] Scheduled means the next track in the queue started itself [18:24] check line 120 and 122 [18:25] the list of interprerations is here http://zeitgeist-project.com/docs/0.6/ontology.html#symbol-interpretation [18:25] and list of Manifestation http://zeitgeist-project.com/docs/0.6/ontology.html#symbol-manifestation [18:25] please choose one from these list [18:25] next is Subject Interpration [18:25] Subject Interpretation: AUDIO : 135 - "what is this" [18:25] check line 135 [18:25] What is this? It is Audio :) [18:26] akshatj asked: Are there guides available on how to write a dataprovider or plugin? [18:26] akshatj: there are definitive guide. This session is meant for that :) [18:26] next is Subject Manifestation [18:26] check line 62 [18:26] Manifestion for subject asks [18:26] "how does this item exist" === jhernandez is now known as jhernandez_afk [18:26] it exists as FILE_DATA_OBJECT [18:27] all these values are from the list of Manifestation [18:27] I will give you people one more example to study yourself [18:27] It is EOG Plugin [18:27] the file is [18:27] http://bazaar.launchpad.net/~zeitgeist-dataproviders/zeitgeist-dataproviders/trunk/view/head:/eog/zeitgeist_plugin.py [18:27] Hints are [18:27] Event Interpretation: MODIFY_EVENT/LEAVE_EVENT : 69/87 - "what happened" [18:27] Event Manifestation: USER_ACTIVITY : 70/88 - "how did this happen" [18:27] Subject Interpretation: IMAGE : 61 - "what is this" [18:27] Subject Manifestation: FILE_DATA_OBJECT : 62 - "how does this item exist" [18:28] Keep the questions coming people [18:28] More examples [18:28] Tomboy: [18:28] http://bazaar.launchpad.net/~zeitgeist-dataproviders/zeitgeist-dataproviders/trunk/files/head:/tomboy/ [18:28] Banshee: https://gitorious.org/banshee-community-extensions/banshee-community-extensions/blobs/master/src/ZeitgeistDataprovider/Banshee.ZeitgeistDataprovider/ZeitgeistDataprovider.cs [18:28] I would also like to defuse a few myths [18:28] Zeitgeist is not a file search engine [18:28] it does not track files [18:29] there can be events for zeitgeist which might not need a file [18:29] like you got disconnected from internet [18:29] or you recieved a call [18:29] I would like to recieve some feedback from you people before I advance to next part of the session [18:29] you can keep them coming on -chat channel [18:30] moving to next part [18:31] we have to search for events [18:31] for searching for events [18:31] we can look at the method [18:31] FindEvents [18:31] it contains [18:32] murphy asked: Can you please give an example of some context sensitive situation Zeitgeist helps with? [18:32] murphy: as seiflotfy gave you the link on -chat [18:32] http://www.youtube.com/watch?v=U6YOvVaRWh4 [18:32] this is an example [18:32] it contains [18:32] time_range [18:33] which tells you the range of time when you want to search [18:33] like you want to search for events only between today and 3 days back [18:33] event_templates [18:33] you can create event instances and pass it [18:33] the daemon will compare these templates to the stored events [18:34] and provide you with the results [18:34] storage_state [18:34] it is [18:34] Enumeration class defining the possible values for the storage state of an event subject. [18:34] http://zeitgeist-project.com/docs/0.5.2/datamodel.html#zeitgeist.datamodel.StorageState [18:34] contains 3 values [18:34] StorageState.NotAvailable [18:34] The storage medium of the events subjects must not be available to the user. (Integer value: 0) [18:34] StorageState.Available [18:34] The storage medium of all event subjects must be immediately available to the user. (Integer value: 1) [18:34] StorageState.Any [18:34] The event subjects may or may not be available. (Integer value: 2) [18:35] then we have [18:35] num_events [18:35] maximal amount of returned events [18:35] last one is [18:35] order [18:35] which asks in which order the results have to be returned [18:36] result type is an enumration [18:36] http://zeitgeist-project.com/docs/0.5.2/datamodel.html#zeitgeist.datamodel.ResultType [18:36] it has many values [18:36] have a look [18:36] this contains the full list of events [18:36] http://zeitgeist-project.com/docs/0.5.2/dbus_api.html#org-gnome-zeitgeist-log [18:37] now we have an example of how we can pull the events [18:37] we have a gedit plugin [18:37] here [18:37] http://paste.ubuntu.com/593228/ [18:37] written by seiflotfy and others [18:37] seiflotfy: can you explain this [18:37] sure [18:37] so [18:37] template = Event() [18:37] template.actor = "application://gedit.desktop" [18:37] here you can see an Event being created [18:38] and the actor of the event is being set to "gedit" [18:38] we then ask Zeitgeist for [18:38] "Get me most used files with gedit" [18:38] this will look like this [18:38] CLIENT.find_events_for_templates([template], handle_most, num_events = 100, result_type = ResultType.MostUsed) [18:38] where handle_most is a method for the callback [18:39] and then you get the "most used" files with gedit === ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - https://wiki.ubuntu.com/Classroom || Support in #ubuntu || Upcoming Schedule: http://is.gd/8rtIi || Questions in #ubuntu-classroom-chat || Event: Ubuntu App Developer Week - Current Session: Zeitgeist API & Zeigeist Application Integration - Instructors: m4n1sh, seiflofty, manish, seiflotfy [18:39] Logs for this session will be available at http://irclogs.ubuntu.com/2011/04/12/%23ubuntu-classroom.html following the conclusion of the session. [18:39] the whole code can be found http://paste.ubuntu.com/593238/ [18:40] if you people want a documentation of zeitgeist for mono/C# it can be found here [18:40] http://launchpad.net/zeitgeist-sharp/0.1/0.1.0.1/+download/zeitgeist-sharp-0.1-api-doc.tar.gz [18:41] and for C/Vala library [18:41] documentation is here http://people.canonical.com/~kamstrup/projects/libzeitgeist/doc [18:41] akshatj asked: So, dataprovider is used for pushing data to zg and plugin is used for pulling from it? [18:42] akshatj: dataprovider is out term for those plugins which only push events to the daemon [18:42] they dont pull [18:42] dataproviders are a subset of plugins [18:42] we have many dataproviders already [18:42] they are hosted here [18:42] https://launchpad.net/zeitgeist-dataproviders [18:43] you can see the list here [18:43] http://bazaar.launchpad.net/~libzeitgeist-developers/libzeitgeist/trunk/files [18:43] sorry [18:43] wrong link [18:43] http://bazaar.launchpad.net/~zeitgeist-dataproviders/zeitgeist-dataproviders/trunk/files [18:43] like [18:43] bzr, chrome, emacs, eog, firefox, chrome, geany, gedit, rhythmbox, telepathy, tomboy [18:44] vim xchat [18:44] banshee [18:44] banshee is upstream [18:44] not in this repo [18:44] we need more dataproviders [18:44] to make more sensible logging [18:44] or say more detailed logging [18:45] then we can have plugins which pull information [18:45] example.. I can show a snapshot [18:45] of how gedit source code you checked actually looks like [18:45] http://wiki.zeitgeist-project.com/images/2/2b/GEditOpenZeitgeist.png [18:45] if you use synapse [18:45] it shows results from zeitgeist [18:46] it asks zeitgeist for info [18:46] asked : this may be a silly question but, what is the purpose of zietguiest, and how is it different than syslog? [18:46] syslog logs system events [18:46] it does not log that you opened a file [18:46] syslog contains things like [18:47] Apr 12 23:17:01 Foo CRON[4852]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) [18:47] how much sense did it make to you :) [18:47] it stores info in unstructured format [18:47] does not have any dedicated API (as far as I know) [18:48] so what all you can do via Zeitgeist [18:48] you can have a complete list [18:48] of which file you opened when, closed when, saved when [18:48] started from when it was created [18:48] you can have a list of all the calls you recieved via empathy [18:48] when you recieved, when you dropped [18:48] when you got a call [18:48] the whole list [18:49] we people are still working in that area [18:49] we have telepathy work in progress [18:49] we need to integrate it even more [18:49] you can even list down all recent used files [18:49] sort them based on [18:49] 1) Application [18:50] 2) Which all were related to file creation [18:50] 3) Time Range [18:50] etc etc [18:50] Windows has a logging facility that was originally an attempt to store more structured log data that could be better parsed. Is that about the same idea here? So the idea is to replace .xsession-errors with a more structured log system? [18:50] psusi asked this question [18:50] I don't remember if windows had such a thing [18:50] I think it is there somewhere in Control Panel [18:50] I dont think anyone uses it [18:51] it is about Logs [18:51] same like we have in Linux [18:51] syslog, messages, kern.log etc [18:51] but zeitgeist is meant not only for storing data [18:51] There are 10 minutes remaining in the current session. [18:51] but having a nice API such that any application can get back the data [18:52] Zeitgeist is a thing under development [18:52] and its development is very fast [18:52] we all are learning a lot [18:52] so if you want to learn a lot, you can come to our channel #zeitgeist [18:52] try out some examples [18:52] and get your hands dirty [18:53] I am always there after 16UTC on that channel by this same nick [18:53] murphy asked: So if I copied a file to a server using nautilus I would be able to discover which server and what I copied a few days later (assuming nautilus informed zg)? Stuff for which there is usually no log... What about bash? [18:53] murphy: sadly bash events are not logged now [18:53] since we use GtkRecentManager for logging where plugins are not available [18:53] it is fallback logging [18:54] AFAIK bash doesnt inform GtkRecentManager [18:54] so it is not logged [18:54] we need a dataprovider for bash too [18:54] wrt nautilus [18:54] the source file is surely logged [18:54] nautilus does inform zeitgeist [18:54] using GtkRecentManager [18:54] and it gets logged to zeitgeist [18:54] you can come to know about it [18:55] wrt to "which server" depends on the URI [18:55] which is stored [18:55] zinga60 asked: how does the zeitgeist daemon store the events? in some db I guess? is it going to be more slow with more and more data providers and the longer the daemon runs (over months/years)? [18:55] zinga60: events are stored in sqlite database in ~/.local/share/zeitgeist/activity.sqlite [18:55] please dont try to open it maually [18:55] it is locked by the daemon [18:56] it *might* become slow and slow as millions of events fill up [18:56] but it will take lot of time [18:56] dataproviders dont make it slow [18:56] There are 5 minutes remaining in the current session. [18:56] as dataproviders run with the application [18:56] daemon can run for months and years [18:56] it is perfectly fine for it to work for long time [18:57] it doesnt leak memory [18:57] if it leaks memory, please inform us on #zeitgeist [18:57] yea, sounds like what MS originally intended the windows logging service to be, but it never really caught on and so nobody uses it today... only they had it apply to both system logging and user application logging. Zeitgeist has no intention of handling syslog events as well? Only desktop events? Or will there be a version on the system dbus as well as the session? [18:57] we handle mostly desktop events [18:57] which helps the *user* [18:57] our area of focus is desktop events [18:57] depends what apps push into it [18:58] even though zeitgeist can log any event [18:58] even system events [18:58] exactly [18:58] a user doesnt care much about system events [18:58] ubuntu one uses zeitgeist to log sync events [18:58] :) [18:58] closing time [18:58] please join us at #zeitgeist channel [18:58] we are happy to mentor anyone who wants to work with us [18:59] zeitgeist team is an extremely rocking and close-knit team [18:59] if you join us, you will really enjoy [18:59] take my word :) [18:59] we have a list of things we need to get done [18:59] so feel free to join [18:59] so there is no instance running on the system wide dbus? [18:59] zeitgeist runs as SessionBus which is user based [18:59] instead of SystemBus which is system wide [19:00] makes sense to run on per user basis [19:00] we have many many more things to do [19:00] lots of applications needs to be touched [19:00] we have a rush of adrenalin whenever we get something done :) [19:00] just for info.. if anyone of you know django we have a good work for you :) [19:00] feel free to poke me [19:01] esp if you know django-piston then you can start contributing to us very easily }}}