Webkit

Ubuntu Opportunistic Developers Week March 2010 - Web browsing and rapid UI with WebKit - Ryan Paul - Mar 5 2010

(03:01:33 PM) seg|ars: Hello!
(03:01:49 PM) seg|ars: I'm going to be discussing application development with WebKit
(03:02:28 PM) seg|ars: For those who don't already know, WebKit is an HTML rendering engine that is developed by Apple. It was originally based on KDE's KHTML renderer
(03:02:57 PM) seg|ars: WebKit is lightweight and easy to embed, which makes it useful for integrating HTML rendering capabilities into conventional desktop applications
(03:03:32 PM) seg|ars: There is a Gtk+ variant of WebKit that uses Cairo for drawing and can be used seamlessly in standard Gtk+ and GNOME programs
(03:04:15 PM) seg|ars: It's really easy to use if you want to display a web page in your application
(03:04:31 PM) seg|ars: but it's also increasingly useful for building rich user interfaces
(03:05:20 PM) seg|ars: If you want to present content in more dynamic way in your application, HTML can sometimes be a very compelling shortcut
(03:05:32 PM) seg|ars: Making a custom Gtk+ widget can be hard, but writing HTML is very easy
(03:06:19 PM) seg|ars: Gwibber, my open source microblogging client, uses WebKit and HTML for several aspects of its user interface
(03:06:55 PM) seg|ars: http://s3.amazonaws.com/scrnshots.com/screenshots/225884/gwibber-230png
(03:07:21 PM) seg|ars: The main message stream in Gwibber is rendered with HTML, allowing us to have sophisticated theming
(03:07:51 PM) seg|ars: But that's not the only part that's HTML. The new sidebar and the target bar at the bottom of the window are also rendered with WebKit
(03:08:30 PM) seg|ars: WebKit supports some really sophisticated CSS features that make it conducive to interface design
(03:09:15 PM) seg|ars: For example, it supports CSS gradients, transparency, reflections, and even animated transitions
(03:09:50 PM) seg|ars: You can see several of those features in the previous screenshot
(03:11:25 PM) seg|ars: The Gtk+ port of WebKit is maintained by the WebKitGtk+ developers: http://webkitgtk.org/
(03:11:41 PM) seg|ars: the port is under very heavy active development and is gaining new features with every release
(03:12:27 PM) seg|ars: Some of the key stakeholders in the porting effort recently gathered for a hackfest where they worked on some key improvements
(03:12:48 PM) seg|ars: You can find more details about the hackfest here: http://arstechnica.com/open-source/news/2010/01/webkitgtk-hackfest-improves-html-renderer-for-gnome-apps.ars
(03:13:30 PM) seg|ars: They are adding a lot of great HTML5 features, such as HTML5 video playback via GStreamer.
(03:14:19 PM) seg|ars: You can use WebKitGtk+ in several different programming languages thanks to the availability of bindings
(03:14:40 PM) seg|ars: Python is my programming language of choice, so that's what we will be using for the code examples during this session
(03:15:15 PM) seg|ars: The Python bindings are available through the PyWebKitGtk project: http://code.google.com/p/pywebkitgtk/
(03:16:01 PM) seg|ars: The lead developer behind that project recently halted activate maintenance, so those bindings will not be used much in the future
(03:16:31 PM) seg|ars: PyGi, which provides Python bindings to Gtk+ libraries through GObject-introspection, will be used in the future
(03:16:57 PM) seg|ars: PyGi is already largely usable, but it wasn't quite mature enough yet for us to use in an LTS release
(03:17:25 PM) seg|ars: As such, Gwibber still uses the conventional PyWebKitGtk+ bindings in Lucid but will be using PyGi in the next major release
(03:17:45 PM) seg|ars: Fortunately, it's not that difficult to switch from one to the other. In fact, you can write code that supports both, as I will demonstrate later
(03:18:40 PM) seg|ars: In our first code example, you can see how to instantiate a WebKit widget and embed it into a Gtk+ application: http://gist.github.com/323077
(03:19:21 PM) seg|ars: The line "web = webkit.WebView()" will create the WebKit renderer
(03:19:44 PM) seg|ars: You can use the WebView widget's "open" method to cause it to load a web page
(03:20:05 PM) seg|ars: in the example, the WebView is embedded in a ScrolledWindow so that the user can scroll around the content
(03:20:37 PM) seg|ars: Notice the gtk.gdk.threads_init() call at the beginning of the script. You need to do that *before* you instantiate a WebView, otherwise the program will crash
(03:21:03 PM) seg|ars: That code snippet is all you need if you just want to embed a web page in your application
(03:21:28 PM) seg|ars: But there are also many cases where you want to generate the HTML yourself, like we do in Gwibber
(03:21:48 PM) seg|ars: to make the WebKit renderer display your own HTML, you use the load_html_string method.
(03:22:06 PM) seg|ars: web = webkit.WebView(); web.load_html_string("<p>test</p>", "file:///")
(03:22:18 PM) seg|ars: The first parameter is an HTML string that you want to display in the renderer
(03:22:43 PM) seg|ars: The second parameter is the base path that you want to use for the html content
(03:23:05 PM) seg|ars: specifying a specific base path can be useful if you want to include images or other content in the html using relative paths
(03:23:36 PM) seg|ars: When you use WebKit to make a user interface, there are several tricks that you can use to make the experience more seamless
(03:23:51 PM) seg|ars: This code example demonstrates several of those tricks: http://gist.github.com/323080
(03:24:13 PM) seg|ars: By default, an image that is embedded in a WebView widget can be dragged around the screen and out of the program
(03:24:28 PM) seg|ars: You can actually see this in action if you open up the software center and start dragging the big icons in the main pane
(03:24:38 PM) seg|ars: That's obviously an undesirable behavior in a regular user interface
(03:24:58 PM) seg|ars: Fortunately, WebKit has a custom CSS attribute for controlling that behavior
(03:25:13 PM) seg|ars: if you set "-webkit-user-drag: none;" on your images, they will not be draggable
(03:25:27 PM) seg|ars: we had to do that to avoid issues in the Gwibber navigation bar
(03:25:47 PM) seg|ars: Another similar issue is that all content in a WebView is selectable, just like text content in a browser
(03:26:00 PM) seg|ars: that might not be desirable on something like a button label
(03:26:30 PM) seg|ars: when you want to inhibit text selection of content in a WebView, you can apply "-webkit-user-select: none" in the css
(03:27:03 PM) seg|ars: You can see both of those in the code example that I pasted
(03:27:24 PM) seg|ars: If you run that code, it will display a button with a shiny gradient and an icon
(03:27:57 PM) seg|ars: The user-select and user-drag CSS attributes are used to make it so that the icon can't be dragged or text-selected
(03:29:00 PM) seg|ars: When you use WebKit in a Gtk+ application, you will probably want to make it use colors from the user's Gtk+ theme
(03:29:14 PM) seg|ars: we do that for the navigation elements in Gwibber so that they fit better with the rest of the program
(03:29:35 PM) seg|ars: for example, it's really important to make sure that the background of the web page matches the color of the user's gtk window background
(03:29:52 PM) seg|ars: You can see how to do that in the last code example
(03:30:24 PM) seg|ars: In the HTML, we use string interpolation so we can programmatically set a custom value for the page "background" attribute
(03:30:46 PM) seg|ars: In the Python part of the code, you can see a line where we get the background color and assign it to the "bgcolor" variable
(03:31:05 PM) seg|ars: You can extract style information from any gtk widget by using w.get_style()
(03:31:21 PM) seg|ars: to get the background, we do "w.get_style().bg[gtk.STATE_NORMAL]"
(03:31:44 PM) seg|ars: The state part tells Gtk+ what state you want to get the color from
(03:31:56 PM) seg|ars: if you wanted to get the selection color, for example, you could use gtk.STATE_SELECTED
(03:32:37 PM) seg|ars: In addition to "bg" you can also get the foreground ("fg") and text colors ("text")
(03:32:50 PM) seg|ars: QUESTION: What does WebView refer to exactly? Rendering of any url, or any piece of html code? I'm confused.
(03:33:11 PM) seg|ars: The WebView is a WebKit rendering widget. It's like an HTML renderer that you can plop into a Gtk+ application
(03:33:27 PM) seg|ars: you can use it to load either a remote url or to display any specified html content
(03:34:19 PM) seg|ars: ok, back to the color setting
(03:34:59 PM) seg|ars: In the code example, we use "%s" as a placeholder in the html code to denote the place where we are going to insert the color data
(03:35:19 PM) seg|ars: when we apply the html, we use "html % bgcolor", which will replace that "%s" with the actual value of the bgcolor variable
(03:36:03 PM) seg|ars: if you were going to programmatically set multiple values in various places in the html, you can use a more sophisticated approach like Python's string "Format" method. It will let you do more complex interpolation and value replacement
(03:36:19 PM) seg|ars: In Gwibber, we needed even more control over the html
(03:36:36 PM) seg|ars: we use a template engine called Mako that lets you generate HTML with bits of code interspersed
(03:36:55 PM) seg|ars: You can learn more about Mako here: http://www.makotemplates.org/
(03:37:19 PM) seg|ars: Mako is really great for theming because you can create a set of programmable templates that can be inherited and overridden
(03:37:50 PM) seg|ars: In Gwibber, we supply a base template that defines functions for rendering all of the items in the message stream. Each theme inherits that base template and overrides parts of it to create their own desired look and feel
(03:38:30 PM) seg|ars: This is what the Gwibber base template looks like: http://bazaar.launchpad.net/~gwibber-committers/gwibber/trunk/annotate/head%3A/ui/templates/base.mako
(03:39:00 PM) seg|ars: You can also do some programming in the HTML itself
(03:39:39 PM) seg|ars: in Gwibber, for example, we use the jquery library to make it so that the user can expand grouped messages. It has a nice animated transition
(03:39:52 PM) seg|ars: Using jquery in embedded HTML can be really useful for building a rich UI
(03:40:20 PM) seg|ars: One of the big challenges of using an embedded HTML renderer in a Gtk+ application is facilitating communication between the contents of the HTML and the outside program
(03:40:31 PM) seg|ars: you want your program to be able to respond to click events inside of the html
(03:41:07 PM) seg|ars: In our code example (http://gist.github.com/323080) we use navigation events to do that
(03:41:55 PM) seg|ars: You can see that the button in the html is wrapped in an "a" tag that points to "program:/test" as its URL
(03:42:33 PM) seg|ars: What I tend to do when I want to detect a click event in WebKit is intercept all navigation events and filter the ones that have a pre-determined prefix
(03:42:59 PM) seg|ars: in the code example, the on_click_link function gets bound to the "navigation-requested" event
(03:43:05 PM) seg|ars: web.connect("navigation-requested", on_click_link)
(03:43:42 PM) seg|ars: what this means is that any time something causes a navigation action in the WebView, like if somebody clicks a link, that Python function will be called
(03:44:03 PM) seg|ars: In the Python function, we analyze the URI to see if it starts with "program:/"
(03:44:15 PM) seg|ars: when it does, we look at what the rest of it says and then we perform whatever action
(03:44:48 PM) seg|ars: you could have a set of if/elif checks on the value of the rest of the URI after "program:/" to determine exactly what the program should do
(03:45:51 PM) seg|ars: for example, in Gwibber, clicking the reply icon in a message bubble will load the url "gwibber:/reply?msg=${msgid}"
(03:46:15 PM) seg|ars: The template library is used to replace the ${msgid} part with the actual id of the individual message
(03:47:02 PM) seg|ars: when the user clicks that icon, the python side of the program will detect it and then execute gwibber's reply function with the specified id value as a parameter
(03:47:15 PM) seg|ars: QUESTION: navigation-requested is a webkit event, right?
(03:47:24 PM) seg|ars: yes, navigation-requested is a webkit event
(03:47:51 PM) seg|ars: By default, if you don't set a function to the navigation-requested signal, clicking a link will cause the url to load inside of the WebView
(03:48:28 PM) seg|ars: in most cases where we are using WebKit for a user interface, we want to capture those clicks to perform an event and we want regular links to load in the user's external browser
(03:49:19 PM) seg|ars: You can see that in action in the code example. When the URI doesn't match "program:/" or anything else we are tracking, we just pass the whole thing to python's "webbrowser.open" method so that it will launch in the user's browser
(03:50:06 PM) seg|ars: As you can see, using WebKit makes interaction a little bit more complex, but you get a tremendous amount of control over how content is presented
(03:50:24 PM) seg|ars: and it's great for rapid development because it's often easier to write HTML than to build a real Gtk+ UI
(03:50:32 PM) seg|ars: now for some questions before we finish up
(03:50:40 PM) seg|ars: QUESTION: How do I refresh a webkit display?  Say a website pane that displays the weather.
(03:51:19 PM) seg|ars: The WebView has a reload method that you can use to get it to refresh the current contents
(03:51:44 PM) seg|ars: If you wanted to do it on an interval, you would use something like gtk.timeout_add
(03:51:58 PM) seg|ars: and then you could just call the refresh every time the timeout hits
(03:52:13 PM) seg|ars: If you are setting the HTML yourself, refreshing will cause the contents to disappear
(03:52:49 PM) seg|ars: so if you are using your own html, you just have to set the content with load_html_string every time you want to refresh it
(03:53:38 PM) seg|ars: QUESTION: can you embed JS too? You mentioned JQuery, what about JS?
(03:54:00 PM) seg|ars: Yes, it works just like JavaScript in a regular web page
(03:54:15 PM) seg|ars: if you are generating your own html, you can include a script block that defines JavaScript functions
(03:54:49 PM) seg|ars: you can also externally cause the WebView to run any arbitrary JavaScript code by using the "execute_script" method
(03:54:57 PM) seg|ars: if you pass it javascript code in a string, it will just work
(03:55:53 PM) seg|ars: You can seen an example of how I use JavaScript with Mako in the Gwibber navigation bar: http://bazaar.launchpad.net/~gwibber-committers/gwibber/trunk/annotate/head%3A/ui/templates/navigation.mako
(03:56:27 PM) seg|ars: QUESTION: In Gwibber for example, how do you attach the latest content? Reload everything or you just add the new info to the end? Is the 2nd option possible?
(03:56:46 PM) seg|ars: In Gwibber, we regenerate the entire stream. We do that primarily because we need to update the timestamps in addition to adding new messages
(03:57:05 PM) seg|ars: if you wanted to just add a new bit of content at the top, for example, you could actually do that with javascript
(03:57:31 PM) seg|ars: in the very early days of gwibber, we would take the message data and convert it into a json string which was then passed to a JavaScript function in the HTML
(03:57:47 PM) seg|ars: the JavaScript function would process the json, generate the HTML for the message blocks, and then inject that directly into the page
(03:58:09 PM) seg|ars: That approach proved to be a bit difficult and slow with highly complex data, but I think it would probably work better now
(03:58:23 PM) seg|ars: WebKitGtk+ was quite brittle a few years ago when I first started using it
(03:58:34 PM) seg|ars: it's a lot more robust today and it's javascript handling is more reliable
(03:58:44 PM) seg|ars: QUESTION: Since all these frameworks are undergoing rapid development, how do I keep my new application from breaking in a month?
(03:59:22 PM) seg|ars: that can be very difficult. One of the biggest problems we have had in Gwibber is that WebKitGtk+ is a moving target and different versions render things in slightly different ways
(03:59:44 PM) seg|ars: if you are going to be targeting a wide range of distros including some with very old webkit, you will need to keep the html VERY simple and avoid using scripting
(03:59:57 PM) seg|ars: it has stabilized a lot, so it's not as much of a problem as it used to be

MeetingLogs/OpWeek1003/Webkit (last edited 2010-03-05 21:18:55 by pool-71-182-100-128)