== Dev Week -- Introduction to LAVA -- Zygmunt Krynicki -- Tue, Jul 12th, 2011 == {{{#!irc [19:01] welcome everyone :) [19:01] I'm glad to be able to tell you something about LAVA today [19:02] my name is Zygmunt Krynicki, I'm one of the developers working on the project [19:02] feel free to ask questions at any time, check the topic for instructions on how to do so [19:02] okay [19:02] let's get started [19:02] So first off, what is LAVA? [19:03] LAVA is an umbrella project, created by Linaro, that focuses on overall quality automation [19:04] you can think of it as a growing collection of small, focused projects that work well together [19:05] the overall goal of LAVA is to improve quality that developers perceive while working on ARM-based platforms [19:05] we're trying to do that by building tools that can be adopted by third party developers and Linaro members alike [19:06] okay [19:06] As I mentioned earlier LAVA is a collection of projects, I'd like to enumerate the most important ones that exist now [19:06] first of all all of our projects can be roughly grouped into two bins "server side" and "client side" [19:07] where client is either client of the "server" or a unrelated non-backend computer (like your workstation or a device being tested) [19:08] the key project on the server is called lava-server, it acts as a entry point for all other server side projects [19:08] essentially it's an extensible application container that simplifies other projects [19:09] next up we have lava-dashboard - a test result repository with data mining and reporting [19:09] lava-scheduler - scheduler for "jobs" for lava-dispatcher [19:09] lava-dispatcher - automated deployment, environment control tool that can remotely run tests on ubuntu and android images [19:10] the last one is really important and is getting a lot of focus recently [19:10] essentially it's something that knows how to control physical devices so that you can do automated image deployment, monitoring and recovery on real hardware [19:11] on the client side we have a similar list: [19:11] lava-tool is a generic wrapper for other client side tools, it allows you to interact with server side components using command line instead of the raw API exposed by our services [19:11] lava-dashboard-tool talks to the dashboard API [19:12] lava-scheduler-tool talks to the scheduler API [19:12] and most important of all: lava-test, it's a little bit different as it is primarily an "offline" component [19:12] it's a wrapper framework for running tests of any kind and processing the results in a way that lava-dashboard can consume [19:13] all of those projects are full of developer friendly APIs that allow for easy customization and extensions [19:13] we use the very same tools to build additional features [19:13] lava-test is also important because it is a growing collection of wrapper for existing tests [19:14] using our APIs you can easily wrap your test code so that the test can be automated and processed in our stack [19:14] some test definitions are shipped in the code of lava-test but more and more are using the out-of-tree API to register tests from 3rd party packages [19:15] if you are an application author you could easily expose your test suite to lava this way [19:15] okay [19:15] that's the general overview [19:15] now for two more things: [19:15] 1) what can LAVA give you today [19:15] 2) how can you help us if you are interested [19:16] While most of our focus is not what typical application developers would find interesting (arm? automation? testing? ;-) [19:16] some things are quite useful for a general audience [19:16] you can use lava-server + lava-dashboard to trivially deploy a private test result repository [19:17] the dashboard has a very powerful data system, you could store crash reports, user-submitted benchmark measurements, results from CI systems that track your development trees [19:18] in general anything that you'd like to retain for data mining and reporting that you (perhaps) currently store in a custom solution that you need to maintain yourself [19:18] all of lava releases are packaged in our ppa (ppa:linaro-validation/ppa) and can be installed on ubuntu lucid+ with a single command [19:19] the next thing you could use is our various API layers: you could integrate some test/benchmark code in your application and allow users to submit this data to your central repository for analysis [19:20] if you are really into testing you could wrap your tests in lava-test and benefit from the huge automation effort that we bring with the lava-dispatcher [19:21] in general, as soon as testing matters to you and you are looking for a toolkit please consider what we offer and how that might solve your needs [19:22] during this six month cycle a few interesting thins are planned to land [19:22] first of all: end user and developer documentation [19:23] overview of lava project, various stack layers, APIs and examples [19:23] jykae asked: any projects that use successfully lava tools? [19:23] jykae, linaro is our primary consumer at this time but ubuntu QA is looking at what we produce in hope for alignment [19:24] jykae, next big users are ARM vendors (all the founding members of linaro) that use lava daily and contribute to various pieces [19:25] jykae, finally I know of one big user, also from the ARM space, expect some nice announcement from them soon - they are really rocking (with what they do with LAVA and in general) [19:25] jykae, but I hope to build LAVA in a way that _any_ developer can just deploy and start using, like a bug tracker that virtually all pet projects have nowdays [19:25] jykae, we need more users and we will gladly help them get started [19:26] ok, back to the "stuff coming this cycle" [19:26] so documentation is the number one thing [19:26] another thing in the pipe is email notification for test failures and benchmark regressions [19:26] this will probably land next month [19:27] we are also looking at better data mining / reporting features, currently it's quite hard to use this feature, this will be somewhat improved with good documentation but we still think it can be more accessible [19:27] the goal is to deliver a small IDE that allows users to do data mining and reporting straight from their browsers [19:27] this is a big topic but small parts of it will land before 11.10 [19:28] finally we are looking at some build automation features so that LAVA can help you out as a CI system [19:29] and of course: more tests wrapper in lava-test, more automaton (arm boards, perhaps x86) [19:29] jykae asked: do you have irc channel for lava? [19:29] jykae, yes, we use #linaro for all lava talks [19:30] jykae, a lot of people there know about it or use it and can help people out [19:30] jykae, also all the core developers are lurking there so it's the best place to seek assistance and chat to us [19:30] ok [19:30] so [19:30] a few more things: [19:31] 1) I already mentioned our PPA, we have a policy of targeting Ubuntu 10.04 LTS for our server side code [19:31] you should have no problems in installing our packages there [19:32] if you want more modern system (we also support all the subsequent ubuntu releases, except for 11.10 which will be supported soon enough) [19:32] if you want most of the code is also published on pypi and can be installed on any system with pip or easy_install [19:32] 2) We have a website at http://validation.linaro.org where you can find some of the stuff we are making in active production [19:33] The most prominent feature there is lava-server with dashboard and scheduler [19:33] (the dispatcher is also there but has no web presence at this time) [19:34] There is one interesting thing I wanted to show to encourage you to browse that site more: http://validation.linaro.org/lava-server/dashboard/reports/benchmark/ [19:34] this is a simple report (check the source code button to see how it works) that shows a few simple benchmarks we run daily on various arm boards [19:35] there are other reports but they are not as interesting (pictures :-) unless you know what they show really [19:36] another URL I wanted to share (it's not special, just one I selected now): http://validation.linaro.org/lava-server/dashboard/streams/anonymous/lava-daily/bundles/7c0da1d8765e806102c6f8a707ff22b99a43c485/ [19:36] this shows a "bundle" which is the primary thing that dashboard stores [19:37] bundles are containers for test results [19:37] from that page click on the bundle viewer tab to see how a bundle really looks like [19:38] in the past whenever we were talking about "dashboard bundles" people had a hard time understanding what those bundles are and this is a nice visual way to learn that [19:38] okay [19:38] one more thing before I'm done [19:38] what we'd like from You [19:39] 1) Solve your problem, tell us about what you need and how LAVA might help you reach your goal (or what is preventing you from using it effectively), work with us to make that happen [19:40] 2) Testing toolkit authors: consider allowing your users to save test results in our format [19:41] 3) Application authors: if you care about quality please tell us what features you'd like to see the most [19:41] 4) Coders: help us implement new features, we are a friendly and responsible upstream [19:41] okay [19:42] that's all I wanted to broadcast, I'm happy to answer any questions now [19:45] nobody into quality it seems :-) [19:49] okay, guess that's it -- thanks everyone :-) }}}