Author: Dorileo

GSoC: Spice Payload report

Yeah! it`s came the time to write another report on GSoC status. In fact I`ve – intentionally – postponed it for quite some time and it doesn`t exactly mean there was a lack of informative emails between me and Marc(my mentor).

The need to finish some stuffs justifies – in some ways – the aforementioned delay. I understand you don`t need to report you aren`t done with something, a mail stating “I`m not done yet” would be enough – well, maybe not anyway…

OpenEmbedded Journey

With the second half of my project I jumped in the OpenEmbedded ecosystem and believe it, I`ve loved to get in touch with.

Putting my hands on OE is something I`ve planned for some time, I just hadn`t had the time to do so.

OpenEmbedded is something amazing, and it does what I realized years ago when I worked with gentoo. I always saw gentoo as a great meta-distribution, something you can bend and forge as you need – customizing it according to your needs.

Despite all the conceptual things touching OE wasn`t as easy as I initially tought it would. Bitbake(the great maestro behind OE) was designed with portage in mind and theoretically it was I good advantaged to me – look, theoretically.

Nothing is exactly smooth as you plan, you`ll always get troubles in the way – with OE wasn`t an exception.

OE transitions and yocto project

One of the biggest problems I faced was mainly due the transition the OE project is getting through. The docs(Getting started wiki page for example) are out dated and you get conducted by the old code base, and trust me, it`s not a good way to get started.

My first two weeks was full of crazy hacks, searching for old tarballs, setting up local source repositories, doing everything I could to make that thing to work – it was a bad race doing my best to proof the howtos.

The true is OpenEnbedded has moved to what we name bblayer, it`s a bitbake feature to ease to extend a base system. The intention(as I see) is to keep a minimal, clean and stable set of core packages and yet make it possible to “third party” vendors to append it to fit their needs.

The yocto project has extensively used OpenEmbedded as their base system, both the projects have exchanged a lot and sometimes you loose yourself if you`re touching one or the other. One of the tools provided by yocto project is Poky – which`s actually an OE layer.

There isn`t plenty of docs describing how the bblayer and bbappend work – the bitbake docs aren`t much precise and the OpenEmbedded barely mention it, yocto just describes how it`s fit within poky(or something close to that).

I would really like to recommend newcomers to first play with poky then later consider starting a new third-party layer.

The project as a bblayer

A third party layer is what best fits my project, not exactly a full yocto/poky layer, maybe and extension of it or not even that but an own layer itself, to accomplish that I had to experiment a lot, setting the environment up and watching how everything get together.

Packaging

After many years not touching a single ebuild and having never touched a bb package I jumped in the task to pack some components. The spice client has a bunch of dependencies – of course, I hadn`t to pack everything myself, a great number of things were already done.

Among the things that got me longer then I expected was cyrus-sasl, the old OE tree had it packaged but it was an old version – should I mention it was broken as well?

So, bringing the recipes wouldn`t be enough but I would have to fix things up, once fixing stuffs was the only alternative I decided to upgrade it to the latest version 2.1.31.

Anyway, it brought me a lot of work to pick patches to fix its building and fixing what hadn`t got fixed already. My final PR was 177 what means I came through 177 builds, debugging, testing and working everything around.

The cyrus-sasl code has a bug introduced after 2.1.21, it wasn`t possible to build it –with-static. I did an ugly and ridiculous fix. Everything I found out there – searching the internet – was even uglier. Suggestions to run make twice was one of them. The build system was kind of messed up.

The other packages weren`t so painful and I could quickly move forward.

Slimming the image

I still have to slim few things, I need to cut some X11 packages I included in the image, append the yocto kernel with my own .config and write a small shell script(or something smarter than that) to launch the spice client.

BuildRom

The first thing I worked on in the beginning of my project was buildRom, I wanted to bring all the tasks involved on building the OS image and bios/firmware into to it. But, with my move to OE for building the OS image I realized I could go the reversed way and bring the tasks for building the bios/firmware to OE.

Now I`ve manually packaged the things but have already started to write bbclass to controll bios/firmware + image building and packaging them. I see it as a second generation to BuildRom project, a OE layer with coreboot bb packages and recipes plus the needed bb classes.

Conclusion

After the great effort I had, getting in touch by the first time with OE, I feel comfortable to say it was a good experience to me, I realized many possibilities. I`m really happy with everything I learned on the path and I`m sure I still have a lot to contribute to Coreboot and OE as well.

GSoC2011: coreboot spice payload, OE and rootfs

As stated in my midterm report I’m working this first 2 weeks after midterm to work on building the payload image. I wanted to hardcode everything but we(my mentor and I) understood it would be better to use something like OpenEmbedded for that.

 
This first week I studied OpenEmbedded. I tested many distributions starting with angstrom, they showed too heavy for our purpose. After those tests I found micro and realized it would be a better start.

 
With that I’m working around micro to build our own distro, I’m adding X11 dependencies and right now I’m packing the spice client and its dependencies. In the end of the week I must have sorted every details.

GSoC2011 spice payload midterm report

My project mostly involves to build an image with a small linux systemrunning a spice client, the system is attached  with/as a LAB tocoreboot.

What was done

My buildrom tree received some updates and small changes to the building system. The bigest problem to solve was make it build coreboot with the new [k]build system. For that I changed build rom to incorporate the coreboot options so the user can configure every aspects of coreboot – all from buildrom menus. 🙂

For prototyping I used a chroot with a debian bootstrap(debootstrap), this surely doesn`t fit well to the project but I didn`t much attention for that, I thought I could leave it for second half – in the last days talking to my mentor I realized I should have paid more attention to the image things.

With a debootstrap I hacked I small script to package copy that root to my final image, what, once again, doesn`t fit our needs and requirements and here comes the tasks to be taken in the second half.

Second half

The project has been designed to be fully installed in/loaded from flash, It`s loaded with filo(but like discussed with my mentor any other bootload would fit prefectly).

Busybox will be our base system and a x-server. To acomplish that I have to take and finish the following tasks:

  • design the building strategies which involves;
    • building the busybox(well, this one is already done once we have a busybox package in buildrom system – I just need to base on that);
    • select the needed x-server componentes(modules – Xlibs, Xprogs, Xvesa);
    • select the alsa packages;
    • include the spice client and its dependencies;
  • define the kernel features;
  • define how filo is about to load the final image;
  • define the best compression algorithm;
  • integrate the whole thing to buildrom;
  • alix board support;

The base building scripts will be packaged separetely and then integrated to buildrom like the other packages(i.e busybox, filo, grub and so on).

There`re just 6 weeks to the final evaluation and I`m leaving the alix board support to the last 2 weeks.

The build scripts and kernel prototyping will take the 2 first weeks, while the third one I`ll be working to put everything to be built by buildrom, the system can be tested as a proper system from an iso image till I get to integrate it to buildrom and package it as a coreboot payload.

spice payload: Before MidTerm GSoC report

My very first task on gsoc was to make buildrom work with latest codes from coreboot, with that I changed how it used to build coreboot. First I introduced the coreboot kconfig options to buildrom so the user can configure and choose the options for coreboot in the same mainmenu as buildrom.

 
Some small changes was introduced as a consistent pattern to name the kernel mk files and the possibility to checkout source code with https:// or svn://.
Updated some packages mk files to comply some patterns I introduced.

 
I still have some changes to introduce but I think it`s already usable. I might push the final bits in the next few days whenever I get a chance to work out my patches.

GSoC 2011: (week 2) Coreboot Spice Payload

I still haven’t had a good progress on my project, my first step is to work around the buildrom system.

I got to bring fully the coreboot Kconfig options to buildrom, since I don’t want to manually update it after coreboot updates I decided to write a small program to parse the coreboot Kconfig files. I found some api’s to help but I god stuck with that first task since none of them actually help me so much. A simple task became so complex and took me more time than I wanted(I started to update the api’s and change it acordingly to my needs, but well my project is not a kconfig api).

For a better and simpler solution I sat down and wrote a small shell script to run all the Kconfig’s and merge it into a single Kconfig.coreboot.

Now the next task is to change the building flow, how coreboot is fetched, patched and built.

For the final image and package I still want to get it all on the flash, I’m working on finding a solution to fit it as small as possible.

I found it would be interesting to use Kdrive to make the image smaller since it’s a minimal X server, but it’s unmaintained and we can build a thin X server from upstream.
For code sharing and analisys I setup a git repository so my mentor can comment on what I’m writing.
Now I have to move I bit faster.

GSoC 2011: Coreboot Spice Payload

This year I was accepted on gsoc – and I`m excited – to work in a project to make corebook a spice client. The idea is to use buildrom to pack everything.

Once buildrom has been unmaintained for quite some time now I`ll need to work around it before working on the spice bits.

 

Spice

Yeah! when I say spice I mean the remote and virtualized desktop protocol, originally developed by Qumranet now acquired by Red Hat.
The basic idea is to make a client run from a minimal environment. We`ll be working on a LAB(Linux as bootloader) solution.

 

Initial steps

As I said above, we need some work around buildrom system, the coreboot buildsystem has migrated to kbuild on the stable code and lots of things have changed with it.
Both buildrom and coreboot work with kbuild with that I`m writing a small python piece of code to parse coreboot`s Kconfig files and then arrange it into buildrom source code. I also need to 1) monitor the coreboot Kconfig files changes so I run the parser again and 2) change the coreboot build calls.
With all the kconfig arranged the buildrom user needs just to set the rom properties including the coreboot options(since those options are passed to coreboot build).

The kernel configs also need some attention it`s statically set for each board, I haven`t come with a better solution for this till now.

 

Hardware and new toys

I have ordered an ALIX.3D3, a flexyICE and a new desktop computer. The first 2 are comming from EU and will be dispatched later on May. The new desktop computer I get till the end of the week(before the gsoc bonding period end).

 

Now, let`s rock.