Author: Antonello Dettori

[GSOC] Panic Room, Recap

Hello everyone, in the following post I’m going to recap all that I’ve managed to accomplish during the GSoC of this year.

As a disclaimer: I plan to maintain these patches until they are fit to be mainlined or rejected altogether in case of design flaws.

libpayload: replace cbfs_media api with region_device api

This patchset should be complete, I tested it quite thoroughly by using the API to update the bayou payload (separate patch). I'm still waiting on a review on this one so I suspect there will be some cause for nitpicks.
bayou: Adapt to the coreboot-v4 era

Changed the design of the payload to make it integrate better with the current infrastructure of coreboot. The majority of the changes have been documented in the appropriate wiki entry. The patchset is complete and working.
console/serialice: Add SerialICE

The patchset was started and the initial work was done by Patrick Rudolph.
I picked it up from there, fixed the problems that it had and tried to get it in shape. It currently works as expected, waiting on more review/feedback.
serialice: update QEMU to version 2.5
serialice: adapt serialice to work with QEMU stable-2.5

In its current state this patchset produces a working build, all the functionality seems to be intact and SerialICE produces the same output as with the old implementation. There could be some unforeseen problems with the changes in the QEMU infrastructure and it could use a cleanup.
[WIP] src: replace device_t type with pnp/pci_defn_t

This is one of the latest commits that I've worked on, it's still a work in progress but I plan to remove all the the improper uses of device_t and replace it with the appropriate type. It's probably gonna take a while and I suspect the patch is going to be massive. If anyone has any suggestion on how to handle this transition feel free to tell me.
tint: Fix tint and add Kconfig option

The payload works as intended. Only nit is that maintaining a separate patch to apply to the tint code is tricky/messy.
lib/selfboot: Replace rdev_mmap_full()

The code worked, even though it's been a while since I rebased the change to check for conflicts. Should be an easy fix anyway.
payloads: add support lz4 compression

selfboot: add sequential lz4 payload decompression

The functionality works as expected.
coreinfo: Add support to read timestamps

cbmem: share additional time stamps IDs

Code works as intended, the only thing missing are the commas that should split the timestamps in groups of 3 digits. I couldn't port that part since it relied on a recursive function that no longer worked.
elfwriter: Fix multi-phdrs ELFs parsing

cbfstool: Require "-m ARCH" to extract payloads and stages

cbfstool: Extract payload in ELF

One of the oldest patchset that I wrote, took a while to get it right but it seems to be working wonderfully.
region: Add writeat and eraseat support

Below are a bunch of minor patches that I’ve written to fix the bugs that I’ve encountered while working on the patches above or browsing the source tree:

i945.h: fix #include path

emulation/qemu-i440fx: add cmos.default file

nvramcui: refactor code

pc80/mc146818rtc.h: Replace leftover macro token

lenovo/x60: add GPIOs initialisation before dock check

serialice: update lint filters

payloads: add Kconfig option for bayou

libpayload: split "Drivers" config section in Kconfig

bayou: delete pbuilder utility

filo: Specify libpayload path

cbfstool: Allow to easily build the individual tools

If you have any question regarding these patches you can find me on IRC (avengerf12).

I wanna conclude this post by expressing my gratitude towards the coreboot’s GSoC admins and mentors for the wonderful opportunity that they have given me and to the coreboot community for all the help and suggestions.

Sincerely,

Antonello

[GSOC] Panic Room, week #8,*

What have you worked on during the past few weeks?

I tried to finish one of the optional goals of my proposal:

a way to access a payload (possibly a recovery) after the hardware initialisation is complete but the OS has still not taken over.

To define it in more practical terms, I wanted to replace the running payload at the press of a predefined button.

I started by looking into the SMM (System Management Mode) and in particular the SMI (System Management Interrupts) handlers defined by each target board in the coreboot tree.

That was the easy part, it took just a bit of probing my board (Lenovo x60) and a serial cable to retrieve some of the SMI button codes (I planned on using the ThinkAdvantage button, code 0x19).
I added it to the appropriate smihandler.c file, defined the behaviour and that was about it.

The more difficult part was actually implementing a way to make it possible to boot a new payload while in SMM.
SMM can be considered a separate “module” from romstage, etc, this consequently means that it does not have access to all of the same functions.
In particular it cannot access the load_payload() and run_payload() functions defined inside prog_loaders.c, which, as the name implies, are necessary in order to use a payload.

In order to solve the problem I came up with two designs:

  • The first design is quite simple, but in my opinion a bit messy.
    It involves including all of the missing source files into the SMM module at compile time, a.k.a adding more entries inside the Makefile.inc scattered around the source tree.
    I went down this route for a while until I stumbled on a function that needed to use a malloc(), this was expected so I included the definition of that function.
    It then asked to have access to the heap (_eheap) in order to allocate the memory requested, the problem is that the heap is not defined inside SMM.
    I grepped through the source code and found very little about what I was supposed to do in order to define it, so I had to let it go.
    (To add a little bit of context, this was yesterday and if you take a look at this post’s date you’ll realise that there is only 1 day left of GSoC, panic!)
  • The second approach consists of leaving SMM mostly as it is now, and instead find a way to communicate with ramstage, in order to boot the desired payload.
    The thought process was that, if cannot call a function directly from SMM, I would have to use the functions already in place in ramstage, unfortunately those functions are called only once, after the initialisation is done.
    This is where the CMOS memory comes in, I modified the payload loading process and instead of using an hard-coded path defined at compile-time, I made it check for a string inside CMOS and use that as the payload path.
    This method, however, also presented some problems:
    – It seems that currently cmos.defaults is completely ignored and the CMOS string values are not initialised as intended (at least on my board), therefore the payload path resulted always blank.
    – There needs to be quite a bite of space in the CMOS to keep the path there.
    The usual path (fallback/payload), occupies around 160 contiguous bits of the 1024 available ones, which also need to share space with other important configuration bits such as RTC, etc.
    In order to circumvent these limitations it would be possible to replace the string path in the CMOS with an enumeration of the possible paths, this however would need to be defined, maintained and shared both in the CMOS and the rest of coreboot.
    However, same as above, I ran out of time.

That about sums it up for the past few weeks.

Thank you for reading and have a nice week.

[GSOC] Panic Room, week #7

What have you procrastinated on this past week?

As usual, I’m glad you asked.

During the past week I worked every day(/night) on the porting of the region_device API (commit) as I probably mentioned last week. The port is now complete and it should be bug-free (?).
It’s been tested pretty thoroughly since I immediately used it to adapt Bayou from the old LAR/coreboot-v3 paradigm to the new CBFS/coreboot-v4 one.
A bunch of bugs cropped up with the new API during the transition (debugging the API wasn’t fun at all) but they have been solved and unless for some corner-cases that I missed it should just work.

The second part of the week, as you might have guessed read, has been devoted to fixing Bayou (commit). This has been quite fun, almost everything was broken or relied on some old function that has since been dropped from libpayload.
It now works quite well and supports all of the original features.
The only problem now are with the other payloads since they aren’t all able to exit cleanly and give back control to bayou (i.e. coreinfo reboots before returning, tint halts, etc).
If you want to know more details take a look at the commit message.

Apart from this two items I wrote a couple of patches concerned with libpayload and a patch that adds support for sequential lz4 decompression during the boot process (Thank you, Julius Werner, for pointing that out).
I also just finished updating the wiki entry for Bayou, so if someone wants to test it out… 🙂

What’s next?

With less than a month left of my GSOC time I plan to dedicate solely to maintain the patches that I currently have and work on the coreboot/serialice integration (so as to finally do something pertinent to the original proposal).

Have a nice weekend!

 

P.S.

I had to drop the idea of working on the FILO (flashupdate) project since it relies on a series of patches that have not been finished yet (libflashrom). It will be a project for another time perhaps.

P.S.S.

I think I lost count of a few weeks here and there…

[GSOC] Panic Room, week #4/5/6

What have you been up to these past few weeks?

My facetious alter ego, I must admit I sinned, I strayed oh so much from my original timeline. (Quite obvious if you read the previous posts)

Since my last post I mainly kept focusing on the region_device API, I attempted to go through with my week #3 plan to continue the phasing out of rdev_mmap_full() from the selfload code (described in previous post).

I think I overestimated what I could accomplish regarding that project, I tried to modify the current lzma API used inside to coreboot to make it possible to easily load each chunk of data from memory/chip and decompress it gradually.
Theoretically it shouldn’t be too difficult, in practice the decompression code is particularly nasty and it seems it is a customised version of the LZMA SDK code.
I, not so quickly, realised that I didn’t have the time to delve into that, so I momentarily dropped the idea. (It could be a nice project to do after the GSOC ends.)

I also spent a chunk of my time in porting the time stamps reading functionality from the cbmem utility into the coreinfo payload (commit).
It is now possible to read how much time the coreboot boot sequence takes without having a working OS environment.
I needed to measure if there had been any boot time improvements after the rdev_mmap_full commit. (Still haven’t got around to doing that though…)

What are you working on now then?

I’m currently working on porting the region_device API from coreboot to libpayload, meanwhile replacing all the functionality that was provided by its predecessor: cbfs_media.

Today I finished (in before my code has to be reworked completely ) replacing the old api inside libpayload and started converting the only payload that used cbfs_media: depthcharge, the payload used to boot ChromeOS.
Hopefully it will be a painless process.

What’s next?

So, after I finish these current patch (possibly for the weekend), I need to re-focus my attention on my actual timeline.
I plan to test the current FILO master branch to check for possible problems and show-stoppers before an eventual new release in the (near?) future.
It would be quite helpful in that the current stable is not stable at all, it doesn’t even compile and neither does the master tag (unless you apply this patch which is still pending approval).
It would also be the first release that includes the new flashupdate command and would allow for some interesting things to be done (i.e. path to the recovery from a bad coreboot flash/configuration).

Furthermore I would like to get the SerialICE firmware-side code to be merged into the main coreboot codebase (commit), so I plan to work out the current problems with the time that I still have at my disposal. (Uh, sounds a bit cliche’ by now)

See you next week time! (Just to be on the safe side 🙂 )

[GSOC] Panic Room, week #3

What happened during the past week ?

After many iteration of patches and bug hunting I finished writing and testing the code that added to cbfstool allows to convert between SELF and ELF.
The code has been now merged.

One of the most problematic things has been to get GRUB to work after the conversion to ELF whereas all of the other payloads were working wonderfully.
It turned out it is the only payload (that I tested) that used more than two segments to describe the memory image of  the program.
This also uncovered a bug contained inside the elf_writer code that was probably never triggered given that the majority of payloads only contain one segment (commit).

I also received the replacement mobo for my Lenovo X60 target, so I can get back on track with the SerialICE part of the project.

What are your plans for next week?

I am currently investigating a bug in the serial communication between QEMU and the target while using the most recent version of the patch that integrates SerialICE into coreboot.
I am also looking into some work related to selfboot.c and the region api; the objective is to avoid loading the payload all at once while it is being executed and allow for the various parts of the payload to be loaded when needed.
Hopefully I’ll manage to finish all of this before next week. (Sometimes I definitely feel too optimistic)

What?! Didn’t you have any mishaps© this week?

It’s indeed quite fascinating how my equipment keeps breaking… this week was my Raspberry Pi’s turn. It won’t boot anymore.
Fortunately I have a Bus Pirate and a BeagleBone Black to use for SPI flashing, so it’s all good.
EDIT: Scratch that… apparently you just need to wait for a while for it to reset… strange.

See you next week!

[GSOC] Panic Room, week #2

How was your last week?

Let’s say that it was a bit unexpected.

I spent the majority of it trying to wrap my head around the ELF (Executable Linkable Format) specification.
I used this new acquired knowledge to improve the utility cbfstool and allow it to extract payloads contained inside a CBFS directly into ELF instead of SELF (commit).

In order to achieve this cbfstool has to do a few things:

  • Extract the payload from the coreboot image
  • Parse the segment table contained inside the SELF payload in order to find out how many and which segments are present.
  • Using the elf_writer API generate a compliant ELF header
  • Take the content from each segment and copy it to the correspondent ELF section header and configure it accordingly
  • Once the section table is filled out, use elf_writer to generate the program header table and write out the final ELF

The final results would allow to, for example, easily move payloads from a CBFS to another one without having to re-build the payload, coreboot rom or mess with the build system configuration.
Right now the implementation it’s not complete yet but it works quite well with a good chunk of the payloads commonly used with coreboot such as SeaBIOS, coreinfo, nvramcui and others.
The major hurdles right now are to get the GRUB payload to work and add a way to handle the extraction of a compressed payload.

Wait a minute! Weren’t you working on SerialICE?

You are quite the inquisitive type, aren’t you?

Yes, my main goal is still to continue integrating SerialICE and coreboot.
Unfortunately there have been a few showstoppers this week, first my only test clip broke and now my target, Lenovo x60, stopped working and I am no longer able to flash its BIOS chip.
I already ordered a replacement but it’ll probably take a bit more than a week to arrive.

In the meantime my mentor (adurbin) kindly pointed out the task above to keep me busy while waiting.

What are your plans for the next week?

I plan to finish implementing the functionality described above and test all the remaining payloads.
Hopefully I will also be able to start looking at some of the other tasks that have been suggested to me by my mentors.

That’s it for today, see you next week!

[GSOC] Panic Room, week #1

Who are you?

Hello everyone, I’m Antonello Dettori (avengerf12 on IRC) and I’m the student currently working on improving SerialICE.

What are you working on?

I’m glad you asked.

As I said just a bunch of lines before I’m working on SerialICE, which is one of the main tools used in reverse engineering an OEM BIOS and therefore in understanding the initialisation process that coreboot will have to perform in order to properly run on a target.

The original idea of my proposal was to work towards:

  • Incorporating the functionality of SerialICE into coreboot.
  • Allowing for a way to flash a coreboot-running target without a working OS environment.

The situation has changed a bit in the few months after the proposal was written and part of the goals have already been worked on by some of the wonderful contributors in the coreboot community.
I still have plenty of work to do and my mentors already pointed out some of the areas of the project with which I could spend my time.

How was your first week?

Oh boy, you had to go there, didn’t you?

I’ve been kind of a late bloomer regarding this project since only from this week I came to truly appreciate all of the work that goes into making coreboot and SerialICE tick.
I’m therefore still knee-deep in the learning process, but don’t worry, progress is being made on this front.
Unfortunately, this also means that I don’t have any actual code to reach my goals yet.

What will you do during the next week?

I will, hopefully, manage to wrap up my learning “session” with SerialICE and get to finally write some actual (possibly useful) code.
In particular I hope to fix the problem regarding the conflicts in managing the cache and its related registers that occur when coreboot initialises the target but SerialICE is used as the romstage.

That’s pretty much it  for now, see you next week!