coreboot changelog

This report covers commits 2f953d3 to b142b84afb, up to early Monday, 2015-10-05

Last week has seen the removal of FSP support for Sandybridge and Ivy Bridge, in favor of the native raminit code and the originally supported MRC binary, with the native code path becoming the default code path on these chipsets.

The microcode handling was reworked further so that Intel microcode is now stored as binary files in 3rdparty/blobs, instead of being kept in the coreboot tree and processed by the compiler.

Development on cbfstool continued, adding the capability to relocate FSP binaries on add, and the capability to store hashes of file data to the file header, which can be a useful primitive for a verified boot process.
For this, the recently introduced commonlib was used to share code between coreboot and cbfstool.

The verstage code for Chrome OS’ vboot was refactored to allow this separate verification stage on x86. verstage was introduced to allow moving the romstage to updateable storage (that can be used only after verification which is what verstage is doing), and the idea is to make this also available on x86.
The change allowed code deduplication for several ARM chipsets, too.
For this, our Cache-as-RAM initialization code also became more independent from romstage code.

Kconfig’s strict mode (akin to gcc’s “warnings-as-errors”) is now available in more of its frontends. This should help fix Kconfig issues.

Several bug fixes also made it in, improving support for older gcc versions, for building on Mac OS X, to get native VGA init to build again for multiple Intel chipsets and more.

coreboot changelog, most-of-september edition

This report covers 410f9ad to 2f953d304e

The biweekly cadence messed up my timing so this report covers five weeks and for that reason gets to report more development (314 commits).

The biggest chunk (over one third of the commits) covers Intel Skylake development, where boards and chipset code saw misc improvements and tons of clean ups (eg. dropping ACPI code and fragments in the devicetrees that were inherited from older chipset development and aren’t appropriate anymore).
There also was a notable effort of unifying common code across the more recent Intel SoCs, removing lots of duplicated code all over the place.

As far as cleaning up the code base goes, we also got rid of a number of Kconfig definitions and uses across the tree (Intel, AMD, MIPS – something for everybody). Some definitions were booleans that were always enabled with no reason to ever disable, other symbols were selected even though they didn’t even exist anymore.

Another field of clean-up work was the EDID code used for native graphics init, which is the default on ARM SoCs (simply because there is nothing like VGABIOS there). Two structs that both defined standard fields of this “Extended Display Identification Data” standard were reworked to avoid the duplication, avoiding issues where a field is filled in for one representation but not the other.

On x86, the romstage is now relocated for its final location in CBFS by cbfstool, obsoleting the old approach that had us link it twice, once to determine its final size and then to the actual location it’s supposed to run from. This reused code from our rmodule support that already contained knowledge about relocation fields.
In the future this same approach may be extended to other files that need to be executed in place such as the FSP binary.
This effort also had x86’s linking setup become more similar to what all other architectures are doing, for romstage and ramstage, as well as some more clean-ups in the general area.
Finally, the way the romstage is linked together was simplified, resulting in less “magic” in the build system to collect the various pieces of initialization code.

The romstage change eliminated the need for cbfstool’s “locate” command, and so it was removed. cbfstool also saw other extensions, the biggest one a compatible change to the format to allow for per-file attributes in CBFS. These attributes can contain additional information about a file, currently the compression method and uncompressed size of a file. cbfstool and the build system were extended to allow compressing files, libpayload is able to uncompress these files.
The libpayload change leads to a change in behavior for some specialized cbfs access routines (eg. cbfs_load_optionrom) in that they return malloc’d memory instead of a pointer into CBFS, which may lead to a memory leak in payloads.
cbfstool’s support for flashmap “partition tables” was also improved: It already could create flashmap partitioned images, and now can create a backward compatible CBFS master header, so an fmap-based image can still be used by older CBFS parsers (which right now means: pretty much all of them).
If an fmap is available, cbfstool will prefer it over a master header, but that should have no visible effect yet.
The changes also seeked to improve the internal structure of the cbfstool codebase, although much more work is necessary in that area.

libpayload saw the addition of a video_printf function to write colored text with support for text justification.
Its USB device controller driver for DeviceWare based controllers became more robust, too.
It also got back a POSIX-compatible ffs() implementation that was dropped for a while, got coreboot-compatible write8/16/32 implementations and saw a couple of bugfixes.

Another clean-up topic is the reduction of code that is #ifdef’d away for invidivual stages. To this end, we now expose declarations (but no definitions) for various functions even in romstage, so files compile. Use of ramstage-level functions like malloc in romstage still fail, but at link time. The linker is smart enough these days to pinpoint the error to a line in a source file, so no information is lost. The upside is that the code flow isn’t broken by preprocessor conditionals every few lines.
In a similar light, more #if CONFIG_FOO statements were replaced by if (IS_ENABLED(CONFIG_FOO)), exposing the guarded code to the compiler, which avoids errors from accumulating in non-standard configurations: Now the code needs to pass the compiler at least syntactically even in a standard configuration.
Finally, Chrome OS vboot related code was abstracted away in SoC code, to make it more useful in non-Chrome OS configurations more easily.

To reduce code duplication, we also started building a common library (src/commonlib/), containing things like CBFS data structure declarations or endianess handling functions that are useful in both coreboot and its utilities.

On the AMD side, there were various bugfixes both for new (merlin falcon) and old (Fam10) chipsets.

ARM64 and Tegra210 saw various bugfixes and improvements to power use. For the latter, coreboot also learned how to reserve memory for other functions than the main processor.
Rockchip’s RK3288 ARMv7 SoC also saw a number of bug fixes and the code was restructured to use a single mainboard directory for a large number of very similar Google Veyron mainboards based on that SoC.

Our RISCV support now boots on the Spike simulator which (besides supporting a wider variety of emulators) is notable because unlike the QEmu RISCV support, Spike supports RISCV’s revised ABI.
Speaking of emulators, recent versions of qemu-x86 expect the firmware to initialize the LAPIC, which we now do.

The timestamps recording code was extended to also store the timebase (ticks) so that the userspace tool in the OS doesn’t need to try to figure it out – probably wrongly because by the time the OS started, the clock changed and with it, on some CPUs, the tick duration.
cbmem also gained an option “-T” to display timestamps in a machine readable format.
During the course of that development, some timestamps code was simplified, relying more on generic code over per-chipset implementations.

The ongoing effort to move CPU microcode into CBFS (and to store these as binaries in 3rdparty/blobs instead of header files in the main sources) saw some progress.

Our crosscompiler generator script is more helpful on errors and learned building gcc (the “GNU compiler suite”) for non-C languages, including some extensions to support Ada in particular.
abuild now produces proper logs when building the tools (below util/), making it easier to track down issues in that area from error reports on our continuous build system.

The standard seabios released used by our build system is now 1.8.2, up from 1.7.5. In other payload related news, coreboot doesn’t initialize PS/2 keyboards by itself by default, but expects the payload to do so. The option to re-enable the initialization (as well as the code it controls) is still around.

The intel-related utilities saw some improvements, with ifdtool supporting the new IFD v2 better (as used on Skylake), and inteltool able to detect Intel’s 4-Series.

The tree also saw some improvements with regard to portability and should behave better when building on MinGW, Cygwin, Mac OS X and FreeBSD host.

Update: coreboot conference in Bonn, Germany, October 2015

TL;DR: Book your flights/hotel now and submit talks! More info at http://coreboot.org/Coreboot_conference_Bonn_2015

Dear vendors, developers, users and interested parties,

on behalf of the Federal Office for Information Security (BSI) Germany I would like to invite you to the coreboot conference and developer meeting on October 9-11 2015 in Bonn, Germany.

This conference and developer meeting is geared towards manufacturers of hardware (processors, chipsets, mainboards and servers/ laptops/ tablets/ desktops/ appliances) as well as developers of firmware with an interest in coreboot and the possibilities it offers as well as (potential) coreboot users. Both professionals and hobbyists are invited.

The Federal Office for Information Security (BSI) in Germany will host the conference in Bonn, Germany. As the national cyber security authority, the goal of the BSI is to promote IT security in Germany. For this reason, the BSI has funded coreboot development in the past for security reasons.

The date of the coreboot conference is Friday October 9 to Sunday October 11, 2015. This is scheduled directly after Embedded Linux Conference Europe to make travel arrangements easier for people attending both events.

For the rest of the invitation letter, travel information, a call for presentations and more please see http://coreboot.org/Coreboot_conference_Bonn_2015

 

coreboot changelog – Weeks of 2015-08-10 and 2015-08-17

this report covers commits 1cbef1c to 410f9ad

The vast majority of changes in these two weeks were upstreamed from Chrome OS and cover work on the Intel Skylake chipset and two mainboards based on it.

QEmu and Getac P470 saw a couple of improvements.
On AMD, there were some bugfixes to Fam10h concerning VGA memory and SMM initialization. The latter was in response to the Memory Sinkhole vulnerability, although it is as yet unclear if it even affects AMD.
Finally, an important memory structure used on pre-AGESA AMD code is now also usable outside Cache-as-RAM.
There was more progress on fixing 64bit issues across the codebase.

Our reference compiler was updated to gcc 5.2. This became necessary to support an update to the RISC-V specification.

Our other tools also saw a couple of improvements: ifdtool now works for descriptors on Skylake and newer platforms. cbfstool saw some refactorings that allow us to extend the format. cbmem now emits the accumulated boot time.

In our configuration system, the Kconfig definitions were cleaned up, so that boards don’t define symbols that their code never uses, that Chrome OS capable boards define “MAINBOARD_HAS_CHROMEOS” (which defines the capability) instead of “CHROMEOS” (which defines that this mode should be
used) and that dependencies between Kconfig options become more consistent.
There is a pending commit on gerrit to enforce clean dependencies by making errors out of kconfig’s warnings, that the latter changes prepare for.

On the build system side, it is now possible to build SeaBIOS as part of our build system even with an enabled ccache. The payload config and revision can also be stored in CBFS for better reproducibility. Finally, it’s possible to override the location from where the vboot source code for Chrome OS-style verified boot is taken from.

In libpayload, the non-accelerated memmove implementation now also works with size == 0 (instead of trying to move 4GB), and there were a couple of bug fixes to the DWC2 (some ARM) and XHCI (USB3) controller drivers, including support for the newer XHCI 1.1 specification.

[GSoC] End user flash tool – week #13 – summary

Hello!

During week 13 I worked on:

  • writing project documentation
  • bug fixing
  • code cleanup
  • changing debug messages to information popups
DOCUMENTATION

All functions and variables are now documented in javadoc style, I also attached some comments in code, documentation can be generated with doxygen. Besides documenting code I prepared documentation for functional tests which I executed. It contains described test cases and test results.

NEW WORKING CONFIGURATIONS

To make automatic building of coreboot image feature more useful it is necessary to add more data about working configurations. Every added configuration also needs to be tested to check if tool correctly recognizes hardware on target system and builds working coreboot image. I can do this for my hardware, so of course I did, I can also add some fake configurations for testing purposes, but I can’t do this for hardware which I do not have.

Here is description of application, building process and information about data I need to add a working configuration: link. I will be grateful for every configuration which you will send!

GUI IMPROVEMENTS

Tool is targeted mostly for users which are not familiar with coreboot and flashrom details and also dont know how to proceed with building a working coreboot image. Taking it into consideration I changed most debug/error messages to appear in a form of a popup with description what action is needed from application user to proceed or what went wrong.

end_user_flash_tool_dialogs

SUMMARY

End user flash tool is my first experience with coreboot and flashrom. Both of them are not easy projects, especially for someone who does not have great experience with firmware programming, because most code involves serious low level implementations, but in End user flash tool project I have been working always few layers above it because my work involved GUI programming, system programming, integration of external tools like bios_extract or cbfs_tool and adding few features to libflashrom. By doing it I learned a bit how these tools work internally. I am now also more familiar with coreboot itself. This work gave me good basics and smooth entry to the coreboot world. As all of this is interesting and working for such project is very satisfying I want to dive in more and also maintain and extend the tool.

CCCamp 2015

We had an table on the cccamp 2015 in Mildenberg. The cccamp (chaos communication camp) is one of Europe big hacker events every 4 years where several hackers come together and do camping. Everybody could relax a little bit and talk to each other in a nice background. The camp infrastructure contained own wifi, dect, gsm network and a 10 GBit uplink in the middle of nowhere. You could give your tent a 1 Gbit uplink to the internet ;).

Our table was in the BER village, named to the never finished airport near Berlin, Germany. Several people from the coreboot community showed up (CareBear, felix, mue, paulk-collins, tnias, zaolin, […]) and we shared a lot of ideas to each other. In that way we flashed several laptops, replaced some WSON chips with SOIC-8’s. Also we’ve found another bug in the sandybridge ram init, a fix is waiting for merge on gerrit #11248.
paulk-collins came by to talk about a EC open source firmware for the ENE KBxxx embedded controllers.
One of the MAME hackers visited us to get ideas how to port a Dell notebook (ENE KBxxx based EC).
Felix did some work on ME as well other hackers joined him.
Tnias and zaolin started the idea of a raspberry pi doing all the flashing including detection of the device. This could let us drink more Mate while other do the flashing themself.
In the end a thunderstorm reached our tent and we had to evacuate it.

Hopefully everybody can come to Bonn this year.

[GSoC] End user flash tool – week #10 #11 #12

Hello!

During weeks 10, 11 and 12 I worked on:

  • functions for gathering hardware specific data
  • automating process of building coreboot image
  • GUI improvements
GATHERING HARDWARE SPECIFIC DATA

As I mentioned in my last post, if coreboot image should be built automatically then application needs to collect hardware specific data. During last weeks I added functions responsible for:

  • dumping VGABIOS from memory: Some systems (like Lenovo T60 with ATI graphics) require adding VGABIOS dumped from memory to coreboot image, because factory BIOS patches it at runtime, so in certain cases using option rom extracted from factory BIOS may not work
  • getting EDID data: to know what type of display panel is used in system
  • getting motherboard model name: this is mandatory to recognize if we can install coreboot in system or if we have known working configuration
AUTOMATING PROCESS OF BUILDING COREBOOT IMAGE

It is now possible to build and flash coreboot image by clicking few application buttons without taking care which options are correct for system. Of course this is not possible for all hardware configurations, for now this is very limited, but with time database of known good configurations will grow and more users will be able to flash coreboot in such easy way.

flash_tool_auto_tab

Process of automated image building:

  1. Check if working configuration for system is known.
  2. Check if configuration requires additional option rom.
  3. If necessary, add option rom extracted from factory bios or extracted from running system memory.
  4. Build image.
GUI IMPROVEMENTS

I decided to change a look of ‘ROM options’ tab. Previously just raw output of cbfs_tool with rom contents info was redirected to GUI log window. Now it is visible in a form of table, what in my opinion is more readable.

cbfs_tool_GUI

LAST WEEK AND FURTHER PLANS

GSoC “pencils down” date is coming up on Friday, so only few days are left. I want to use this time for:

  • writing project documentation
  • looking for bugs and fix them
  • code cleanup
  • adding new working configurations and testing them (I need your help with it)
  • change debug messages to information popups

What after GSoC? I would like to still contribute, there is place for many improvements and possible extensions.

[GSoC] End user flash tool – week #7 #8 #9

Hello!

During weeks 7, 8 and 9 I worked on:

  • functions for gathering hardware specific data
  • extending libflashrom
  • GUI improvements
  • testing
GATHERING HARDWARE SPECIFIC DATA

Main purpose of End user flash tool project is to provide an easy way to build and flash coreboot ROM. To achieve it there is a need to collect hardware specific data such as:

  • lspci -nn output: information about all PCI buses and devices in the system, it is possible to recognize a graphic card, its vendor and device codes
  • dump of factory BIOS: it is important to make a copy of factory BIOS in case something will go wrong, but not only in this case, very often there is a need to use a VGABIOS extracted from factory BIOS if particular graphic card or display panel is present in our system, it is the best to make dump two times and then check if files are the same (for example by comparing hashes)
EXTENDING LIBFLASHROM

Sometimes when flashrom probes for all known chips there are multiple chips found. I needed to implement a function which will return all such chips to GUI. Now in this case it is possible to just select which chip is correct by clicking a proper one in dialog box.

choose_chip

GUI IMPROVEMENTS

I decided to change a bit visual design of the app. There is a new (but most important for the project)  tab – ‘Auto’. In this tab it is possible to gather hardware specific data, which will be then used in a process of automatic building of coreboot image and flashing. I also decided to move programmer selection combobox from ‘Flash tab’ to main application window and  add edit text field for parameters.

LOOKING FOR TESTERS

GSoC ends in next week, application is almost done (but will be improved and extended also after GSoC), so this is time for some testing! I would be very grateful if some of you could help me with it. It would be the best if you have Lenovo T60 and external programmer. First there is a need to collect some hardware specific data and then it would be possible to check if application creates working coreboot image basing on this data. So it is not only about testing, but also about making white list of hardware configurations bigger to let more users flash their hardware with coreboot in easy way!

Please contact me on:

  • IRC #flashrom #coreboot: lukaszdm
  • e-mail: lukasz.dmitrowski@gmail.com

Thanks in advance!

coreboot changelog – Weeks of 2015-07-27 and 2015-08-03

This covers commits ef0158ec up to commit 1cbef1c
Development is typically slower during the summer and 2015 is no exception, so the report switches to a biweekly installment for a while.

The last two weeks have seen improvements in our development tools:
coreboot upstream can now build Chrome OS boards with Chrome OS features (verified boot, interaction with Chrome EC, flash based error logging) enabled, and the projects builders at http://qa.coreboot.org/ are now routinely building these configurations alongside the regular default configs for all boards.
The builders now run ‘make what-jenkins does’ (see coreboot/Makefile.inc) instead of a hard-coded set of commands, which provides the community the capability to adapt the test build without admin intervention.
When adding the .config used for building an image into said image, it’s now minimized which gives visibility to the relevant changes to the config compared to the board’s defaults.
Kconfig features a strict mode, which acts as a ‘warnings-as-errors’ equivalent and fails the build if kconfig would emit any warning. Since we still have a couple of those in the tree, it’s not enabled yet.
For users of cscope or ctags, we now have new make targets to create tree-wide indexes (make ctags-project cscope-project).

Reproducible builds got a boost by fixes to the build.h generator script, which can finally emit stable timestamps based on the git revision, instead of the local time.

External payload integration was coalesced within payloads/external, with more work in progress. The integrated SeaBIOS build can now also be used when building with ccache. libpayload gained robustness in different developer environments, being smarter about looking for compilers, configs and include files in all the right places.

On the Free Software side, more microcode blobs were moved to the 3rdparty/blobs repository and one false positive that libreboot’s blob detector tripped over was eliminated, and with a little more progress, it should soon be possible to build from a fully blob-free coreboot tree. Before you get your hopes up, please note that the result may not be very useful on a lot of boards, so more care must be taken.

The effort to make coreboot capable of booting in 64bit mode on x86-64 is still ongoing and saw the integration of more commits.

coreboot should have an easier time again when building on Cygwin and BSD systems.

Skylake was the chipset with the largest amount of work in the 2 weeks, but there was also the addition of a coreboot port for RISC-V’s Spike ISA Simulator, contributions to the AMD Bettong mainboard and its chipset drivers, as well as fixes and cleanups to AMD K8 and Intel i945.

In terms of style, a bunch of extraneous whitespaces, indenting errors and FSF addresses were also dealt with.