coreboot changelog, most-of-september edition

This report covers 410f9ad to 2f953d304e

The biweekly cadence messed up my timing so this report covers five weeks and for that reason gets to report more development (314 commits).

The biggest chunk (over one third of the commits) covers Intel Skylake development, where boards and chipset code saw misc improvements and tons of clean ups (eg. dropping ACPI code and fragments in the devicetrees that were inherited from older chipset development and aren’t appropriate anymore).
There also was a notable effort of unifying common code across the more recent Intel SoCs, removing lots of duplicated code all over the place.

As far as cleaning up the code base goes, we also got rid of a number of Kconfig definitions and uses across the tree (Intel, AMD, MIPS – something for everybody). Some definitions were booleans that were always enabled with no reason to ever disable, other symbols were selected even though they didn’t even exist anymore.

Another field of clean-up work was the EDID code used for native graphics init, which is the default on ARM SoCs (simply because there is nothing like VGABIOS there). Two structs that both defined standard fields of this “Extended Display Identification Data” standard were reworked to avoid the duplication, avoiding issues where a field is filled in for one representation but not the other.

On x86, the romstage is now relocated for its final location in CBFS by cbfstool, obsoleting the old approach that had us link it twice, once to determine its final size and then to the actual location it’s supposed to run from. This reused code from our rmodule support that already contained knowledge about relocation fields.
In the future this same approach may be extended to other files that need to be executed in place such as the FSP binary.
This effort also had x86’s linking setup become more similar to what all other architectures are doing, for romstage and ramstage, as well as some more clean-ups in the general area.
Finally, the way the romstage is linked together was simplified, resulting in less “magic” in the build system to collect the various pieces of initialization code.

The romstage change eliminated the need for cbfstool’s “locate” command, and so it was removed. cbfstool also saw other extensions, the biggest one a compatible change to the format to allow for per-file attributes in CBFS. These attributes can contain additional information about a file, currently the compression method and uncompressed size of a file. cbfstool and the build system were extended to allow compressing files, libpayload is able to uncompress these files.
The libpayload change leads to a change in behavior for some specialized cbfs access routines (eg. cbfs_load_optionrom) in that they return malloc’d memory instead of a pointer into CBFS, which may lead to a memory leak in payloads.
cbfstool’s support for flashmap “partition tables” was also improved: It already could create flashmap partitioned images, and now can create a backward compatible CBFS master header, so an fmap-based image can still be used by older CBFS parsers (which right now means: pretty much all of them).
If an fmap is available, cbfstool will prefer it over a master header, but that should have no visible effect yet.
The changes also seeked to improve the internal structure of the cbfstool codebase, although much more work is necessary in that area.

libpayload saw the addition of a video_printf function to write colored text with support for text justification.
Its USB device controller driver for DeviceWare based controllers became more robust, too.
It also got back a POSIX-compatible ffs() implementation that was dropped for a while, got coreboot-compatible write8/16/32 implementations and saw a couple of bugfixes.

Another clean-up topic is the reduction of code that is #ifdef’d away for invidivual stages. To this end, we now expose declarations (but no definitions) for various functions even in romstage, so files compile. Use of ramstage-level functions like malloc in romstage still fail, but at link time. The linker is smart enough these days to pinpoint the error to a line in a source file, so no information is lost. The upside is that the code flow isn’t broken by preprocessor conditionals every few lines.
In a similar light, more #if CONFIG_FOO statements were replaced by if (IS_ENABLED(CONFIG_FOO)), exposing the guarded code to the compiler, which avoids errors from accumulating in non-standard configurations: Now the code needs to pass the compiler at least syntactically even in a standard configuration.
Finally, Chrome OS vboot related code was abstracted away in SoC code, to make it more useful in non-Chrome OS configurations more easily.

To reduce code duplication, we also started building a common library (src/commonlib/), containing things like CBFS data structure declarations or endianess handling functions that are useful in both coreboot and its utilities.

On the AMD side, there were various bugfixes both for new (merlin falcon) and old (Fam10) chipsets.

ARM64 and Tegra210 saw various bugfixes and improvements to power use. For the latter, coreboot also learned how to reserve memory for other functions than the main processor.
Rockchip’s RK3288 ARMv7 SoC also saw a number of bug fixes and the code was restructured to use a single mainboard directory for a large number of very similar Google Veyron mainboards based on that SoC.

Our RISCV support now boots on the Spike simulator which (besides supporting a wider variety of emulators) is notable because unlike the QEmu RISCV support, Spike supports RISCV’s revised ABI.
Speaking of emulators, recent versions of qemu-x86 expect the firmware to initialize the LAPIC, which we now do.

The timestamps recording code was extended to also store the timebase (ticks) so that the userspace tool in the OS doesn’t need to try to figure it out – probably wrongly because by the time the OS started, the clock changed and with it, on some CPUs, the tick duration.
cbmem also gained an option “-T” to display timestamps in a machine readable format.
During the course of that development, some timestamps code was simplified, relying more on generic code over per-chipset implementations.

The ongoing effort to move CPU microcode into CBFS (and to store these as binaries in 3rdparty/blobs instead of header files in the main sources) saw some progress.

Our crosscompiler generator script is more helpful on errors and learned building gcc (the “GNU compiler suite”) for non-C languages, including some extensions to support Ada in particular.
abuild now produces proper logs when building the tools (below util/), making it easier to track down issues in that area from error reports on our continuous build system.

The standard seabios released used by our build system is now 1.8.2, up from 1.7.5. In other payload related news, coreboot doesn’t initialize PS/2 keyboards by itself by default, but expects the payload to do so. The option to re-enable the initialization (as well as the code it controls) is still around.

The intel-related utilities saw some improvements, with ifdtool supporting the new IFD v2 better (as used on Skylake), and inteltool able to detect Intel’s 4-Series.

The tree also saw some improvements with regard to portability and should behave better when building on MinGW, Cygwin, Mac OS X and FreeBSD host.

Update: coreboot conference in Bonn, Germany, October 2015

TL;DR: Book your flights/hotel now and submit talks! More info at http://coreboot.org/Coreboot_conference_Bonn_2015

Dear vendors, developers, users and interested parties,

on behalf of the Federal Office for Information Security (BSI) Germany I would like to invite you to the coreboot conference and developer meeting on October 9-11 2015 in Bonn, Germany.

This conference and developer meeting is geared towards manufacturers of hardware (processors, chipsets, mainboards and servers/ laptops/ tablets/ desktops/ appliances) as well as developers of firmware with an interest in coreboot and the possibilities it offers as well as (potential) coreboot users. Both professionals and hobbyists are invited.

The Federal Office for Information Security (BSI) in Germany will host the conference in Bonn, Germany. As the national cyber security authority, the goal of the BSI is to promote IT security in Germany. For this reason, the BSI has funded coreboot development in the past for security reasons.

The date of the coreboot conference is Friday October 9 to Sunday October 11, 2015. This is scheduled directly after Embedded Linux Conference Europe to make travel arrangements easier for people attending both events.

For the rest of the invitation letter, travel information, a call for presentations and more please see http://coreboot.org/Coreboot_conference_Bonn_2015

 

2015-08-28 Librem 13: Weekly BIOS Update

Author: Larry.Moberg@puri.sm

This post gives some details on the Librem 15 rev2 prototype. One challenge with developing BIOS is finding parts that can be reused; coreboot makes heavy reuse of certain pieces of code.

Very Similar

Starting with the chips on the back of the mainboard, the 15 prototype uses the same ene KB3930QF-A1 as the Librem 13, and it is configured to read from an external Macronix MX25L512 SPI flash for firmware. The 15 has this SPI chip on the front side of the board between the DIMMs and the USB3 ports.

The 15 prototype uses a Realtek ALC269 codec via the AC’97. The Librem 13 should have a very similar codec.

On the front (the side visible just by removing the back laptop case), the 15 prototype uses an MX25L6406E SPI flash for the BIOS. The Librem 13 prototype uses a GD25Q64B, but other than the Intel Firmware Descriptor fields for JEDEC ID etc, these chips are interchangeable.

Both CPUs are Broadwell-U. They use the same FSP. They both have the LPC bus exposed on pads.

These similarities help us by reducing the amount of variation between the board subdirs in coreboot and can use the same development rig.

Acceptance Test Matrix

We’ve put together the following tests to validate coreboot builds:

  1. Cold boot: memory controller works.
  2. Cold boot: all installed DRAM is online.
  3. Cold boot: graphics controller works.
  4. Cold boot: SATA controller succeeds.
  5. Cold boot: EC controller responds ok to init code.
  6. Cold boot: LCD backlight turns on.
  7. Cold boot: linux boots ok in text mode.
  8. Cold boot: linux boots ok in framebuffer (boot splash) mode.
  9. Cold boot: X initializes the LCD at full native resolution.
  10. Cold boot: X enables hardware acceleration.
  11. Boot time: Cold boot to grub succeeds in less than a set timeout.
  12. Boot time: Reboot from linux back to linux succeeds in less than a set timeout.
  13. Boot time: Power down succeeds in less than a set timeout.
  14. SeaBIOS test: keyboard works.
  15. Grub test: keyboard works.
  16. Grub test: text mode and framebuffer graphics work.
  17. Cold boot to USB linux succeeds. (We plan to use SeaBIOS for boot device selection, barring major bugs.)
  18. Reboot to USB linux succeeds.
  19. EC test: fan spins.
  20. EC test: holding power for >5 seconds forces a power down.
  21. ACPI test: lid switch works.
  22. ACPI test: power button event received ok.
  23. ACPI test: AC power on/off event received ok.
  24. ACPI+EC+battery test: battery percentage works.
  25. Media keys on keyboard work in linux.
  26. Device tests: internal mic, internal speakers, webcam, webcam mic, wifi, bluetooth, hard drive, SSD, SD card, each USB port, headphone jack.
  27. prime95 (one instance bound to each hyperthread) for a fixed time to test CPU thermal management.
  28. glxgears for a fixed time to test GPU thermal management.
  29. During prime95 test, CPU digital thermal sensor should give reasonable results.
  30. Linux suspend ok.
  31. LCD backlight adjustable in linux.
  32. Linux kernel boot messages should not contain too many errors.

The effort to write Free Software implementations for all binary blobs will continue in parallel.

Secondary items would include further tweaks to PCI IRQ routing, additional ACPI tables, and optimizing battery life/power use.

coreboot changelog – Weeks of 2015-08-10 and 2015-08-17

this report covers commits 1cbef1c to 410f9ad

The vast majority of changes in these two weeks were upstreamed from Chrome OS and cover work on the Intel Skylake chipset and two mainboards based on it.

QEmu and Getac P470 saw a couple of improvements.
On AMD, there were some bugfixes to Fam10h concerning VGA memory and SMM initialization. The latter was in response to the Memory Sinkhole vulnerability, although it is as yet unclear if it even affects AMD.
Finally, an important memory structure used on pre-AGESA AMD code is now also usable outside Cache-as-RAM.
There was more progress on fixing 64bit issues across the codebase.

Our reference compiler was updated to gcc 5.2. This became necessary to support an update to the RISC-V specification.

Our other tools also saw a couple of improvements: ifdtool now works for descriptors on Skylake and newer platforms. cbfstool saw some refactorings that allow us to extend the format. cbmem now emits the accumulated boot time.

In our configuration system, the Kconfig definitions were cleaned up, so that boards don’t define symbols that their code never uses, that Chrome OS capable boards define “MAINBOARD_HAS_CHROMEOS” (which defines the capability) instead of “CHROMEOS” (which defines that this mode should be
used) and that dependencies between Kconfig options become more consistent.
There is a pending commit on gerrit to enforce clean dependencies by making errors out of kconfig’s warnings, that the latter changes prepare for.

On the build system side, it is now possible to build SeaBIOS as part of our build system even with an enabled ccache. The payload config and revision can also be stored in CBFS for better reproducibility. Finally, it’s possible to override the location from where the vboot source code for Chrome OS-style verified boot is taken from.

In libpayload, the non-accelerated memmove implementation now also works with size == 0 (instead of trying to move 4GB), and there were a couple of bug fixes to the DWC2 (some ARM) and XHCI (USB3) controller drivers, including support for the newer XHCI 1.1 specification.

[GSoC] coreboot for ARM64 Qemu – Week #9 #10

In the last post I talked about using aarch64-linux-gnu-gdb and debugging in qemu. In these two weeks I was intensely involved in stepping through gdb, disassembly and in-turn debugging the qemu port. I summarise the major highlights below.

Firstly, the correct instruction to invoke qemu is as follows

./aarch64-softmmu/qemu-system-aarch64 -machine virt -cpu cortex-a57 -machine type=virt -nographic -smp 1 -m 2048 -bios ~/coreboot/build coreboot.rom -s -S

After invoking gdb, I moved onto tracing the execution of the instructions step by step to determine where and how the code fails. A compendium of the code execution is as follows

gdb) target remote :1234
Remote debugging using :1234
(gdb) set disassemble-next-line on
(gdb) stepi
0x0000000000000980 in ?? ()
=> 0x0000000000000980: 02 00 00 14 b 0x988
(gdb)
0x0000000000000988 in ?? ()
=> 0x0000000000000988: 1a 00 80 d2 mov x26, #0x0                    // #0
(gdb)
0x000000000000098c in ?? ()
=> 0x000000000000098c: 02 00 00 14 b 0x994
(gdb) c
Continuing.
^C
Program received signal SIGINT, Interrupt.
0x0000000000000750 in ?? ()
=> 0x0000000000000750: 3f 08 00 71 cmp w1, #0x2

The detailed version can be seen here.

The first sign of error can be seen here, where the instruction is 0 and the address is way off.

0x64672d3337303031 in ?? ()
=> 0x64672d3337303031: 00 00 00 00 .inst 0x00000000 ; undefined

To find insights as to why this is happening, I resorted to tracing in gdb. This can be done by adding the following in the qemu invoke command. This creates a log file in /tmp which can be read to determine suitable information.

-d out_asm,in_asm,exec,cpu,int,guest_errors -D /tmp/qemu.log

Looking at the disassembly, it can be seen that execution of instructions till 0x784 is correct and it goes bonkers immediately after it. Looking at the trace, this is where the code hangs

IN:
0x0000000000000784:  d65f03c0      ret
The ret goes to somewhere bad. So the stack has been blown or it has executed into an area it should have prior to this. Next, I did a objdump on the bootblock.debug file. Relating to the code at this address, it could be determined that the code fails at “ret in 0000000000010758 <raw_write_sctlr>:”
Next up was determining where the stack gets blown or corrupt. For this, while stepping through each instruction, I looked at the stack pointer. The output here shows the details. Everything seems to function correctly till 0x00000000000007a0 (0x00000000000007a0: f3 7b 40 a9 ldp x19, x30, [sp] ), then next is 0x00000000000007a4: ff 43 00 91 add sp, sp, #0x10 . This is where saved pc goes corrupt. This code gets called in the “raw_write_sctlr_current” (using objdump)
From the trace, we have the following information : The ret goes bad at 0000000000010758 <raw_write_sctlr>:
0x0000000000000908:  97fffe06      bl #-0x7e8 (addr 0x120)
0x0000000000000120:  3800a017      sturb w23, [x0, #10]
0x0000000000000124:  001c00d5      unallocated (Unallocated)
Taking exception 1 [Undefined Instruction]
…from EL1
…with ESR 0x2000000
Which is here:
0000000000010908 <arm64_c_environment>:
   10908: 97fffe06  bl 10120 <loop3_csw+0x1b>
   1090c: aa0003f8  mov x24, x0
This finally gave some leads in the qemu debug. There seems be some misalignment in smp_processor_id.
While tracing in gdb, we have
0x0000000000000908 in ?? ()
=> 0x0000000000000908: 06 fe ff 97   bl  0x120
(which is actually bl smp_processor_id (from src/arch/arm64/stage_entry.S))
Under arm64_c_environment (in objdump) we have;
10908:       97fffe06        bl      10120 <loop3_csw+0x1b>
Also in the trace we have
IN:
0x0000000000000908:  97fffe06      bl #-0x7e8 (addr 0x120)

Now loop3_csw is defined at (from objdump)
0000000000010105 <loop3_csw>:

So this + 0x1b = 10120

Thus it wants to branch and link to 0x120 but smp_processor_id is at 121.

smp_processor_id is at (from objdump)
0000000000010121 <smp_processor_id>:

This gives us where the code is failing. Next up is finding out the reason for this misalignment and rectifying it.

 

 

 

[GSoC] End user flash tool – week #13 – summary

Hello!

During week 13 I worked on:

  • writing project documentation
  • bug fixing
  • code cleanup
  • changing debug messages to information popups
DOCUMENTATION

All functions and variables are now documented in javadoc style, I also attached some comments in code, documentation can be generated with doxygen. Besides documenting code I prepared documentation for functional tests which I executed. It contains described test cases and test results.

NEW WORKING CONFIGURATIONS

To make automatic building of coreboot image feature more useful it is necessary to add more data about working configurations. Every added configuration also needs to be tested to check if tool correctly recognizes hardware on target system and builds working coreboot image. I can do this for my hardware, so of course I did, I can also add some fake configurations for testing purposes, but I can’t do this for hardware which I do not have.

Here is description of application, building process and information about data I need to add a working configuration: link. I will be grateful for every configuration which you will send!

GUI IMPROVEMENTS

Tool is targeted mostly for users which are not familiar with coreboot and flashrom details and also dont know how to proceed with building a working coreboot image. Taking it into consideration I changed most debug/error messages to appear in a form of a popup with description what action is needed from application user to proceed or what went wrong.

end_user_flash_tool_dialogs

SUMMARY

End user flash tool is my first experience with coreboot and flashrom. Both of them are not easy projects, especially for someone who does not have great experience with firmware programming, because most code involves serious low level implementations, but in End user flash tool project I have been working always few layers above it because my work involved GUI programming, system programming, integration of external tools like bios_extract or cbfs_tool and adding few features to libflashrom. By doing it I learned a bit how these tools work internally. I am now also more familiar with coreboot itself. This work gave me good basics and smooth entry to the coreboot world. As all of this is interesting and working for such project is very satisfying I want to dive in more and also maintain and extend the tool.

CCCamp 2015

We had an table on the cccamp 2015 in Mildenberg. The cccamp (chaos communication camp) is one of Europe big hacker events every 4 years where several hackers come together and do camping. Everybody could relax a little bit and talk to each other in a nice background. The camp infrastructure contained own wifi, dect, gsm network and a 10 GBit uplink in the middle of nowhere. You could give your tent a 1 Gbit uplink to the internet ;).

Our table was in the BER village, named to the never finished airport near Berlin, Germany. Several people from the coreboot community showed up (CareBear, felix, mue, paulk-collins, tnias, zaolin, […]) and we shared a lot of ideas to each other. In that way we flashed several laptops, replaced some WSON chips with SOIC-8’s. Also we’ve found another bug in the sandybridge ram init, a fix is waiting for merge on gerrit #11248.
paulk-collins came by to talk about a EC open source firmware for the ENE KBxxx embedded controllers.
One of the MAME hackers visited us to get ideas how to port a Dell notebook (ENE KBxxx based EC).
Felix did some work on ME as well other hackers joined him.
Tnias and zaolin started the idea of a raspberry pi doing all the flashing including detection of the device. This could let us drink more Mate while other do the flashing themself.
In the end a thunderstorm reached our tent and we had to evacuate it.

Hopefully everybody can come to Bonn this year.

[GSoC] End user flash tool – week #10 #11 #12

Hello!

During weeks 10, 11 and 12 I worked on:

  • functions for gathering hardware specific data
  • automating process of building coreboot image
  • GUI improvements
GATHERING HARDWARE SPECIFIC DATA

As I mentioned in my last post, if coreboot image should be built automatically then application needs to collect hardware specific data. During last weeks I added functions responsible for:

  • dumping VGABIOS from memory: Some systems (like Lenovo T60 with ATI graphics) require adding VGABIOS dumped from memory to coreboot image, because factory BIOS patches it at runtime, so in certain cases using option rom extracted from factory BIOS may not work
  • getting EDID data: to know what type of display panel is used in system
  • getting motherboard model name: this is mandatory to recognize if we can install coreboot in system or if we have known working configuration
AUTOMATING PROCESS OF BUILDING COREBOOT IMAGE

It is now possible to build and flash coreboot image by clicking few application buttons without taking care which options are correct for system. Of course this is not possible for all hardware configurations, for now this is very limited, but with time database of known good configurations will grow and more users will be able to flash coreboot in such easy way.

flash_tool_auto_tab

Process of automated image building:

  1. Check if working configuration for system is known.
  2. Check if configuration requires additional option rom.
  3. If necessary, add option rom extracted from factory bios or extracted from running system memory.
  4. Build image.
GUI IMPROVEMENTS

I decided to change a look of ‘ROM options’ tab. Previously just raw output of cbfs_tool with rom contents info was redirected to GUI log window. Now it is visible in a form of table, what in my opinion is more readable.

cbfs_tool_GUI

LAST WEEK AND FURTHER PLANS

GSoC “pencils down” date is coming up on Friday, so only few days are left. I want to use this time for:

  • writing project documentation
  • looking for bugs and fix them
  • code cleanup
  • adding new working configurations and testing them (I need your help with it)
  • change debug messages to information popups

What after GSoC? I would like to still contribute, there is place for many improvements and possible extensions.

2015-08-14 Librem 13: Weekly Progress Update

A question coreboot developers are commonly asked is this: “can you port coreboot to my board?”

For my first coreboot post I’d like to show some of the steps required to port coreboot to the Librem 13. In particular, this post is a good example of some of the challenges involved in such a port.

This post is also the first weekly progress update for the Librem 13. Please email me with questions or comments: larry.moberg@puri.sm.

LPC Bus

The Librem 13 has convenient test points for the LPC bus. This allows a bed-of-nails test setup to quickly diagnose problems during manufacturing. But it has the added bonus of facilitating coreboot development.

The earliest coreboot stages are the most important to get right. Debugging using port 0x80 writes on the Librem 13 is possible because port 0x80 writes are configured as LPC writes, which can be traced by connecting to the LPC pins.

And…It’s Gone

BIOS development is hard. I applied a little too much force on the SPI flash chip and tore the solder pads off the board.

Pads...gone

I attached the LPC connection to a test setup and didn’t check using a multimeter before applying power. LAD2 was shorted to LAD3. This immediately bricked the laptop without even releasing any smoke. Remembering to double check for shorts is a tedious but important lesson.
Don't Cross The Streams

The LPC bus wires go under the board. Don’t Cross The Streams!

Why It Matters

Imagine a laptop where the LPC bus is only available by soldering directly to the pins of the EC. Yes, they exist! That level of fine soldering is a significant barrier for future coreboot hackers. (The Librem 13’s external USB ports are all USB 3, which makes an EHCI debug port harder, but the LPC bus is a good substitute.)

Porting coreboot to a new laptop takes a lot of time and work. Even a good laptop design like the Librem 13 where the LPC pads are available still has a non-trivial level of engineering work to get to a Free Software BIOS.

Next week, I’ll document the engineering considerations around writing to the SPI flash chip, and how that affects coreboot development.