[GSOC] Panic Room, week #3

What happened during the past week ?

After many iteration of patches and bug hunting I finished writing and testing the code that added to cbfstool allows to convert between SELF and ELF.
The code has been now merged.

One of the most problematic things has been to get GRUB to work after the conversion to ELF whereas all of the other payloads were working wonderfully.
It turned out it is the only payload (that I tested) that used more than two segments to describe the memory image of  the program.
This also uncovered a bug contained inside the elf_writer code that was probably never triggered given that the majority of payloads only contain one segment (commit).

I also received the replacement mobo for my Lenovo X60 target, so I can get back on track with the SerialICE part of the project.

What are your plans for next week?

I am currently investigating a bug in the serial communication between QEMU and the target while using the most recent version of the patch that integrates SerialICE into coreboot.
I am also looking into some work related to selfboot.c and the region api; the objective is to avoid loading the payload all at once while it is being executed and allow for the various parts of the payload to be loaded when needed.
Hopefully I’ll manage to finish all of this before next week. (Sometimes I definitely feel too optimistic)

What?! Didn’t you have any mishaps© this week?

It’s indeed quite fascinating how my equipment keeps breaking… this week was my Raspberry Pi’s turn. It won’t boot anymore.
Fortunately I have a Bus Pirate and a BeagleBone Black to use for SPI flashing, so it’s all good.
EDIT: Scratch that… apparently you just need to wait for a while for it to reset… strange.

See you next week!

[GSoC] Better RISC-V support, week #3

Last week, after updating GCC (by applying Iru Cai’s patch) and commenting out uses of outdated instructions and CSRs (most notably eret and the HTIF CSRs), I noticed that coreboot crashed when it tried to access any global variables. This was because the coreboot build system thought coreboot would live near the start of the address space.

I found spike-riscv/memlayout.ld, and adjusted the starting offset. But then I got a linker error:

build/bootblock/arch/riscv/rom_media.o: In function `boot_device_ro': [...]/src/arch/riscv/rom_media.c:26:(.text.boot_device_ro+0x0): relocation truncated to fit: _RISCV_HI20 against `.LANCHOR0'

I played around with the start address and noticed that addresses below 0x78000000 worked, but if I chose an address that was too close to 0x80000000, it broke. This is, in fact, because pointers to global variables were determined with an instruction sequence like lui a0, 0xNNNNN; addi a0, a0, 0xNNN. On 32-bit RISC-V, the LUI instruction loads its argument into the upper 20 bits of a register, and ADDI adds a 12-bit number. On a 64-bit RISC-V system, lui a0, 0x80000 loads 0xffffffff80000000 into a0, because the number is sign extended.

After disassembling some .o files of coreboot and the RISC-V proxy kernel, I finally noticed that I had to use the -mcmodel=medany compiler option, which makes data accesses pc-relative.

Now that coreboot finally ran and could access its data section, I finished debugging the UART block that I promised last week. Coreboot can now print stuff, but it stops running pretty soon.

Plans for this week

This week I will debug why coreboot hangs, and will hopefully get it to boot until the “Payload not found” line again, which worked with an older version of Spike.

Also, Ron Minnich will be giving a talk about coreboot on RISC-V at the coreboot convention in San Francisco, in a few hours.

[GSOC] Panic Room, week #2

How was your last week?

Let’s say that it was a bit unexpected.

I spent the majority of it trying to wrap my head around the ELF (Executable Linkable Format) specification.
I used this new acquired knowledge to improve the utility cbfstool and allow it to extract payloads contained inside a CBFS directly into ELF instead of SELF (commit).

In order to achieve this cbfstool has to do a few things:

  • Extract the payload from the coreboot image
  • Parse the segment table contained inside the SELF payload in order to find out how many and which segments are present.
  • Using the elf_writer API generate a compliant ELF header
  • Take the content from each segment and copy it to the correspondent ELF section header and configure it accordingly
  • Once the section table is filled out, use elf_writer to generate the program header table and write out the final ELF

The final results would allow to, for example, easily move payloads from a CBFS to another one without having to re-build the payload, coreboot rom or mess with the build system configuration.
Right now the implementation it’s not complete yet but it works quite well with a good chunk of the payloads commonly used with coreboot such as SeaBIOS, coreinfo, nvramcui and others.
The major hurdles right now are to get the GRUB payload to work and add a way to handle the extraction of a compressed payload.

Wait a minute! Weren’t you working on SerialICE?

You are quite the inquisitive type, aren’t you?

Yes, my main goal is still to continue integrating SerialICE and coreboot.
Unfortunately there have been a few showstoppers this week, first my only test clip broke and now my target, Lenovo x60, stopped working and I am no longer able to flash its BIOS chip.
I already ordered a replacement but it’ll probably take a bit more than a week to arrive.

In the meantime my mentor (adurbin) kindly pointed out the task above to keep me busy while waiting.

What are your plans for the next week?

I plan to finish implementing the functionality described above and test all the remaining payloads.
Hopefully I will also be able to start looking at some of the other tasks that have been suggested to me by my mentors.

That’s it for today, see you next week!

[GSoC] Multiple status registers, block protection and OTP support, week #1 and #2

Hi, I am Hatim Kanchwala (hatim on IRC) from India. I am the GSoC student working with flashrom this year. Stefan Tauner (stefanct) and David Hendricks (dhendrix) will be mentoring me (thanks a lot for the opportunity). The pre-midterm phase of my project comprises three sub-projects – multiple status registers, block protection and OTP support. Each of these projects deals with SPI flashchips.

As of writing this post, flashrom supports over 300 SPI flashchips. Around 10% have multiple status registers (most have two but there is one with three). Almost all have some sort of block protection in place. Around 40% have some variation of OTP or security registers. A combination of BP (Block Protect, first status register) and SRP bits (usually first, but sometimes second status register as well) in the status register determine the range and type of protection in effect. A few have a TB bit (Top/Bottom) in addition to BP bits. Some also have a CMP bit (Complement Protect, second or third status register) to add more flexibility to range available. Few chips have a WPS bit (Write Protect Scheme, second or third status register) that define which scheme of access protection is in use. Chips with security registers have corresponding LB bits (Lock Bits, second status register) which are one-time programmable and, when set, render the corresponding security register read-only. Chips with a separate OTP sector(s) have opcodes to enter/exit OTP mode and, within OTP mode usual read, page program and sector erase opcodes can be used.

Previously, flashrom could only read/write the first status register. For writes, all block protect bits were unset (this configuration corresponds to block protection), if the type of protection allowed it. Once unset, flashrom couldn’t revert the BP bit configuration. The ChromiumOS fork of flashrom has some support for locking/unlocking block access protection in place. A lot of the work is done around specific families of chips, but they are moving towards generalising it. For chips with OTP support, flashrom simply printed a warning.

In these two weeks I sifted through around 5-6 dozen datasheets and developed models for multiple status registers, block protection and OTP/security registers. I discussed with mentors and the community over mailing list (link to thread) the infrastructural changes and use cases corresponding to the models. To substantiate these ideas, I wrote separate prototype code. In the process, Stefan introduced me to a powerful tool, Coccinelle. This tool will make applying changes to the large struct flashchips easier while being safe. As a byproduct of studying existing flashrom infrastructure, I had the opportunity to explore the history of flashrom through git log – evolution of flashrom from its humble beginnings in coreboot/util to flash_and_burn to flash_rom to finally flashrom today!

My broad targets for the following few weeks will be to finish up with the pending dozen or two datasheets, polish the models and start transforming the prototype code into merge-worthy code. Following the infrastructure changes, I will update existing chips to make use of the new infrastructure, add support for a bunch of new chips and finally test on actual hardware.

Thanks. See you later!

[GSoC] Better RISC-V support, week #2

Last week, I updated my copy of spike (to commit 2fe8a17a), and familiarized myself with the differences between the old and the new version:

  • The Host-Target Interface (HTIF) isn’t accessed through the mtohost and mfromhost CSRs anymore. Instead, you have to define two ELF symbols (tohost and fromhost). Usually this is done by declaring two global variables with these names, but since the coreboot build system doesn’t natively produce an ELF file, it would get a little tricky.
  • Spike doesn’t implement a classic UART.
  • The memory layout is different. The default entry point is now at 0x1000, where spike puts a small ROM, which jumps to the start of the emulated RAM, at 0x80000000. One way to run coreboot is to load it at 0x80000000, but then it can’t catch exceptions: The exception vector is at 0x1010.
  • Within spike’s boot ROM, there’s also a text-based “platform tree”, which describes the installed peripherals.

“Why does coreboot need a serial console?”, you may ask. Coreboot uses it to log everything it does (at a configurable level of detail), and that’s quite useful for debugging and development.

Instead of working around the problems with HTIF, I decided to implement a minimal, 8250-compatible UART. I’m not done yet, but the goal is to use coreboot’s existing 8250 driver.

Plans for this week

This week, I will rewrite the bootblock and CBFS code to work with RISC-V’s new memory layout, and make sure that the spike UART works with coreboot’s 8250 UART driver. Booting Linux probably still takes some time.

[GSOC] Panic Room, week #1

Who are you?

Hello everyone, I’m Antonello Dettori (avengerf12 on IRC) and I’m the student currently working on improving SerialICE.

What are you working on?

I’m glad you asked.

As I said just a bunch of lines before I’m working on SerialICE, which is one of the main tools used in reverse engineering an OEM BIOS and therefore in understanding the initialisation process that coreboot will have to perform in order to properly run on a target.

The original idea of my proposal was to work towards:

  • Incorporating the functionality of SerialICE into coreboot.
  • Allowing for a way to flash a coreboot-running target without a working OS environment.

The situation has changed a bit in the few months after the proposal was written and part of the goals have already been worked on by some of the wonderful contributors in the coreboot community.
I still have plenty of work to do and my mentors already pointed out some of the areas of the project with which I could spend my time.

How was your first week?

Oh boy, you had to go there, didn’t you?

I’ve been kind of a late bloomer regarding this project since only from this week I came to truly appreciate all of the work that goes into making coreboot and SerialICE tick.
I’m therefore still knee-deep in the learning process, but don’t worry, progress is being made on this front.
Unfortunately, this also means that I don’t have any actual code to reach my goals yet.

What will you do during the next week?

I will, hopefully, manage to wrap up my learning “session” with SerialICE and get to finally write some actual (possibly useful) code.
In particular I hope to fix the problem regarding the conflicts in managing the cache and its related registers that occur when coreboot initialises the target but SerialICE is used as the romstage.

That’s pretty much it  for now, see you next week!

[GSoC] Better RISC-V support, week #1

Hi, I’m Jonathan Neuschäfer (jn__ on IRC) and my GSoC project for this year is to improve coreboot’s support for RISC-V platforms. RISC-V is a new instruction set architecture (ISA) that can be implemented without paying license fees and is relatively simple.

Coreboot has already been ported to RISC-V in 2014, and has since received a bunch of patches, but since the RISC-V Privileged ISA Specification (which defines things like interrupt handling and virtual memory) is still in flux, it has become unbootable again.

My first first goal last week was to run coreboot in SPIKE, the official RISC-V emulator, and get some console output. I checked out commit 419f1b5f3 (current master) of the riscv-tools repository and built SPIKE from there.

After I patched a few outdated instructions and worked around the fact that the RISC-V binutils port currently included in coreboot targets a newer version of the RISC-V Privileged Spec by hardcoding some Control and Status Register numbers, I finally got coreboot booting until the point where it would jump into a payload, had I specified one.

All patches can be found under the riscv topic on gerrit.

Plans for this week

This week I will update my SPIKE to a version that supports the upcoming Privileged Spec 1.9, which will be released in the next couple weeks. This has the advantage that I don’t need to patch instructions because GCC encodes them differently than SPIKE decodes them. Additionally, I’ll try to get Linux to boot in SPIKE, under coreboot.

Announcing coreboot 4.4

We are happy to announce the release of coreboot 4.4.  This is our fourth quarterly release.  Since the last release, we’ve had 850 commits by 90 authors adding 59000 lines to the codebase.

The release tarballs are available at https://www.coreboot.org/releases/
There is a 4.4 tag and branch in the git repository.

Log of commit 3141eac900 to commit 588ccaa9a7

Major areas that received significant changes in for this release:

  • Build system (30 commits) – Add postcar stage, ‘timeless’ builds, extend site-local, test toolchain by version string, update dependencies, catch ACPI errors, l add additional macros.
  • Toolchain updates (40+ patches) – Update IASL to v20160318 , LLVM to v3.7.1, add GNU make, add nds32le GCC compiler
  • Lint tools (30 patches) – Update existing lint utilities, add lint tests for executable bit, make sure site-local isn’t committed, add test to break all lint tests.
  • Payloads (60 commits) – Fixes for libpayload, coreinfo and nvramcui, add new payloads, see below.
  • Maintainers file – (8 patches) – continue adding maintainers for various areas.
  • Documentation for adding Intel FSP-based platforms (20 commits)

Mainboards

Added 9 mainboards

  • asus/kcma-d8
  • emulation/qemu-power8
  • google/auron_paine
  • google/gru
  • intel/amenia
  • intel/apollolake_rvp
  • intel/camelbackmountain_fsp
  • intel/galileo
  • lenovo/t420

Existing boards with significant updates

  • asus/kgpe-d16
  • google/oak
  • google/chell
  • intel/kunimitsu

Changes in chips

Added 1 new architecture

  • power8

Added 1 processor

  • qemu-power8

Added 5 socs

  • intel/apollolake
  • intel/fsp_broadwell_de
  • intel/quark
  • marvell/armada38x
  • rockchip/rk3399

Existing chip areas with many changes

  • cpuamd/mct_ddr3
  • drivers/intel/fsp2_0
  • northbridge/intel/sandybridge/raminit
  • soc/intel/apollolake
  • soc/intel/fsp_baytrail
  • soc/intel/skylake
  • soc/mediatek/mt8173

Added 1 new vendorcode directory

  • siemens

Submodules

Added 1 submodule

  • chromeec

Updated 3 submodules

  • 3rdparty/arm-trusted-firmware (329 commits)
  • 3rdparty/vboot (28 commits)
  • util/nvidia/cbootimage (13 commits)

Other

Added 4 payloads

  • depthcharge: For ChromeOS verified boot
  • iPXE: For network booting
  • Memtest86+: Updated with fixes for correctly testing coreboot with payloads
  • U-Boot (Experimental): Alternate payload for booting an OS

Added 6 utilities

  • archive – Concatenates files into a single blob with an indexed header
  • chromeos – Download and extract blobs from a ChromeOS image
  • futility – vboot Firmware utility
  • intelmetool – Shows information about the Intel ME on a platform.
  • marvell/doimage_mv – No usage notes
  • post – Simple utility to test post cards

coreboot statistics

  • Total Commits:    850
  • Total authors:        90
  • New authors:         28
  • Total Reviewers:   40
  • Total Submitters:  17
  • Total lines added:       74054
  • Total lines removed: -15056
  • Total difference:          58998

coreboot changelog March 2 – March 15

This changelog covers 187 commits in the two week period between March 2, 2016 and March 15, 2016. (c77e0419 – 80547369)

Once again this time, we had many changes in the payloads area. We added a memtest86+ git repository, and set it up as a secondary payload within the coreboot build process. SeaBIOS updated the stable version from 1.9.0 to 1.9.1 and has a new option to build from any specified commit instead of just master or stable branches. Google’s depthcharge payload was added for ChromeOS builds, and the coreinfo payload started getting some updates – removing obsolete pieces, fixing the makefile, and correcting issues with cbfs.

The MediaTek MT8173 ARM based SOC and the Google OAK board using it received a significant number of patches, adding trusted firmware support, and initialization routines for memory, USB, audio, TPM, GPIOs, I2c and RTC.

Several other groups of patches were to perform cleanup for various chipsets. One series unified and fixed up the UDELAY settings, many of which were incorrectly specifying TSC delays which weren’t supported by those platforms. Other sets removed code #includes of C files, merged the MRC cache implementations into a single common version, and combined Sandybridge & Ivybridge LVDS implementations. The FSP version of Intel’s Bay Trail was updated to mirror the non-FSP implementation, enabling LPE and LPSS in ACPI mode. The plan with Bay Trail is to make the two versions as similar as possible, then work to combine the directories and use common code for both.

Intel has started adding support for their Xeon D (Broadwell DE) processor. So far only the vendorcode has been merged.  The coreboot code is another 4700 lines of chipset code and 800 lines of mainboard code, so that’s taking some time to get reviewed.

The patches bringing up the Quark and Apollo Lake Intel chips continued, with Quark getting minor updates and Apollo Lake continuing to add core functionality like memory init and the various calls into the FSP.

Additional work was done on Skylake as well, updating the FSP parameter table, adding a Voltage Regulator mailbox command, and adding clock gating for the 8254 timer.

Utilities only got a few changes this time. The cbmem utility got a fix a regression and correctly scale the timestamp values and an option to change the SPI ROM chip sizes was added to ifdtool. Cbfstool got a couple of fixes as well, making sure the structure sizes are the same whether compiled for 32-bit or 64 bit platforms, and zeroing out unused Linux parameters.

AMD’s native memory initialization got some more cleanup and several fixes, restoring DQS delay values on a failed loop, and making sure that both read and write training pass before proceeding to the next training phase instead of continuing when either one passed.

SMBIOS changes included a patch to add SMBIOS type 17 (Memory) fields to the Sandy Bridge / Ivy Bridge platforms, and another patch to fix the length calculated for those fields for every platform. A third patch added the names of several different DIMM vendors.

The X86 bootblock renamed several symbols for clarification, removed some unused code, and marked the reset vector as executable so it would show up in objdump.

We had a slew of patches from new authors merged in the past two weeks. Welcome to all new authors and thank you to everyone.

Antonello Dettori had 3 patches merged, allowing SeaBIOS to be build from any revision, and cleaning up early serial on the roda rk9 and amd thatcher platforms.
Bayi Cheng wrote a patch adding NOR flash DMA read routines for the Mediatek MT8173.
Georg Wicherski updated and added Google’s auron paine board.
Huki Huang modified the ChromeOS wifi regulatory domain to use the region key from VPD.
Jan Tatje updated the Intel Firmware Descriptor tool (iftdool) to allow the SPI rom sizes to be updated.
Jitao Shi added the parade ps8640 MIPI-to-eDP video format converter driver.
Jonathan Neuschäfer had an astounding 7 patches merged in his first couple of weeks submitting to coreboot. He fixed a syntax error in buildgcc, and updating several areas in coreinfo.
Jun Gao did I2C work on Mediatek MT8173 and on Google’s Oak board,
Lance Zhao had a pair of patches for Intel’s Apollo Lake reference board, setting up the devicetree, and adding memory training configuration.
Medha Garima added runtime SD card detection to Intel’s Kunimitsu board.
Milton Chiang had a patch updating the infracfg register map for the Mediatek MT8173.
Peter Kao wrote a pair of patches adding DRAM initialization to the Mediatek MT8173 and Google’s Oak board.
PH Hsu set up 4GB mode on Mediatek MT8173 and Google’s Oak board.

coreboot statistics

- Total commits: 187
- Total authors: 44
- New authors: 13
- Total lines added: 15724
- Total lines removed: -1750
- Total difference: 13974

Added 1 mainboards: google/auron_paine
Added 1 new driver: parade/ps864C

=== Top Authors - Number of commits ===
Martin Roth                  27 (14.439%)
Stefan Reinauer              24 (12.834%)
Andrey Petrov                18 (9.626%)
Aaron Durbin                 15 (8.021%)
Yidi Lin                      8 (4.278%)
Timothy Pearson               8 (4.278%)
Jonathan Neuschäfer           7 (3.743%)
Patrick Rudolph               7 (3.743%)
Leroy P Leahy                 6 (3.209%)
Alexander Couzens             5 (2.674%)
Duncan Laurie                 5 (2.674%)
Total Authors: 44

=== Top Authors - Lines added ===
Peter Kao                  3750 (23.849%)
Andrey Petrov              2536 (16.128%)
York Yang                  2509 (15.956%)
Georg Wicherski            2214 (14.080%)
Alexandru Gagniuc           409 (2.601%)
Ben Gardner                 406 (2.582%)
Leroy P Leahy               384 (2.442%)
Daisuke Nojiri              373 (2.372%)
Bayi Cheng                  314 (1.997%)
Martin Roth                 256 (1.628%)

=== Top Authors - Lines removed ===
Alexander Couzens           309 (17.657%)
Leroy P Leahy               255 (14.571%)
Stefan Reinauer             207 (11.829%)
Aaron Durbin                162 (9.257%)
Jonathan Neuschäfer         156 (8.914%)
Timothy Pearson             127 (7.257%)
Julius Werner                93 (5.314%)
Zheng Bao                    87 (4.971%)
Martin Roth                  66 (3.771%)
Andrey Petrov                58 (3.314%)

=== Top Reviewers - Number of patches reviewed ===
Martin Roth                  82 (43.850%)
Stefan Reinauer              62 (33.155%)
Paul Menzel                  45 (24.064%)
Aaron Durbin                 28 (14.973%)
Andrey Petrov                13 (6.952%)
Patrick Georgi               12 (6.417%)
Furquan Shaikh                6 (3.209%)
Timothy Pearson               4 (2.139%)
Ronald G. Minnich             4 (2.139%)
Alexander Couzens             4 (2.139%)
Total Reviewers: 22

=== Submitters - Number of patches submitted ===
Martin Roth                  85 (45.455%)
Patrick Georgi               47 (25.134%)
Aaron Durbin                 24 (12.834%)
Stefan Reinauer              20 (10.695%)
Vladimir Serbinenko           4 (2.139%)
Werner Zeh                    2 (1.070%)
Timothy Pearson               2 (1.070%)
Zheng Bao                     1 (0.535%)
Leroy P Leahy                 1 (0.535%)
Ronald G. Minnich             1 (0.535%)
Total Submitters: 10

GSoC 2016

The coreboot project is proud to announce that it has been selected as one of the 2016 Google Summer of Code (GSoC) mentor organizations. GSoC is a program designed to encourage university students age 18 and older to participate in open source projects. This is done by paying students to partner with experienced mentors from the selected open source projects to work on a specific project chosen by the student. This helps the mentor organization by encouraging participation and getting tasks done while helping the student get involved in open source, add projects to their resume, and learn from experienced open-source participants.

Student applications begin next Monday, March 14th, 2016, and close on Friday, March 25th.  Accepted student projects will be announced on April 22nd. Any students who are interested in applying for a coreboot, flashrom, or SerialICE GSoC project should look at the GSoC student terms page, and at both coreboot’s GSoC page and the coreboot / Flashrom / SerialICE projects page.  Projects are not limited to what is currently listed here. Students typically select from these, but if you have other ideas of projects in our space, we’d love to hear about them.

As noted above, coreboot acts as an umbrella organization for other firmware related open-source projects, currently supporting Flashrom and SerialICE. If there are other firmware related projects who would like to be included under the coreboot project for GSoC, please contact the project administrators, Patrick Georgi or Martin Roth.

Finally, if you are a developer who would like to volunteer as a mentor, please contact us. First year volunteers will generally be teamed up with experienced mentors, so don’t worry about not having previous experience. If you’re interested, you can read more about mentoring expectations in the GSoC mentoring Guide.