[GSoC] Ghidra firmware utilities, week 4

During the previous week, I worked on additional filesystem loaders to support parsing Flash Map (FMAP) images and the coreboot file system (CBFS). As of this week, these FS loaders are mostly complete, and can be used to import raw binaries within compiled coreboot ROMs. Support for CBFS file compression (with either LZMA or LZ4) is also implemented; compressed files will be automatically extracted. Here are some screenshots of the new FS loaders:

While these might not be the most useful FS loaders (as FMAP and CBFS are mainly used by coreboot itself), I gained additional familiarity with Ghidra’s plugin APIs for FS loaders. This will be useful, as I will be writing additional FS loaders for this project.

Plans for this week

I’ll continue to make minor changes to the existing FS loaders (various cleanups/etc). I’ll also start to write a FS loader for parsing ROMs with an Intel firmware descriptor (IFD), which shouldn’t be too complicated. After this is completed, I plan on writing a FS loader for UEFI firmware volumes (ideally similar to UEFITool or uefi-firmware-parser). I anticipate that this loader will be more complex, so I’ve reserved additional time to ensure its completion.

[GSoC] Coreboot Coverity, Week 3

Hello again! This is a continuation of my posts about fixing the Coverity issues in coreboot. This week’s plan was to tackle the 28 issues in northbridge/intel, which turned out to be much easier than I expected, since I’m already done! With that out of the way, I’m going to begin working on northbridge/via and southbridge. For the curious, here is the project timeline for entire summer. (I had wanted to include this in last week’s post, but hadn’t figured out how to do tables in WordPress yet.)

Week Components Issues
May 6 to 10 util 22
May 13 to 17 util, payloads 22
May 20 to 24 arch, drivers 20
May 27 to 31 commonlib, cpu, lib, mainboard 22
June 3 to 7 northbridge/amd 21
June 10 to 14 northbridge/intel 28
June 17 to 21 northbridge/via, southbridge 22
June 24 to 28 soc/intel 21
July 1 to 5 soc/rockchip, soc/nvidia 20
July 15 to 19 soc/misc, vendorcode/cavium 26
July 22 to 26 vendorcode/amd 21
July 29 to Aug 2 vendorcode/amd 21
Aug 5 to 9 vendorcode/amd 20
Aug 12 to 16 vendorcode/amd 20
Aug 19 to 23 vendorcode/amd 20

As you can see, there are a lot of issues in the AMD vendorcode. This consists primarily of AGESA, AMD’s framework for initialization of their 64 bit platforms (somewhat similar to Intel’s FSP). This code is somewhat … dense (someone on IRC described it as a “sea of abstraction”), so I made sure to leave plenty of time for it. As always, you can keep up to date on my current progress on Gerrit.

PS: As an extra bonus, here is a picture of my new BeagleBone Black!

I recently got a ThinkPad T500 to practice installing coreboot on, and I needed some sort of external programmer to flash the SOIC. There are many options available (flashrom has a whole list here), but a single-board computer like this is one of the closest you can get to “plug-and-play.” There are many other popular boards (notably the Raspberry Pi), but the BBB doesn’t require any binary blobs to boot, and is open source hardware too. The only thing I’m waiting for now is an Atheros ath9k wireless card, which runs without any binary firmware. (Hey, if you’re gonna go freedom, you gotta go all the way.)

[GSoC] Ghidra firmware utilities, week 3

Last week, I finalized my work on the PCI option ROM loader, which was the first part described in my initial proposal for this project. This consists of a filesystem loader for hybrid/UEFI option ROMs and a binary loader for x86 option ROMs.

Background information on PCI option ROMs

Option ROMs may contain more than one executable image; for example, a graphics card may have a legacy x86 option ROM for VGA BIOS support as well as a UEFI option ROM to support the UEFI Graphics Output Protocol. x86 option ROMs are raw 16-bit binaries. The entry point is stored as a short JMP instruction in the option ROM header; the BIOS will execute this instruction to jump to the entry point. In contrast, UEFI images contain an UEFI driver, which is a PE32+ binary. This binary can be (and frequently is) compressed with the EFI compression algorithm, which is a combination of Huffman encoding and the LZ77 algorithm.

Filesystem loader

The filesystem loader allows hybrid/UEFI option ROMs to be imported. It also transparently handles the extraction of compressed UEFI executables.

Initially, I attempted to write a Java implementation of the EFI Compression Algorithm for use in the FS loader, but ran into several issues when handling the decompression of certain blocks. I eventually decided to reuse the existing C decompression implementation in EDK2, and wrote a Java Native Interface (JNI) wrapper to call the functions in the C library.

With the FS loader, UEFI drivers in option ROMs can be imported for analysis with Ghidra’s native PE32+ loader.

x86 option ROM binary loader

This loader allows x86 option ROMs to be imported for analysis. Various PCI structures are automatically defined, and the entry function is resolved by decoding the JMP instruction in the option ROM header.

PCI option ROM header data type
PCI data structure data type
Disassembled entry point

Plans for this week

I’ve started to work on filesystem loader for FMAP/CBFS (used by coreboot firmware images). After that, I plan on working on additional FS loaders for Intel flash images (IFD parsing) and UEFI firmware volumes.

As usual, the source code is available in my GitHub repository. Installation and usage instructions are included in the README; feel free to open an issue report if anything goes awry.

[GSoC] Coreboot Coverity, Introduction

Hello everyone! My name is Jacob Garber, and I am a student in this year’s GSoC 2019! My project is on making coreboot Coverity clean. Coverity is a free static-analysis tool for open source projects that searches for common coding mistakes and errors, such as buffer overruns, null pointer dereferences, and integer overflow. Coverity automatically analyzes the coreboot codebase and flags issues it finds, and my job is to classify them into bugs and false-positives and patch them if I can. You can check the Coverity overview for coreboot here, though seeing the issue tracker itself requires registration. At the beginning of the summer, coreboot had over 380 flagged issues, but it’s now down to 303, so we’re making progress! I plan to address 20-30 issues per week depending on the source component, which so far has gone surprisingly well (surprising, in the sense that coming into the summer I knew very little about coreboot or firmware development in general). For the curious, you can see the history and progress of all my changes on Gerrit. My mentors for this project are Patrick Georgi, Martin Roth, and David Hendricks, who have all been extremely helpful in guiding me through the development process, reviewing my patches, and answering my many questions. Thank you all.

Now, fixing Coverity bugs isn’t the only thing I’d like to do this summer. As I said before, I’d like to learn more about coreboot, and what better way to do that than installing it on a laptop! My current laptop is an old 2011 Macbook Air, which is surprisingly close to getting coreboot support (many thanks to Evgeny Zinoviev). However, I am (slightly) hesitant about installing yet-experimental firmware on my one and only development machine, so until then I picked up an old Thinkpad T500 to practice on. This laptop has the advantage of being able to run blob-free, and if in the very unlikely event I end up bricking it, who cares! (I mean, I’ll care, but it was a worthy sacrifice.) I also bought a BeagleBone Black to try out external flashing and was hoping to include a picture today, but the shipping was delayed. You’ll have to wait until next week!

[GSoC] Ghidra firmware utilities, weeks 1-2

Hi everyone. I’m Alex James (theracermaster on IRC) and I’m working on developing modules for Ghidra to assist with firmware reverse engineering as a part of GSoC 2019. Martin Roth and Raul Rangel are my mentors for this project; I would like to thank them for their support thus far.

Ghidra is an open-source software reverse engineering suite developed by the NSA, offering similar functionality to existing tools such as IDA Pro. My GSoC project aims to augment its functionality for firmware RE. This project will consist of three parts: a loader for PCI option ROMs, a loader for firmware images, and various scripts to assist with UEFI binary reverse engineering (importing common types, GUIDs, etc).

The source code for this project is available here.

Week 1

During my first week, I started implementing the filesystem loader for PCI option ROMs. This allows option ROMs (and their enclosed images) to be loaded into Ghidra for analysis. So far, option ROMs containing uncompressed UEFI binaries can be successfully loaded as PE32+ executables in Ghidra. The loader also calculates the entry point address for legacy x86 option ROMs.

Plans for this week

So far this week, I’ve worked on writing a simple JNI wrapper for the reference C implementation of the EFI decompressor from EDK2, and have used this to add support for compressed EFI images to the option ROM FS loader. Additionally, I plan on making further improvements to the option ROM loader for legacy option ROMs; while the entry point address is properly calculated, they still have to be manually imported as a raw binary.

[GSoC] Multiple status registers, block protection and OTP support, wrap-up (1/2)

Hello! ๐Ÿ™‚

GSoC 2016 coding period has come to an end and mentor’s evaluating students this week. It has been an enriching 13 weeks of reading datasheets, designing structures, coding, learning and hanging out over IRC! ๐Ÿ˜› I’d like to take this opportunity to present my work and details on how to use it. ๐Ÿ™‚

Firstly, to offer context to the work, here is a list of public mails and blog posts. These should give an idea as to how the discussions and work evolved. A lot of the discussions have happened over IRC, but #flashrom does not keep any logs.

The patch sets that I sent to the mailing list can be found at –

  1. Multiple status register and access protection infrastructure
  2. OTP/Security registers infrastructure
  3. Dummy chips

You can also find these over at flashrom’s patchwork. The mailing list is where the review happens (although a better alternative, IMHO, is Gerrit which coreboot uses). The patches aren’t currently merged and are under review. In any case, you are most welcome to join review (which will likely be very helpful for me). ๐Ÿ™‚ If you’d like to look at something more on the bleeding edge, then I invite you to my GitHub.

Now, moving on how to use the work. The most exhaustive documentation on how to use it is the code itself :P, but in the following list I attempt to list scenarios –

  • For SPI chips that have multiple status registers, flashrom’s verbose output will print the status register bits and there values. Most bits are named, i.e., the datasheet refers to the bit by an abbreviation, for instance, WEL for Write Enable Latch, WIP for Work In Progress, BP for Block Protect, LB for Lock Bit and so on. The verbose output will print these names, both in abbreviated and long forms, for most chips (and these abbreviations tend to be generic across many manufacturers). However, the process for adding new chips that leverage this, and adding new bits, is a fairly easy task (I would invite you to have a look at the code ๐Ÿ˜‰ for more details). The verbose output also prints the write protection mode for status register(s) in effect (software protected, hardware protected, power cycle lock down and so on).
  • In case you want to disable or enable (a particular type) write protection for status register, you can use the --wp-disable or --wp-enable[=MODE] respectively (where MODE is either of software, hardware, power cycle or permanent – you are encouraged to have a look at the man page ๐Ÿ™‚ for more details)
  • In case you want to protect a particular range of an SPI chip from writes or erases, you will need to alter the BP, TB or SEC bits. Currently, there is a CLI that will enable you to accomplish all that. ๐Ÿ˜› First, you’ll want to look at the list of ranges your SPI chip supports – run flashrom with --wp-list. Take note of the start address and the length of the memory range you want to protect. Then again run flashrom with --wp-set-range start=0xfff000,len=4 (0xfff000 and 4 are for representational purpose only). By now the memory range is protected, but you can additionally enable status register write protection by following what the foregoing point described.
  • For SPI chips that support OTP, you can read, write and erase OTP regions (of course for supported chips :P). For OTP operations, you have at your disposal --print-otp-status, --read-otp [,reg=], --write-otp file=[,reg=], --erase-otp [reg=] and --lock-otp [reg=]. You can read the OTP memory to a file, or you can write to the OTP region from a file, very much like reading and writing from/to SPI chip. For more details, I would again like to point you to the man page. ๐Ÿ™‚

Since this is a work-in-progress, the CLI may change (and is very likely). Currently around 10% of SPI chips use this new infrastructure. Models of a few manufacturers (and especially exotic ones like Atmel) are yet to be fully incorporated. You are most welcome to add support for new chips or update the existing ones to support new infrastructure. ๐Ÿ™‚

I would like to sincerely thank my mentors Stefan and David for their support and help. I am indebted to them for this opportunity and I hope that we continue to share this relationship in the future while I continue to explore and contribute to flashrom. It has been a pleasure getting to know each of them. I’d also like to thank Urja for pitching in from time to time ๐Ÿ™‚ It was fun hanging out over IRC and helping folks asking questions there. And I am looking forward to it for years to come. ๐Ÿ˜›

In the next and final part of this post, I will highlight how we intend to improve upon this work in the future, where it will be headed and what more we have in store, so please stay tuned. ๐Ÿ˜‰ Phew, this was a long one, and rightly so as it attempts to summarise a great deal of experiences. If you have any feedback, questions or comments on the blogs or code, please feel to ping me on #flashrom where I am known as hatim. You can also email me at hatim@hatimak.me.

Thanks, and looking forward to hearing from you. ๐Ÿ™‚ See you in the next and final part.

[GSoC] Better RISC-V support, wrap-up

In less than an hour, Google Summer of Code 2016 will be over (at least for us students). In this blog post, I will describe how you can use coreboot on RISC-V.

You can find the complete list of commits that I made during GSoC with this gerrit query.

The details

Compiling spike, the RISC-V instruction-set-level simulator

Spike, also known as riscv-isa-sim, is the reference implementation of RISC-V, and the only RISC-V platform that is currently known to work with coreboot (QEMU is nominally also supported, but the corresponding coreboot code has not been updated in a while).

First, you need to build and install libfesvr:

Then, you can compile and install spike:

  • Clone the spike git repository.
  • Apply the patch in this pull request to make console output possible.
  • Open riscv/processor.cc in a text editor and find processor_t::get_csr. Add a line that reads case CSR_MTIME: return 0;.
  • run ./configure --prefix="$HOME" --with-fesvr="$HOME" && make && make install

Compiling coreboot for RISC-V

  • Clone the coreboot git repository, if you haven’t already
  • Apply Iru Cai’s patch that updates the toolchain to GCC 6.1. You will currently have to fix a merge conflict when you apply this patch, but it’s an easy one.
  • Run make crossgcc-riscv
  • Run make menuconfig to configure coreboot. Select Emulation and SPIKE usb riscv in the Mainboard menu.
  • Run make
  • Run util/riscvtools/make-spike-elf.sh build/coreboot.rom build/coreboot.elf
  • Start coreboot by running spike build/coreboot.elf. You should see a few pages of output, ending in Payload not loaded.

Compiling and running Linux

  • Clone the riscv-linux git repository and check out the priv-1.9 branch
  • Apply this patch that allows linux to be started in machine-mode.
  • Download a copy linux 4.6.x from kernel.org, and unpack it. I’ll assume version 4.6.2 is used.
  • cd into linux-4.6.2/arch and symlink the arch/riscv directory from riscv-linux
  • Back in linux-4.6.2, run make O=build ARCH=riscv defconfig; cd into the newly created build directory.
  • Run make ARCH=riscv menuconfig. In the “General Setup” menu of the linux menuconfig, enter path/to/coreboot/util/crossgcc/xgcc/bin/riscv64-elf- as the cross-compiler tool prefix.
  • Run make ARCH=riscv vmlinux to compile linux.
  • Open vmlinux in a hex editor, such as hexer. Change the 8-byte number at 0x18 to 00 00 00 90 00 00 00 00; Add 00 00 00 90 00 00 00 00 to the numbers at 0x58 and 0x90, to load linux at physical addresses within RAM. It’s unfortunate that I have to recommend this step, but I did not come up with a better fix yet.

Next, you need to add vmlinux to coreboot:

  • Return to the coreboot directory.
  • Apply the remaining coreboot patches that are tagged riscv.
  • In the Payload menu, select An ELF executable payload. Instead of payload.elf, select the vmlinux file.
  • Run make and util/riscvtools/make-spike-elf.sh build/coreboot.rom build/coreboot.elf again.
  • Run spike build/coreboot.elf. You should now see a Linux boot, at least partially.

Future work

Even though my GSoC is over, coreboot’s support for RISC-V can still be improved, and I intend to fix at least some of the following things:

  • As you can see above, running coreboot and linux on RISC-V currently isn’t straight-forward, but involves a few manual steps.
  • There are other RISC-V platforms that I’d like to see coreboot on, such as lowRISC.
  • Linux does not completely boot, i.e. into userspace. There are still some bugs to be ironed out.
  • Automatic testing could be used to detect regressions.
  • I only tested coreboot on RISC-V with Linux; support for other operating systems or payloads is welcome.

Acknowledgements

I’d like to thank Ron Minnich and Furquan Shaikh for being good mentors, and everyone in the coreboot community for being helpful and friendly.

[GSOC] Panic Room, week #8,*

What have you worked on during the past few weeks?

I tried to finish one of the optional goals of my proposal:

a way to access a payload (possibly a recovery) after the hardware initialisation is complete but the OS has still not taken over.

To define it in more practical terms, I wanted to replace the running payload at the press of a predefined button.

I started by looking into the SMM (System Management Mode) and in particularย the SMI (System Management Interrupts) handlers defined by each target board in the coreboot tree.

That was the easy part, it took just a bit of probing my board (Lenovo x60) and a serial cable to retrieve some of the SMI button codes (I planned on using the ThinkAdvantage button, code 0x19).
I added it to the appropriate smihandler.c file, defined the behaviour and that was about it.

The more difficult part was actually implementing a way to make it possible to boot a new payload while in SMM.
SMM can be considered a separate “module” from romstage, etc, this consequently means that it does not have access to all of the same functions.
In particular it cannot access the load_payload() and run_payload() functions defined inside prog_loaders.c, which, as the name implies, are necessary in order to use a payload.

In order to solve the problem I came up with two designs:

  • The first design is quite simple, but in my opinion a bit messy.
    It involves including all of the missing source filesย into the SMM module at compile time, a.k.a adding more entries inside the Makefile.inc scattered around the source tree.
    I went down this route for a while until I stumbled on a function that needed to use a malloc(), this was expected so I included the definition of that function.
    It then asked to have access to the heap (_eheap) in order to allocate the memory requested, the problem is that the heap is not defined inside SMM.
    I grepped through the source code and found very little about what I was supposed to do in order to define it, so I had to let it go.
    (To add a little bit of context, this was yesterday and if you take a look at this post’s date you’ll realise that there is only 1 day left of GSoC, panic!)
  • The second approach consists of leaving SMM mostly as it is now, and instead find a way to communicate with ramstage, in order to boot the desired payload.
    The thought process was that, if cannot call a function directly from SMM, I would have to use the functions already in place in ramstage, unfortunately those functions are called only once, after the initialisation is done.
    This is where the CMOS memory comes in, I modifiedย the payload loading process and instead of using an hard-coded path defined at compile-time, I made it check for a string inside CMOS and use that as the payload path.
    This method, however, also presented some problems:
    – It seems that currently cmos.defaults is completely ignored and the CMOS string values are not initialised as intended (at least on my board), therefore the payload path resulted always blank.
    – There needs to be quite a bite of space in the CMOS to keep the path there.
    The usual path (fallback/payload), occupies around 160 contiguous bits of the 1024 available ones, which also need to share space with other important configuration bits such as RTC, etc.
    In order to circumvent these limitations it would be possible to replace the string path in the CMOS with an enumeration of the possible paths, this however would need to be defined, maintained and shared both inย the CMOS and the rest of coreboot.
    However, same as above, I ran out of time.

That about sums it up for the past few weeks.

Thank you for reading and have a nice week.

[GSoC] Better RISC-V Support, week #10

This past week, I worked on the virtual memory initialization code of coreboot on RISC-V. The first part of this was to update encoding.h a file that defines constants such as bit masks which are necessary for interacting with RISC-V’s Control and Status Registers. As a result, I also had to change a few files that relied on outdated constants. Then I wrote some code to walk the page table structures, and fixed one or two bugs in the page table setup code. Unfortunately my patches aren’t as finished as I would like them to be.

When I tested my changes with a Linux payload (which I had to edit the ELF headers of, because Linux uses virtual addresses, while cbfstool and the SELF loader use physical addresses), I stumbled upon another strange error: I get a store access fault in the instruction of the trap handler that saves the first register, even if I initialize the stack pointer to a value that’s within the RAM. When the trap handler runs again to handle this store access fault, it runs without any problems. This fault is especially confusing, because machine mode should always be able to access RAM through its physical addresses.

What’s next?

During the next week, I’ll be traveling and won’t be able to work on coreboot. When I return, I will rework my patches so they can be merged, and hopefully understand the aforementioned access fault problem. Properly set-up page tables should bring me a step closer to running Linux on coreboot/RISC-V (without bbl in the middle).

If time permits, I will start porting coreboot to the Nexys 4 DDR devboard.

[GSoC] Multiple status registers, block protection and OTP support, week #6, 7, 8 and 9

Hello! I have been away for some time now, so this is going to be a longer post. I hope you have missed hearing from me ๐Ÿ˜› In this post I will talk about my work in the weeks post-midterm evaluations. After a discussion with my mentors in the midterm evaluations week, we decided to shift focus towards the first three phases of my GSoC proposal for the remainder of the duration. Work on the final phase will be done after GSoC along with the more long-term goals that have come up as I have been making progress.

I submitted patches (finally ;)) to the mailing list. The set of patches adds multiple status register, and block protection infrastructure. I have also added a command-line interface to expose the new functionality. Although I am not sure that the exact wording of the CLI is most optimum, but I did not spend a lot of time on that because IMO it is a rather subjective issue and altering it is not a difficult task. The set of patches also adds support for new infrastructure to around 90 existing chips. I am still waiting to receive feedback and review on them. (My mentors had been slightly busier then.) I am also investigating adding support for access protection to non-SPI chips. This isn’t on the highest priority (more like a long-term goal), but once the SPI infrastructure gets merged, I will begin writing code for that.

Based on the initial prototypes I built (here, here and here), we had decided to use pointers to new structs instead of fully embedding them in struct flashchip. This decision really started to show when I was adding support for existing chips – with only 25 unique struct definitions we were able to support those 90 chips! ๐Ÿ˜‰ One of the problems I faced was that I needed to test the new infrastructure, but doing so on a physical chip would be cumbersome. So that problem was solved by adding a dummy chip to use with flashrom’s dummy programmer. (At that time the code was the dummy chip was messy and something I would be ashamed to put up publicly, but now I have improved upon it! :P)

Currently I am working on finalising the OTP/security register(s) patches – more specifically, adding support to existing chips, code cleanup and documentation. I will be able to send them to the mailing list in a few days. In my research on Eon, GigaDevice and Winbond chips, 2 distinct models for OTP were observed – the GigaDevice and Winbond model with security register(s), and the Eon model with a security sector.

The Security Register(s) model has 3 separate opcodes for read, program and erase – 0x48, 0x42 and 0x44 respectively. A chips can have multiple security registers (most commonly 3, but as high as 4) with each register being anywhere between 128 bytes to 1024 bytes in size (most commonly 512 bytes and then 256 bytes). Usually chips have a lock bit (LB1, LB2, …) in the status register that correspond to respective security registers. These one-time programmable bits are changed using the standard WRSR instruction. Some chips have a single lock bit that controls OTP status for all security registers.

The Security Sector model has a separate sector which can be operated in the OTP mode. OTP mode is entered with opcode 0x3A and exited by sending WRDI (0x04) instruction. While in the OTP mode, the sector behaves just like any other sector – normal read, program and erase instructions apply. The SRP/SRWD bit is served as OTP bit while in OTP mode. Issuing the WRSR command (irrespective of the data sent along with it) will cause the one-time programmable OTP bit to be set.

One of the recurrent issues (for the lack of a better word, I don’t think of it to be an issue really ;)) is that many chips I have based my research on, are not originally supported by flashrom (perhaps unfortunate siblings of the same family that didn’t find support in flashrom earlier xD). I don’t call it an issue per se because after I have submitted my patches flashrom will end up supporting even more chips, but since I have to write more code it might take slightly longer to submit the patches.

There is a third model which is dominantly followed by Spansion chips and a couple of AMIC chips (some AMIC chips follow one of the earlier models – it’s like AMIC couldn’t decide which one to stick to or they probably had different teams working on it! :P). Similar to the security sector design, these chips also have a separate OTP sector but instead of storing configuration in the register, a byte within the sector is allocated for storing the configuration data. I have planned to support this model in the next revision of patches, after the upcoming ones get reviewed and merged.

Thanks for your time, it was nice to get back in touch with you! ๐Ÿ™‚
(Phew, that was long! :P)