[GSoC] Ghidra firmware utilities, week 5

Hi everyone. As stated in my previous blogpost, I have been working on a FS loader for Intel Flash Descriptor (IFD) images. The IFD is used on Intel x86 platforms to define various regions in the SPI flash. These may include the Intel ME firmware region, BIOS region, Gigabit ethernet firmware region, etc. The IFD also defines read/write permissions for each flash region, and it may also contain various configurable chipset parameters (PCH straps). Additional information about the firmware descriptor can be found in this helpful post by plutomaniac on the Win-Raid forum, as well as these slides from Open Security Training.

For a filesystem loader, the flash regions are exposed as files. FLMAP0 in the descriptor map and the component/region sections are parsed to determine the base and limit addresses for each region; both IFD v1/v2 (since Skylake) are supported. Ghidra supports nested filesystem loaders, so the FMAP and CBFS loaders that I’ve previously written can be used for parsing the BIOS region.

If you encounter any issues with the IFD FS loader, please feel free to submit an issue report in the GitHub repository.

Plans for this week

I have started working on a filesystem loader for UEFI firmware volumes. In conjunction with the IFD loader, this will allow UEFI firmware images to be imported for analysis in Ghidra (behaving somewhat similar to the excellent UEFITool).

[GSoC] Ghidra firmware utilities, week 4

During the previous week, I worked on additional filesystem loaders to support parsing Flash Map (FMAP) images and the coreboot file system (CBFS). As of this week, these FS loaders are mostly complete, and can be used to import raw binaries within compiled coreboot ROMs. Support for CBFS file compression (with either LZMA or LZ4) is also implemented; compressed files will be automatically extracted. Here are some screenshots of the new FS loaders:

While these might not be the most useful FS loaders (as FMAP and CBFS are mainly used by coreboot itself), I gained additional familiarity with Ghidra’s plugin APIs for FS loaders. This will be useful, as I will be writing additional FS loaders for this project.

Plans for this week

I’ll continue to make minor changes to the existing FS loaders (various cleanups/etc). I’ll also start to write a FS loader for parsing ROMs with an Intel firmware descriptor (IFD), which shouldn’t be too complicated. After this is completed, I plan on writing a FS loader for UEFI firmware volumes (ideally similar to UEFITool or uefi-firmware-parser). I anticipate that this loader will be more complex, so I’ve reserved additional time to ensure its completion.

[GSoC] Ghidra firmware utilities, week 3

Last week, I finalized my work on the PCI option ROM loader, which was the first part described in my initial proposal for this project. This consists of a filesystem loader for hybrid/UEFI option ROMs and a binary loader for x86 option ROMs.

Background information on PCI option ROMs

Option ROMs may contain more than one executable image; for example, a graphics card may have a legacy x86 option ROM for VGA BIOS support as well as a UEFI option ROM to support the UEFI Graphics Output Protocol. x86 option ROMs are raw 16-bit binaries. The entry point is stored as a short JMP instruction in the option ROM header; the BIOS will execute this instruction to jump to the entry point. In contrast, UEFI images contain an UEFI driver, which is a PE32+ binary. This binary can be (and frequently is) compressed with the EFI compression algorithm, which is a combination of Huffman encoding and the LZ77 algorithm.

Filesystem loader

The filesystem loader allows hybrid/UEFI option ROMs to be imported. It also transparently handles the extraction of compressed UEFI executables.

Initially, I attempted to write a Java implementation of the EFI Compression Algorithm for use in the FS loader, but ran into several issues when handling the decompression of certain blocks. I eventually decided to reuse the existing C decompression implementation in EDK2, and wrote a Java Native Interface (JNI) wrapper to call the functions in the C library.

With the FS loader, UEFI drivers in option ROMs can be imported for analysis with Ghidra’s native PE32+ loader.

x86 option ROM binary loader

This loader allows x86 option ROMs to be imported for analysis. Various PCI structures are automatically defined, and the entry function is resolved by decoding the JMP instruction in the option ROM header.

PCI option ROM header data type
PCI data structure data type
Disassembled entry point

Plans for this week

I’ve started to work on filesystem loader for FMAP/CBFS (used by coreboot firmware images). After that, I plan on working on additional FS loaders for Intel flash images (IFD parsing) and UEFI firmware volumes.

As usual, the source code is available in my GitHub repository. Installation and usage instructions are included in the README; feel free to open an issue report if anything goes awry.

[GSoC] Ghidra firmware utilities, weeks 1-2

Hi everyone. I’m Alex James (theracermaster on IRC) and I’m working on developing modules for Ghidra to assist with firmware reverse engineering as a part of GSoC 2019. Martin Roth and Raul Rangel are my mentors for this project; I would like to thank them for their support thus far.

Ghidra is an open-source software reverse engineering suite developed by the NSA, offering similar functionality to existing tools such as IDA Pro. My GSoC project aims to augment its functionality for firmware RE. This project will consist of three parts: a loader for PCI option ROMs, a loader for firmware images, and various scripts to assist with UEFI binary reverse engineering (importing common types, GUIDs, etc).

The source code for this project is available here.

Week 1

During my first week, I started implementing the filesystem loader for PCI option ROMs. This allows option ROMs (and their enclosed images) to be loaded into Ghidra for analysis. So far, option ROMs containing uncompressed UEFI binaries can be successfully loaded as PE32+ executables in Ghidra. The loader also calculates the entry point address for legacy x86 option ROMs.

Plans for this week

So far this week, I’ve worked on writing a simple JNI wrapper for the reference C implementation of the EFI decompressor from EDK2, and have used this to add support for compressed EFI images to the option ROM FS loader. Additionally, I plan on making further improvements to the option ROM loader for legacy option ROMs; while the entry point address is properly calculated, they still have to be manually imported as a raw binary.

2015-08-28 Librem 13: Weekly BIOS Update

Author: Larry.Moberg@puri.sm

This post gives some details on the Librem 15 rev2 prototype. One challenge with developing BIOS is finding parts that can be reused; coreboot makes heavy reuse of certain pieces of code.

Very Similar

Starting with the chips on the back of the mainboard, the 15 prototype uses the same ene KB3930QF-A1 as the Librem 13, and it is configured to read from an external Macronix MX25L512 SPI flash for firmware. The 15 has this SPI chip on the front side of the board between the DIMMs and the USB3 ports.

The 15 prototype uses a Realtek ALC269 codec via the AC’97. The Librem 13 should have a very similar codec.

On the front (the side visible just by removing the back laptop case), the 15 prototype uses an MX25L6406E SPI flash for the BIOS. The Librem 13 prototype uses a GD25Q64B, but other than the Intel Firmware Descriptor fields for JEDEC ID etc, these chips are interchangeable.

Both CPUs are Broadwell-U. They use the same FSP. They both have the LPC bus exposed on pads.

These similarities help us by reducing the amount of variation between the board subdirs in coreboot and can use the same development rig.

Acceptance Test Matrix

We’ve put together the following tests to validate coreboot builds:

  1. Cold boot: memory controller works.
  2. Cold boot: all installed DRAM is online.
  3. Cold boot: graphics controller works.
  4. Cold boot: SATA controller succeeds.
  5. Cold boot: EC controller responds ok to init code.
  6. Cold boot: LCD backlight turns on.
  7. Cold boot: linux boots ok in text mode.
  8. Cold boot: linux boots ok in framebuffer (boot splash) mode.
  9. Cold boot: X initializes the LCD at full native resolution.
  10. Cold boot: X enables hardware acceleration.
  11. Boot time: Cold boot to grub succeeds in less than a set timeout.
  12. Boot time: Reboot from linux back to linux succeeds in less than a set timeout.
  13. Boot time: Power down succeeds in less than a set timeout.
  14. SeaBIOS test: keyboard works.
  15. Grub test: keyboard works.
  16. Grub test: text mode and framebuffer graphics work.
  17. Cold boot to USB linux succeeds. (We plan to use SeaBIOS for boot device selection, barring major bugs.)
  18. Reboot to USB linux succeeds.
  19. EC test: fan spins.
  20. EC test: holding power for >5 seconds forces a power down.
  21. ACPI test: lid switch works.
  22. ACPI test: power button event received ok.
  23. ACPI test: AC power on/off event received ok.
  24. ACPI+EC+battery test: battery percentage works.
  25. Media keys on keyboard work in linux.
  26. Device tests: internal mic, internal speakers, webcam, webcam mic, wifi, bluetooth, hard drive, SSD, SD card, each USB port, headphone jack.
  27. prime95 (one instance bound to each hyperthread) for a fixed time to test CPU thermal management.
  28. glxgears for a fixed time to test GPU thermal management.
  29. During prime95 test, CPU digital thermal sensor should give reasonable results.
  30. Linux suspend ok.
  31. LCD backlight adjustable in linux.
  32. Linux kernel boot messages should not contain too many errors.

The effort to write Free Software implementations for all binary blobs will continue in parallel.

Secondary items would include further tweaks to PCI IRQ routing, additional ACPI tables, and optimizing battery life/power use.

2015-08-14 Librem 13: Weekly Progress Update

A question coreboot developers are commonly asked is this: “can you port coreboot to my board?”

For my first coreboot post I’d like to show some of the steps required to port coreboot to the Librem 13. In particular, this post is a good example of some of the challenges involved in such a port.

This post is also the first weekly progress update for the Librem 13. Please email me with questions or comments: larry.moberg@puri.sm.

LPC Bus

The Librem 13 has convenient test points for the LPC bus. This allows a bed-of-nails test setup to quickly diagnose problems during manufacturing. But it has the added bonus of facilitating coreboot development.

The earliest coreboot stages are the most important to get right. Debugging using port 0x80 writes on the Librem 13 is possible because port 0x80 writes are configured as LPC writes, which can be traced by connecting to the LPC pins.

And…It’s Gone

BIOS development is hard. I applied a little too much force on the SPI flash chip and tore the solder pads off the board.

Pads...gone

I attached the LPC connection to a test setup and didn’t check using a multimeter before applying power. LAD2 was shorted to LAD3. This immediately bricked the laptop without even releasing any smoke. Remembering to double check for shorts is a tedious but important lesson.
Don't Cross The Streams

The LPC bus wires go under the board. Don’t Cross The Streams!

Why It Matters

Imagine a laptop where the LPC bus is only available by soldering directly to the pins of the EC. Yes, they exist! That level of fine soldering is a significant barrier for future coreboot hackers. (The Librem 13’s external USB ports are all USB 3, which makes an EHCI debug port harder, but the LPC bus is a good substitute.)

Porting coreboot to a new laptop takes a lot of time and work. Even a good laptop design like the Librem 13 where the LPC pads are available still has a non-trivial level of engineering work to get to a Free Software BIOS.

Next week, I’ll document the engineering considerations around writing to the SPI flash chip, and how that affects coreboot development.

Flashrom 0.9.4 released – Flashing BIOS/ROM chips from the Unix/Linux command line using various programmers

flashrom logo

Forgot to mention this here: We released flashrom 0.9.4 a few days ago, the latest release of the open-source, GPL'd ROM chip flashing software for Linux, *BSD, DOS, and partially also Windows (work in progress, though).

Here's a quick summary of the release announcement. Some of the noteworthy news items include:

  • Support for new programmers: OpenMoko Neo1973/Neo FreeRunner debug board version 2 or 3, Olimex ARM-USB-TINY, ARM-USB-TINY-H, ARM-USB-OCD, and ARM-USB-OCD-H, Open Graphics Project development card (OGD1), Angelbird Wings PCIe SSD/88SX7042, ITE IT85xx embedded controllers, Intel NICs with parallel flash.
  • Dozens of added flash chips, chipsets, mainboards.
  • Improved Dediprog SF100 support.
  • Add support for more than one Super I/O or EC per machine.
  • Always read the flash chip before writing, for improved error checking and faster programming.
  • Enable write support on NVIDIA MCP6x/MCP7x.
  • Lots of bugfixes, documentation fixes, internal improvements, etc.

Get the latest release tarball, or download and build the most recent version via Subversion:

  $ svn co svn://flashrom.org/flashrom/trunk flashrom
  $ cd flashrom
  $ make

I already updated the Debian package to 0.9.4 (it has also already migrated to Debian testing and Ubuntu), other people have updated Fedora, Gentoo, NetBSD etc. etc.

There's already a huge amount of patches queued for the next release, including support for even more programmers, PowerPC support (tested on Mac Mini and others), and of course the usual "more boards, more chips" items...

u-boot as coreboot payload

U-boot is bootloader on ARMs, PowerPCs and other platforms, it has a nice set of commands and in general it feels like a small operating system. I’m not certainly sure if it is good direction, please feel free to compare with UEFI 😉 but I simply miss it on x86. I work at SYSGO with u-boot in daily basis and even port it to different boards/platforms. The x86 is no easy to init and I think this is the reason why there is only one x86 board in whole u-boot tree. This board is called eNET and it has a AMD ELAN SC520 SOC. But luckily, with coreboot we can init much more x86 boards and this leads to natural conclusion to have the u-boot as the coreboot payload. I would like to share with you part of this “fantastic” hacking journey to make it happen. Continue reading u-boot as coreboot payload

openbiosprog-spi, a DIY Open Hardware and Free Software USB-based SPI BIOS chip flasher using flashrom

openbiosprog-spi device

If you're following me on identi.ca you probably already know that I've been designing a small PCB for a USB-based SPI chip programmer named openbiosprog-spi.

The main use-case of the device is to help you recover easily from a failed BIOS upgrade (either due to using an incorrect BIOS image, due to power outages during the flashing progress, or whatever). The device only supports SPI chips, as used in recent mainboards (in DIP-8 form factor, or via manual wiring possibly also soldered-in SO-8 variants). It can identify, read, erase, or write the chips.

Of course the whole "toolchain" of software tools I used for creating the hardware is open-source, and the hardware itself (schematics and PCB layouts) are freely released under a Creative Commons license (i.e., it's an "Open Hardware" device). The user-space source code is part of flashrom (GPL, version 2), the schematics and PCB layouts are licensed under the CC-BY-SA 3.0 license and were created using the open-source Kicad EDA suite (GPL, version 2).

openbiosprog-spi schematics
openbiosprog-spi Kicad PCB layout

The schematics, PCB layouts, and other material is available from gitorious:

  $ git clone git://gitorious.org/openbiosprog/openbiosprog-spi.git

You can also download the final Gerber files (ZIP) for viewing them, or sending them to a PCB manufacturer.

Some more design notes:

  • The device uses the FTDI FT2232H chip as basis for USB as well as for handling the actual SPI protocol in hardware (MPSSE engine of the FT2232H).
  • Attaching the SPI chip:
    • There's a DIP-8 socket on the device so you can easily insert the SPI chip you want to read/erase/program.
    • Optionally, if you don't want a DIP-8 socket, you can solder in a pin-header with 8 pins, which allows you to connect the individual pins to the SPI chip via jumper wires or grippers/probes.
  • The PCB board dimensions are 44mm x 20mm, and it's a 2-layer board using mostly 0603 SMD components.

Basic usage example of the device on Linux (or other OSes supported by flashrom):

  $ flashrom -p ft2232_spi:type=2232H,port=A -r backup.bin (reads the current chip contents into a file)

openbiosprog-spi PCBs
openbiosprog-spi parts list

Over at the main projects page of openbiosprog-spi at

  http://randomprojects.org/wiki/Openbiosprog-spi

I have put up a lot more photos and information such as the bill of materials, the Kicad settings I used for creating the PCBs, the Gerber files and the Excellon drill files and so on.

The first few prototype boards I ordered at PCB-POOL.COM (but you can use any other PCB manufacturer of course), the bill of materials (BOM) lists the Mouser and CSD electronics part numbers and prices, but you can also buy the stuff elsewhere, of course (Digikey, Farnell, whatever).

I already hand-soldered one or two prototypes and tested the device. Both hardware and software worked fine basically, you just need a small one-liner patch to fix an issue in flashrom, but that should be merged upstream soonish.

In order to make it easy for interested users to get the PCBs I'll probably make them available in the BatchPCB Market Place soonish, so you can easily order them from there (you do still need to solder the components though). Note: I'm not making any money off of this, this is a pure hobby project.

All in all I have to say that this was a really fun little project, and a useful one too. This was my first hardware project using Kicad (I used gEDA/PCB, also an open-source EDA toolsuite, for another small project) and I must say it worked very nicely. I didn't even have to read any manual really, it was all pretty intuitive. Please consider not using Eagle (or other closed-source PCB software) for your next Open Hardware project, there are at least two viable open-source options (Kicad, gEDA/PCB) which both work just fine.