Tag: flashrom

[GSoC] Optimize Erase Function Selection, wrap-up

GSoC 2022 coding period is about to come to an end this week. It has been an enriching 12 weeks of reading old code, designing algorithms and structures, coding, testing, and hanging out over IRC! I’d like to take this opportunity to present my work and details on how it has impacted Flashrom. 🙂

You can find the complete list of commits I made during GSoC with this gerrit query. Some of the patches aren’t currently merged and are under review. In any case, you are most welcome to join the review (which will likely be very helpful for me).

Definitions

Let me clarify some terminologies used hereafter

  • Region: A list of contiguous addresses on the chip
  • Layout: List of regions
  • Locked region: There are some regions on the chip that cannot be accessed directly or in some cases cannot be accessed at all.
  • Page: The entire flash memory is divided into uniform pages that can be written on. You have to write on an entire page at once.
  • Erase block/sector: According to each erase function, flash memory is divided into regions (not necessarily uniform sized) that can be erased. You have to erase an entire erase block at once. So each erase function has a different erase block layout.
  • Sub-block: An erase block is a sub-block of another erase block if it is completely inside the latter.
  • Homogenous erase layout: All erase blocks in a layout are of uniform size.
  • Nonhomogenous erase layout: There might be erase blocks of varying sizes in this layout.

Need for this project

To change any 0 to a 1 in the contents of a NOR-flash chip, a full block of data needs to be erased. So to change a single byte, it is sometimes required to erase a whole block, which we’ll call erase overhead. Most flash chips support multiple erase-block sizes. Flashrom keeps a list of these sizes and ranges for each supported flash chip and maps them to internal erase functions. Most of these lists were sorted by ascending size. I added a simple test to ensure this and corrected the ones which gave errors.

Earlier, Flashrom tried all available erase functions for a flash chip during writes. Usually, only the first function was used, but if anything went wrong on the wire, or the programmer rejected a function, it falls back to the next. As the functions are sorted by size, this results in a minimum erase overhead.

However, if big portions of the flash contents are re-written, using the smallest erase block size results in transaction overhead, and also erasing bigger blocks at once often results in shorter erase times inside the flash chip.

So in a nutshell, the erase function selection in Flashrom was sequential, i.e. they started from the function with the smallest erase block and moved to the next one only in case of an error. This was inefficient in cases where bigger chunks of the flash had to be erased.

Current implementation

After rigorous discussion with my mentors and with the community, we decided on a look-ahead algorithm for selecting the erase functions or the erase blocks to be more precise. From the layout of erase blocks as seen by different erase functions, we make an optimal selection.

We start with the first erase function (remember this erase function has the smallest erase blocks) and mark those blocks which lie completely inside the region to be erased. Then moving on to subsequent erase layouts and marking those erase blocks that have more than half of the sub-blocks marked and then unmarking the marked sub-blocks. Finally, erase all the marked erase blocks. This strategy ensures that if we have a large contiguous region to be erased then we would use an appropriately large erase function to erase it.

The above-mentioned strategy had some challenges and required a few tweaks:

  • The region to be erased may not be aligned to any erase blocks. This could be easily overcome by extending the ‘to be erased’ region to align it with some erase block.
  • The above solution of extending the region brought a new challenge – the newly aligned region might overlap with write-protected regions. In this case, the erase function would fail to erase the erase block containing this region.
  • It may happen that some erase functions are not supported by the programmer. This would lead to catastrophic failure. To this end, we thought of first getting a list of erase functions supported by the programmer and then running the algorithm on this subset of erase functions.

So with this, we were all set to begin the implementation of the algorithm. Or so I thought. Flashrom had stored information regarding the erase layout in a nested format which was very difficult to use so I had to flatten them out and fit them in a data structure that would give me quick information about sub-blocks as well. 😮‍💨

For a better understanding of the algorithm, I made the following flowchart.

Testing

Testing on hardware is difficult because you need all the different chips and programmers and test on each one of them to be sure that your code is working correctly. On top of that, identifying all possible scenarios adds to the burden. This definitely takes a lot of time and effort and my mentors and the community is helping a lot with it.

The current progress of testing can be seen here. Feel free to suggest more cases and help with the testing. It will definitely help a lot.

Future work

Definitely, the code or algorithm isn’t fully optimized and can be improved further. Also one can fix the progress bar feature to Flashrom, which got broken due to my code😅.

Acknowledgments

I’d like to thank Thomas Heijligen and Simon Buhrow for being excellent mentors, Nico for providing constant code reviews and giving regular inputs, and everyone in the Flashrom community for being helpful and friendly.

[GSoC] Multiple status registers, block protection and OTP support, wrap-up (1/2)

Hello! 🙂

GSoC 2016 coding period has come to an end and mentor’s evaluating students this week. It has been an enriching 13 weeks of reading datasheets, designing structures, coding, learning and hanging out over IRC! 😛 I’d like to take this opportunity to present my work and details on how to use it. 🙂

Firstly, to offer context to the work, here is a list of public mails and blog posts. These should give an idea as to how the discussions and work evolved. A lot of the discussions have happened over IRC, but #flashrom does not keep any logs.

The patch sets that I sent to the mailing list can be found at –

  1. Multiple status register and access protection infrastructure
  2. OTP/Security registers infrastructure
  3. Dummy chips

You can also find these over at flashrom’s patchwork. The mailing list is where the review happens (although a better alternative, IMHO, is Gerrit which coreboot uses). The patches aren’t currently merged and are under review. In any case, you are most welcome to join review (which will likely be very helpful for me). 🙂 If you’d like to look at something more on the bleeding edge, then I invite you to my GitHub.

Now, moving on how to use the work. The most exhaustive documentation on how to use it is the code itself :P, but in the following list I attempt to list scenarios –

  • For SPI chips that have multiple status registers, flashrom’s verbose output will print the status register bits and there values. Most bits are named, i.e., the datasheet refers to the bit by an abbreviation, for instance, WEL for Write Enable Latch, WIP for Work In Progress, BP for Block Protect, LB for Lock Bit and so on. The verbose output will print these names, both in abbreviated and long forms, for most chips (and these abbreviations tend to be generic across many manufacturers). However, the process for adding new chips that leverage this, and adding new bits, is a fairly easy task (I would invite you to have a look at the code 😉 for more details). The verbose output also prints the write protection mode for status register(s) in effect (software protected, hardware protected, power cycle lock down and so on).
  • In case you want to disable or enable (a particular type) write protection for status register, you can use the --wp-disable or --wp-enable[=MODE] respectively (where MODE is either of software, hardware, power cycle or permanent – you are encouraged to have a look at the man page 🙂 for more details)
  • In case you want to protect a particular range of an SPI chip from writes or erases, you will need to alter the BP, TB or SEC bits. Currently, there is a CLI that will enable you to accomplish all that. 😛 First, you’ll want to look at the list of ranges your SPI chip supports – run flashrom with --wp-list. Take note of the start address and the length of the memory range you want to protect. Then again run flashrom with --wp-set-range start=0xfff000,len=4 (0xfff000 and 4 are for representational purpose only). By now the memory range is protected, but you can additionally enable status register write protection by following what the foregoing point described.
  • For SPI chips that support OTP, you can read, write and erase OTP regions (of course for supported chips :P). For OTP operations, you have at your disposal --print-otp-status, --read-otp [,reg=], --write-otp file=[,reg=], --erase-otp [reg=] and --lock-otp [reg=]. You can read the OTP memory to a file, or you can write to the OTP region from a file, very much like reading and writing from/to SPI chip. For more details, I would again like to point you to the man page. 🙂

Since this is a work-in-progress, the CLI may change (and is very likely). Currently around 10% of SPI chips use this new infrastructure. Models of a few manufacturers (and especially exotic ones like Atmel) are yet to be fully incorporated. You are most welcome to add support for new chips or update the existing ones to support new infrastructure. 🙂

I would like to sincerely thank my mentors Stefan and David for their support and help. I am indebted to them for this opportunity and I hope that we continue to share this relationship in the future while I continue to explore and contribute to flashrom. It has been a pleasure getting to know each of them. I’d also like to thank Urja for pitching in from time to time 🙂 It was fun hanging out over IRC and helping folks asking questions there. And I am looking forward to it for years to come. 😛

In the next and final part of this post, I will highlight how we intend to improve upon this work in the future, where it will be headed and what more we have in store, so please stay tuned. 😉 Phew, this was a long one, and rightly so as it attempts to summarise a great deal of experiences. If you have any feedback, questions or comments on the blogs or code, please feel to ping me on #flashrom where I am known as hatim. You can also email me at hatim@hatimak.me.

Thanks, and looking forward to hearing from you. 🙂 See you in the next and final part.

[GSoC] Multiple status registers, block protection and OTP support, week #6, 7, 8 and 9

Hello! I have been away for some time now, so this is going to be a longer post. I hope you have missed hearing from me 😛 In this post I will talk about my work in the weeks post-midterm evaluations. After a discussion with my mentors in the midterm evaluations week, we decided to shift focus towards the first three phases of my GSoC proposal for the remainder of the duration. Work on the final phase will be done after GSoC along with the more long-term goals that have come up as I have been making progress.

I submitted patches (finally ;)) to the mailing list. The set of patches adds multiple status register, and block protection infrastructure. I have also added a command-line interface to expose the new functionality. Although I am not sure that the exact wording of the CLI is most optimum, but I did not spend a lot of time on that because IMO it is a rather subjective issue and altering it is not a difficult task. The set of patches also adds support for new infrastructure to around 90 existing chips. I am still waiting to receive feedback and review on them. (My mentors had been slightly busier then.) I am also investigating adding support for access protection to non-SPI chips. This isn’t on the highest priority (more like a long-term goal), but once the SPI infrastructure gets merged, I will begin writing code for that.

Based on the initial prototypes I built (here, here and here), we had decided to use pointers to new structs instead of fully embedding them in struct flashchip. This decision really started to show when I was adding support for existing chips – with only 25 unique struct definitions we were able to support those 90 chips! 😉 One of the problems I faced was that I needed to test the new infrastructure, but doing so on a physical chip would be cumbersome. So that problem was solved by adding a dummy chip to use with flashrom’s dummy programmer. (At that time the code was the dummy chip was messy and something I would be ashamed to put up publicly, but now I have improved upon it! :P)

Currently I am working on finalising the OTP/security register(s) patches – more specifically, adding support to existing chips, code cleanup and documentation. I will be able to send them to the mailing list in a few days. In my research on Eon, GigaDevice and Winbond chips, 2 distinct models for OTP were observed – the GigaDevice and Winbond model with security register(s), and the Eon model with a security sector.

The Security Register(s) model has 3 separate opcodes for read, program and erase – 0x48, 0x42 and 0x44 respectively. A chips can have multiple security registers (most commonly 3, but as high as 4) with each register being anywhere between 128 bytes to 1024 bytes in size (most commonly 512 bytes and then 256 bytes). Usually chips have a lock bit (LB1, LB2, …) in the status register that correspond to respective security registers. These one-time programmable bits are changed using the standard WRSR instruction. Some chips have a single lock bit that controls OTP status for all security registers.

The Security Sector model has a separate sector which can be operated in the OTP mode. OTP mode is entered with opcode 0x3A and exited by sending WRDI (0x04) instruction. While in the OTP mode, the sector behaves just like any other sector – normal read, program and erase instructions apply. The SRP/SRWD bit is served as OTP bit while in OTP mode. Issuing the WRSR command (irrespective of the data sent along with it) will cause the one-time programmable OTP bit to be set.

One of the recurrent issues (for the lack of a better word, I don’t think of it to be an issue really ;)) is that many chips I have based my research on, are not originally supported by flashrom (perhaps unfortunate siblings of the same family that didn’t find support in flashrom earlier xD). I don’t call it an issue per se because after I have submitted my patches flashrom will end up supporting even more chips, but since I have to write more code it might take slightly longer to submit the patches.

There is a third model which is dominantly followed by Spansion chips and a couple of AMIC chips (some AMIC chips follow one of the earlier models – it’s like AMIC couldn’t decide which one to stick to or they probably had different teams working on it! :P). Similar to the security sector design, these chips also have a separate OTP sector but instead of storing configuration in the register, a byte within the sector is allocated for storing the configuration data. I have planned to support this model in the next revision of patches, after the upcoming ones get reviewed and merged.

Thanks for your time, it was nice to get back in touch with you! 🙂
(Phew, that was long! :P)

[GSoC] Multiple status registers, block protection and OTP support, week #1 and #2

Hi, I am Hatim Kanchwala (hatim on IRC) from India. I am the GSoC student working with flashrom this year. Stefan Tauner (stefanct) and David Hendricks (dhendrix) will be mentoring me (thanks a lot for the opportunity). The pre-midterm phase of my project comprises three sub-projects – multiple status registers, block protection and OTP support. Each of these projects deals with SPI flashchips.

As of writing this post, flashrom supports over 300 SPI flashchips. Around 10% have multiple status registers (most have two but there is one with three). Almost all have some sort of block protection in place. Around 40% have some variation of OTP or security registers. A combination of BP (Block Protect, first status register) and SRP bits (usually first, but sometimes second status register as well) in the status register determine the range and type of protection in effect. A few have a TB bit (Top/Bottom) in addition to BP bits. Some also have a CMP bit (Complement Protect, second or third status register) to add more flexibility to range available. Few chips have a WPS bit (Write Protect Scheme, second or third status register) that define which scheme of access protection is in use. Chips with security registers have corresponding LB bits (Lock Bits, second status register) which are one-time programmable and, when set, render the corresponding security register read-only. Chips with a separate OTP sector(s) have opcodes to enter/exit OTP mode and, within OTP mode usual read, page program and sector erase opcodes can be used.

Previously, flashrom could only read/write the first status register. For writes, all block protect bits were unset (this configuration corresponds to block protection), if the type of protection allowed it. Once unset, flashrom couldn’t revert the BP bit configuration. The ChromiumOS fork of flashrom has some support for locking/unlocking block access protection in place. A lot of the work is done around specific families of chips, but they are moving towards generalising it. For chips with OTP support, flashrom simply printed a warning.

In these two weeks I sifted through around 5-6 dozen datasheets and developed models for multiple status registers, block protection and OTP/security registers. I discussed with mentors and the community over mailing list (link to thread) the infrastructural changes and use cases corresponding to the models. To substantiate these ideas, I wrote separate prototype code. In the process, Stefan introduced me to a powerful tool, Coccinelle. This tool will make applying changes to the large struct flashchips easier while being safe. As a byproduct of studying existing flashrom infrastructure, I had the opportunity to explore the history of flashrom through git log – evolution of flashrom from its humble beginnings in coreboot/util to flash_and_burn to flash_rom to finally flashrom today!

My broad targets for the following few weeks will be to finish up with the pending dozen or two datasheets, polish the models and start transforming the prototype code into merge-worthy code. Following the infrastructure changes, I will update existing chips to make use of the new infrastructure, add support for a bunch of new chips and finally test on actual hardware.

Thanks. See you later!

flashrom: Which hardware do you want supported?

flashrom is growing rapidly and we are constantly adding support for new SATA/PATA controllers, network cards, graphics cards, USB devices, JTAG cables, DIY hardware hacks, desktop/server mainboards and even some laptops.

If you own a piece of hardware which has a flash chip and lives in a PC (embedded devices with NAND flash are already covered by other software), and if you want to reflash that piece of hardware, please comment here (with some info on how to reach you) or mail us at flashrom@flashrom.org. We will use this to prioritize support for hardware people are interested in.

Continue reading flashrom: Which hardware do you want supported?

flashrom supports PowerPC and MIPS

You can now use flashrom on PowerPC and MIPS with most programmers. I added the last bit of infrastructure in version 0.9.2-r1111, and we got a few success reports since then. All USB/serial/network drivers work, and quite a few PCI drivers work fine as well.

Added bonus: If your flashrom driver selection does not need direct hardware access (e.g. USB programmers, serial programmers, dummy, …) you can now compile flashrom on all architectures, even those which are not explicitly supported. That feature was added in version 0.9.2-r1116.

This is a big step forward for flashrom which had been x86 centric since the beginning.

Continue reading flashrom supports PowerPC and MIPS

Nvidia MCP6x/MCP7x SPI support merged

I merged my Nvidia MCP61/MCP65/MCP67/MCP73/MCP78S/MCP79 SPI support patch a few days ago in version 0.9.2-r1113, but right now flashrom will refuse to erase/write on those chipsets for safety reasons. Details about the patch can be found in my earlier blog post: First successful Nvidia MCP6x/MCP7x SPI access.

Current flashrom is thus well-equipped to handle any x86 mainboard you throw in its way.

Continue reading Nvidia MCP6x/MCP7x SPI support merged

RayeR SPIPGM support in flashrom

A few days ago I added flashrom support for the RayeR SPIPGM hardware by Martin Rehak. It is basically one capacitor and a few resistors attached to a classic parallel port cable, and you can use it to reflash SPI chips. Many recent mainboards from MSI, Gigabyte, VIA and other vendors have either removable SPI flash chips or a JSPI/JSPI1/SPI header where you can attach RayeR’s programmer to perform a BIOS recovery. See http://rayer.ic.cz/elektro/spipgm.htm for schematics and instructions.

If you want to test it, compile flashrom version 0.9.2-r1093 or later. RayeR support is enabled by default. To invoke the RayeR driver, run

flashrom -p rayer_spi

Continue reading RayeR SPIPGM support in flashrom

First successful Nvidia MCP6x/MCP7x SPI access

Since a few hours, my Nvidia MCP61/MCP65/MCP67/MCP73/MCP78S/MCP79 SPI driver is tested and it works well. Only probing for a flash chip was tested, but still… this means my SPI bitbanging code is correct, and Michael Karcher’s reverse engineered docs are correct, and my implementation of the Nvidia GPIO interface used for bitbanging SPI is correct as well.

This is big news because with this patch flashrom finally has 100% support for all x86 chipsets we saw in the last ten years.

Huge thanks go to Michael Karcher for reverse engineering the interface and writing up cleanroom documentation which I could use for implementing the interface.
Huge thanks to Johannes Sjolund for testing my patch on his hardware although it was completely untested before.

Get the patch here: http://patchwork.coreboot.org/patch/1520/ (click on the “patch” link on that page to get a download).
Continue reading First successful Nvidia MCP6x/MCP7x SPI access