coreboot

[GSoC] Multiple status registers, block protection and OTP support, wrap-up (1/2)

Hello! 🙂

GSoC 2016 coding period has come to an end and mentor’s evaluating students this week. It has been an enriching 13 weeks of reading datasheets, designing structures, coding, learning and hanging out over IRC! 😛 I’d like to take this opportunity to present my work and details on how to use it. 🙂

Firstly, to offer context to the work, here is a list of public mails and blog posts. These should give an idea as to how the discussions and work evolved. A lot of the discussions have happened over IRC, but #flashrom does not keep any logs.

From the mailing list, this and this
And of course my blog posts 😉

The patch sets that I sent to the mailing list can be found at –

You can also find these over at flashrom’s patchwork. The mailing list is where the review happens (although a better alternative, IMHO, is Gerrit which coreboot uses). The patches aren’t currently merged and are under review. In any case, you are most welcome to join review (which will likely be very helpful for me). 🙂 If you’d like to look at something more on the bleeding edge, then I invite you to my GitHub.

Now, moving on how to use the work. The most exhaustive documentation on how to use it is the code itself :P, but in the following list I attempt to list scenarios –

For SPI chips that have multiple status registers, flashrom’s verbose output will print the status register bits and there values. Most bits are named, i.e., the datasheet refers to the bit by an abbreviation, for instance, WEL for Write Enable Latch, WIP for Work In Progress, BP for Block Protect, LB for Lock Bit and so on. The verbose output will print these names, both in abbreviated and long forms, for most chips (and these abbreviations tend to be generic across many manufacturers). However, the process for adding new chips that leverage this, and adding new bits, is a fairly easy task (I would invite you to have a look at the code 😉 for more details). The verbose output also prints the write protection mode for status register(s) in effect (software protected, hardware protected, power cycle lock down and so on).
In case you want to disable or enable (a particular type) write protection for status register, you can use the --wp-disable or --wp-enable[=MODE] respectively (where MODE is either of software, hardware, power cycle or permanent – you are encouraged to have a look at the man page 🙂 for more details)
In case you want to protect a particular range of an SPI chip from writes or erases, you will need to alter the BP, TB or SEC bits. Currently, there is a CLI that will enable you to accomplish all that. 😛 First, you’ll want to look at the list of ranges your SPI chip supports – run flashrom with --wp-list. Take note of the start address and the length of the memory range you want to protect. Then again run flashrom with --wp-set-range start=0xfff000,len=4 (0xfff000 and 4 are for representational purpose only). By now the memory range is protected, but you can additionally enable status register write protection by following what the foregoing point described.
For SPI chips that support OTP, you can read, write and erase OTP regions (of course for supported chips :P). For OTP operations, you have at your disposal --print-otp-status, --read-otp [,reg=], --write-otp file=[,reg=], --erase-otp [reg=] and --lock-otp [reg=]. You can read the OTP memory to a file, or you can write to the OTP region from a file, very much like reading and writing from/to SPI chip. For more details, I would again like to point you to the man page. 🙂

Since this is a work-in-progress, the CLI may change (and is very likely). Currently around 10% of SPI chips use this new infrastructure. Models of a few manufacturers (and especially exotic ones like Atmel) are yet to be fully incorporated. You are most welcome to add support for new chips or update the existing ones to support new infrastructure. 🙂

I would like to sincerely thank my mentors Stefan and David for their support and help. I am indebted to them for this opportunity and I hope that we continue to share this relationship in the future while I continue to explore and contribute to flashrom. It has been a pleasure getting to know each of them. I’d also like to thank Urja for pitching in from time to time 🙂 It was fun hanging out over IRC and helping folks asking questions there. And I am looking forward to it for years to come. 😛

In the next and final part of this post, I will highlight how we intend to improve upon this work in the future, where it will be headed and what more we have in store, so please stay tuned. 😉 Phew, this was a long one, and rightly so as it attempts to summarise a great deal of experiences. If you have any feedback, questions or comments on the blogs or code, please feel to ping me on #flashrom where I am known as hatim. You can also email me at hatim@hatimak.me.

Thanks, and looking forward to hearing from you. 🙂 See you in the next and final part.

Rewriting history (of flashrom)

Sorry to interrupt the stream of GSoC posts! 😉 Some of you may have wondered why there were no flashrom commits at all since the 0.9.9 release in March (i.e. about 5 Months). There wasn’t much direct flashrom development going on on the vanilla branch of flashrom apart from Hatim’s GSoC efforts of course.

Instead I’ve been working on moving our repository to git. I have created some client- and server-side hooks + other infrastructure (with the help of Patrick) that will be used in the future to host the vanilla flashrom repository. This part was actually only the smaller part of time I spent on flashrom in the last months (which was not as much as I would have wished for, obviously). More on this and future development later on the flashrom mailing list…

The remaining time I’ve been working on a script that rewrites history. Not all of history, only the flashrom repository’s history. Continue reading Rewriting history (of flashrom)

[GSoC] Multiple status registers, block protection and OTP support, week #6, 7, 8 and 9

Hello! I have been away for some time now, so this is going to be a longer post. I hope you have missed hearing from me 😛 In this post I will talk about my work in the weeks post-midterm evaluations. After a discussion with my mentors in the midterm evaluations week, we decided to shift focus towards the first three phases of my GSoC proposal for the remainder of the duration. Work on the final phase will be done after GSoC along with the more long-term goals that have come up as I have been making progress.

I submitted patches (finally ;)) to the mailing list. The set of patches adds multiple status register, and block protection infrastructure. I have also added a command-line interface to expose the new functionality. Although I am not sure that the exact wording of the CLI is most optimum, but I did not spend a lot of time on that because IMO it is a rather subjective issue and altering it is not a difficult task. The set of patches also adds support for new infrastructure to around 90 existing chips. I am still waiting to receive feedback and review on them. (My mentors had been slightly busier then.) I am also investigating adding support for access protection to non-SPI chips. This isn’t on the highest priority (more like a long-term goal), but once the SPI infrastructure gets merged, I will begin writing code for that.

Based on the initial prototypes I built (here, here and here), we had decided to use pointers to new structs instead of fully embedding them in struct flashchip. This decision really started to show when I was adding support for existing chips – with only 25 unique struct definitions we were able to support those 90 chips! 😉 One of the problems I faced was that I needed to test the new infrastructure, but doing so on a physical chip would be cumbersome. So that problem was solved by adding a dummy chip to use with flashrom’s dummy programmer. (At that time the code was the dummy chip was messy and something I would be ashamed to put up publicly, but now I have improved upon it! :P)

Currently I am working on finalising the OTP/security register(s) patches – more specifically, adding support to existing chips, code cleanup and documentation. I will be able to send them to the mailing list in a few days. In my research on Eon, GigaDevice and Winbond chips, 2 distinct models for OTP were observed – the GigaDevice and Winbond model with security register(s), and the Eon model with a security sector.

The Security Register(s) model has 3 separate opcodes for read, program and erase – 0x48, 0x42 and 0x44 respectively. A chips can have multiple security registers (most commonly 3, but as high as 4) with each register being anywhere between 128 bytes to 1024 bytes in size (most commonly 512 bytes and then 256 bytes). Usually chips have a lock bit (LB1, LB2, …) in the status register that correspond to respective security registers. These one-time programmable bits are changed using the standard WRSR instruction. Some chips have a single lock bit that controls OTP status for all security registers.

The Security Sector model has a separate sector which can be operated in the OTP mode. OTP mode is entered with opcode 0x3A and exited by sending WRDI (0x04) instruction. While in the OTP mode, the sector behaves just like any other sector – normal read, program and erase instructions apply. The SRP/SRWD bit is served as OTP bit while in OTP mode. Issuing the WRSR command (irrespective of the data sent along with it) will cause the one-time programmable OTP bit to be set.

One of the recurrent issues (for the lack of a better word, I don’t think of it to be an issue really ;)) is that many chips I have based my research on, are not originally supported by flashrom (perhaps unfortunate siblings of the same family that didn’t find support in flashrom earlier xD). I don’t call it an issue per se because after I have submitted my patches flashrom will end up supporting even more chips, but since I have to write more code it might take slightly longer to submit the patches.

There is a third model which is dominantly followed by Spansion chips and a couple of AMIC chips (some AMIC chips follow one of the earlier models – it’s like AMIC couldn’t decide which one to stick to or they probably had different teams working on it! :P). Similar to the security sector design, these chips also have a separate OTP sector but instead of storing configuration in the register, a byte within the sector is allocated for storing the configuration data. I have planned to support this model in the next revision of patches, after the upcoming ones get reviewed and merged.

Thanks for your time, it was nice to get back in touch with you! 🙂
(Phew, that was long! :P)

[GSoC] Multiple status registers, block protection and OTP support, week #1 and #2

Hi, I am Hatim Kanchwala (hatim on IRC) from India. I am the GSoC student working with flashrom this year. Stefan Tauner (stefanct) and David Hendricks (dhendrix) will be mentoring me (thanks a lot for the opportunity). The pre-midterm phase of my project comprises three sub-projects – multiple status registers, block protection and OTP support. Each of these projects deals with SPI flashchips.

As of writing this post, flashrom supports over 300 SPI flashchips. Around 10% have multiple status registers (most have two but there is one with three). Almost all have some sort of block protection in place. Around 40% have some variation of OTP or security registers. A combination of BP (Block Protect, first status register) and SRP bits (usually first, but sometimes second status register as well) in the status register determine the range and type of protection in effect. A few have a TB bit (Top/Bottom) in addition to BP bits. Some also have a CMP bit (Complement Protect, second or third status register) to add more flexibility to range available. Few chips have a WPS bit (Write Protect Scheme, second or third status register) that define which scheme of access protection is in use. Chips with security registers have corresponding LB bits (Lock Bits, second status register) which are one-time programmable and, when set, render the corresponding security register read-only. Chips with a separate OTP sector(s) have opcodes to enter/exit OTP mode and, within OTP mode usual read, page program and sector erase opcodes can be used.

Previously, flashrom could only read/write the first status register. For writes, all block protect bits were unset (this configuration corresponds to block protection), if the type of protection allowed it. Once unset, flashrom couldn’t revert the BP bit configuration. The ChromiumOS fork of flashrom has some support for locking/unlocking block access protection in place. A lot of the work is done around specific families of chips, but they are moving towards generalising it. For chips with OTP support, flashrom simply printed a warning.

In these two weeks I sifted through around 5-6 dozen datasheets and developed models for multiple status registers, block protection and OTP/security registers. I discussed with mentors and the community over mailing list (link to thread) the infrastructural changes and use cases corresponding to the models. To substantiate these ideas, I wrote separate prototype code. In the process, Stefan introduced me to a powerful tool, Coccinelle. This tool will make applying changes to the large struct flashchips easier while being safe. As a byproduct of studying existing flashrom infrastructure, I had the opportunity to explore the history of flashrom through git log – evolution of flashrom from its humble beginnings in coreboot/util to flash_and_burn to flash_rom to finally flashrom today!

My broad targets for the following few weeks will be to finish up with the pending dozen or two datasheets, polish the models and start transforming the prototype code into merge-worthy code. Following the infrastructure changes, I will update existing chips to make use of the new infrastructure, add support for a bunch of new chips and finally test on actual hardware.

Thanks. See you later!

GSoC 2014 [flashrom] Support for Intel Bay Trail, Rangeley/Avoton and Wildcat Point

While we were busy updating our AMD driver code to accommodate the new SPI controller found in Kabini and Temash, Intel has also changed their SPI interface(s) in a way that required quite some effort to support it in flashrom. A pending patch set is the result of the work of a number of parties and I will shortly explain some details below.

All started with a patch for ChromiumOS’s flashrom fork in fall 2013 that introduced support for Intel’s Bay Trail SoCs which are used in a number of currently shipping or announced Chromebooks. Bay Trail is part of the Silvermont architecture also in other SoCs intended for different use cases like mobile phones (Merrifield and Moorefield) or special-purpose servers (Avoton and Rangeley). At least for the latter two the SPI interface is equivalent to Bay Trail’s. This was handy for Sage when they were developing their support package for Intel’s Mohon Peak (Rangeley reference board) which was upstreamed to the coreboot repository shortly before this blog post was written. They ported the patch to vanilla flashrom, added the necessary PCI IDs and submitted the result to our mailing list at the end of May.

Because we, the flashrom maintainers, are very picky, the code could not be incorporated as is. I took the patch and completely reworked and refactored it so that more code could be shared and we are hopefully better prepared for future variations of similar changes. Additionally, I have also backported the Intel Wildcat Point support that ChromiumOS got already in May.

The major part of my contribution was not simply integrating foreign code, but refactoring and refining it where possible as well as verifying it against datasheets. While digging these numerous datasheets and SPI programming guides I have also fixed the problem of hardware sequencing not working on Lynx Point (and Wildcat Point) as it was reported in March when I had no time to correct it.

All of this is not committed to the main repository yet, but will be soon. It is mostly untested so far and I would very welcome any testers with the respective hardware.

GSoC 2014 [flashrom] Rise like a phoenix

The best plan is no plan at all, then everything goes according to the plan. (Jacek Bukowski)

I am 142% on track according to my proposal so far if you take the quote above seriously. But even if you don’t the progress so far is way better than I would have hoped for when I applied. I made 22 commits so far during the official coding period (starting with 2014-05-19 or within ~16 days), and 49 if you count from the date of project acceptance (2014-04-21, i.e. ~37 days). That’s pretty significant for a project that normally does 100-150 commits per year. Not all of the patches were authored by myself, and some of them were bitrotting for an extended period of time before I revived them. I have also reviewed, rebased and merged patches sent in by foreigners as well as long-term contributors. Even Carl-Daniel awakes periodically from hiatus and helps where he can. So flashrom seems to be pretty vivid again, yay.

Two of the most interesting sets of changes are the eventually committed support for AMD’s Yangtze-based SPI controller (found in Kabini and Tamesh), and finer-grained display of support/test status of hardware. For example we can now clearly indicate if a flash chip is actually a ROM that can only be probed and read. This has become necessary when we added our first ROM chip, Macronix MX23L3254. The results can be seen in the wiki as well as flashrom’s -L output. Besides that there were also various new chips and even two programmers committed with another 1-3 waiting.

It is still a bit early to nail down any fundamental changes I want to tackle, but I think I’ll manage to rewrite the probing algorithm completely this year. I have made one huge step towards that by getting rid of the .probe_timing field of struct flashchip. It was used by some probing functions of parallel chips but there were only about 5 concrete values so they could easily be wrapped into stub functions. This is – as Carl-Daniel correctly stated – not what we normally do… replacing data in tables with code, but in this case I think it is completely justified. The next big step will be to turn the probing loop inside-out. Currently we iterate over all flash chips, filter the ones compatible with the selected programmer, and execute the respective probing function stored. Naturally this creates the same results over and over again because there is only about a dozen different probing functions. That does not only take a lot of unnecessary time but also makes it hard to support more than one probing function. It should be the other way round: we should iterate over all possible probing functions and match the results with the stored flash chips. I will report back how far I got with that in my next post.

GSoC 2014 [flashrom] Do I need to introduce myself?

Hello, my name is Stefan Tauner and I am participating for the third time in GSoC with a flashrom-related project. If you don’t know me then you have to be new here and should write an introduction post instead of me. 🙂 If you really have no idea then please take a brief look at my introductory posts from last year.

I finally managed to finish my master thesis shortly after this year’s accepted GSoC proposals were announced, so this year will be the last time I’ll participate as a student – unless I find time to do it again during my potential PhD project…

Anyway, instead of writing about my goals for this year, I want to share a few graphs related to flashrom development. The first one draws the number of lines added by individual contributors over time. Note that this includes all patches a developer submitted to our Subversion repository even if he is not the original author. The more horizontal a lines is the less active the committer is. Sadly there are a lot of horizontal lines…

I was curious how my hiatus during my thesis work was affecting flashrom development so I created two graphs spanning the last year. This includes not only the final months of my thesis but also last year’s GSoC. One diagram shows the number of average commits per day each month and the other one depicts the emails sent to the flashrom mailing list on average per day each month. The bars for May 2014 are based on only 4 days worth of data, and I really hope that at the end of May it would not show about 25 Emails per day anymore or I’ll have to work on my project from a rehab facility. 😉

In my next post I’ll lay out my plans for this summer. My (in)formal proposal can be found here. I was quite surprised that it has been accepted instead of my second and more focused proposal for a GUI tool to prepare coreboot images. OTOH, after a short trip into C++ land at the end of my thesis, I am quite delighted that I don’t have to wrestle with Qt and the like…

GSoC 2013 [flashrom] blog post #8

After my last post I continued to improve our flashrom build bot and it now uses VirtualBox’ safe state feature which allows to bring up and down VMs in a matter of seconds. Also, I was able to fully parallelize the builds. Using the safe state feature alone improves the duration of a full build from 4m55.930s to 2m10.496s. The fully parallel execution gets it down to 25.807s, yay.

This week I have merged a mix of old and new patches that fix a few things, add support for other things, and most importantly… gets rid of the dependency on dmidecode: I have finally committed my internal DMI decoder. Compiling for DOS and under various BSDs should be easier now, the AMD patches are slowly getting reviewed… clearly there is some progress, but obviously not enough to reach my original goals for GSoC…

GSoC [early debugging] More connectivity

A substantial cleanup on CBMEM initialisation is now under review. Goal is to get timestamps and CBMEM console supported on more/most mainboards, but I do not expect to complete this during the official GSoC period, or within the next two weeks time.

One of the goals I originally had set was to have means to re-program the flash chip from pre-OS environment. It is now clear I will not have time to finish (or even start) this part of the project. The decision to delay this part was made quite early on, actually. I learned similar work had already been developed as a combination of FILO+libflashrom, and I hope Stefan’s efforts on another GSoC project will help get this code published in near future. I might still try to get the FILO console appear over usbdebug, adding support of USB communication class (CDC / ACM) in libpayload should not be very difficult.

I have saved some of the most interesting and challenging parts last: having SerialICE and GDB run over usbdebug. Hopefully I get to report about those next week.

GSoC 2013 [flashrom] blog post #7

Nothing too fancy to report this week. I have added NetBSD and DragonFlyBSD support to our buildbot, which took quite a while: I have never set up a VirtualBox VM on a headless machine. It is quite easy actually, but there are lots of options one can configure with the CLI so that the most important/unconditional ones are not obvious at all. For example, storage controllers need to have a user-defined name given to address it in later commands, one has to specify the bus and device number when attaching a hd to an IDE storage controller (even if you don’t care at all) etc. It does not sound very problematic, but…
$ VBoxManage | wc -l 495
That’s quite a wall of text. Luckily there is good documentation and lots of howtos online.

In general it wasn’t too bad and the RDP console support of VirtualBox in conjunction with Remmina made up for it: I did not want to set up port forwarding on the remote host nor ssh tunnels myself, but Remmina can do the latter automatically on connect which is really handy.

Configuring the BSDs was way worse. Mostly because I was completely unfamiliar with them and pkgsrc, but also because they have a long way to go regarding usability. Not only that the DragonFly installer does not even try to set a correct keymap, even after installation I could not get that to work and the installer has a few other quirks that thwarts some functionality completely.

Anyway, build testing works on these hosts now and will hopefully prevent future breakage when tinkering with OS-dependent preprocessor #ifdefs (unlike before…). Next: some refactoring of the build bot and fixing issues with getrevision’s use of date on BSDs. Then back again to layout patches and libflashrom.