GSoC 2013 [flashrom] blog post #5

The running gag that is not funny at all: new flashrom release will be out very soon! 😛

There are mainly two reasons why it was not released yet. First we have found a regression due to the inclusion of flash chips bigger than 16 MB and then I was working on adding decent support for a new SPI programmer before release.

The big flash chips were known to not be readable or writable yet because of the 4 B addressing they use, but I thought just probing for them would not break anything… well it does because flashrom unconditionally tries to map the top <flash chip size> bytes of the forth gigabyte of memory even if the programmer does not need that functionality at all.

Why is that a problem? Because apparently there is or was an (unwritten?) rule that (only) the top 16 MB are reserved for this kind of thing but no more, and that the address space below might be used by other things (e.g. memory mapped registers) or marked to be completely unusable. So on some boards trying to map more than 16 MB will explode (i.e. flashrom aborts while probing for a chip) – bummer. We merged a trivial fix that got rid of the new big flash chips for now until I finish my 4 B addressing patches (well, or until I start to write them to be precise).

The other thing that held the release back has a lot to do with flows. No, no esoteric bullshit about energy flows and what not that made flashrom feel indisposed… 🙂 I am talking about rivers, big rivers… like Hudson or Yangtze (two AMD code names for southbridges/FCHs). AMD has done an awful job when changing PCI IDs for the LPC bridge in their newer chipsets, which does also control most of the SPI interface all new boards use for firmware storage. They have not changed the IDs for years (almost 4 years to be exact), but did so with the introduction of Hudson the second iteration of Hudson (A55 Hudson-D2, A60M Hudson-M2). Also, the datasheet of the second (and third, and forth(!?)) Hudson generation is not public (yet?) so I don’t know exactly why they did change it, but apparently the core features of the SPI controller remained the same and flashrom continued to work, when we took the new IDs into account (in r1422 in 2011).

Fast forward to this summer when they released a new chip, an SoC with an APU and the FCH combined which they named Kabini (and Tamesh for tablets). The FCH inside is codenamed Yangtze. This time they did not change the ID, _*BUT*_ as you might have guessed this time the SPI controller changed… a lot. Parts of the old interface are still in there but there is also a completely new interface and it is not very well documented. The previous generations all used an 8 byte FIFO ring buffer where the host has no access via indices but one fixed address to read or write a byte with an internal pointer incrementing on every access (and a register somewhere to query and reset the pointer). The new interface (named SPI 100) got a much bigger buffer (which is a great idea, because in the 8 bytes of the old one 3 bytes are used for the address bytes in most performance-critical transactions). But the buffer is a little bit odd: First of all it is 71 bytes long. Please stop shaking your head now so that you can start shaking it again. Secondly it is documented as a FIFO but it is completely memory-mapped and freely accessible, and it is no longer a ring buffer.

Thankfully I had the hardware to test this out and we now have (still unmerged) full Yangtze support. But not only that but I took the challenge a bit personal and studied the half dozen datasheets of AMD chipsets with SPI controllers to find out how to distinguish them reliably. It turns out that the standard PCI revision byte of the SMBus device is a good choice at least according to the datasheets. Of course the datasheets lie and contain the wrong values but some googling later I had gathered the needed information to rewrite our programmer module completely while taking the partly small partly huge differences between chipsets into account… which I did. Also, there was a preexisting patch to specify the SPI clock on the command line and we wanted to use safer defaults for the SPI clock anyway… combine everything, clean it up and separate it into smaller chunks and you have a nice patch set with 500 changed lines… which have to be tested and reviewed. There goes the release deadline again.

The changes affect about 7 generations of AMD chipsets and it would be very difficult to test them all, especially without having the hardware. But then Martin Roth surprised me again quite a bit. He came up with the idea of using the idle time of Sage‘s test department for flashrom testing. He did also convince his boss and has arranged for everything else and so we are now able to send test assignments directly to Sage’s test apartment and they will come back to us when they have had time to do it – awesome. The first round showed (apart from a few cosmetic problems) that my chipset generation detection works fine, but we decided to postpone the whole AMD patch set and release 0.9.7 without it. It just changes too much and there is a huge potential for regressions. Unlike coreboot flashrom is shipped by a wide range of distributors, from the big daily-use distributions like Ubuntu, Fedora, Arch etc. to rescue and disaster recovery tools like SystemRescueCd and grml. Also, it is quite easy for end users to shoot themselves in the foot with flashrom and we don’t want to make that worse by shipping code that we suspect to have regressions. In 0.9.7 we will only detect Yangtze and bail out.

We will use git!

But not as our main VCS. I am preparing better git support in our build system though. David Hendricks has written a patch long ago that adds way more detailed version information to builds and which had also some support for git-svn. I have reworked his script and it just needs a bit more of testing before we decide on the final version information to be printed on runtime. The output of my current prototype looks like this:

flashrom v0.9.7-r1706-b75bc95-6-dirty available from git@github.com:stefanct/flashrom.git git, modified 2013-08-12T02:19:39+0200 on Linux 3.8.0-6-generic (x86_64)

Separated by hyphens it prints the (traditional) stable version of flashrom, the svn revision (on native and cloned git-svn repositories too), the git hash of the last commit, the number of patches since the last upstream/subversion commit, and a dirty flag in case of uncommitted changes in the working directory. It also tries to find a remote repository that contains the last commit and prints its URL and the commit date (or the current date for dirty repositories). This should make it way easier to interpret log files and ends the era of “-runknown” prints.

Buildbot

Our buildbot is slowly getting into shape thanks to the server space provided by Anton Kochkov and the time Carl-Daniel invested into building cross compilers and setting up virtual machines. I have migrated it to a shared directory and be able to push a local git commit to it and let it build it by just issuing a single command, yay. We still need a better interface for it – currently there is only an nginx autoindex of the generated files, and we need to hook it up in subversion to get automatic builds on commits, but I really like it already.

GSoC

6 weeks remain for GSoC work… will be interesting how much of my initial proposal I can still achieve. More on that next week hopefully.