The truth about Purism: Why Librem is not the same as libre

 

Purism Librem is an interesting project which promises to produce a completely libre laptop, designed to respect the user’s privacy and the user’s rights. The indivduals behind the project seem so confident as to “promise that a Purism system and all its components will be free according to the strictest of guidelines set forth by the Free Software Foundation’s Free Software Definition.” Can they deliver on this bold claim?

I first heard about Purism Librem sometime last October, when someone on #coreboot posted a link to Purism’s ideas/ideology page. It promised a fully libre system, from the ground up, with a high emphasis on libre firmware.

I only glanced over that page quickly before dismissing it as another over-ambitious and mis-informed project. There was no way that the Intel CPU and chipset they wanted could run libre, given Intel’s tight grip on the low level boot process. Coupled with their desire to include an Nvidia GPU — another high-profile offender in the libre software world — their ideas looked doomed.

The first red flag was that we, the coreboot hackers, were never contacted by Purism about what it would take to get such a design up and running on the firmware side. We could have immediately told them that there are major pieces of the initialization path for their CPU which were missing source. That is, they were only available as blobs.

The possibility of reverse engineering those blobs existed at the time. Although that takes a lot of effort, we’ve done it numerous times before. But they never asked. Had they done so, we would have also told them about another major offender. That’s the microcontroller in the chipset, which needs a signed firmware binary. By “signed” I mean a state-of-the art cryptographic verification mechanism. The chipset will refuse to run any firmware unless it was signed by a secret key held deep within Intel’s most secure dungeons. In short, this blob isn’t going away.

It was obvious to me from those first versions of Librem’s web page that the designers behind it had no idea about the binary situation. My impression was that they never contacted knowledgeable parties about it. They were reinventing the wheel and they were doomed to fail.

But that wasn’t to be. A month or so later, the subject popped up again on #coreboot. This time, they were considering crowd-funding. The information they provided via their website has changed as well. They were now claiming to be working with Intel about releasing source code for the early initialization blobs, and talking about disabling the chipset microcontroller entirely.

That sounded like a reasonable plan, with only one deadly fault: Intel doesn’t release this type of source code, and the Intel decision-makers do not talk with low-volume customers directly. Not even Google could get the source code out of Intel after shipping millions of Intel Chromebooks. The few hundred units that Purism was planning to build was definitely not going to cut it.

By that point, I made up my mind that the people behind Purism were either naive, or full of it. Deep in my heart, I wanted them to succeed, and I wanted to personally congratulate them for said success. I’m a coreboot developer; I know how this business rolls. I can make your firmware email me a daily digest of your passwords and Facebook activity, and you wouldn’t even know about it. I know what I’m talking about.

Fast-forward to the middle of January. Librem’s crowd-funding campaign had been extended from December to the end of January. A lot of new information also appeared on their web pages. Besides a quote from Richard Stallman, suggesting the FSF endorses their project, their campaign page was filled with buzz-words that seemed to have come straight out of Stallman’s most iconic speeches. The claim was that Librem is “designed to respect your privacy”.

There was also a graphic representation of the software stack. A green square meant libre, while a white square meant closed. There was so much green on the reverse pyramid of the graph, that it was too easy to miss the two or three white squares at the base. They were the platform firmware. That’s the part that emails me your passwords without any of the green dots above it being aware.

Finally, by January 22, a ZDNet article appeared which confirmed what I had predicted all along: that Librem will ship with proprietary firmware. That’s announcement came just a week shy of the crowd-funding campaign drawing to a close. With proprietary firmware, Librem is just as free as any other laptop on the market with GNU/Linux. Or in other words, it’s not any more free. It is certainly not libre.

So at the end of the day, Librem is bringing nothing new to the market. Laptops with libre operating systems have existed for decades. The only real innovators in this area have been Google and GluGlug. Google ships partially free firmware, although insufficiently libre to be able to provide the “respect your privacy” guarantee. GluGlug can make this claim, and it ships laptops with fully libre firmware. The downside of GluGlug is that it’s an aftermarket add-on. GluGlug and Google have been in business far longer than Purism. So, what has Purism brought in that’s new and exciting and libre? Nothing.

Intel Boot Guard

So some innocent post on the coreboot mailing list managed to make some waves.

The problem they try to solve…

Intel Boot Guard is the latest effort in a long series by Intel and others to allow computers to provide some reliable information about the state a computer is in. They're working on int since at least 2003, with projects and trade groups named Palladium, TCPA, and now TCG, and some of them faced scrutiny in the past already because the freedom of computing was deemed under attack (partly realistic, but with some hyperbole). The scheme they developed ultimately requires having a chip in system that keeps track of the system state and is able to keep secrets from the main CPU until it proves that the system is in a safe state, called TPM (short for Trusted Platform Module). From the beginning the TPM was designed as a passive component: Some other part of the system needs to update the TPM's view of the platform, the TPM is not able to lock down any component in the system except access to its own memory. The TPM consists of some way to keep track of the system state, some non-volatile memory (for the "secrets"), a way to bind secrets to system states, and it also provides some cryptographic operations - among them: creating RSA keypairs, and working with them. One major design issue is where the trust is rooted in: The first verification of signatures happens by code on the CPU, so if you are able to intercept that and replace it with your own, it's trivial to emulate a "properly" booted system (by just sending the right values to the TPM). Moving that issue ever earlier in the boot process, the last frontier is eventually the bootblock, the part of the firmware that contains the first instructions executed by the CPU: Since it comes first, it verifies the part that comes after it, which again verifies its successor, and so on. Analogous to a proof by induction, the entire system state remains well-known as long as the first component tests the next component, and every other component does likewise. But if you can't trust the bootblock to send a truthful state into the TPM, you have already lost. Enter Boot Guard: It allows the hardware vendor to lock down the boot block so the machine only starts if the code stored there matches a key they wrote into the computer.

… and its very own problems

The main grief with this approach is that this key can't be rewritten once it is in. So when the hardware vendor (say Lenovo) sets this key, they are the only ones that can provide firmware to the machine, even if the owner of the machine wants something else. In combination with the complexity of UEFI, which commonly includes a network stack (and thus the capability to communicate with the world) and the ability to load and execute code (eg. through the net), some people are uncomfortable with that prospect. Others just want the ability to replace all code on a system, including firmware, as a matter of principle - and since they own their machines, I find it really hard to argue that they shouldn't be able to.

There's more to it: Verified Boot vs. Measured Boot

But this isn't the whole story to Intel's Boot Guard. The option of installing a key is what Intel refers to as "Verified Boot". It's described in some short words on their product brief for these CPUs. That document also talks about another mode, "Measured Boot". In this mode, Boot Guard creates a hash over the bootblock and sends it off to the TPM. The value is stored in one of TPM's plenty registers, and in particular in a register that isn't writable by code running on the CPU (there's some circuitry to make sure of that). This is supposed to prevent replay attacks in which it would be possible to fake a certain Boot Guard state if it's ever possible for an attacker to disable Boot Guard altogether.

User friendly security, powered by Boot Guard

Since the TPM is able to bind data it stores to those registers, it's possible to verify the system state against a key stored in the TPM that is bound against a known good state: In the factory, have the TPM create a keypair, and bind it against the bootblock that is installed. Export the public part of the key (the TPM won't relinquish the private one, so this operation is safe). When trying to assert if the system is still in a good state, encrypt a random value (nonce) with the public key, and send it to the system to test. If it can decrypt the value and send it back, the state is known, and everything is fine. This could be a measure for Windows to employ on a Domain login, to assert that the system wasn't tampered with. Or the Windows Account / Store / Update so it can report suspect events (you already have to register the machine with Microsoft when using Windows, so let's use it for something user-friendly). Or bind the encryption key for the disk against that state, so the data is only readable in that computer with that firmware. (Since TPM crypto is so slow, this is somewhat more involved, but conceptually that's what can be done with it) A user wanting to install their own firmware (including bootblock), will face loss of access to an encrypted disk that is bound to the bootblock in this way, and to a Domain that does this verification. From a security perspective, both are desirable. But they can use Boot Guard with their own TPM state, and encrypt the disk with their own secret stored within that chip. With "Verified Mode", overwriting the firmware just transforms their computer into a nice, expensive, dead brick.

So what went wrong?

The issue isn't that Intel added Boot Guard to their platform. It's that Intel provides the Verified Boot mode. Had they not done that, the effort would be universally lauded as a technology that improves the security of their users (and without ripping holes in their security fabric like Intel TXT did). But as is, as Matthew Garrett states, "vendors are forced to choose between security and freedom". And that's an exercise most vendors routinely fail to do properly. So Intel, please: Protect the vendors from themselves, and your users from the vendors between yourself and your users, and do the right thing. Drop Verified Boot in future chipsets, and discourage vendors from using it now.

Request for Participation: coreboot consortium for business

We want to make coreboot more accessible and usable for businesses. In order to coordinate efforts, we plan to create a business organization (a.k.a. a coreboot consortium) that helps commercial players out in the field to become a part of the coreboot community, share marketing materials, participate in working groups, and other efforts to promote the use of coreboot in the market place and be successful with coreboot based solutions.

If you work for an organization that is using coreboot, developing coreboot, or is just interested in coreboot, please help us learn about your needs and wishes and fill out the following form: http://goo.gl/glhNh5

coreboot Code of Conduct

coreboot has grown and matured a lot in the last few years and we have many new contributors and users. This growth has been something I have been thinking about for some time. I often ask myself what makes a good community? How does our community grow and flourish? How do we get new contributors and new students for GSOC. How can I contribute to growing our community?  These questions were on my mind as I attended the GSOC reunion in San Jose last fall and again later at the coreboot meetup in Prague. Discussions at both events helped shape my ideas, but GSOC reunion had some key elements to shaping a good community.

A reoccurring topic at GSOC gatherings is how to  expand project communities, keeping students engaged after GSOC, and how to get new students and contributors involved with projects. Normally, these discussions are among mentors, who are well entrenched in the FLOSS culture, are experienced developers, and have developed strong relationships within their project. Unlike previous mentor summits, the GSOC reunion was open to past students and mentors to attend, which brought a different perspective. Many of the students  were not involved in the projects after GSOC and we wanted to know why. This was an opportunity to hear from students about what worked for them and what didn’t work. Most of the students I met still used lots of open source software, but there were a number of reasons that they no longer developed on their GSOC project. Some students lost interest in the project, but still developed on other projects. Some students got jobs that didn’t deal with open source, but would do it again given the opportunity. Finally, there were some students (and mentors) that said that they didn’t like working in open source, that they were mistreated, forced or flamed out, or never felt welcome and appreciated. It is the last group that I wanted to make a difference in.

While most developers don’t experience something as terrible as the harassment that happened in the context of Gamergate or other terrible behavior documented at technical and open source conferences the last several years, it does indicate that there is a problem in our open source communities. This terrible behavior online and offline is unacceptable.  You shouldn’t be required to have a thick skin to develop open source. It should be a collaborative, not combative environment. We can’t lose valuable contributors because they felt uncomfortable or unwanted. A few mentors of large communities have had to deal with some really ugly incidents  (I’m trying to recall, but I think it was folks from Debian and Ubuntu among others) and they provided some great advice. The first was that mature communities should have some rules for behavior.  The second, and maybe more important, is that community members must point out when someone is behaving poorly. I felt that these were important point and started to consider how to make coreboot a more safe and welcoming place.

Typically, harassment, trolling, flaming, etc  is not a problem in the coreboot community.  We have a group that is generally friendly and helpful. We try to encourage good behavior and discourage bad behavior, but we could do more.  With a growing and maturing community we are going a step farther and describing what is unacceptable within our community with the coreboot Code of Conduct. It is a simple and straight forward statement about what we expect from out community members. The code of conduct is based on the Citizen Code of Conduct and similar documents in the open source community. Feel free to comment and provide feedback either publicly or privately.  Please note that this isn’t directed at anyone in particular or for a specific instance. It is part of “growing up” as an active and maturing community.

Reverse engineering blobs: adding diff to the toolkit

Last time I talked about the benefits of using sed to transform repetitive low-level patterns into meaninful function calls.  And still, doing all that regex magic did not get us a fully working replay. A great portion of the hardware initialization flow is based on situational awareness. What hardware is connected? What are our capabilities? What if …?

That means a simple sequence of writes is, in most cases, not sufficient. We may need to modify registers, wait on other hardware, or respond differently to hardware states. While that seems daunting and tedious, it gives us an unexpected advantage: that every execution of the blob produces a different trace.

This is where diff comes in. By getting a bunch of traces and diffing them, we can see the points where the firmware takes different decisions, and the states which determine those decisions.  It won’t tell us what condition triggers path A or path B, but it allows us to infer that by comparing the hardware states. Let’s have a look:

@@ -305,28 +305,28 @@ void run_replay(void)
 radeon_read_sync(0x6430); /* 04040101 */
 radeon_write_sync(0x6430, 0x04000101);
 radeon_write_sync(0x3f50, 0x00000000);
- radeon_read(0x3f54); /* 000dda12 */
- inl(0x2004); /* 000dda8a */
- inl(0x2004); /* 000ddb1a */
- inl(0x2004); /* 000ddb9c */
- ...
- inl(0x2004); /* 000de3a0 */
- inl(0x2004); /* 000de41a */
+ radeon_read(0x3f54); /* 000d9efb */
+ inl(0x2004); /* 000d9f73 */
+ inl(0x2004); /* 000da003 */
+ inl(0x2004); /* 000da081 */
+ ...
+ inl(0x2004); /* 000da883 */
+ inl(0x2004); /* 000da8fd */
 radeon_read_sync(0x611c); /* 00000000 */
 radeon_write_sync(0x611c, 0x00000002);
 radeon_write_sync(0x6ccc, 0x00007fff);

I’ve chosen an example that shows similarities rather than differences, as I find this to be a more interesting case. Since we’ve already established that 0x2004 is our data port, as long as we don’t touch the index port, we’ll be reading from the same register, in this case 0x3f54.

Now the values returned by this register are completely different in every trace, yet the behavior of the blob is strikingly similar every time. The first key observation is that this register increase monotonically in every trace. It also increases by roughly the same amount on successive reads. The differences between the last read and first read of the register are also strikingly similar in both traces: 0xa08 and 0xa02 respectively.

This register looks to be a monotonic timer, and the loop has all the elements of a delay loop. To determine the actual delay, we could try to extract absolute timing information when collecting the trace; however, in this specific case, I had the AtomBIOS tables handy. By comparing register accesses around this loop, I was able to figure out where in the tables this delay is occuring:

 0200: 0d250c1901 OR reg[190c] [...X] <- 01
 0205: 54300c19 CLEAR reg[190c] [.X..]
 0209: 5132 DELAY_MicroSec 32

The ’32’ in the delay is a hex number. Doing a bit of hex math we see we’re waiting about 51 ticks per microsecond. Comparing more loops, we get between 50 to 52 ticks per microsecond. Since a delay loop normally waits until the minimum time has elapsed, we now have a very convincing case that register 0x3f54 implements a 50MHz monotonic timer. Every time before accessing this register, we also poke register 0x3f50. That looks very much like the timer control register.

We now extend our sed script with:

timerctl=0x3f50
timer=0x3f54
...
sed "s/radeon_read($timer);[^$hex\r]*\([$hex]*\)[^\r]*\(\r\tinl($dport);[^$hex\r]*\([$hex]*\)[^\r]*\)*/radeon_delay(0x\3 - 0x\1);/g" |
sed "s/radeon_write_sync($timerctl, 0x0\{1,8\});[^\r]*\r\tradeon_delay(/radeon_delay(/g"

Now when we rerun our logs through the script, the results decrease in size from 20K lines to 13K lines. The diffs between processed logs also decrease in size significantly. All the more proof we were right!

There’s another way in which diff is excellent for our purpose. We can implement our helpers to generate the same output as the processed logs. That allows us to poke the replay from userspace, yet get the same output format. Now we can diff the replay and original log, and observe how the hardware state changes. We can even go as far as implementing our delay with usleep() instead of the timer at 0x3f54. When the diff is independent of the delay method we use, we have another strong proof our assumptions are true. This is the case here.

‘diff’ is an extremely powerful tool. Despite its name, it can show similarities just as well as differences. While regular expressions exaust their usability with simple patterns, diff can take us a lot further. Now that we’ve cracked the delay implementation of the blob, we can more easily see delay and wait loops — again, using diff. Complex, multivariable patterns are too awkward to handle with sed. I’ll go over those some other time. However, once such patterns are simplified to a function call, diff can once again show the story. Different GPU model? diff. New display? diff. HDMI connected? diff. It’s almost as versatile as det cord .

Reverse engineering blobs with replay and sed

I recently started re-visiting the HP Pavilion m6 1035dx with a recent coreboot master. As usual the benign VGA BIOS got in the way again. This time I decided to use coreboot’s YABEL realmode emulator to tell the story. Let’s dive in:

Executing Initialization Vector...
[0000f2cb]c000:3b69 inb(0x03c3) = 0x20
[0000f2cc]c000:3b6f inl(0x204c) = 0x00000000
[0000f2ce]c000:3e06 inl(0x2000) = 0x9ffffffc
[0000f2cf]c000:3b69 inb(0x03c3) = 0x20
[0000f2d0]c000:3b6f inl(0x204c) = 0x00000000
[0000f2d3]c000:3b69 inb(0x03c3) = 0x20
[0000f2d4]c000:3b6f inl(0x204c) = 0x00000000
[0000f2d5]c000:3e61 outl(0x00001728, 0x2000)
[0000f2d5]c000:3e67 outl(0x0008c000, 0x2004)
[0000f2d8]c000:3b69 inb(0x03c3) = 0x20
[0000f2d9]c000:3b6f inl(0x204c) = 0x00000000
[0000f2da]c000:3e49 outl(0x00003f54, 0x2000)
...

That’s how the coreboot log looks when we enable YABEL traces. Enabling direct hardware access produces a cleaner log to start with. We want to clean that up a little bit. A bit of simple regex substitution gets us there. Excuse the wrap-around:

egrep "c000:[0-9,a-f]{4}|x86emuOp_halt|runInt[0-9,a-f]{2}.*starting" $1 |
grep -v "Running option rom at c000:0003" |
sed "s/c000\:[0-9,a-f]\{4\}\ /\t/g" |

tr '\n' '\r' |

sed "s/\[[0-9,a-f]\{8\}\]//g" |
sed "s/)/);/g" |
sed "s/=\ 0x\([0-9,a-f]*\)/\/\*\ \1\ \*\//g" |

tr '\r' '\n'

That’s enough to get our trace to something that looks more like C code, which could be used as a replay function (HINT!):

 inb(0x03c3); /* 20 */
 inl(0x204c); /* 00000000 */
 inl(0x2000); /* 9ffffffc */
 inb(0x03c3); /* 20 */
 inl(0x204c); /* 00000000 */
 inb(0x03c3); /* 20 */
 inl(0x204c); /* 00000000 */
 outl(0x00001728, 0x2000);
 outl(0x0008c000, 0x2004);
 inb(0x03c3); /* 20 */
 inl(0x204c); /* 00000000 */
 outl(0x00003f54, 0x2000);
...

We’ve neatly kept the return values of IO input operations for future reference, but the current form doesn’t tell us much. We do, however see patterns emerging. IO to 0x2000 followed by 0x2004 is fairly common. That looks like an index/data pair. Also, access to 0x03c3 and 0x204c before poking the above pair is all too common. Let’s extend our script with:

iport=0x2000
dport=0x2004
stsport=0x204c

..

sed "s/outl(0x0\{0,4\}\([0-9,a-f]*\), $iport);[^\r]*\r\toutl(0x\([0-9,a-f]*\), $dport);/radeon_write(0x\1, 0x\2);/g" |
sed "s/outl(0x0\{0,4\}\([0-9,a-f]*\), $iport);[^\r]*\r\tinl($dport);/radeon_read(0x\1);/g" |

sed "s/inb(0x03c3);[^\r]*\r\tinl($stsport);[^\r]*/sync_read();/g" |

Since we’ve converted all our newlines to carriage returns, we can match patterns from multiple lines. It doesn’t matter what we think these patterns do, or how we call the new functions. We’re just interested in grouping them to see the bigger picture. I could as well have called these hamburger() and french_fries().

 sync_read();
 inl(0x2000); /* 9ffffffc */
 sync_read();
 sync_read();
 radeon_write(0x1728, 0x0008c000);
 sync_read();
 radeon_read(0x3f54); /* 002badc3 */
...
 sync_read();
 radeon_read(0x0670); /* 0000fe04 */
 sync_read();
 radeon_write(0x0670, 0x0000fe04);

Much better! We’ve replaced the low-level patterns with more meaningful descriptions. We could grep out the sync_read(), or, if we were using this for replay code, incorporate the _sync() into another function with radeon_(). Even in this form, we can start looking for higher-level patterns. If we look into the disassembly of the video BIOS provided by AtomDis, we see:

 0009: 07a59c01fc AND reg[019c] [.X..] <- fc
 000e: 0d659c0180 OR reg[019c] [..X.] <- 80

Since the registers are 32-bit, then their address would be reg << 2. Thus [019c] becomes 0x0760. These read-modify-write sequences appear in the replay trace.

Now we have an idea where we are with respect to the AtomBIOS tables, we see, at a low level, how those tables translate to hardware accesses. As more patterns are identified, they can be transformed into more meaningful function calls

Now we have the opportunity to look for more advanced patterns, and even identify any code that is not in the AtomBIOS tables. This, in turn can allow us to figure out what actually turns on the display. There is still a huge gap before having anything close to native init. What we’ve done here is develop a coreboot-level understanding of the init process, and made the first tiny step towards native VGA initialization.

While there is only so much regex substitution can do for us, it is a necessarry first step towards a larger understanding of the problem. Transforming 0x0760 to 0x019c is ill suited for regex.  Identifying more complex patterns such as waiting for a condition, or delaying a set amount of time is also a more demanding task; however, the power lies in the ability to script and automate the conversion from a nonsensical log into something more human-friendly. It then becomes trivial to diff several traces and see higher-level patterns. I’ll discuss those some other time.

The pink room – a coreboot developer meeting in Prague

A highly unusual sight? Yes. A pink room filled to the brim with hardware, hackers and pizza boxes.
An unexpected place? Indeed. The architecture faculty building of the Czech Technical University in Prague.
An unusual schedule for four days in Prague? Of course. Intense hacking, talks, discussions and hardware destruction/modding.
Highly unusual sightseeing? Absolutely. A steam-powered wastewater treatment plant and the Prague Signal Festival.
Excellent food? Oh yes. Czech specialties at really low prices.
Disasters? Not really. There were canceled trains/planes from and to Prague affecting quite a few of the attendees, but those were eventually overcome.
Bad things? Possibly. We did unspeakable things to BIOS and EFI images, and we’re proud of it, so I’m not sure if that qualifies as bad.
Great meeting? Awesome meeting!

So what did we do?
– Found a way forward for the dozens of boards which haven’t been tested in ages.
– Overhauled the board status reporting model, so be implemented soon.
– Evaluated ways to get board testing into a state where people can do so easily without manual work.
– Discussed hardware vendor interaction.
– Agreed to ship verified boot functionality by default in a way which still keeps all the freedom for the user.
– coreboot will get a compelling security story out of the box.
– Fixed bugs in some ports.
– Ported a few new laptops.
– Learned new tricks to improve suspend-to-RAM functionality the easy way.
– Made a plan to upstream essential drivers in the chromiumos branch of flashrom.
– Adjusted the flashrom development model to something that works with the scarcity of reviewer time currently available.
– Found out that a video projector is a nice way to test VGA output in case the only VGA capable monitor is in use.
– Noticed that pretty much every question in the form of “does anybody have X” can be answered with “yes” if X is something that may be connected to a computer or is remotely useful for coreboot.
– Didn’t get a lot of sleep.
– Listened to talks about the present and future of coreboot.
– Met some longtime community members for the first time.

Photos of the meeting and sightseeing will be added soon.

coreboot.org website updates

Woot! A new look for coreboot.org.  We have shifted the landing page from the mediawiki to WordPress. DON’T PANIC!, we are still using the wiki as the primary location for developer content. The new landing page and WordPress site is more visually appealing and is the location for news, blogs,  and other basic information for those that are just discovering and learning about coreboot.

GSoC [infrastructure] : Along the way, something went terribly wrong

I started working with AMD platforms’ infrastructure with high hopes of being able  to better manage the CBMEM setup. While I have a selection of family14 boards to work with, things did not continue so well:

First off, tree had literally thousands of lines of copy-pasted or misplaced AGESA interface code remaining in the tree. A lot of that should have been caught in the reviews, but it appears a few years back the attitude was that if coreboot project was lucky enough to get some patches from an industry partner, the code must be good (as the development was paid for!) and just got rubber-stamped and committed.

Second, the agreement I have for chipset documentation is not open-source friendly. It contains a clause saying all documentation behind the site login is to be used for internal evaluation only. I was well aware of this at the beginning of GSoC and at that time I expected my mentor organisation would be able to get me in contact with right people at AMD to get this fixed. But that never happened. I am also concerned of the little amount of feedback received as essentially nobody in community has fam16kb to test.

So currently I am balancing which parts of my work on AGESA I should and can publish and what I cannot. Furthermore, first evidences that vendor has decided to withdraw from releasing  AGESA sources have appeared for review. In practice there has been zero communication with the coreboot community on this so I anticipate the mistakes that were done with FSP binary blobs will get repeated.

Needless to say the impact this has had on my motivation to further work on AGESA as my efforts are likely to go wasted with any new boards using blobs. I guess this leaves pleanty of opportunities for future positive surprises once we have things like complete timestamps, CBMEM and USBDEBUG consoles and generally any working debug output from AGESA implemented. I attempted to initiate communication around these topics already in fall 2013 without success and it is sad to see the communications between different parties interested in overall tree maintenance have not improved at all since then.