Another week bites the dust, and coffee supplies are running low. The swarm of zombies is restless, but they seem to be active mostly at night. The barricaded windows are holding up well for the time being. I can venture small distances during the day, but not far enough to reach other survivors.  I was able to recover a package from the mailbox this week. Its unannounced appearance is still a mystery, but its contents are most enthralling in this forlorn aeon. I was able to use the waterblocks in the package to reduce my heat signature. There are fewer of THEM trying to break through the barricades. Last night, the leader of the pack did not disturb my hiding place. I don’t know if help will ever come, but I owe it. I owe it to myself, and to the flight engineers. I do not know if they survived, but I owe it to them to finish the flight plans. We must leave this planet.

If you are listening to this transmission, you are a survivor. I have spent this last week in improving the flight plans; I have annotated and documented them in glorious detail. Get them here:

$ git clone git://git.qiprog.org/qiprog.git
$ git clone git://git.qiprog.org/vultureprog.git

 How QiProg works

QiProg was originally designed to be a pure USB protocol, specialized in driving flash chip programmers. Peter Stuge’s original QiProg specification is just that: a USB protocol. But as Peter suggested, once that protocol is converted into API calls, it stops being a protocol, and becomes a full-featured API. It’s amazing how each USB control request can be mapped to one and just one API call. Most USB dependencies become invisible in the API, with a very limited number of exceptions, where the dependence on USB could be inferred from the size of the data structures. Even in these limited cases (hint: there are exactly three), the dependency on the USB bus can trivially be abstracted away. My original reaction was to modify the spec to remove them, but that patch now sits lonely on a forgotten github branch.

QiProg initialization

Initializing the QiProg logic is a very boring boilerplate operation. Luckily, this can be done with just one or two API calls. In fact, I have only included three functions to take care of this. They can create a context, free a context, or set the verbosity of debug messages. That’s it: the very standard boilerplate.

QiProg device discovery

The discovery phase is yet again, boilerplate, albeit a very smart suggestion from Peter. With a single, lightweight call,  qiprog_get_device_list(), QiProg scans all devices and presents them in a flat list. This gives us a bunch of qiprog_device pointer. The qiprog_device pointers are at the heart of QiProg. The public API only presents them as pointers, as opaque as the dictionary allows.

Once we have a device pointer, we can try to open the device, ask the device what it can do, and decide whether or not to hire it. Once we hire a device, the real fun begins

The QiProg core

Remember how I said QiProg devices are presented as opaque pointers? This makes them full-fledged objects. Anytime we want to do anything with the device, we have to perform a device operation. This is exactly where the core come is. It makes sure that the operation is dispatched to the correct handler (more on that later), and makes sure that we don’t crash because of programming mistakes. If I had written QiProg in C++, I would have made the qiprog_device an abstract class, and would have hidden the derived classes and their constructors away from the API.

So, what’s in the core?

qiprog_do_action_x(device, action_parameters...);

That’s about it. action_x can be any of the actions in the original QiProg specification. While it might seem that a _lot_ of logic is needed to make this happen, the core is actually ludicrously lightweight.

Inside the core

QiProg is designed to handle more than just USB programmers. This brings the need for different code paths for each class of devices. Internally, QiProg implements a “driver” for each class. This driver is a structure with function pointers. QiProg asks each of these drivers to scan for available devices, and append them to a context-global device list. This is the exact list we get with qiprog_get_device_list().

So, back to the core. Since the drivers are invisible to the outside world, we can’t get those function pointers. This is the job of the core. The core dereferences the device pointer, and sanity-checks it. This sanity checking removes most boilerplate from the application. And now the magic: each device stores a pointer to its associated driver. All the core has to do is dereference the driver and call the appropriate function with the device as the parameter.

Each device gets a void pointer to store private data. The driver decides what to store there and how to use it. That is sufficient to carry all necessary context information, and why the device pointer is passed to each member of the driver. Since there is no need to look up context information, the core is essentially an O(1) operation. This is the reason we can run the core on the embedded QIProg device.

The hidden QiProg core

Yes, QiProg is running on the VultureProg device as well, not just the host. We don’t care about discovery, or any function that does not need a qiprog_device. Those steps can be handled by standard USB requests; all information is in the USB descriptors. The situation, once again, turns interesting when we have a qiprog_device. VultureProg has a qiprog_device as well (and it can have several).

From USB to the core

Any USB transaction will come in through some sort of hardware-specific channel. It’s the nature of the beast. So, the first thought is: “OK, let’s write a bus IO, hook it into the USB handler, and be done”. However, we can make our USB dispatcher forward control requests to QiProg. And this is where a little file that never seems to be included in the build comes in. qiprog_usb_device.c is never compiled in host code, but is our bridge to QiProg on VultureProg devices. It takes USB requests, and forwards them to a real QiProg driver.

Ok, let’s pause for a second:

Yes, we run QiProg drivers inside the little Cortex-M processor, and with QiProg drivers comes the slick QiProg core. There are a few more tricks we use for making several drivers use the same hardware, but they are far too technical. For the curious, I have to words: “doxygen documented”.