Software Reuse Success Story

I want to tell a little success story about using my Win32 threading and messaging code.

I recently completed a project for Thomson, where I reused the code here. The project started out as a simple Win32 Dialog application, but as is so often the case expanded. The project was to create an application that would upgrade the flash image in the MobiliTV settop box. The requirements are pretty simple connect to the box over its USB interface (unfortunately custom rather than a serial emulation, but that will become significant in our reuse story.) retrieving version information from the box. Also connect to a HTTP server to discover if there is a new software version available. If there is a newer version pull it down and enable upgrade. This application is to be customer friendly, and encourage updates.

The code makes threading and messaging trivial and it allows trivial communication between a windows application doing traditional windows messages and threads using the buffered messaged scheme implemented here (see message.h, etc.) The CThread class allows one to create a thread by instantiating the “thread” method. Messaging is implemented via two classes, CMessage, and CMessageReceiver. CMessage is the body of a message, and is primarily a buffer of a size to be determined at construction time. CMessage are “owned” by the CMessageReceiver class. CMessageReceiver is a communication primitive. When a thread tries to receive a message it blocks if there are none. If this alarms you, forced blocking, there are a number of ways to avoid this. The thread can quarry the class for the number of outstanding messages, implementations of the class may implement a non-blocking protocol in which case the “receive” method will return null forcing the application to check the returned message pointer before use.

CMessageReceiver is abstract, however I provide several implementations. The basic implementation is CMessageQueue which provides a set of buffers of a size determined at construction time. The number of buffers can be fixed, or can allowed to grow either a fixed amount an unbounded about. Once a message is allocated it belongs to the queue. The user calls the “release” method to return a received or allocated message back to the queue. The call HWndReceiver is a CMessageQueue that adds Win32 Post or Send Message with the delivery of a message, waking up the windows thread. The windows code must handle retrieve the message from the queue, which is delivered as one of the Win32 message values.

To create a communication structure the programmer decides on one or more enumerations, paired or global, between communicating threads. The buffer size can be tuned to the needs of the receiver or can be big enough. (Other implementations with the same API would be straight forward, e.g. an underlying circular buffer but are not called for in the Win32 environment.) Each enumerated value will have some data type associated with it by agreement; if desired this can be a common base class that uses dynamic type casting to discover the actual type. In this later, dynamic type, case one would want to construct objects in the message buffer.

For this project I used the basic implementation linked to above, but I also have a version that allows the same interface for both Linux and Win32 where the types have been abstracted, i.e. to IMessage, IMessageReceiver, also IEvent, and extended to provide a more general capability. Let me know if you are interested in this version. I chose the less abstract version for this project in part because it is more self contained and easier to explain but if I had not been thinking quick and easy I would have used the other version. In any case the reuse story here is independent of this choice.

Now that I have provided some idea of the capabilities let me explain how this went and why having easy access to threading and messaging is so powerful. First there is an independent justification for messaging in this project. If the communication to the device or the Internet had been implemented in the GUI thread each time the application communicated with the device, or the web, it would go into an unresponsive mode possibly confusing and frustrating the user. By putting the communication in separate threads and running them at a lower priority than the GUI unresponsiveness can be almost completely eliminated; of course we are talking windows here so completely is not possible.

The basic design encapsulates via a simple message, check-for-new-device, or get-latest-version-info, to a thread that “knows” how to do that. BTW, this message is also encapsulated into an abstract API but I believe this has less to do with the success than the threading and messaging. The key I believe is having a high level independent object that knows how to deal with and owns the communication. This object is fundamentally reusable because it encapsulates both the data aspects and the activity aspect of the desired functionality.

Thomson typically has two versions of its code. It has the EPG version and the CLM version. The EPG is the Electronic Program Guide version, i.e. the version that is deployed to a user, and the CLM is the Command Line Monitor version that used for testing and validating the hardware and software components. In previous Thomson STBs the CLM version is available over a serial port, and is always available over a JTAG interface, but unlike versions this STB did not have an RS-232 port it only has the USB port. This is where the power of the data and activity encapsulation comes to the fore.

As an aside, some of this reuse power would have gone unutilized if we had reused an other paradigm, i.e. the serial port. Because it would have required driver work on the PC side we chose to go with the USB chip vendors default PC drivers, and these did not implement a serial port. For the record, I recommend the serial port solution.

All along Thomson was planning on a second project that would duplicate some of the effort of interfacing to the device. In fact, their first efforts in talking to a device, before I got on the project, failed to abstract access to the device all together; this is also where the decision to not do the interfaces as a COM port came from. However, since I had my thread and messaging abstractions already, I knew I could abstract access to the device without concern about schedule (in point of fact I knew that abstracting device access would accelerate the schedule).

One of the Thomson engineers, John K., kept on saying all I really want is HyperTerm, he said this because this is the way they traditionally communicated with a STB with the CLM image. Unfortunately the device side of the USB interface was quite dumb. It can handle one message at a time and relies on the PC side being smart about when and how it sends the data, again I decision made before I arrived. Limitation include not being able to send multiple message without a delay and that a message must fit within one USB packet. Once you get this kind of kludge working you really don’t want to redo it.

It was at the point that an other engineer was going to try and reuse my interfaces in an other windows app that would include the HyperTerm capability that I proposed implementing a console app that simply echoed device data to the terminal and took standard in and sent it to the device. How long would this take. The get the basic function up, about a day! There were caveats so that it was about three days before there was a working application that could really be handed off to anyone else. It instantly became the preferred tool, doing a better job than HyperTerm ever did, the console gave command line history, and logging function already the windows app gave a log. The only problem was that had to reboot the STB when you disconnected.

One extreme programming session later John and I had solved this by modifying the device software to notice disconnect and do a proper reconnect and asked by the host. By the way we called the program iziecho. Iziecho was not done. Iziecho would talk to three different images, the boot loader, the EPG, and CLM image. Initially, iziecho could only tell the boot loader to reboot into the download image. If for instance you want to change the download image you need to leave the program, start the upgrade program, and usually power cycling the STB, before doing the “upgrade.”

“It would sure be cool if we didn’t could upgrade from iziecho,” was bemoaned by John. Iziecho is used by the manufacturing line, and currently they needed to do all o f this for each device, requiring a whole bunch of manual intervention. Really pretty unacceptable but not that different from what they did with other STBs. Remember the other STBs have less capability and they can only be upgraded with the JTAG connected. This means staying connected to JTAG through much of the process. I responded, “that’s not a problem, I can do that in a day. The capability is in the interface, I don’t have to reinvent it.” Again there are things one does not consider when making these off the cuff estimates. For instance, to make it all play nicely I needed to add the same reboot capability to the EPG and CLM code that existed in the boot loader.

So two days later there is capability from iziecho to download any image, to write memory to reconnect to the new image after download. All capabilities that were not intended, and never planned and done in about one week of work.

All of this capability was enabled by creating an active object encapsulation.

To top this off, in the middle of the process I also create an API for calling iziecho from a windows program in much the same way that one would connect to a serial port. This allowed legacy code to talk to the new STB in the same way it did older STBs. An interesting thought, I could now rewrite my windows app to use iziecho rather than talking to the API directly!