Interlude II

It’ s the second time through the lifetime of this blog that development has slowed almost down to a halt. Just like in my November 2008 posts, both project-related and real-life conditions contribute to this. From the project’s standpoint, I have been offered a proposal for volunteer help, and have decided to construct a few more prototypes, so I can send over (a working) one to start development in parallel [BTW, should anyone else be interested in volunteering, please drop me a note]. So, I have swept the dust off my stash of PCBs, and ordered again the necessary materials to build three more boards (my only prototype is showing some signs of bad behavior on occasion, so building a few more prototypes to have handy does not sound like a bad idea anyway).

There are news from the real life front as well: our newborn second son has just arrived home! As you can imagine, a lot of non-openusbfxs-related tasks keep me busy most of the day, and for the rest of the time, I have to try to fill some gaps for my real-life job. Although all this leaves little time for the project now, I hope that things will gradually settle down and I ‘ll have the chance to pick up again the soldering iron.

In the meantime, I have uploaded all project-related code, this time to Google Code for people to see, and possibly comment, use, or contribute to (I decided moving off, however the policy there is to keep code around, so the old — and possibly, copyright violating — code is still around; confusing, isn’t it?). Some of the uploading work included removing parts of code that are clearly Microchip’s and just provide just a few source patches to indicate my own changes (all binary PIC code is readily available however). That, plus adding appropriate copyright and license notes everywhere kept me a bit busy the last few days.

In the trial-and-error front, the situation is much as I described it in my previous post. This means that I cannot get a good voice quality yet: initially, and for one or two seconds, audio comes out OK, but then it looks as if synchronization is lost or packet losses occur. Then, if the audio file is sufficiently long, it seems to synch again for a while, then I get losses again, and so forth. This is weird, because I have been mirroring the sequence numbers of the OUT packets (the ones sent from the PC to my board) into the IN packets. By observing the mirrored sequence numbers, it seems that no loss occurs. Not a single packet is missed or reordered (well… almost; see below). I cannot tell anything about strict packet transmission timing, since the sniffer does not give me any real visibility into what happens on-the-wire.

In addition, I got a weird finding, which could be related to the intermittent audio quality. During the first two libusb isochronous receive (reap) operations, the first two packet slots are empty (remember that, depending on the buffer size, a buffer consists of several 16-byte packets, and each 16-byte packet contains 8 bytes of audio data, plus other control information). It is normal to see the first two slots empty: because of ping-pong buffering on the PIC, it takes two packets’ time to get the first OUT sequence number mirrored into an IN packet. Again, this happens for the first two reap operations. However, from the third reap operation and onwards, the first packet slot in each receive (IN) buffer seems to be occupied with the packet that comes next to the last one in that buffer! In other words, it seems that libusb returns the IN buffer one packet too late, while it resets some internal pointer, so that one packet more than the expected number appears, and this extraneous packet is placed at the beginning of the buffer. Unfortunately (from a debugging standpoint), the sniffer cannot tell me whether this is happening too in the OUTgoing direction: the sniffer “sniffs” data at the user-kernel interface, and not on-the-wire, so it is hard to tell actual data gets transmitted on the wire. Obviously, this is why debugging this issue is hard.

All this remains a mystery to me. There are two plausible scenarios: either my board loses synchronization with the SOF pulse, or libusb occasionally shufles OUT packet data, much the same way as it does with the IN packet data. I am in favor of the second scenario, not just because I am the proud designer and builder of my board, but because it makes more sense. I have measured inter-packet time using TMR3 (you have to look at the ISR code to see how), and the time is constantly 24000 PIC cycles per each two packets (I am reporting the time only on even packets — or is it only on odd ones, it eludes me now), which amounts to 2 ms exactly.

On the other hand, these 2 ms are counted using the PIC’s own crystal-based clock frequency as a time reference, and drifts between that and the PC clock may well exist. But my feeling is that such drifts would not be so important as to cause a loss-of-synch condition every few seconds.

Moreover, if it were a mis-synch issue, periodic re-synch would recur every now and then in fixed time intervals. This is not the case, though: re-synch occurs in seemingly random periods. Sometimes, many seconds of audio comes out without distortion, while in still other cases, I get garbled sound right away. Moreover, by fiddling with libusb buffer sizes, I have managed to get better or worse results. All these tend to put the blame on the side of libusb (or my userspace code) rather than on the board’s side.

In any case, my plans are to abandon further development using libusb and see how I can write a linux kernel driver. Presumably, there I will have less undesirable effects such as the .NET garbage collector or some libusb bug getting in my way. On the other hand, the code debugging cycle will be considerably longer. But I am not worrying about this. Time will show.

In any case, switching to a Linux kernel driver will take some time; in addition, I need to assemble a few boards, so more time there. Hang on, though, it will happen in the end. And, let me not forget to mention that all kinds of help are greatly appreciated!


2 Responses to “Interlude II”

  1. openusbfxs Says:

    A nice idea from a friend was to run my driver program (including of course the libusb driver) in a Virtual Machine running some version of Windows, and run the sniffer in the supervisor OS (the one that hosts the VM). This will get past the user-kernel interface of libusb, so the data I ‘ll be able to sniff will be the exact same data going to and coming from the wire.

  2. openusbfxs Says:

    I tried the VM idea that is outlined above. Performance is significantly worse when the driver program runs inside the VM. The sniffer (running in the host OS, not inside the VM) worsens performance even more. As a result, when the sniffer starts capturing, packet losses occur in both the IN and the OUT directions. The driver program’s code does not tolerate such losses in the IN direction (the program gets stuck forever waiting for a full buffer’s worth of IN packets). So, once more, there is no verdict as to what is causing distortion in the audio. I also created a command-line version of the code, which is based solely on win32 API (no .NET CLR, no garbage collector, etc.) [the code is to be posted soon on the project’s Google code page — BTW, this is a useful thing in any case, because it does not need the .NET runtime to be installed, and it can produce a textual log in a file for post-mortem analysis]. With that version, the audio file plays a bit longer than with the .NET version, however eventually it becomes distorted after three to four seconds or so. All this proves nothing in reality. Halas, still more testing is needed…

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: