Todo This Week

• Make 6 parallel connections bring up same webpage as single connection browsers.
• Handle http messages larger than 8KB
• Complete all the TODOs in http.c

Completed This Week

• Support large http packets
• Fixed some crashes
• Fixed some data loss
• Fixed a bunch of TODOs

Todo Next Week

• Fix things

This week consisted of bug fixing and tracking down broken code. I’m not proud to say that things are no really working. I now realize that things only worked a few weeks ago because of another bug. I thought TCP and USB were requiring known message sizes but really they were still passing data back & forth as packets came in.

I’ve fixed many of the bugs but I’m still hitting bugs like this:

01. Note: USB: Getting 262144 bytes of 0
02. Note: USB: Got 320 bytes
03. Note: USB: Getting 8704 bytes of 8953
04. Note: USB: Got 2048 bytes
05. Note: USB: Getting 6656 bytes of 8953
06. Note: USB: Got 2048 bytes
07. Note: USB: Getting 260096 bytes of 0
08. Note: USB: Got 2048 bytes
09. Note: USB: Getting 4608 bytes of 8953
11. Note: USB: Got 2048 bytes
12. Note: USB: Getting 2560 bytes of 8953
13. Note: USB: Got 441 bytes
14. Note: USB: Getting 2048 bytes of 8953
15. Note: USB: Got 336 bytes
16. Note: USB: Getting 260096 bytes of 0
17. Segmentation fault (コアダンプ)

On line 1 of this fresh server responce we see USB did not know what size to expect so it asked libusb for enough data to fill its buffer.

On line 2 USB received  from the printer the message’s header. We can guess this because 320 bytes is an average header size.

Line 3 confirms our header guess because now USB knows how large the HTTP packet will be.

On line 7 USB somehow forgets how large the packet will be, so it asks for a full buffer.

Lines 8 to 15 look normal. Keep in mind in the “Getting X” the X has been rounded up to the nearest multiple of 512. This is something required by libusb so a precaution incase the usb device sends a full 512 byte packet instead of what we expect.

Then on line 16 USB forgets the message size again, just as it did on line 7. Also note how it thinks the buffer has the same amount of spare capacity as it did on line 7.

Then we segfault.

The above bug is something I hit while tracing a different bug where USB was getting data and could not find the packet size at all.

This past week I’ve been adding more and more tracing statements and it has helped. I cannot print the messages themselves since they are too large and the binary requests often break my terminal. Then since data has been left in the USB pipe any new requests are going to get malformed responses, which requires rebooting the printer to clear.

So, I’ve got four full weeks until GSoC ends. I plan to work more afterwards but I don’t want to ask my mentor to “Just trust me, I promise to fix it afterwards”. Instead ippusbxd needs to work. As well there is the Openprinting Summit starting on the 12th and I want to demo ippusbxd there.

The core things which need to get done are:

1. ippusbxd needs to work
2. udev-config-printer needs to startup ippusbxd
3. udev-config-printer needs to add a print queue to cups
4. cups should identify the printer between reboots

Its hard to estimate how long fixing ippusbxd will take but I cannot see it taking less than 1 week and I cannot afford for it to take more than 3 weeks.

So I’ll plan for it to take 2 weeks.

Todo For This Week

• Make tcp.c support and require known http size
• Fix atleast %50 of the TODOs
• Figure out how ippusbxd should fit in with udev-configure-printer

Completed This Week

• Made tcp.c now require knowable http message size
• There are more TODOs than ever

Todo For Next Week

• Make 6 parallel connections bring up same webpage as single connection browsers.
• Handel http messages larger than 8KB
• Complete all the TODOs in http.c

Slow week, every time I completed a TODO I find one or two more things to mark TODO. Thinking through the buffer handling edge cases can be a bit confusing and not very much fun.

Todo For This Week

• Get new printer working group website in shipable state
• Do not touch ippusbxd codebase

Completed This Week

• Finished the todo list for the new PWG website
• Only touched ippusbxd once

Todo Next Week

• Make tcp.c support and require known http size
• Fix atleast %50 of the TODOs
• Figure out how ippusbxd should fit in with udev-configure-printer

Perhaps the biggest news this week was I got approved from the Linux Foundation for travel funding. So thank you to the Linux Foundation’s members, this will save a decent chunk of the GSoC stipend from going to travel and hotel costs.

After the week off I’m looking forward to working again on ippusbxd. The codebase is in workable prototype stage and I’ve got 2 months to make it robust.

Todo For This Week

• Add back the size finding code
• Release usb interface when we know a complete message has left the printer
• Fill out Google Summer of Code mid-term evaluation

Completed This Week

• Main loop and USB now again inspect packet size, tcp not yet
• USB interface is now acquired then released before and after every HTTP request and response
• GSoC mid-term evaluation completed and passed

Todo Next Week

• Get new printer working group website in shipable state
• Do not touch ippusbxd codebase

About a year ago I started work on the new printer working group website.  The website is close to ship-able so next week’s work will be getting the new site in shape for approval to go live.

Then the week afterwards I can return to the ippusbxd without this burnout. For the past 3 weeks I’ve worked over the weekend to maintain a streak of acivity on my github profile. I thought I was being productive, an reaction to the time I didn’t do anything for two days. In hindsight skipping weekends may have burned me out a bit. After this short break I promise not to skip weekend breaks.

Todo For This Week

• Rewrite USB and TCP to not need knowledge of message size
• Book flight and hotel for OpenPrinting Summit

Completed This Week

• Flight and hotel booked
• USB and TCP no longer depend on message size
• Removed then unused message size finding code
• Discovered printer did not support HTTPS after all
• Added usb interface pool
• Spawned thread for every tcp connections, which grabs a usb interface

Todo Next Week

• Add back the size finding code
• Release usb interface when we know a complete message has left the printer
• Fill out Google Summer of Code mid-term evaluation

This week brought a new and more fundamental limitation ippusbxd must surmount. When a web browser opens a webpage it wants to open multiple HTTP connections and reduce page load times. Many years ago this meant using 2 connections. Now browsers open 6!

In addition these 6 connections are kept open and persistent. If a ipp over usb driver allocates one usb interface per tcp connection the web page will not load. Part of the page may load if the files have been requested over a lucky connection. The unlucky connections will hang and never get responses.

Early in the week I was working under the assumption the printer needed HTTPS. Once that code was working I learned this was wrong. The printer does not even support HTTPS. The “please use HTTPS” message on the webpage was a sporadic bug caused by lost resources.

Instead handling 6 TCP connections with only 2 or more real connections is our fundamental challenge which stumped me this week.

The solution I found was to support both encrypted HTTP, and read-able HTTP. When handling encrypted HTTP we’ll default to a ratio of one real connection to one TCP connection. Read-able HTTP is the nice case where we can know when a real connection will be unused. This lets us release the real connection, the usb interface, back to the interface pool. Then new requests over the TCP connection cause us to pull an interface out of the pool.

By releasing the interface after every HTTP message we will cycle through the TCP connections. Thus no TCP connection will stale.

Anyway ippusbxd is where I hope we would be for the mid-term. Provided clients do not exhaust the usb interfaces ippusbxd will bring up the printer’s webpage without issue!

The second half of GSoC will be less focused on ippusbxd. Instead my todo list consists of:

• Finish the new pwg website
• Write and practice the ippusbxd presentation
• Make udev-configure-printer start ippusbxd
• Make udev-configure-printer configure the CUPS queue
• Make CUPS match ipp over usb printers between restarts, best to ask msweet about this