GSoC 2014 Friday Report #10

Todo For This Week

  • Get new printer working group website in shipable state
  • Do not touch ippusbxd codebase

Completed This Week

  • Finished the todo list for the new PWG website
  • Only touched ippusbxd once

Todo Next Week

  • Make tcp.c support and require known http size
  • Fix atleast %50 of the TODOs
  • Figure out how ippusbxd should fit in with udev-configure-printer

Perhaps the biggest news this week was I got approved from the Linux Foundation for travel funding. So thank you to the Linux Foundation’s members, this will save a decent chunk of the GSoC stipend from going to travel and hotel costs.

After the week off I’m looking forward to working again on ippusbxd. The codebase is in workable prototype stage and I’ve got 2 months to make it robust.

GSoC 2014 Friday Report #9

Todo For This Week

  • Add back the size finding code
  • Release usb interface when we know a complete message has left the printer
  • Fill out Google Summer of Code mid-term evaluation

Completed This Week

  • Main loop and USB now again inspect packet size, tcp not yet
  • USB interface is now acquired then released before and after every HTTP request and response
  • GSoC mid-term evaluation completed and passed

Todo Next Week

  • Get new printer working group website in shipable state
  • Do not touch ippusbxd codebase

About a year ago I started work on the new printer working group website.  The website is close to ship-able so next week’s work will be getting the new site in shape for approval to go live.

Then the week afterwards I can return to the ippusbxd without this burnout. For the past 3 weeks I’ve worked over the weekend to maintain a streak of acivity on my github profile. I thought I was being productive, an reaction to the time I didn’t do anything for two days. In hindsight skipping weekends may have burned me out a bit. After this short break I promise not to skip weekend breaks.

GSoC 2014 Friday Report #8

Todo For This Week

  • Rewrite USB and TCP to not need knowledge of message size
  • Book flight and hotel for OpenPrinting Summit

Completed This Week

  • Flight and hotel booked
  • USB and TCP no longer depend on message size
  • Removed then unused message size finding code
  • Discovered printer did not support HTTPS after all
  • Added usb interface pool
  • Spawned thread for every tcp connections, which grabs a usb interface

Todo Next Week

  • Add back the size finding code
  • Release usb interface when we know a complete message has left the printer
  • Fill out Google Summer of Code mid-term evaluation

This week brought a new and more fundamental limitation ippusbxd must surmount. When a web browser opens a webpage it wants to open multiple HTTP connections and reduce page load times. Many years ago this meant using 2 connections. Now browsers open 6!

In addition these 6 connections are kept open and persistent. If a ipp over usb driver allocates one usb interface per tcp connection the web page will not load. Part of the page may load if the files have been requested over a lucky connection. The unlucky connections will hang and never get responses.

Early in the week I was working under the assumption the printer needed HTTPS. Once that code was working I learned this was wrong. The printer does not even support HTTPS. The “please use HTTPS” message on the webpage was a sporadic bug caused by lost resources.

Instead handling 6 TCP connections with only 2 or more real connections is our fundamental challenge which stumped me this week.

The solution I found was to support both encrypted HTTP, and read-able HTTP. When handling encrypted HTTP we’ll default to a ratio of one real connection to one TCP connection. Read-able HTTP is the nice case where we can know when a real connection will be unused. This lets us release the real connection, the usb interface, back to the interface pool. Then new requests over the TCP connection cause us to pull an interface out of the pool.

By releasing the interface after every HTTP message we will cycle through the TCP connections. Thus no TCP connection will stale.

Anyway ippusbxd is where I hope we would be for the mid-term. Provided clients do not exhaust the usb interfaces ippusbxd will bring up the printer’s webpage without issue!

2014-06-17-222055_3600x1080_scrot

The second half of GSoC will be less focused on ippusbxd. Instead my todo list consists of:

  • Finish the new pwg website
  • Write and practice the ippusbxd presentation
  • Make udev-configure-printer start ippusbxd
  • Make udev-configure-printer configure the CUPS queue
  • Make CUPS match ipp over usb printers between restarts, best to ask msweet about this

 

GSoC 2014 Friday Report #7

Todo For This Week

  • Send multiple usb packets worth of server response to client
  • Refactor everything to be better

Completed This Week

  • Multiple usb packets sent to client
  • Many functions refactored, thank you to Eric Fiselier (https://github.com/EricWF) for valuable help
  • Support content-length http messages
  • Support transport-encoding chunked messages
  • Rename project to IPPUSBXD

Todo Next Week

  • Rewrite USB and TCP to not need knowledge of message size
  • Book flight and hotel for OpenPrinting Summit

This Monday I got content-length messages supported. Which was fantastic news to present on Tuesday’s OpenPrinting conference call. Earlier on the weekend I noticed Mac OSX’s version of ippusbd was also called ippusbd. I mentioned this and Ira suggested renaming my ippusbd to ippusbxd.

So now I’m working on ippusbxd! I like the new name, it looks pretty good on the logo:

ippusbxd_logo

Also on the conference call I mentioned the “only” feature left to implement was supporting chunked transport http messages.

I was wrong.

Just today I landed chunked transport support upon which this greeted me in my browser:

2014-06-13-184835_3600x1080_scrot

Ha! This is a major set back! HTTPS wraps the entire HTTP messages in encryption. Thus preventing us from knowing the HTTP message’s size. In the current architecture this in turn prevents us from know when to stop asking for more data. If we went as far as integrating OpenSSL or a similar we could make a half-solution and man-in-the-middle attack the SSL connection, which would not be pretty.

Instead I’ll have to rewrite a decent chunk of the HTTP handling to not need the message size. We can avoid attacking the SSL connection and simplify the code if instead of using blocking TCP receives calls we use non-blocking calls. This lets us ferry data between the client and the server without worrying if any data is even coming from the client or the server.

This will require some strategic sleep()s to prevent 100% CPU usage but should work well in practice.

The other option could be to ask HP to disable HTTPS since ipp over usb requests never hit the internet. I like having everything encrypted so I’m glad this option is avoidable.

Still looking back this does mean all of the week’s work will be redone. The driver will be better for the rework. The codebase will be smaller and safer. So I’m not sad about the lost effort, plus it gives me something to mention at the OpenPrinting Summit.

In fact unless I knew ahead of time we needed HTTPS I might still think inspecting the HTTP for the message size was the best solution as it provides lower latency.

GSoC 2014 Friday Report #6

Todo For This Week

  • Open the found interfaces
  • Push the http message over bulk transport
  • Echo to command line any response from bulk transport

Completed This Week

  • Interfaces opened and stored in struct usb_sock_t
  • Client’s http request pushed to printer
  • First USB packet of Printer’s echo’d to command line
  • Refactored the tcp handling out of the http stuff
  • Detaches kernel driver if it has already claimed the printer
  • Option to direct error and warning messages to syslog

Todo Next Week

  • Send multiple usb packets worth of server response to client
  • Refactor everything to be better

More good progress this week. The USB interface stuff went faster than expected so I started on refactoring. Then on Wensday afternoon I decided to relax and play a new game I bought: Factorio,  a fantastic game of factory building.

Back when Command & Conqour 3 was released I thought it would be cool if instead of mining crystals and building tanks, you just mined crystals and built more crystal mining stuff. Factorio is that game times ten and this is a problem for me.

Factorio is so fantastic and fits my interests so well I played almost none stop from Wednesday afternoon, to 5am in the morning on Tuesday, woke up at 13:00 then played all day until 4am in the morning on Friday. In total over 20 hours in two days. I would look at the clock and notice time had jumped 5 hours ahead, and then resume playing.

You can see this in the git log. Progress is good until Wednesday afternoon at which point the addiction hits and nothing gets done for a day and a half.

The part that makes me feel bad is I was taking time off making things (ippusbd) to make imaginary things. Other games I’ve played have never been this bad. Ones like Fallout 3 are fun for 2-3 hours sessions then I have a natural inclination to take a break, thus I never risk letting myself get carried away. Meanwhile in Factorio there was always something to make, design, or tune be it the Oil supply line or the iron plate production line.

Thus to me Factorio is too good and I cannot handle it. This morning I deleted the game and all savefiles and I intend to not play ever again. I’d recommend Factorio to others except it brings up the MMO catch-22: liking it is the worse case scenario.