Category Archives: Life

All ready, let GSoC 2014 begin!

Project

This summer the project is developing a userland driver for ipp class usb devices. At the most basic level this involves proxying http between usb bulk transfers and tcp. For the real world we also need the usb subsystem to auto-start us and to notify the spooler of the new printer.

I have taken the liberty of naming said new driver ippusbd. My high-level design goals are, in order of importance from most important to least:

  1. Low idle resource usage
  2. security over the usb interface
  3. Portability to non-Linux Unixes

Linux’s udev can handle the auto-start. Libusb gives us userland access to the usb device. Till should know how linux’s cups auto-discovery will work. The usb standard itself is well written and I enjoyed reading it.

 

Timeline

In theory Google Summer of Code does not start for another 20 some days. This official timeline is driven by the american university’s semester dates. In Canada our universities let out almost a month earlier. Thus I’m taking the extra month to work ahead.

I hope to have ippusbd printing by the mid-summer review. The summer’s second half can then get spent polishing and porting ippusbd to the BSDes.

Workstation

IMG_20140501_165238

This summer I’m trying out a standing desk setup. I’ve borrowed my sister’s room while she is in germany so I get a work room. My hope is having a separate room will let me focus better. When I was at microsoft getting things done was easy since the office created a physical seperation between play time and work time, this might replicate that.

The setup consists of two laptops, a Lenovo Thinkpad X1 Carbon, and a Lenovo Ideapad Yoga 2. The Yoga has a touch screen and the X1 gets the full keyboard and mouse. Together they create something like the multimonitor setup I have downstairs.

Gettext’s bindtextdomain() ignoring directory

If you use strace on a program using the gettext internationalization library and you see something like the below.

open("/usr/lib/locale/ru/LC_MESSAGES", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

This means gettext is looking in the default locale directory which is /usr/lib/locale on my ubuntu machine. This was a bug for me because I had used bindtextdomain() to change the directory for the locale’s .mo files.

The cause for gettext ignoring my custom locale directory was that I had set my LANG enviromental variable to just the language code. For example LANG=ja

Instead LANG should be a full xx_CC format, for example:

LANG=ja_JP

You can check what your terminal’s locale environmental variables are set to use `locale`. An example of this is below.

danieru@danieru-x1:~$ locale
LANG=ja_JP.UTF-8
LANGUAGE=ja:en_CA:en
LC_CTYPE="ja_JP.UTF-8"
LC_NUMERIC=en_CA.UTF-8
LC_TIME=en_CA.UTF-8
LC_COLLATE="ja_JP.UTF-8"
LC_MONETARY=en_CA.UTF-8
LC_MESSAGES="ja_JP.UTF-8"
LC_PAPER=en_CA.UTF-8
LC_NAME=en_CA.UTF-8
LC_ADDRESS=en_CA.UTF-8
LC_TELEPHONE=en_CA.UTF-8
LC_MEASUREMENT=en_CA.UTF-8
LC_IDENTIFICATION=en_CA.UTF-8
LC_ALL=

Above was a correct and valid set of locale variables. What caused gettext to not find my translation’s .mo files was the following broken set.

danieru@danieru-x1:~$ locale
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=ja
LANGUAGE=ja:en_CA:en
LC_CTYPE="ja"
LC_NUMERIC=en_CA.UTF-8
LC_TIME=en_CA.UTF-8
LC_COLLATE="ja"
LC_MONETARY=en_CA.UTF-8
LC_MESSAGES="ja"
LC_PAPER=en_CA.UTF-8
LC_NAME=en_CA.UTF-8
LC_ADDRESS=en_CA.UTF-8
LC_TELEPHONE=en_CA.UTF-8
LC_MEASUREMENT=en_CA.UTF-8
LC_IDENTIFICATION=en_CA.UTF-8
LC_ALL=

Bypassing ShawOpen Wifi’s Captive Portal

Shaw, a local cable monopoly, has started offering businesses a free wifi with free internet connection if they allow the installation of a public wifi access point. If you are a Canadian in the western provinces you might have seen these networks under the name “ShawOpen”. The term open in this case is misleading since only customers with shaw accounts can access the internet. The entire thing is a clever branding of the 2.4GHz frequency and businesses love being able to “offer” wifi with no cash out of their pocket.

 

Meanwhile I’m sitting here in the doctor’s waiting room and I am bit less enthusiastic over the ordeal. If Shaw wasn’t offering this service there is a good chance my doctor would be providing usable internet. Instead all I can do is poke around inspecting things.

 

There is more to inspect than initial impressions give. ShawOpen allows unauthenticated users to browse Shaw’s webportal and other Shaw owned web properties.

 

The easy bypass is through DNS which is unrestricted. While most protocols are blocked for unauthenticated users DNS requests are often not. In general restricted wifi access points do not restrict DNS because it requires setting up a special DNS server and isn’t the easiest solution. Since it is a common work around you can use iodine to proxy your traffic. To my discredit I don’t have a dns2ip proxy setup on my server so this is no use to me.

 

Instead with an unknown period of waiting ahead I set about inspecting those Shaw owned web properties. The best list for our needs is found at the bottom of Shaw Connect Lite. The full version of connect can be found here over the full internet. http://www.shawconnect.ca/ which also contains the list.

 

What you want to use for this task is either Firefox Development Tools or Chrome Development Tools. The panel of interest is the network connections tab. This will show what resources have been requested by the page.

 

What we want to find is a request which contains another url in the get parameters. These requests may be going to a proxy which is allowed to access the real interenet. You will find some false starts and will want to ignore anything going to Twitter or Facebook as these requests are for the social media buttons.

 

What you want is something like this: http://www.globaltv.com/images/blank.gif?size=100×70&src=http://media.globaltv.com/uploadedimages/content/shows/remedy/remedy_castdillon_400x500.jpg

Notice how it includes a full url to another domain. This is what you want to look for and there are a few of them on different accessible websites. This exact image proxy requires us to specify a size value in the get request but will accept 0x0 to skip resizing.

 

Thus allowing us to request: http://www.globaltv.com/images/blank.gif?size=0x0&src=http://www2.warnerbros.com/spacejam/movie/img/p-jamlogo.gif

In theory you could tunnel IP through these image proxies to your own proxy similar to how iodine functions. Of course it is more fun to find these loop-holes than use them so you may just want to do that.

Notes from Day 5 of UCalgary CPSC 585 Winter 2014

Final day!

Tuesday 7th:

Console Architecture

Our lecturing developers are quite console focused so they should know lots of interesting tidbits.

We’ll talk about

  • what is a console
  • console component
  • differences between consoles and pcs
  • benifits of console development
  • development environment
  • console game design
  • ps3 in detail

What is a console

  • dedicated game machine
  • nintendu wii(U)
  • gamecube
  • n64
  • nds
  • etc

Console history

  • Playstation (1995) – 33MHz mips, 2MB ram, cd storage
  • Plystation 2 (2000) – 300MHz mips, 32MB ram, dvd storage
  • Platstaion 3 (2006) – 3GHz PowerPC, 256MB ram and 256MB vram, dual layer Blu-ray
  • Playstation 4 (2013) – 1.6GHz 8 core x86, 8GB ram, four layer blu-ray + HDD

Differences between consoles and tv

  • TV vs higher resolution monitor
  • tv vs high color accuracy monitor
  • No HDD on some consoles
  • No virtual memory (on linux malloc will never fail since the memory is not allocated until you write to a page).
  • No keyboard nor mouse – makes FPS and RTS hard to implement

Console benefits

  • Fixed target, you can make assumptions without crying in a month
  • “dedicated” hardware, not sure how Xbone falls under this
  • consoles are “cheaper”
  • consoles are “more secure”, “less” copyright infringement “Please ignore the dreamcast”
  • more people buy games, meanwhile I have 144 games in steam. Yes, I’m not a console gamer
  • “Consoles are where the money is for games developers, certainly compared to pc”

Console liabilities

  • underpowered
  • little or no operating system
  • Lots of hardware level programming, DMA, task scheduling
  • closed production and distribution models, I agree

Development environment

  • Games are written on host machine
  • cross-compiled, visual-studios for xbox, GCC and SN for PS3, clang for PS4, codewarrior for nintendo wii.
  • Download to dev console over network or usb.

The Wii was a bit crazy. During development it would be connected over ethernet, usb, scsi, and serial. All at the same time.

Testing and debugging

  • game runs on dev console
  • debugging is done on host connected by network
  • xbox through visual studios
  • ps3 debugged over SN Debugger
  • Emulate DVD instead of burning disk

Development libraries

  • No Operating system but often have libraries
  • Lots of variation between consoles
  • early generation support will be weak
  • sometimes poorly translated from japanese
  • “On the PS2 sony had a nack for writing documentation which was correct, and on time, and useless”. “This sentence describes the exact behaviour for this instructions but not what or why you might use it. It could be useful for clipping but they don’t tell you”. “The assembly documentation gave everything in opcode order. If you knew the opcode of the instruction then fine, but otherwise you’d be doing a linear search. Oh, and there was no index.”

Game design on consoles

  • limited memory
  • lower resolution
  • played in a living room and sometimes in a party

“The GPU in the ps2 was called the emotion engine, granted it did make us feel emotions. Like rage, sadness, and hopeless-ness”

PS3 architecture

1 PPE (CPU)

  • powerpc
  • Two hardware threads required to capture full performance
  • 2 x 32K L1

7 “synergistic” processing elements SPU, co-processor

  • custom instruction set
  • 256K embedded SRAM
  • 128 x 128bit SIMD registers
  • main memory access via DMA only

 

High latency but high thurough-put. Every PS3 has 8 SPUs but they were having problems with silicon yield. On average every chip had an error killing at least one SPU.

Graphics

  • fast to read from GPU (22 GB/s)
  • fast to write from CPU (4GB/s)
  • slow to read from CPU (16MB/s!)

RSX graphics chip

  • 550MHz
  • based on GeForce 7800
  • 8 vertex shaders
  • 24 pixel shaders
  • 24 texture filtering units
  • 8 texture addressing units
  • peak theoretical pixel fill rate 4.4 Gpixel/s

GPU was what held the PS3 back. It betrayed the complex architecture. You cannot use the SPUs to interesting things because they are being used to make up for the power GPU. Each SPU in theory is equal to a 360’s main CPU core but the SPUs are all being used else-where.

Development Environment

  • Compilers, was GCC fork by sony, now SNC made by sony
  • IDEs, Visual Studios plugin available
  • Debuggers, ProDB from sony
  • OpenGL like PSGL
  • GCM used for high-performance

Playstation 3 issues

  • Memory dichotomy, everything is split
  • heterogeneous CPU architecture is hard to program
  • PSX performance is poor. RSX was an afterthought and designed from the beginning.
  • 360 had a more gradual and less rocky path to multithreading

Console Transitions

Warning that he cannot talk much about our current transition due to NDA and not knowing all the interesting tidbits.

  • transitions tend to happen at the same time
  • this generation some exclusive games are launching on both an old and new console.
  • current transition started by WiiU, this might have sparked Sony and Microsoft to release a year later
  • business turmoil, some one might get “dreamcasted”.

Loading times on consoles

  • Each generation RAM increases by 10-16x but optical disk bandwidth only by 2x
  • So for the PS4 and Xbone the goal is getting data on the hdd
  • But a full install takes an hour or more so now you can play a game while installing

We took a break at 10:45 until 11:00, now we’re talking about future of gaming.

Lunch at 12:00

Coming back at 13:00 I asked him about Mantle. He said he could not talk about that, which I think is good news since we can talk about the Ouya.

Had lecture on Memory & Data

Now lecture on Project Management

Took 15 minute break at 15:00. At 15:15 we started looking at Fifa 14 and NBA Live 14.

For our last lecture is a talk on getting a game in the games industry. If you’re interested in working in the games industry you’ll want to take this course. Sorry I will not be blogging this part, I’ll leave it as a treat for future students.