USB HID host side Perl libraries

Yaron Velner, Shahar Lev

http://courses.cs.tau.ac.il/embedded/projects/fall2009/usb_eth/images/LPC2148_EDU_350x221-board.gif

 

Content

USB HID host side Perl libraries. 1

Content 1

Introduction. 2

HID device. 2

HID communication concept 2

Win32 API. 3

High Level Design. 3

Host side. 3

Device side. 3

Implementation. 4

Host side. 4

Device side. 5

API Guide. 6

Overview.. 6

Getting Started. 6

Returned status value. 7

API functions. 7

Demonstration. 10

Preparing the environment for demonstration. 10

Running the demonstration. 10

Further enhancements. 14

Support Hardware Unplug/Replug detection. 14

Support standard data formats. 14

Additional API. 14

Another Action Items. 14

Files. 14

Host Side files. 14

Device Side files. 16

License. 17

Introduction

Our final project in "Embedded Systems" course at Tel-Aviv University was to design, implement, and test host-side native libraries that will allow Perl programs to communicate with USB HID devices.
The implementation was done for Windows XP 32 bit OS, and tested with LPC2148 board.
The idea was to encapsulate Win32 API C methods with Perl interface.

HID device

A human interface device or HID is a type of computer device that interacts directly with, and most often takes input from, humans and may deliver output to humans.

Click here to learn more.

HID communication concept

Overview

In short, HID device has controls that send or receive reports. The meaning of the reports (or alternatively – the functionality of the control) is determined by its control descriptor.

Controls

A HID control is a data source or data sink associated with a control descriptor. An example of a data source, or input control, is the thermometer. An example of a data sink, or output control, is an LED. Control data is obtained and sent to a device by using HID reports.

 

In addition there is a feature control which is a data source and a data sink. However, this kind of control should be use only for configuration information of the device (in our example, a timer value).

Reports

A HID report is used to transmit HID control data to and from a control. A report descriptor defines the format of a report. Input and output reports specify control data and feature reports specify configuration data. If a device supports more than one report of the same type, each report is assigned a unique report ID.

Control descriptor

A control descriptor specifies the protocol and meaning assigned to HID control data.

One of the important fields in the descriptor is the usage field.

Usages

HID usages identify the intended use of HID controls and what the controls actually measure.

Sending/Requesting reports

There are two ways:

·         Interrupt transfer – WriteFile / ReadFile

o   Possible only for report data.

o   Reports are not lost, as long as buffer is not full.

o   ReadFile hangs application until report is ready.

o   This is the main approach to continuously requesting/sending report.

·         Control transfer – SetOutputReport/GetInputReport, SetFeature/GetFeature.

o   Reports may lose.

o   Requesting report does not hang application.

Win32 API

Win32 API, informally WinAPI, is Microsoft's core set of application programming interfaces (APIs) available in the Microsoft Windows operating systems.

Click here to learn more.

High Level Design

Host side

The foundation is Win32 API C functions for HID.

Next layer is a partial functionality of the Windows Driver Kit Host Client example for HID.

Next layer is our own C HID driver abstraction, with interface that includes only ANSI C types.

Next layer consist of DLL and Perl package which are interface between Perl and C. This was done using SWIG.

Last layer is a Perl package that is an abstraction of HID driver.

 

Device side

The software of the device side contains firmware of a USB HID device.

It contains the basic USB functionality that fulfills the standard USB protocol, and in addition it has the HID functionality.

The HID functionality

As part of the USB protocol the device identifies itself as an HID device.

This causes the host application to request additional descriptor. In this descriptor the device sends an HID descriptor which contains a series of control descriptors.

 

In addition this layer handles the HID requests for receiving and transmitting the reports of the controls.

 

Controls we implemented (for the example)

Input controls

·         Thermometer

·         Joystick

Output controls

·         Motor

·         LCD screen

·         Pins 7-15 of GPIO (all under one control)

Feature control

·         Timer value

Implementation

Host side

Our C Hid Driver

First we had to download Windows SDK. Using the "Windows Driver Kit"->"Building environment" we built the Microsoft HID Client Example. At this point, we were able to divide our efforts, and simultaneously develop both Host side, and Device side using the MS Client example to debug it.

 

Second step was to create our own abstraction of an HID driver. We decided to use some of the MS client example functionality to handle an internal data structure, and use some abstractions Microsoft already implemented in their example. In order to do it, we changed the resource file of the example, so it will not include the main file, and made the target output to be a Lib file – rather than EXE.

 

Since we had troubles compiling the usbhid.h file in Visual Studio (later we found out, that officially Visual Studio does not support Windows SDK) we made our own h file that declares the relevant Win32 API functions, and copies the definitions of the needed structure types.

 

Finally, we implemented some simple data structure and basic logic for Hid C API.The C API header does not include any additional headers, so one does not need Win SDK headers to use it.

Since we have better knowledge in C (than Perl), we tried to make this layer as thick as possible, putting there all the validations and logic.

Hid Driver in Perl

We used SWIG tool to create a wrapper from C to Perl. Since our C Hid driver used only integral data types (int, char etc), the only additional effort we made was handling arrays and pointers.

So we had to add some C functionality for create/delete new array/pointer, and get/set array/pointer values. These functions are under #ifdef SWIG

 

We decided to add another layer on top of the SWIG Perl module output for two reasons:

1.      Some annoying handling with create/delete array/pointer was still needed.

2.      Use Perl flexibility – Various size arrays (data), multiple return values etc.

Our Own Perl Client Example

Our example program uses:

·         Hid Perl driver – DLL, Wrapper module, Perl driver.

·         Perl TK – for GUI implementation.

Device side

We used WinARM as our ARM IDE for Windows. WinARM was used both for compiling of the firmware code and loading it to the board.

After successfully running the USB exercise code from the course labs, we download LPC2148 USB HID Example code from Keil website.

 

We managed to compile the Keil code but we weren't able to get it to the point where Windows OS would identify a working HID device (Device Manager did show a newly created device but it failed with "The device cannot start (Error 10)").

 

We decided to try fixing the Keil code. What we did was:

·         Examining execution flow by debug prints.

·         Comparing chain of events between lab code and Keil code

·         Migrating cardinal parts of the driver from the lab code to the Keil code.

 

Perhaps the most significant problem we encountered while trying to make the Keil code work on our board is the faulty handling of alignment and packaging of structures for USB descriptors, which caused faulty data to be sent on control transfers. Despite out attempt to fix this problem, we couldn't get to fix this device error.

According to information on the Keil website, this firmware should, in fact, work with LPC2148 microcontrollers.

This leads us to believe that perhaps we can't get this code to work because it was not meant to be compiled using WinARM.

 

After many failed attempts, we started off with the code of the USB mouse exercise (which also abides the HID interface).

 

In order to make this code identifies as a generic HID device (not as a pointer or a keyboard), we migrated the USB and HID descriptors from the Keil code back to the lab code.

 

As a result of these changes, we were able to reach the desired state where the board would be recognized by the OS as an operating generic HID device.

 

After that we modified the USB descriptor to enable bidirectional interrupt transfers (through endpoint 1), and the HID descriptor as well in correspondence with the devised host-device protocol and using:

1.      The Device Class Definition for HID 1.11 documentation from USB.org.

2.      The HID Descriptor Tool software from USB.org.

 

We later added the skeleton of a handling code for certain HID-class requests concerning incoming and outgoing HID reports.

By our own host-device protocol we filled these handling methods of the HID device firmware using code from lab exercises.

API Guide

Overview

Driver includes the following files:

·         HidDriver.pm – Module for HID API functions

·         HidInterface.pm – Wrapper between Perl and C DLL

·         HidInterface.dll – Implementation of HID functionality using Win32 API

·         msvcr100.dll – file needed for programs that were compiled in visual studio

User must verify it has all the above files in its working directory.

The driver was tested on Windows XP 32 bits, with Active Perl v5.10.1 built for MSWin32-x86-multi-thread.

NOTE: We know for sure that it will not work in x64 version of Active Perl.

 

Driver supports multithreading, all accesses to global variables are done under lock of mutex.

Getting Started

Driver requires initialization by calling HidDriver::Init(), at the end of use, user must call HidDriver::Destroy() to recycle all resources.

 

After initialization, user can get a list of all plugged in devices (during initialization stage) by calling HidDriver::GetListOfDecives(). Every device is identified by a unique id. The id has no meaning, to get some clue about the actual device, user should call HidDriver::GetDeviceDescription().

 

Now user can open the device and read or write data using the appropriate functions.

 

User should use HidDriver::GetDeviceCapabilities() and HidDriver::GetInputControlCapabiliy(), HidDriver::GetOutputControlCapabiliy(), HidDriver::GetFeatureControlCapabiliy() to obtain control usages.

 

For Host Application Example see HAE.pl

Returned status value

Most of the API functions return a status value. Status 0 means that there were no errors, other values means error. In order to get a description of last error, call: HidDriver::LastErrorToString($LastError).

API functions

HidDriver::Init

·         Input:              None

·         Return value:   Status

·         Description:     Initialize internal data structure and allocates resources.

HidDriver::Destroy

·         Input:              None

·         Return value:   Status

·         Description:     Free all resources

HidDriver::GetListOfDecives

·         Input:              None

·         Return value:   Status, Array of Device Ids

·         Description:     Returns an array of all plugged in USB devices (during init stage).

HidDriver::OpenDevice

·         Input:              Device Id, Boolean – open for read, Boolean – open for write.

·         Return value:   Status

·         Description:     Open device for read and/or write

HidDriver::CloseDevice

·         Input:              Device Id

·         Return value:   None

·         Description:     Close device

HidDriver::WriteFile

·         Input:              Device Id, Report Id, Data (as bytes array)

·         Return value:   Status

·         Description:     A user-mode application should use WriteFile as its main approach to continuously send output reports to a device.

HidDriver::SetOutputReport

·         Input:              Device Id, Report Id, Data (as bytes array)

·         Return value:   Status

·         Description:     application can also use SetOutputReport routine to send output reports to a device. However, an application should only use this routine to set the current state of a device.

HidDriver::SetFeature

·         Input:              Device Id, Report Id, Data (as bytes array)

·         Return value:   Status

·         Description:     Sends a feature report to a device.

HidDriver::ReadFile

·         Input:              Device Id, Report Id

·         Return value:   Status, Data (as bytes array)

·         Description:     A user-mode application should use ReadFile as its main approach to continuously obtain input reports.

 

HidDriver::GetInputReport

·         Input:              Device Id, Report Id

·         Return value:   Status, Data (as bytes array)

·         Description:     An application can also use GetInputReport routine to obtain input reports from a collection. However an application should only use these routines to obtain the current state of a device. If an application attempts to use GetInputReport to continuously obtain input reports, reports can be lost.

HidDriver::GetFeature

·         Input:              Device Id, Report Id

·         Return value:   Status, Data (as bytes array)

·         Description:     An application can also use GetFeature routine to obtain feature report from a collection. However an application should only use these routines to obtain the current state of a device.

HidDriver::GetDeviceCapabilities

·         Input:              Device Id

·         Return value:   Status, Input report length, Output report length, Feature report length, Number of input controls, Number of output controls, Number of feature controls.

·         Description:     Returns the input report length, output report length and feature report length supported by the device. In addition, returns the number of input/output/feature controls.

HidDriver::GetInputControlCapabiliy

·         Input:              Device Id, Control index (0 – number of input controls – 1).

·         Return value:   Status, Report id, Usage page, Min usage id, Max usage id.

·         Description:     Return the supported report id and usages for an input control.

HidDriver::GetOutputControlCapabiliy

·         Input:              Device Id, Control index (0 – number of output controls – 1).

·         Return value:   Status, Report id, Usage page, Min usage id, Max usage id.

·         Description:     Return the supported report id and usages for an output control.

HidDriver::GetFeatureControlCapabiliy

·         Input:              Device Id, Control index (0 – number of feature controls – 1).

·         Return value:   Status, Report id, Usage page, Min usage id, Max usage id.

·         Description:     Return the supported report id and usages for a feature control.

HidDriver::GetDeviceDescription

·         Input:              Device Id

·         Return value:   Status, Product string, Manufacture string, Serial number string.

·         Description:     Returns 3 strings that describe the device.

HidDriver::GetInputBufferNumber

·         Input:              Device Id

·         Return value:   Status, Input number.

·         Description:     returns the current size, in number of reports, of the ring buffer that the HID class driver uses to queue input reports from a specified device.

HidDriver::SetInputBufferNumber

·         Input:              Device Id, Input Number

·         Return value:   Status.

·         Description:     sets the maximum number of input reports that the HID class driver ring buffer can hold for a specified device.

HidDriver::LastErrorToString

·         Input:              Last error value

·         Return value:   Error string.

·         Description:     Returns a description of last error

Demonstration

Preparing the environment for demonstration

Host side

·         Make sure you have WINDOWS XP 32bit and Active Perl v5.10 for 32bit.

·         If you don't have TK Perl module, run the EnvInstall.pl to install it. We recommend running this script anyway.

·         Make sure you have HidDriver.pm, HidInterface.dll, HidInterface.pm and msvcr100.dll files in your working directory.

·         Run HAE_GUI.pl from your working directory.

Device side

·         In the HID device firmware, lies a makefile which allows you to compile the code (by running "make") and loading it to the board (by running "make program").

·         Please make sure that the following settings in makefile.local file are correct: GCCBIN should point location of gcc program and LPC21ISP_PORT should hold the serial port identifier which will be used

·         to load the firmware to the board. Example values for this environment variables:

o   GCCBIN = C:/Inst/WinARM/bin/

o   LPC21ISP_PORT = com4

Running the demonstration

·         Run HAE_GUI.pl

·         Select the "Yaron and Shahar" device, and click on Open device button.

·         Capabilities widget:

o   After you opened the device, a capabilities widget replaces the devices widget.

·         Pins widget:

o   You can turn on/off Pins 7-15 of GPIO in the device. This is done using SetOutputReport method, since this operation is very short.

o   Pins 8 – 15 controls the LEDs.

o   Pins 7-8 controls the RGB LED.

o   Pin 12 controls one polarity of the motor.

 

·         Motor Widget:

o   You can turn the motor to the left or to the right. This is done using WriteFile method, since this operation might take a while.

·         Timer Widget:

o   You can set/get the value of an internal timer in the device. This is done using SetFeature/GetFeature, since this is the closest thing to a "feature" that we could think of.

·         Joystick Widget:

o   Click the "Launch joystick window" button.

o   Click the "Start reading device joystick" button.

o   Move the device joystick, and see the "X" moves accordingly.

o   Press on the josytick to stop the reading. We decided to stop reading this way since Perl TK does not supports multithread, so we can’t stop it without external intervention.

o   This was implemented using ReadFile, a report is send only when user moves the joystick.

 

·         LCD Widget:

o   Write text, and click on "Submit" button.

o   This is done using several calls to WriteFile (since operation might take a while).

§  First call cleans the LCD screen.

§  Next calls write only one char each.

·         Get Temp Widget:

o   Click on the "Get Temp" button.

o   The device example, periodically (when ever it is not busy) measures temperature and update global variables.

o   Upon request it sends a report of the variable value.

o   This was implemented with GetInputReport.

·         Command Widget:

o   Shows all calls to HID driver, and error messages if any.

 

 

Further enhancements

Support Hardware Unplug/Replug detection

Implement a method that invokes Perl Callback when device is unplugged/replugged.

Win32 API has a registration function for Window handle to receive a message when certain device is unplugged (or replugged).

 

However in order to use it, one must have a window. Since our application is a console application, this is not easy. In addition one should be able to run a Perl function from a C code.

 

We investigate the subject for a while, and we think that there following should work:

1.      Create a hidden window, in a different thread.

2.      When ever window gets unplug/replug message, invoke a callback from a global variable.

3.      It is possible (but no so simple) to active Perl code from C program, using PerlCall.

 

We decided not to implement it, also because it is not so easy, but mainly because we believe that creating additional thread is not something a driver should do (unless it has to).

Support standard data formats

There are predefined data formats for known devices. They are decided according to usage page and id.

It is possible to supply API for packing and packing data according to usage value.

We decided to communicate using a raw input (i.e bytes buffer).

Additional API

Microsoft supplies much more complicate API than we reveille here. We decided to supply an abstraction of this API that will be enough for basic communication between device and host.

Another Action Items

1.      Use different Perl GUI library. TK does not support multithreading, Make different thread for joystick that always calls ReadFile.

2.      Optimize performance.

3.      Test on additional devices.

4.      Test with multi-thread application.

5.      Support additional program languages (Java, Python).

Files

All host side files are at HIDProject/Host directory.

All device side files are at HIDProject/Device directory

Host Side files

Recall the high level design diagram:

C sources

Host Side files

Microsoft Client example

·         Microsoft Client Example\sources – this is the only file we changed. It is used in the make file. If you want to use it, obtain the Microsoft Client example, and copy it to <WinDDK>\7600.16385.0\src\hid\hclient

·         Microsoft Client Example\hclient.lib – the result of compilation.

Our Hid C driver

·         Our Hid C Driver\hid_sdk.h – this h file includes Win32 API declarations of types and functions, as described here.

·         Our Hid C Driver\HidInterface.c – Implementation of driver.

·         Our Hid C Driver\HidInterface.h – Header of driver.

·         Our Hid C Driver \HidInterface.lib – Result of compilation.

C to Perl interface

·         C to Perl Interface\HidInterface_wrap.c – file generated by SWIG.

·         C to Perl Interface\Obtained files\perl.h – Obtained from Active Perl.

·         C to Perl Interface\Obtained files\EXTERN.h – Obtained from Active Perl.

·         C to Perl Interface\Obtained files\XSUB.h – Obtained from Active Perl.

·         C to Perl Interface\HidInterface.dll – result of compilation. In building process, need to link with:

o   Our Hid C Driver\HidInterface.lib – Our Hid C driver.

o   Microsoft Client Example\hclient.lib – Modified Microsoft Client Example.

o   C to Perl Interface\ Obtained files\perl510.lib – Obtained from Active Perl.

o   C to Perl Interface\Obtained files\hid.lib – Obtained from Win32 API.

o   C to Perl Interface\Obtained files\setupapi.lib – Obtained from Win32 API.

o   C to Perl Interface\HidInterface.pm – Perl module. File generated by SWIG.

HID Driver in Perl

·         HID Driver in Perl\HidInterface.pm – file generated by SWIG.

·         HID Driver in Perl\msvcr100.dll – file needed to run visual studio compiled programs.

·         File from C to Perl Interface:

o   HID Driver in Perl\HidInterface.dll

o   HID Driver in Perl\HidDriver.pm

Perl Host Example

·         Perl Host Example\EnvInstall.pl – script that setup tk module. Need to run for GUI HAE.

·         Perl Host Example\HAE.pl – console line Host Application Example. Attached as a reference for developers

·         Perl Host Example\HAE_GUI.pl – GUI Host Application Example. Attached to show that things really work.

Device Side files

HID Device Firmware

·         HID Device Firmware\lpc2000 – LPC214x header directory.

·         HID Device Firmware\main.c – Main module, holds initialization procedure.

·         Files NOT-related to USB HID interface:

o   HID Device Firmware\auxiliary.c – Contains fine timing and step motor control methods.

o   HID Device Firmware\char-lcd.c – LCD module.

o   HID Device Firmware\char-lcd-imp.c – LCD module's core implementation.

o   HID Device Firmware\fifo.c – Very basic module of a FIFO data structure.

o   HID Device Firmware\print.c – Contains thin printing layer.

o   HID Device Firmware\uart0-polling.c – UART module.

o   HID Device Firmware\i2c.c – I2C module.

o   HID Device Firmware\vic.c – VIC (Vector Interrupt Controller) module.

·         Files related to USB HID interface:

o   HID Device Firmware\usb.c – USB module.

o   HID Device Firmware\usb.h – Contains USB definitions and macros.

o   HID Device Firmware\hid.h – Contains HID definitions and macros.

o   HID Device Firmware\desc.c – Contains USB and HID descriptors.

o   HID Device Firmware\user.c – Implementation of USB and HID event handlers.

 

License

Copyright (c) 2009-2010, Yaron Velner, Shahar Lev and the

School of Computer Science in Tel Aviv University.

All rights reserved.

Redistribution and use in source and binary forms, with or without

modification, are permitted provided that the following conditions

are met:

1. Redistributions of source code must retain the above copyright

   notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright

   notice, this list of conditions and the following disclaimer in the

   documentation and/or other materials provided with the distribution.

3. The name of the author may not be used to endorse or promote

   products derived from this software without specific prior

   written permission. 

THIS SOFTWARE IS PROVIDED BY THE AUTHOR `AS IS' AND ANY EXPRESS

OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED

WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE

ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY

DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL

DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE

GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS

INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,

WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING

NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS

SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.