Toolkit Advantages

The toolkit offers five unique features and advantages that are not available in any other source code library (either on mobile or desktop platforms).  These are fully outlined in the following sections.

Access to Geographic Data in Vector format

The ability to interact with map data haptically or audibly requires that the data is available in a form that does not rely on pre-rendered image tiles. This is the primary advantage of the toolkit in contrast to those applications that are purely mono-modal (i.e. visual). For example the data representing hiking trails in a national park must be available in a vector format and differentiable from the other features. This ability to differentiate between map data features (not just using their visual appearance) is essential for multi-modal applications. By acquiring map data in this way and providing HCI functions in the toolkit to represent it through non-visual (as well as visual) interfaces, the toolkit offers a unique multimodal architecture. Over slower wireless networks the transfer of vector data is also considerably faster and local caches can hold more information making the user experience much more responsive. Figure 1 illustrates how vector data can be rendered on mobile devices with greater flexibility than using raster data. Vector data can also be combined with other data, for example height data from different sources. 

Figure 1: Vector data

Infrastructure for Acquiring Geographic Data from Multiple Sources

A typical HaptiMap application requires the use of geographical data from multiple sources at any one time.  For example:

• Details of topographic features such as roads, paths, fences and natural features may come from a server provided by a national mapping agency.
• Details of recommended hiking trails may be acquired as a linestring representing a suggested route obtained from an Internet route planning service.
• Information on local points of interest could be retrieved from an additional data source provided by a local tourism agency or be self-generated. 

The toolkit provides an architecture consisting of multiple geographic data plug-ins that can connect to different data sources and retrieve data from them.  It can then merge everything together into a common co-ordinate system and storage format for ease of access by toolkit users.

All the complexities of working with geographical data (e.g. different co-ordinate systems, logical structures, file/transfer formats, attribute models etc.) are dealt with by the plug-ins and are hidden from the user. As such, the user of the toolkit only deals with a simple integer co-ordinate system, which they can treat as a conventional three-dimensional Cartesian system, with simple and intuitive geometric operations for calculating distances, direction etc.

Dynamic Contextualised Map Rendering

A visual map is sometimes necessary and as such it is useful to be able to augment a non-visual display for sighted users. It can also be useful to partially-sighted people, but the default appearance of pre-rendered tiles may not be suitable for these users.  For example, they may need higher contrast or larger font sizes for labels (e.g. street names) shown on the map.

Figure 2: A map drawn using tiles and a custom map rendered using contextualised data derived from raw data.  Maps rendered from raw data offer an easy way to provide multimodal responses such as reading street names and rendering hapic feedback.

Since the Haptimap toolkit stores geographic information as vector data rather than pre-rendered tiles, the fast graphical processors in modern smart-phones can be used to:

• Dynamically render the visual appearance of the map to accentuate features.
• Or only display features of interest.
• Or to provide contextualised rendering based on the needs of the individual user.

The toolkit fully supports this and provides sample rendering styles that can be customised based on the needs of particular groups of users.

This is illustrated in Figure 2 where a toolkit rendering of a map using tens of bytes of data is compared with a raster ‘tile’ rendered map from OpenStreetMap.  The OSM rendering requires a much higher bandwidth for data transmission and its image cannot be easily contextualised (e.g. pointing at a road and having its name read out). 

Cross platform API for mobile and desktop platforms

The market for mobile devices is quite fragmented and the toolkit’s utility would be limited were it to be targeted towards only one of these platforms. Thus the toolkit has been developed, as far as possible, to support a wide range of platforms.

Supported mobile platforms include: 

Android
iPhone OS 3.x & 4.x
Windows Mobile 6.x
Symbian 3rd and 5th editions
Linux phones (e.g. OpenMoko and Maemo 4 & 5)
 

Supported desktop platforms comprise: 

Windows (XP, Vista and 7)
Linux
Mac OS X (Leopard 10.5 and Snow Leopard 10.6)

The toolkit core and mantle compile and run on all these platforms. An important aspect of cross-platform support that the toolkit provides is support for the specialist haptic hardware often used for multi-modal output. Hardware developed by HaptiMap partners uses the Bluetooth standard for wireless communications and the toolkit supports access to this on all platforms using an identical API. Access to other platform-specific functionality such as graphics output is also unified and simplified by the toolkit.  For example, 3D graphical rendering can be done in an analogous way to that used in the GLUT for desktop platforms; The toolkit provides the functionality of setting up the drawing surface and presenting it to the user. The user then has only to use standard OpenGL drawing commands in a callback function.

Pluggable HCI modules for specific platforms

The toolkit provides a large amount of useful functionality across all supported platforms, enabling really useful applications to be built based on it. However, it is important to realise that the target audience for the toolkit is software developers.  As such, development knowledge and experience is certainly required in order to maximise the potential of the toolkit.

In order to ease the learning curve and allow even inexperienced developers to produce useful toolkit-based applications quickly, the toolkit also includes a large number of pluggable HCI modules for the different platforms. As the name suggests, these can be easily “plugged” into existing applications to add advanced multi-modal human-computer interface functionality quickly and easily.

Toolkit Architecture

As illustrated in Figure 3, the toolkit is divided into three layers with each layer using features in the layer below it. The Core and Mantle layers, together with the geographic data plug-ins, form the toolkit library. Code modules, example programs and pluggable task components form the highest layer of the toolkit.  This is known as the Crust and they will be used by developers as templates on which to build applications.

Figure 3: The toolkit layered structure illustrating the components of the three layers.

More details on the components of the core, mantle and crust are described in Deliverable 4.3.  Information is also provided regarding the geographical data plugins. 

Core
The toolkit core provides low-level functionality to the rest of the toolkit. This primarily includes:

• Platform-specific abstractions, so that functionality that operates differently on different operating systems can be accessed using a single API across all supported toolkit platforms.
• Hardware drivers, for certain specific hardware devices that the toolkit includes native support for, e.g. hardware devices developed by HaptiMap partners.
• Geographical data storage—an efficient and optimised implementation of the vector data storage required by toolkit applications.
  

The core is licensed under the LGPL and is written primarily in ANSI C, but includes small amounts of platform-specific code where necessary to implement platform-specific abstractions (e.g. Java for Android or Objective C for iPhone).

Mantle
The toolkit mantle includes functionality that builds on the abstracted interface provided by the core. Mantle modules provide “building blocks” that can be assembled to create user interface components. In general, they either interact with a user via the core’s HCI functions to deliver map information (e.g. a set of directions through vibration patterns), or use the sensor data from the core’s utilities to provide some useful information (e.g. a bearing or a context). The modules within the mantle are all platform independent, written only in ANSI C and licensed under the LGPL.

Geographical Data Plugins
The toolkit geographical data plugins are software modules  that are logically separate from the toolkit core that abstract the complexities of working with geographical data (i.e. different co-ordinate systems, logical structures, file/transfer formats, attribute models etc.) from the toolkit core.  This is necessary because the core has been developed to work with integer Cartesian co-ordinates. It is necessary to write a plugin for each type of data source that has to be supported by the toolkit. These are written in C as far as possible, but may include some platform-specific code if necessary.  At present they are linked into the main toolkit library object and so must also be licensed under the LGPL. The logical separation from the toolkit core means that they could be implemented as separate shared libraries, should licensing considerations make this necessary.

Crust / Applications Layer
The toolkit crust consists of user interface components and advanced functionality built on the lower layers of the toolkit. It contains sample programs and code for platform specific applications, e.g. haptic surface map height rendering on desktop applications. Much of the crust is platform-specific and may be written in any appropriate language. Code in the crust is not compiled into the toolkit library object, and so may be placed under any license.

Implementation and Code Design
For detailed information on the architecture, implementation and code design of the toolkit, please see Chapters 2 and 3 of Deliverable 4.2.