Terminal Mode Technical Architecture Release Candidate v0.9

Terminal Mode Technical Architecture
Release Candidate v0.9

This document is an integral part of the Terminal Mode Specification, and together with “TmSer-
verDevice:1 Device Template, version 0.9”, “TmApplicationServer:1 Service Template, version
0.9”, “TmClientProfile:1 Service Template, version 0.9”, “TmClientDevice:1 Device Template, ver-
sion 0.9”, “TmApplicationClient:1 Service Template, version 0.9”, “TmConnectionManager:1 Ser-
vice Template, version 0.9” define the Specification version 0.9.

Copyright © 2010 Nokia Corporation, Audi AG, BMW AG, Daimler AG, Porsche AG, Volkswa-
gen AG. All rights reserved.
The copyright holders hereby grant you the right to make verbatim copies of the Specification
and to make available and distribute such copies. You may not distribute, make available or copy
only parts of the Specification, nor modify or create any derivative works of the Specification. No
patent rights or other rights not expressly stated as granted, are granted herein.
The above copyright notice and these terms and the following disclaimer must be retained in all
copies of the Specification.
THE SPECIFICATION IS PROVIDED “AS IS”. THE COPYRIGHT HOLDERS GIVE NO
WARRANTIES, EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION THE IMPLIED
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT OF ANY THIRD PARTY INTELLECTUAL PROPERTY RIGHTS
REGARDING THE SPECIFICATION.

Document editor:
Jörg Brakensiek
Jorg.Brakensiek@nokia.com
Nokia Inc.
955 Page Mill Road
94304 Palo Alto, CA
USA

Copyright © 2010 Nokia Corporation, Audi AG, BMW AG, Daimler AG, Porsche AG, Volkswagen AG.
All rights reserved.

-2-

List of Contributors

Name Affiliation
Dennis Fernahl Carmeq (for Volkswagen AG)
Jörg Brakensiek Nokia Corporation
Holger Grandy BMW AG
Kari Kostiainen Nokia Corporation
Keun-Young Park Nokia Corporation
Mark Beckmann Volkswagen AG
Martin Fesefeldt Volkswagen AG
Martti Ala-Rantala Nokia Corporation
Matthias Benesch Daimler AG
Michael Wolf Daimler AG
N. Asokan Nokia Corporation
Raja Bose Nokia Corporation


-3-

TABLE OF CONTENTS

TABLE OF CONTENTS .............................................................................................................................. 3
LIST OF FIGURES ....................................................................................................................................... 5
LIST OF TABLES ......................................................................................................................................... 7
TERMS AND ABBREVIATIONS ............................................................................................................... 9
1 ABOUT ................................................................................................................................................ 10
2 INTRODUCTION TO TERMINAL MODE .................................................................................... 11
3 CONNECTIVITY PROTOCOL STACK......................................................................................... 13
3.1 PHYSICAL & LINK LAYER ............................................................................................................. 13
3.1.1 Universal Serial Bus (USB) ..................................................................................................... 13
3.1.2 Wireless Local Area Networks (WLAN)................................................................................... 14
3.2 NETWORKING AND TRANSPORT LAYER ........................................................................................ 14
3.2.1 USB Networking ...................................................................................................................... 15
3.2.2 WLAN Networking ................................................................................................................... 16
3.2.3 Transport Layer ....................................................................................................................... 16
3.3 SESSION & APPLICATION LAYER .................................................................................................. 16
4 AUTHENTICATION AND SECURITY .......................................................................................... 17
4.1 HOST AUTHENTICATION MECHANISMS......................................................................................... 17
4.1.1 USB Networking ...................................................................................................................... 17
4.1.2 WLAN Networking ................................................................................................................... 17
4.2 CONFIDENTIALITY AND INTEGRITY MECHANISMS ........................................................................ 17
4.2.1 USB Networking ...................................................................................................................... 17
4.2.2 WLAN Networking ................................................................................................................... 17
4.3 DEVICE IDENTIFICATION MECHANISM .......................................................................................... 17
5 DISPLAY OUTPUT AND CONTROL INPUT ............................................................................... 19
5.1 CONNECTION SETUP ..................................................................................................................... 20
5.2 TRADITIONAL VNC PROTOCOL PHASES ....................................................................................... 21
5.2.1 Handshaking Phase ................................................................................................................. 21
5.2.2 Initialization Phase .................................................................................................................. 22
5.2.3 Framebuffer Update and Event Phase ..................................................................................... 23
5.3 VNC TERMINAL MODE EXTENSION MESSAGES ........................................................................... 26
5.3.1 Display Configuration Messages ............................................................................................. 28
5.3.2 Event Configuration Messages ................................................................................................ 31
5.3.3 Event Mapping Messages ........................................................................................................ 34
5.3.4 Key Event Listing Message ...................................................................................................... 36
5.3.5 Virtual Keyboard Trigger Messages ........................................................................................ 39
5.3.6 Device Status Messages ........................................................................................................... 41
5.3.7 Content Attestation Messages .................................................................................................. 44
5.3.8 Framebuffer Blocking Notification .......................................................................................... 47
5.4 ADDITIONAL ENCODINGS AND PSEUDO ENCODINGS..................................................................... 49
5.4.1 Terminal Mode Pseudo Encoding ............................................................................................ 49
5.4.2 Context Information Pseudo Encoding .................................................................................... 49
6 AUDIO OUTPUT AND INPUT......................................................................................................... 51
6.1 RTP PACKET STRUCTURE AND HEADER DEFINITION.................................................................... 52


-4-

6.2 RTP AUDIO PAYLOAD DEFINITION ............................................................................................... 53
6.2.1 16 Bit Audio Payload (Mono) .................................................................................................. 53
6.2.2 16 Bit Audio Payload (Stereo) ................................................................................................. 54
6.3 ESTABLISHING THE RTP CONNECTION ......................................................................................... 54
6.4 RECOMMENDATION FOR CLIENT AND SERVER IMPLEMENTATION ................................................ 55
6.5 INTEROPERABILITY WITH BLUETOOTH.......................................................................................... 56
6.5.1 Bluetooth Profiles relevant for Terminal Mode ....................................................................... 56
6.5.2 Interoperability States –Terminal Mode Server Perspective ................................................... 56
6.5.3 Interoperability States –Terminal Mode Client Perspective .................................................... 58
7 SERVICE NEGOTIATION FRAMEWORK .................................................................................. 61
7.1 UPNP USAGE MODELS ................................................................................................................. 61
7.1.1 2-Box Pull Model ..................................................................................................................... 61
7.1.2 2-Box Push Model.................................................................................................................... 61
7.1.3 3-Box Model............................................................................................................................. 62
7.2 UPNP DEVICE ARCHITECTURE ..................................................................................................... 63
7.3 TMSERVERDEVICE:1 DEVICE ....................................................................................................... 63
7.3.1 TmApplicationServer:1 Service ............................................................................................... 63
7.3.2 TmClientProfile:1 Service ....................................................................................................... 64
7.4 TMCLIENTDEVICE:1 DEVICE ........................................................................................................ 64
8 AUDIO LINK SELECTION .............................................................................................................. 66
8.1 AUDIO LINK OPTIONS ................................................................................................................... 66
8.2 AUDIO LINK SELECTION ............................................................................................................... 67
8.2.1 LaunchApplication (AppID, ProfileID) ................................................................................... 67
8.2.2 TerminateApplication (AppID, ProfileID) ............................................................................... 69
8.2.3 GetApplicationStatus (AppID, ProfileID) ................................................................................ 70
9 DEVICE ATTESTATION ................................................................................................................. 71
9.1 DEVICE ATTESTATION LAUNCH .................................................................................................... 71
9.2 DEVICE ATTESTATION TERMINATION ........................................................................................... 72
9.3 DEVICE ATTESTATION PROTOCOL ................................................................................................ 72
10 REFERENCES .................................................................................................................................... 78
APPENDIX A – EVENT MAPPING ......................................................................................................... 80
KNOB SHIFT AND ROTATE EVENTS ............................................................................................................ 80
KEY EVENT MAPPING ................................................................................................................................ 81
APPENDIX B – APPLICATION CONTEXT INFORMATION ............................................................ 85
TRUST LEVEL ............................................................................................................................................. 85
APPLICATION CATEGORIES ........................................................................................................................ 85
CONTENT CATEGORIES .............................................................................................................................. 86
CONTENT RULES ........................................................................................................................................ 86


-5-

LIST OF FIGURES

Figure 1: Terminal Mode Concept ............................................................................................................ 11
Figure 2: Terminal Mode Networking and Transport Stack................................................................. 15
Figure 3: Session Layer............................................................................................................................... 16
Figure 4: Terminal Mode VNC Setup ...................................................................................................... 19
Figure 5: VNC Connection Setup ............................................................................................................. 20
Figure 6: VNC Handshaking Phase ......................................................................................................... 21
Figure 7: Initialization Phase ..................................................................................................................... 22
Figure 8: Framebuffer Update Phase ....................................................................................................... 23
Figure 9: Server and Client Display Configuration ............................................................................... 28
Figure 10: VNC Server Options for non-scalable Clients ...................................................................... 30
Figure 11: Server and Client Event Configuration ................................................................................. 31
Figure 12: Example Event Mapping Message Flow ............................................................................... 34
Figure 13: Key Event Listing Messages ................................................................................................... 36
Figure 14: Key Event Listing – Incremental Updates ............................................................................ 37
Figure 15: Key Event Listing – Non-Incremental Updates ................................................................... 38
Figure 16: Example Virtual Keyboard Trigger Message Flow ............................................................. 39
Figure 17: Example Device Status Message Flow .................................................................................. 41
Figure 18: Example Content Attestation Message Flow ........................................................................ 44
Figure 19: Example Framebuffer Blocking Notification Message Flow .............................................. 47
Figure 20: Context Information (Example) .............................................................................................. 49
Figure 21: Audio Setup .............................................................................................................................. 51
Figure 22 Sequence for RTP connection: RTP Server by TM Server .................................................... 55
Figure 23: Bluetooth / Terminal Mode IOP States – Terminal Mode Server Perspective ................ 57
Figure 24: Bluetooth / Terminal Mode IOP States – Terminal Mode Client Perspective ................. 59
Figure 25: General UPnP Device Architecture........................................................................................ 61
Figure 26: 2-Box Pull Model ...................................................................................................................... 61
Figure 27: 2-Box Push Model .................................................................................................................... 62
Figure 28: 3-Box Model .............................................................................................................................. 62
Figure 29: Terminal Mode UPnP Service Architecture.......................................................................... 63
Figure 30: Terminal Mode Client Device Architecture .......................................................................... 64
Figure 30: Message Flow – Launch Audio Link ..................................................................................... 68
Figure 31: Message Flow – Terminate Audio Link ................................................................................ 69
Figure 32: Message Flow – Launch Device Attestation Protocol ......................................................... 71


-6-

Figure 33: Message Flow – Terminate Device Attestation Protocol .................................................... 72
Figure 34: Device Attestation certification infrastructure ..................................................................... 72
Figure 35: Device and software attestation protocol overview ............................................................ 73
Figure 36: Coordinate System for Knob Events ...................................................................................... 80


-7-

LIST OF TABLES

Table 1: Bandwidth Requirements vs. Display Update Rate ................................................................ 13
Table 2: Requirements for Handshaking Phase Messages .................................................................... 22
Table 3: Requirements for Initialization Phase Messages ..................................................................... 23
Table 4: Requirements for Framebuffer Update and Event Phase Messages ..................................... 24
Table 5: VNC Extension Type Message Structure .................................................................................. 26
Table 6: New Extension Types for Terminal Mode Messages .............................................................. 26
Table 7: Server Display Configuration Message..................................................................................... 29
Table 8: Client Display Configuration Message ..................................................................................... 29
Table 9: Server Event Configuration Message ........................................................................................ 32
Table 10: Client Event Configuration Message ....................................................................................... 33
Table 11: Event Mapping Message ........................................................................................................... 34
Table 12: Event Mapping Request Message ............................................................................................ 34
Table 13: Key Event Listing Message ....................................................................................................... 37
Table 14: Key Event Listing Request Message ........................................................................................ 38
Table 15: Virtual Keyboard Trigger Message ......................................................................................... 40
Table 16: Virtual Keyboard Trigger Request Message .......................................................................... 40
Table 17: Device Status Message .............................................................................................................. 42
Table 18: Device Status Request Message ............................................................................................... 42
Table 19: Terminal Mode Server Device Status Default Values ........................................................... 43
Table 20: Content Attestation Response Message .................................................................................. 45
Table 21: Content attestation signature content ..................................................................................... 45
Table 22: Content Attestation Request Message ..................................................................................... 46
Table 23: Framebuffer Blocking Notification Message .......................................................................... 47
Table 24: Blocked Rectangle from Framebuffer Update ........................................................................ 48
Table 25: Additional VNC Encodings ...................................................................................................... 49
Table 26: Context Information Pseudo Encoding ................................................................................... 50
Table 27: RTP Packet Header Definition ................................................................................................. 52
Table 28: RTP Payload Format – 16 Bit Mono ......................................................................................... 54
Table 29: RTP Payload Format – 16 Bit Stereo ........................................................................................ 54
Table 30: IOP Transition (from Terminal Mode Server perspective)................................................... 58
Table 31: IOP Transition (from Terminal Mode Client perspective) ................................................... 60
Table 32: UPnP Negotiation for Audio Selection ................................................................................... 67
Table 33: Device Attestation – attestationRequest elements ................................................................. 74


-8-

Table 34: Device Attestation – Component List ..................................................................................... 74
Table 35: Device Attestation – attestationResponse Elements .............................................................. 75
Table 36: Device Attestation – Possible Attestation Result Values ...................................................... 76
Table 37: Knob Shift and Rotate Configuration Settings ....................................................................... 80
Table 38: Key Event Mapping ................................................................................................................... 84
Table 39: Trust Level .................................................................................................................................. 85
Table 40: Application Categories .............................................................................................................. 86
Table 41: Content Categories..................................................................................................................... 86
Table 42: Content Rules to tackle Driver Distraction ............................................................................. 87


-9-

TERMS AND ABBREVIATIONS

A2DP Bluetooth Advanced Audio Distribution Profile
ARP Address Resolution Protocol
BT Bluetooth
CDC Communications Device Class; specified from USB Device Working Group
CE Consumer Electronics; CE devices are referred to as mobile devices within this
specification
DHCP Dynamic Host Configuration Protocol
ECM Ethernet Control Model; part of the CDC device class
HFP Bluetooth Hands-free Profile
HSP Bluetooth Handset Profile
HMI Human Machine Interface"
HU Head-unit (this term is used interchangeably with the Terminal Mode client)
HS Head-set
IP Internet Protocol
NCM Network Control Model; part of the CDC device class
RFB Remote Framebuffer
RTP Real-time Transport Protocol
TCP Transmission Control Protocol
TM Terminal Mode
UDP User Datagram Protocol
UI User Interface
UPnP Universal Plug and Play
USB Universal Serial Bus
VNC Virtual Network Computing


- 10 -

1 ABOUT

This document specifies an interface for enabling remote user interaction of a mobile device via
another device. This specification is written having a car head-unit to interact with the mobile de-
vice in mind, but it will similarly apply for other devices, which do provide a colored display,
audio input/output and user input mechanisms.
This document is aimed at people going to design and develop compliant solutions. The docu-
ment will provide all necessary interface functionality and requirements to implement a fully
compliant device, on both the mobile device and the head-unit side.
The specification lists a series of requirements, either explicitly or within the text, which are man-
datory elements for a compliant solutions. Recommendations are given, to ensure optimal usage
and to provide suitable performance. All recommendations are optional.
The document will focus on the interface functionality, its parameters and protocols only. It does
not provide any guidelines for implementing the protocol. If there is a reference towards an im-
plementation, this is of informative nature only.


- 11 -

2 INTRODUCTION TO TERMINAL MODE

The Terminal Mode provides a concept for integrating the mobile device (hereinafter referred to
as the “Terminal Mode Server”) and the vehicle head-unit (hereinafter referred to as the
“Terminal Mode Client”). In a Terminal Mode context, the control and interaction of
applications and services running on the mobile device will be replicated into the car
environment. Diverting display and audio output to the car head-unit come together with
receiving key and voice control input from it are the main interaction streams, as shown in the
following figure.

Applications User Speaker
Content Display
& Services Input & Micro

Display Control
Consumer
Automotive
Electronics
Head Unit
Device Audio / Voice

Internet

Figure 1: Terminal Mode Concept

The result is a concept somewhere between running the applications natively in the mobile phone
or in the car unit. From the user experience point of view it can offer "the best of the both worlds"
where the large variety of mobile phone applications is complemented and enhanced by the car
system providing convenient and safe means for using (i.e. controlling) these applications.
It is easier to add new consumer electronic functionalities into the vehicle environment via a
mobile device than integrating them into the car infotainment system. In any case, the usage of
those applications will become more convenient if the same device with the same content stored
in it can be used in all the different environments from home to car, and providing Internet
connectivity at the same time. On the other hand, the large displays of the car units can enhance
the user experience from what the mobile device can offer by itself.
In addition the mobile device typically provides the latest technologies, from radio connectivity,
to multimedia codecs. At the same time, the openness of the platforms, allows delivery of new
applications and services at any time.
There are no standard methods currently defined for Terminal Mode connectivity. However,
when creating the required solutions, technologies provided by existing open, non-proprietary
standards - like USB, TCP/IP, VNC, UPnP etc. - should be used as the basis. The needed
additional elements should then be developed and agreed in cooperation between the related
industry sectors.
The car systems comprise of several different methods for user interaction, like individual keys,
rotating knobs, touch screen and even voice-activated control. For proper interoperability, the
control method towards the mobile device should be the same regardless of the actual input
mechanism on the car side. Furthermore, to ensure that Terminal Mode does provide


- 12 -

interoperability independent of any application, even legacy ones, it hooks into low-level
abstraction.


- 13 -

3 CONNECTIVITY PROTOCOL STACK

The connectivity between the terminal mode server and client is the basis to provide interopera-
bility between both. The Connectivity stack is specified in the following, starting from the low
layer and going up the protocol stack.
It is not the objective of this specification to provide a detailed overview of the different protocols.
Instead this document should highlight the components and parameter required to ensure proper
connectivity. The connectivity solution is built purely on existing wireless and wired standards.
Therefore detailed information is available in the respective documents.

3.1 Physical & Link Layer
In principle this specification does not intend to limit the use of any wireless and wired technolo-
gy. Nevertheless the connectivity solution must provide reasonable high bandwidth. Minimum
bandwidth on link layer cannot be given, as the user experience depends on the networking &
transport layer performance, as well as on the parameter of the display (resolution and color for-
mat).
The following table gives some indication of the required bandwidth on the display level, i.e. on
top of any transport mechanism. These values assume non-incremental, uncompressed updates.
Full Display Example: QVGA Example: QHD Example: WVGA Example: WVGA
Update / s 320 x 240 x 4 640 x 360 x 4 800 x 480 x 2 800 x 480 x 4
2 614 000 Byte/s 1 843 200 Byte/s 1 536 000 Byte/s 3 072 000 Byte/s
5 1 536 000 Byte/s 4 608 000 Byte/s 3 840 000 Byte/s 7 680 000 Byte/s

Table 1: Bandwidth Requirements vs. Display Update Rate

Wired technologies do have advantages with regard to achievable bandwidth and security over
wireless technologies. In addition wired USB nowadays provides charging capabilities and is in-
deed the preferred charging interface in the mobile industry.

3.1.1 Universal Serial Bus (USB)
USB provides a high-bandwidth connection while allowing charging of the mobile device at the
same time.
Requirement: The USB host must at least support USB 2.0 high-speed.
To simplify the user intervention on the terminal mode server, it should set the right USB profile
automatically1, once connected to the terminal mode client. To facilitate the automatic personality
selection, the USB host should send a specific identification message to the device, prior configur-
ing the device, according the following format.
bmRequestType = 0x40
bRequest = 0xF0

1 A USB personality may include multiple USB device classes, which can be then used from the

USB host simultaneously.


- 14 -

wValue[1] = Terminal Mode major version (e.g. 1)
wValue[2] = Terminal Mode minor version (e.g. 0)
wIndex = USB Host vendorID
wLength = 0
Data = None
The message should be sent before set configuration, since the phone may have wrong personali-
ty loaded before that2. It must also be understood, that as a result of this, if wrong personality is
loaded, the phone will disconnect and reconnect again with a new personality loaded. This will
restart the enumeration3.
Requirement: The USB device must recognize Terminal Mode request message and must
select the respective USB personality.
Requirement: If the Terminal Mode client does not send the described identification mes-
sage, the mobile device must present the user with a list of available USB per-
sonalities.
The Terminal Mode specification does not attempt to specify, which other USB profiles should be
available under the Terminal Mode personality, and which USB personalities are available and
supported from the device. But to support multiple USB profiles under one personality, USB Host
needs to support composite device including IAD (Interface Association Descriptor). IAD is re-
quired to support USB function which requires multiple interfaces, and without IAD, it is not
possible to associate multiple interfaces with single USB functionality.
Requirement: USB host (TM client) must support composite device including IAD.
Recommendation: It is recommended for USB device (TM server) to provide MTP and ACM as
part of the Terminal Mode personality.

3.1.2 Wireless Local Area Networks (WLAN)
Support for Wireless LAN is optional.

3.2 Networking and Transport Layer
Networking mechanisms are used to abstract the different physical transport mechanisms. The
Internet Protocol is a well-established and known networking solution. IP protocol packets are
encapsulated into Ethernet packages.
Requirement: The networking layer must support IPv4. Support for IPv6 is optional.
This specification does anticipate only USB and WLAN networking. Other wired or wireless links
are supported optionally, as long as they allow carrying IP packets, as shown in Figure 2.

2 According to USB 2.0 specification, a USB device, which does not support a message, must re-

turn a STALL PID (Request Error). As the endpoint is control endpoint, there is no further action
required. USB host can consider device returning STALL as non-terminal mode device and can
proceed with non-terminal mode behavior.
3 A user will be able to manually switch between USB device personalities at any time.


- 15 -

DHCP

UDP TCP

IPv4

WLAN USB 2.0 <Other Link Layer>

Figure 2: Terminal Mode Networking and Transport Stack

3.2.1 USB Networking
Support for USB Networking is mandatory. Three competing Communication Device Class sub-
class drivers are available. In all cases, the USB connection basically looks like an Ethernet 802.3
networking card.
• RNDIS: Remote Network Device Interface Specification is a Microsoft proprietary specifi-
cation. RNDIS is partly Operating System dependent.
• CDC/ECM [3]: Communications Device Class/Ethernet Networking Control Model al-
lows one Ethernet package inside a single USB transfer. ECM has been developed for USB
full-speed.
• CDC/NCM [4]: Communications Device Class/Network Control Model allows multiple
Ethernet packages inside a single USB transfer. NCM can therefore offer a much higher
performance. NCM has been particular developed for high-bandwidth.
Requirement: The USB networking must follow CDC/NCM device class, revision 1.0 speci-
fication.4
Recommendation: The host and client should support a Maximum Transmission Unit (MTU)
size bigger than 1,500 Bytes. It is recommended to support MTU sizes up to
9000 Byte.
Requirement: The USB host must follow the maximum Ethernet frame size supported from
the USB device as discovered from the value wMaxSegmentSize in the Ether-
net Networking Functional Descriptor (For details refer to [3]) and supported
from the host (Least common denominator).
The Dynamic Host Configuration Protocol (DHCP) is used by the terminal mode client (DHCP
client) to obtain configuration information for operation in an IP network from the mobile device
(DHCP server). This protocol reduces system administration workload, allowing devices to be
added to the network with little or no manual intervention. DHCP is using UDP for the negotia-
tion.
• Packets sent from the client have source port 68 and destination port 67.
• Packets sent from the server have source port 67 and destination port 68.
Requirement: A DHCP Server must be available on the mobile device, connected to the
USB interface. The DCHP client must use the standard DHCP port.

4 According to USB CDC/NCM specification, the device and host MUST support 16-bit NTB

structures (NTB-16) and MAY also support 32-bit NTB structures (NTB-32).


- 16 -

Requirement: To minimize IP address conflicts on the terminal mode client, the DHCP
Server must provide an IP address within the 192.168.x.y with x in the range
of 2 to 127 and y in the range of 0 to 254. The netmask must be 255.255.255.0.
Requirement: The DHCP Server may provide a default gateway address for the DHCP
client. Provisioning of the default gateway address must not be interpreted as
if the Terminal Mode server provides Internet connectivity. The Terminal
Mode specification does not intend to specify the setup of IP routing functio-
nality on the Terminal Mode server.
Recommendation: Use ARP to resolve potential IP conflicts on client side.

3.2.2 WLAN Networking
Support for Wireless Local Area Network (WLAN) is optional. IP packets are carried over WLAN
connections, using the Ethernet LLC/SNAP framing. On other network types than Ethernet, LLC
and SNAP headers are required to multiplex different protocols on the link layer.
Requirement: In case WLAN is supported, the mobile device must support ad-hoc and in-
frastructure networks. In latter case, the access point must be implemented in
the terminal mode client.

3.2.3 Transport Layer
The IP protocol enables two transport mechanisms,
• User Datagram Protocol (UDP) to provide connectionless communication, used for ser-
vice advertising, multi-casting, and most real-time streaming protocols
• Transmission Control Protocol (TCP) to provide connection-oriented communication
Requirement: The transport layer must support UDP and TCP transport protocols on top of
IP.

3.3 Session & Application Layer
The Terminal Mode application layer consists of three basic session layer components using ei-
ther UDP or TCP sockets to interact as shown in the figure below.
• Audio, responsible for providing and exchanging audio content, using UDP sockets.
• VNC, responsible for exchanging display and control information, using TCP sockets.
• UPnP, responsible for service negotiation and remote application control, using UDP
broadcasting and TCP sockets.

Audio UPnP VNC

SOAP

RTCP RTP SSDP HTTP 1.1 VNC

UDP TCP

Figure 3: Session Layer

The application layer is specified in the following chapters.


- 17 -

4 AUTHENTICATION AND SECURITY

Giving the terminal mode client control over the server requires addressing security and authen-
tication concerns. Potential threats in the Terminal Mode setup include
1. Attacker can read and/or modify communication between terminal mode client and
server.
2. Terminal mode server (e.g. a mobile device) does not connect to the intended terminal
mode client (e.g. a vehicle head-unit) or vice versa.
3. Terminal mode client connects to a non-compliant server.
Threats 1 is addressed via confidentiality and integrity mechanisms, where as threats 2 is ad-
dressed via host authentication. Threat 3 is addressed with device attestation. The term “security
mechanisms” is used to refer to confidentiality, integrity and authentication mechanisms. Those
mechanisms are described in the following.

4.1 Host Authentication Mechanisms
Host authentication in this context refers to the terminal mode client authenticating itself towards
the terminal mode server, to mitigate the threat that the server connects to unintended client (or
vice versa).

Additional authentication and integrity mechanisms in wired USB networking are not required.

Authentication and integrity mechanisms in WLAN networks are available on Link Layer.
Requirement: Link-Layer authentication mechanisms, like WPA, must be used, if mandated
from the terminal mode server or from the terminal mode client.

4.2 Confidentiality and Integrity Mechanisms
Confidentiality and integrity mechanisms mitigate the threat that an external attacker can read,
change or inject any data.

Additional confidentiality and integrity mechanisms in wired USB networking are not required.

Confidentiality and integrity mechanisms in WLAN networks are available on Link Layer.
Requirement: Link-Layer confidentiality mechanisms, like WPA, must be used if mandated
from the terminal mode server or from the terminal mode client.

4.3 Device Identification Mechanism
Device identification in this context refers to the mobile device identifying itself to the terminal
mode client.


- 18 -

Device identification (proving the identity of the device and or the device manufacturer are avail-
able from different sources during the connection setup.
1. USB standard device descriptor entries idVendor and idProduct
2. UPnP XML device description: <manufacturer> and <modelName>
Requirement: The Device must set the USB vendor and product ID as well as UPnP device
manufacturer and model name accordingly.
Requirement: The device must prevent 3rd party software to change USB vendor ID and
UPnP device manufacturer according to the state-of-the-art of the particular
device platform.


- 19 -

5 DISPLAY OUTPUT AND CONTROL INPUT

The contents of the Terminal Mode server device’s screen are transferred to the Terminal Mode
client device. The control inputs are transferred from the Terminal Mode client to the Terminal
Mode server. Screen copy methods can be used to copy the content of the Terminal Mode server's
display (frame buffer copy) to the Terminal Mode client’s display. The copy operation may in-
clude rotation or color conversion. The frame buffer is used as an abstraction layer, which avoid
any changes to the applications and services running on the mobile device can be avoided. For
this purpose the Virtual Networking Computing (VNC) protocol is used.
The Virtual Networking Computing (VNC) uses the Remote Framebuffer Protocol (RFB) as a
simple protocol for remote access to any sort of framebuffer based user interfaces. The remote
endpoint is called the VNC Client, whereas the endpoint driving the framebuffer is called the
VNC Server. In the Terminal Mode context, the VNC Client resides in the car head-unit (Terminal
Mode client) and the VNC Server is in the mobile device (Terminal Mode server). The VNC client
will show the remote display either on the entire local display or on a subset of it, as shown in
Figure 4.

HU CE User VNC
VNC CE
Display Display Input Client
Server Display

RFB Protocol
Display Control
Consumer
Automotive
Electronics
Head Unit
Device

Figure 4: Terminal Mode VNC Setup

The command and control input is handled as part of the VNC protocol by key and pointer
events. A key or pointer event on terminal mode client will be signaled to the terminal mode
server via a specific key symbol value, which uniquely identifies the event. The mobile device
and/or its application will not necessarily support all possible keys defined. Some applications
may even have a dynamic behavior on the selection of key inputs they expect.
The RFB protocol is originating from the desktop computing world and has been designed as a
thin client protocol, i.e. it assumes a client with only a few requirements, and a server having
access to more processing capabilities. The protocol allows the client to be as simple as possible.
In the Terminal Mode context this assumption needs to be reconsidered, as mobile devices are
experiencing performance limitations as well.
Requirement: The terminal mode client must implement the VNC client functionality.
Requirement: The terminal mode server must implement the VNC server functionality


- 20 -

5.1 Connection Setup
The connection setup is facilitated via UPnP. Once the VNC server is up and running, its service
is announced using UPnP protocol mechanisms. The VNC Server must listen for the VNC client
to connect. Connection is done via establishment of a TCP/IP socket. The Server IP address and
VNC port number are provided as part of the UPnP protocol (see respective chapter).
The established connection is facilitating the execution of the VNC protocol.
Once the TCP/IP connection is disconnected, the VNC server will wait for the VNC client to re-
connect. The VNC protocol does not provide specific messaging to shut down the connection. Be-
fore reconnection, the VNC client has to verify, whether the VNC server is still available (using
UPnP mechanisms).
The connection setup is high-lighted in the following picture. The red dotted lines indicate trigger
points between the Server and Client operation.

Start VNC Client Start VNC Server

Wait for VNC Server to VNC Server available
become available

Connect
Wait for VNC Client to
Connect to VNC Server
(Re-)Connect

VNC
VNC Operation VNC Operation
Protocol

No Still Yes
Close Connection connected
?
Disconnect

Figure 5: VNC Connection Setup

The Server IP address and VNC port number are provided as part of the UPnP protocol (see re-
spective chapter).
Requirement: The VNC Server must listen for the VNC client to connect. If the VNC client
disconnects, the VNC Server must listen for the client to reconnect.
Requirement: The VNC Server must listen on a TCP/IP socket.


- 21 -

5.2 Traditional VNC Protocol Phases
After the connection between the VNC server and client has been established, the VNC protocol
processing will start according to the VNC specification. The VNC protocol basically consists of
three main steps
(1) Exchange of handshaking messages. In this step, the VNC connection between both end-
points is set up. After the handshaking phase, the VNC connection parameters are nego-
tiated and the connection is established.
(2) Exchange of initialization messages. After this phase, both ends have agreed on all
needed parameter for the following operational phase.
(3) Client to server and server to client messages are used to reflect changes of the framebuf-
fer content on the local endpoint and user interaction on the remote endpoint.
These three VNC protocol phases are specified in more detail in the following.

5.2.1 Handshaking Phase
The handshaking phase defines a couple of messages, which are being exchanges between the
VNC Client and the VNC Server, as shown in the following figure. In general the VNC Server
presents its capabilities and the VNC Client selects the best option with regard to its own capabili-
ties.

VNC Server VNC Client

Server Protocol
Version
Client Protocol
Version
Security Type Support
Security Type Security_
Selection type = 0

Security Failure
Reason
Security Result

Security Failure
Reason (3.8 only)

Figure 6: VNC Handshaking Phase

The following parameters have to be supported from the VNC Client and the Server.

Message Origin Parameter Mandatory Values
Protocol Version Server Max. protocol version At least 3.7
Protocol Version Client Max. protocol version At least 3.7
# of security types (as specified in RFB)
Security Type Support Server
Security types 1 (None)
Security Type Selection Client Security type (as specified in RFB)
Reason length
Security Failure Reason Client (as specified in RFB)
Reason string


- 22 -

Security Result Server Security status (as specified in RFB)
Security Failure Reason Reason length
Server (as specified in RFB)
(RFB version 3.8 only) Reason string

Table 2: Requirements for Handshaking Phase Messages

Authentication and security is handled outside the VNC protocol on link-layer and transport-
layer. The VNC Client cannot expect the VNC Server to offer additional security or authentication
features.

5.2.2 Initialization Phase
The initialization phase defines a couple of messages, which are being exchanged between the
VNC Client and the VNC Server, as shown in the following figure. In general the VNC Server
presents its capabilities and the VNC Client selects the best option with regard to its own capabili-
ties.


Client Init

Server Init

Set Encodings

Set Pixel Format

Figure 7: Initialization Phase


Client Init Client Shared-flag (as specified in RFB)
Framebuffer-width
Framebuffer-height
Name-length
Name-string
Server Init
Bits-per-pixel
Server (as specified in RFB)
(using native framebuf- Depth
fer configuration)
Big-endian-flag
True-colour-flag
Red-/Green-/Blue max
Red-/Green-/Blue shift
Number of encodings (as specified in RFB)
Set Encodings Client -223 (Desktop Size Pseudo
Encoding-type Encoding – optional for
client)


- 23 -

-523 (Terminal Mode Pseu-
do Encoding)
Bits-per-pixel 32, 16
Depth 24, 16
Big-endian-flag (as specified in RFB)
Set Pixel Format Client
True-colour-flag 1 (true color)
Red-/Green-/Blue max
RGB888, RGB565
Red-/Green-/Blue shift

Table 3: Requirements for Initialization Phase Messages

The Terminal Mode limits the RFB protocol, as shown in the above table with regard to supported
color formats, to allow for efficient implementations. Some more specific recommendations and
requirements are given below.
Requirement: The VNC Client must not select any other pixel format, unless the server has
indicated support, using the ServerDisplayConfiguration VNC extension
message.
Requirement: The VNC Client must send a Set Pixel Format message, prior to any Frame-
buffer Update Request message.
Recommendation: It is recommended that the VNC Client selects the native color format of the
VNC server. On device color conversion will lead to a significant reduction of
achievable frame rate.

5.2.3 Framebuffer Update and Event Phase
The update and event phase defines a couple of messages, which are being exchanged between
the VNC Client and the VNC Server. The VNC Server only responds to framebuffer update re-
quests, as shown in Figure 8. No response message is sent to any of the other messages.


Framebuffer Update
Request
Framebuffer Update

Figure 8: Framebuffer Update Phase


Incremental
x-position
Framebuffer Update
Client y-position (as specified in RFB)
Request
Width
Height
Number-of-rectangles
Framebuffer Update Server (as specified in RFB)
x-position


- 24 -

y-position
Width
Height
encoding-type 0 (Raw)
first color
number-of-colours
(Message not implemented
Set Colour Map Entries Server Red
at Server)
Green
Blue
Down-flag
Key Event Client (as specified in RFB)
Key
Button-mask
Pointer Event Client x-position (as specified in RFB)
y-position
Length
Client Cut Text Client (as specified in RFB)
Text
Bell Server No parameter (as specified in RFB)
Length
Server Cut Text Server (as specified in RFB)
Text

Table 4: Requirements for Framebuffer Update and Event Phase Messages

The VNC Client can request two types of framebuffer updates, using the incremental flag in the
FramebufferUpdateRequest message.
• A ‘0’ indicates the VNC server to provide a non-incremental update, i.e. the VNC server
must provide the requested area or a superset of it, independent of whether it has
changed or not.
• A ‘1’ indicates the VNC server to provide an incremental update, i.e. the VNC server
must provide only the area(s) containing all framebuffer changes.
Requirement: The VNC Client must support Zero framebuffer update messages, i.e. Fra-
mebuffer Update messages where the width and height equals zero.5
Requirement: The VNC Client must support framebuffer update messages of a bigger fra-
mebuffer area, than the original requested one.6
Requirement: The VNC Client must support that the VNC Server may ignore framebuffer
update request messages, if multiple are sent at a time. Multiple non-
incremental framebuffer update request messages may be combined into one

5 Zero Framebuffer updates are not forbidden from the VNC specification specifically. Though

some existing VNC clients, display warnings.
6 This occurs, if the VNC Client requests a non-incremental framebuffer update for a specific area

and other areas have changed in the mean time as well.


- 25 -

framebuffer update response. It is recommended that the VNC Client sends
only one Framebuffer Update Request message at a time.
Requirement: The VNC Client must support that the VNC Server will not respond imme-
diately to an incremental framebuffer update request. The Server may wait
for the next response until the framebuffer has changed on the Server side.
Requirement: The VNC Server must respond immediately to a non-incremental framebuf-
fer update request.
Recommendation: It is recommended that the VNC Client has a copy of the server side frame-
buffer locally available. Therefore it is recommended that the VNC Client re-
quests incremental framebuffer updates.


- 26 -

5.3 VNC Terminal Mode Extension Messages
The existing RFB protocol specification is not sufficient to cover the mobile device – remote car
display configuration space. Therefore extensions to the current protocol are specified in this
chapter. Extensions are made in compliance with the RFB protocol, introducing a new Terminal
Mode message type (128).
Under the Terminal Mode message type, a couple of additional messages are introduced to the
VNC protocol. These can be distinguished via unique extension-types. All extension messages
must use the Terminal Mode message type and must follow the following fundamental design
principle.

# bytes Type Value Description
1 U8 128 Terminal Mode Message-type
1 U8 Extension-type
2 U16 N Payload length
N U8 array Message specific payload data

Table 5: VNC Extension Type Message Structure

Requirement: The VNC Server and Client must handle Terminal Mode extension messages
with unknown extension types, by reading the whole message (body and
payload) and ignoring it.
The Terminal Mode Message type defines the following set of new VNC messages given in Table
6. All extension messages are mandatory server-side, but the execution of some messages can be
disabled within the Server or Client Event Configuration message.
Exten- Disable
sion- Message Message Name Origin Support Execu-
Type tion
1 Display ServerDisplayConfiguration Server Mandatory No
Configura-
2 tion ClientDisplayConfiguration Client Mandatory No
3 Event Con- ServerEventConfiguration Server Mandatory No
4 figuration ClientEventConfiguration Client Mandatory No
5 Event Map- EventMapping Server Mandatory No
6 ping EventMappingRequest Client Optional No
7 Key Event KeyEventListing Server Mandatory Yes
8 Listing KeyEventListingRequest Client Optional Yes
9 Virtual Key- VirtualKeyboardTrigger Server Mandatory Yes
board Trig-
10 ger VirtualKeyboardTriggerRequest Client Optional Yes
11 Device Sta- DeviceStatus Server Mandatory No
12 tus DeviceStatusRequest Client Optional No
13 Content At- AttestationResponse Server Mandatory No
14 testation AttestationRequest Client Optional No
Framebuffer
16 FramebufferBlockingNotification Client Optional No
Blocking
Table 6: New Extension Types for Terminal Mode Messages


- 27 -

Requirement: The Client must disable the key event listing and the virtual keyboard trigger
support in the Client Event Configuration message, if it has not implemented
the respective request messages.


- 28 -

5.3.1 Display Configuration Messages
The Server and Client Configuration message pair exchanges additional display information be-
tween the client and the server. Based on the received information the client may change the pixel
format, sending a Set Pixel Format message. The server will use this information to optionally
provide higher resolution virtual framebuffer copies. The message flow is shown in Figure 9.


Set Encodings
-523 (Terminal Mode)

Server Display Configuration

Client Display Configuration

Set Pixel Format

Figure 9: Server and Client Display Configuration

Requirement: The Server Display Configuration Message must be sent immediately in re-
sponse to the Set Encoding message, indicating Terminal Mode (-523) sup-
port, prior any other message.
Requirement: The Client Display Configuration Message must be sent immediately in re-
sponse to the Server Display Configuration Message, prior any other mes-
sage.
The Server Display Configuration message is given in Table 7. It will be responded from the
Client with a Client Display Configuration message.

1 U8 128 Message-type
1 U8 1 Extension-type
2 U16 12 Payload length
1 U8 1 Terminal Mode Server Major Version
1 U8 0 Terminal Mode Server Minor Version
Bit Framebuffer configuration (1 = yes, 0 = no)
Server-side framebuffer orientation switch available
1. The VNC server must start in default orientation, which
0
is given in the Server Init message (values width and
height).
2 U16 Server-side framebuffer rotation available
1
• The VNC server must start with no rotation.
Server-side framebuffer scaling available
• The VNC server must be able to scale the framebuffer to
2
the client resolution (as provided from the VNC client in
the Client Display Configuration message)
2 U16 Relative pixel width (set to zero, if relative width not known)
2 U16 Relative pixel height (set to zero, if relative height not known)
Bit RGB Color conversion support (1 = yes, 0 = no)
4 U32
0 32 bit ARGB 888 (mandatory support for VNC server)


- 29 -

7 All other 32 bit formats
8 16 bit RGB 565 (mandatory support for VNC server)
9 16 bit RGB 555
15 All other 16 bit formats
16 bit single color (grayscale)
25 • Client must use red_shift and red_mask to set gray
range
8 bit single color (grayscale)
26 • Client must use red_shift and red_mask to set gray
range
Table 7: Server Display Configuration Message

Recommendation: The relative pixel width and height should be used to compensate for differ-
ent pixel aspect rotation on client and server side.
Requirement: The Server Display Configuration message must be sent only once.
The Client Display Configuration message is shown in the following table.

1 U8 1 Terminal Mode Client Major Version
1 U8 0 Terminal Mode Client Minor Version
Bit Framebuffer Configuration (1 = yes, 0 = no)
Server-side framebuffer orientation switch used
0 • If enabled, the VNC client must use the Device Status
Request message (section 5.3.6)
2 U16 Server-side framebuffer rotation used
1 • If enabled, the VNC client must use the Device Status
Request message (section 5.3.6)
Client-side framebuffer scaling available
• If enabled, the VNC client must be able to scale the
2
server framebuffer (as provided in the Server Display
7
Configuration message) to the client resolution
2 U16 Client display width [pixel] (unknown value must be 0)
2 U16 Client display height [pixel] (unknown value must be 0)
2 U16 Client display width [mm] (unknown value must be 0)
2 U16 Client display height [mm] (unknown value must be 0)
Distance driver – client display [mm] (unknown value must be
2 U16
0)

Table 8: Client Display Configuration Message

7 According to the RFB specification, the client must support any server framebuffer resolution. A

client must not expect the server to support scaling.


- 30 -

Requirement: The Client Display Configuration message must be sent only once.
Requirement: The VNC client must support Desktop Size Pseudo Encoding, if it enables bit
0 or 1 in the framebuffer configuration entry.
Requirement: If the VNC client does not support scaling (disabled bit 2 in the framebuffer
configuration entry), the VNC server must provide the framebuffer content
in the requested client display resolution in one of the following options
shown in Figure 10.

Scaling

Framing

Clipping

Figure 10: VNC Server Options for non-scalable Clients

Scaling, if the VNC server supports scaling (maintaining the server as-
pect ratio – with (0, 0) offset; other client area remains unspecified), or
Framing, if the VNC server does not support scaling and the server reso-
lution is smaller than the client one (with (0, 0) offset; other client area
remains unspecified), or
Clipping, if the VNC server does not support scaling and the server reso-
lution is bigger than the client one (framebuffer content aligned to the
upper left corner).
Requirement: No pixel data must be transmitted for unspecified client area (shown in red
in Figure 10)
Requirement: The VNC client must not provide a Terminal Mode minor and major version,
which is higher than the VNC server supported version.
Requirement: VNC client and server must be backward compatible with regard to different
Terminal Mode versions.


- 31 -

5.3.2 Event Configuration Messages
The Server and Client Event Configuration message pair provides additional information about
the event handling, i.e. which key and pointer events are natively supported on the server and
client. This information helps the server to map specific incoming client events to server events
and helps the client to directly use specific server events. The message flow is shown in Figure 11.


Server Event
configuration
Client Event
configuration

Figure 11: Server and Client Event Configuration

Requirement: The Server Event Configuration Message must be sent immediately in re-
sponse to the Set Encoding message, indicating Terminal Mode (-523) sup-
port. The message may be delayed only until reception of the Client Display
Configuration message.
Requirement: The Client Event Configuration Message must be sent immediately in re-
sponse to the Server Event Configuration Message, prior any other message.
The Server Event Configuration message is given Table 9.

2 U16 Keyboard layout – Language code (according ISO 639-1)
Keyboard layout – Country code (according ISO 3166-1 al-
2 U16
pha-2)
2 U16 UI Language – Language code (according ISO 639-1)
UI Language – Country code (according ISO 3166-1 alpha-
2 U16
2)
Knob shift & rotate configuration (Bitmask according Table
37)
4 U32 Bit l
• ‘1’: Server supports knob key events8
• ‘0’: Server does not support knob key events
Device keys (Bitmask according Table 38)
• Bitmask defined in Table 38
4 U32 Bit m
• ‘1’: Server supports device key events
• ‘0’: Server does not support the key events
Multimedia keys (Bitmask according Table 38)
4 U32 Bit n • Bitmask defined in Table 38
• ‘1’: Server supports multimedia key events

8 The Server can claim e.g. the device cursor keys plus a Device_Ok key to represent a simple

knob, i.e. supporting shift up, down, left right and push knob events.


- 32 -

• ‘0’: Server does not support the multimedia key events
Bit Key related (1 = support, 0 = no support)
0 ITU keypad (T9) events (‘0’, … ,’9’, ‘#’, ‘*”)
1 Virtual keyboard trigger support
4 U32
2 Key event listing support
# additional soft and hard keys, the server requires
8 – 15 • Key events defined in Table 38
• Key events start with TM_Key 0, no subsequent gaps
Bit Pointer related (1 = support, 0 = no support)
0 Single-touch events
4 U32
1 Multi-touch events
8 – 15 Pointer event button mask (according RFB spec)

Table 9: Server Event Configuration Message

Requirement: Server event configuration must be sent only once.
The payload structure of the Client Event Configuration message is fully symmetrical with the
Server Event Configuration message, as shown in Table 10.

2 U16 Keyboard layout – Language code (according ISO 639-1)
Keyboard layout – Country code (according ISO 3166-1 al-
2 U16
pha-2)
2 U16 UI Language – Language code (according ISO 639-1)
UI Language – Country code (according ISO 3166-1 alpha-
2 U16
2)
Knob shift & rotate configuration (Bitmask according Table
37)
4 U32 Bit l
• ‘1’: Client supports knob key events
• ‘0’: Client does not support knob key events
Device keys (Bitmask according Table 38)
4 U32 Bit m • ‘1’: Client supports device key events
• ‘0’: Client does not support device key events
Multimedia keys (Bitmask according Table 38)
4 U32 Bit n • ‘1’: Client supports multimedia key events
• ‘0’: Client does not support multimedia key events
Bit Key related (1 = support, 0 = no support)
0 ITU keypad (T9) events (‘0’, … ,’9’, ‘#’, ‘*”)
1 Virtual keyboard trigger support (ignored at server)
4 U32
2 Key event listing support (ignored at server)
# additional soft and hard keys, the client supports
8 – 15 • Key events defined in Table 38
• Key events start with TM_Key 0, no subsequent gaps
4 U32 Bit Pointer related (1 = support, 0 = no support)


- 33 -

0 Single-touch events
1 Multi-touch events
8 – 15 Pointer event button mask (according RFB spec)

Table 10: Client Event Configuration Message

Requirement: Client Event Configuration must be sent only once.


- 34 -

5.3.3 Event Mapping Messages
The Event Mapping and Event Mapping Request message pair provides the client with informa-
tion about the server mapping of high key symbol values. The Client can send an Event Mapping
Request message at any time, either requesting or setting a specific mapping within the server.
The server must always send an Event Mapping message in response of an Event Mapping Re-
quest message. If the server changes any event mapping locally, the server must inform the client
via an Event Mapping message, if the client has requested the mapping at least once.
The message flow follows the following steps, as shown in Figure 12.

Event Mapping Request

Event Mapping

Server changes
Event Mapping

Event Mapping

Figure 12: Example Event Mapping Message Flow

Requirement: The Server must respond to any Event Mapping Request message imme-
diately.
The Event Mapping message is given in Table 11.

4 U32 Client Key Symbol Value
Server Key Symbol Value
4 U32
(0 = client key value not support from server)
Table 11: Event Mapping Message

The Default Mapping Request message is given in Table 12.

4 U32 Client Key Symbol Value
Server Key Symbol Value
4 U32
(0 = request value from Server)
Table 12: Event Mapping Request Message

Requirement: If the server locally changes any event mapping, it must send an Event Map-
ping message with appropriate Client and Server Key Symbol Values. The


- 35 -

server must send the Event Mapping message only, if the Client has re-
quested the Client Key Symbol Value previously using an Event Mapping
Request message.
Requirement: If the server does not support a new mapping request according to the Event
Mapping Request message, it must send an Event Mapping message, con-
taining the existing mapping. The server key symbol value is set to zero if the
server does not allow any mapping for the client symbol value.


- 36 -

5.3.4 Key Event Listing Message
The Key Event Listing and Key Event Listing Request message pair provides the client with in-
formation about the next valid characters, while entering text information. The VNC Server sup-
port is announced in the Server Event Configuration message. The Client may start the key event
listing at any time. In case a text entry is required, the server will provide the initial list of valid
keys, which is getting updated after each key event, either as incremental or full update. An ex-
ample message flow is shown in Figure 13.


Key Event Listing Request
Start Information Flow

Key Event Listing
Initial List – Counter n

Key Event

Key Event Listing
List Update – Counter n+1

Key Event Listing Request
Stop Information Flow

Figure 13: Key Event Listing Messages

Requirement: The VNC Server must send Key Event Listing messages only, if the VNC
Client has enabled or requested them.
Requirement: The VNC Client must not assume, that the VNC server will send Key Event
Listing messages even if it has indicated support (the current application may
not be able to support this feature).
The Key Event Listing message is given in Table 13.

2 U16 4 + 4*n Payload length
Bit Configuration
0 Update flag (0 = non-incremental, 1 = incremental)
1 Listing flag (0 = black list, 1 = white list)
Default event list flag (0 = normal list, 1 = default list )
• To reference to the default list, set this flag and set #
1 U8
2 key events in list to zero.
• To set the default event list, set this flag and add key
events to the KeySymValue list
Other key event listing follows (0 = no, 1 = yes)
3 • Next key event listing must follow immediately
• Black lists and white lists can be mixed
1 U8 n # key events in list
2 U16 Key event counter


- 37 -

4*n U32 array KeySymValue list used to define the next valid character

Table 13: Key Event Listing Message

The flow using incremental updates is shown in Figure 14. Here, a default layout must be defined
once per VNC session. This list can be used as a reference point prior the incremental update
process. During the incremental update, the white list contains all key symbols to be added to the
key event list, where as black lists contains all key symbols to be removed from the key event list.
Default layout = 1
# key events in list != 0

Wait for Text Event

Default layout = 1
# key events in list = 0

Wait for Key Event

Last Yes
Char?

No

Update flag = 1 Yes No Update flag = 1
White
Listing flag = 1 Listing flag = 0
listing?
Default layout flag = 0 Default layout flag = 0

No Other Yes Yes Other No
List? List?

Figure 14: Key Event Listing – Incremental Updates

In case of non-incremental (i.e. full) updates, the key event listing looks like in Figure 15. Incre-
mental and non-incremental updates can be mixed at any time.


- 38 -

Wait for Text Event

Default layout = 0
Update flag = 0
# key events in list != 0

Other Yes
List?

No

Wait for Key Event

No Last Yes
Char?

Figure 15: Key Event Listing – Non-Incremental Updates

Requirement: The valid key symbol list must not differentiate between upper and lower
case characters
The Key Event Listing Request message is shown in Table 14.

Bit Configuration (0 = Disable, 1 = Enable)
0 Server key event listing
4 U32
1 Incremental updates
2 Reset key event counter

Table 14: Key Event Listing Request Message

The key event counter can be used to synchronize the server key event listing message to key
events. The counter value must represent all received key events (key press events only). The
counter must be set to zero on the reception of the Client Event Configuration message and if the
specific bit is set in the Key Event Listing Request message. The counter must roll over to zero,
once the maximum number is reached.
The VNC Client must not assume that the VNC Server will send a key event list update before the
client sends the next key press event. This specifically can happen if the key events are entered
faster than the Server can provide key event list updates. In such case, the client should use the
default key event list instead.


Terminal Mode Technical Architecture Release Candidate v0.9

Terminal Mode Technical Architecture Release Candidate v0.9

Recomendados

Recomendados

Más contenido relacionado

Similar a Terminal Mode Technical Architecture Release Candidate v0.9

Similar a Terminal Mode Technical Architecture Release Candidate v0.9 (20)

Más de Dev Khare

Más de Dev Khare (20)

Último

Último (20)

Terminal Mode Technical Architecture Release Candidate v0.9