Knowledge-Driven Agile Sensor-Mission Assignment

ACITA 2009

Knowledge-Driven
Agile Sensor-Mission Assignment
Alun Preece, Diego Pizzocaro , Konrad Borowiecki ,

Geeth de Mel , Wamberto Vasconcelos,

Amotz Bar-Noy , Matthew P. Johnson ,

Tom La Porta , Hosam Rowaihy

http://users.cs.cf.ac.uk/D.Pizzocaro D.Pizzocaro@cs.cf.ac.uk

Outline
Sensor-Mission Assignment problem
Previous work

Ontology-based matching

Extensions

1. Task Representation 2. Asset Allocation

Implementation

Conclusion & Future


Main problem

Sensor-Mission
Assignment
is the problem of assigning sensing assets to
missions to cover the information needs (ISR)
of individual tasks in each mission.


Sensors
(or Sensing assets)

Simple sensors

Platforms


Sensors
(or Sensing assets) Missions

Simple sensors composed by different TASKS:

e.g. Peace Support mission

TASK 3
Platforms TASK 4
Area
Surveillance
Area TASK 1
Surveillance
Detect
vehicle TASK 2

Identify
people


Scenario


Scenario

• A network of heterogeneous sensing assets:


Scenario
TASK 3
TASK 7
Localize Detect
Jeep People
TASK 4

Detect
TASK 6 Aircraft
Identify
Tank
TASK 1
TASK 2
TASK 5
Detect Identify
Detect
Ground people
Vehicle
Helicopter


- Support multiple tasks competing for bundles of sensors


Scenario
TASK 3
TASK 7
Localize Detect
Jeep People
TASK 4

Detect
TASK 6 Aircraft
Identify
Tank
TASK 1
TASK 2
TASK 5
Detect Identify
Detect
Ground people
Vehicle
Helicopter



- Sensors are scarce and in high demand.


Scenario
TASK 3
TASK 7
Localize Detect
Jeep People
TASK 4

Detect
TASK 6 Aircraft
Identify
Tank
TASK 1
TASK 2
TASK 5
Detect Identify
Detect
Ground people
Vehicle
Helicopter




- Highly dynamic (sensor failures, change of plan)

Scenario

Where to send that particular bundle?

TASK 1
TASK 2

Detect Identify
Ground people
Vehicle




- Highly dynamic (sensor failures, change of plan)

Problem formulation

• We need schemes to assign bundles of assets to
demonstrates how the core knowledge-based approach can be BT1
the task they best serve. A1
used to drive asset allocation, using the CASS combinatorial B1
auction algorithm as an illustration. In Section VI we review A2
the status of our illustration-of-concept application, SAM T1
•
(Sensor Assignment entities competing for assets.
Tasks: the to Missions), and discuss a variety of
roles this kind of tool can play in supporting the process of
B2
A3

Each task is associated with Information Requirements.
sensor-mission assignment. BT2
While other papers have presented earlier or incomplete B3 A4
• Bundles: collections approach
parts of our knowledge-driven of assets. to sensor-mission
assignment (notably has a[8], [9]) Bundle Type (BT).
Each bundle [6], unique and resource allocation T2 B4 A5
(notably [10], [7]) this is the ﬁrst paper to show how these
• Assets: individual sensors and platforms.
elements can provide an integrated solution.

II. S ENSOR -M ISSION A SSIGNMENT P ROBLEM B5 A6
F ORMULATION
Tasks Bundles Assets
We formulate the sensor-mission assignment problem as a
graph; an example is shown in Figure 1. We distinguish three Fig. 1. Example sensor-mission assignment problem as a grap
kinds of node:
• Tasks denote the entities that are competing for available
assets.1 In our approach, the only important feature of
(MSR) which must be surveilled and protected. Surve
a task is its information requirements — these are what
the border will likely involve, among other things, det
drive the asset matching and allocation processes — so
of suspicious vehicle activity near it: vehicle detectio
the task nodes represent information requirements.
be formalized as an information requirement task T1
• Bundles are collections of individual assets (sensors and
may be accomplished by a variety of means, dependi
http://users.cs.cf.ac.uk/D.Pizzocaroarc between a bundle and a task indicates
platforms). An D.Pizzocaro@cs.cf.ac.uk

Problem formulation

• We need schemes to assign bundles of assets to
demonstrates how the core knowledge-based approach can be BT1
the task they best serve. A1
used to drive asset allocation, using the CASS combinatorial B1
auction algorithm as an illustration. In Section VI we review A2
the status of our illustration-of-concept application, SAM T1
•
(Sensor Assignment entities competing for assets.
Tasks: the to Missions), and discuss a variety of
roles this kind of tool can play in supporting the process of
B2
A3

Each task is associated with Information Requirements.
sensor-mission assignment. BT2
While other papers have presented earlier or incomplete B3 A4
• Bundles: collections approach
parts of our knowledge-driven of assets. to sensor-mission
assignment (notably has a[8], [9]) Bundle Type (BT).
Each bundle [6], unique and resource allocation T2 B4 A5
(notably [10], [7]) this is the ﬁrst paper to show how these
• Assets: individual sensors and platforms.
elements can provide an integrated solution.

II. S ENSOR -M ISSION A SSIGNMENT P ROBLEM B5 A6
F ORMULATION

•
Tasks Bundles Assets
We formulate the sensor-mission assignment problem as a
GOAL: an assignment of bundles to tasks, such that
graph; an example is shown in Figure 1. We distinguish three Fig. 1. Example sensor-mission assignment problem as a grap
1. One-to-One matching between tasks-bundles
kinds of node:
• Tasks denote the entities that are competing for available
2. assets.1 In our approach, the onlybetween bundles-assets
One-to-Many matching important feature of
(MSR) which must be surveilled and protected. Surve
a task is its information requirements — these are what
the border will likely involve, among other things, det
drive the asset matching and allocation processes — so
of suspicious vehicle activity near it: vehicle detectio
the task nodes represent information requirements.
be formalized as an information requirement task T1
• Bundles are collections of individual assets (sensors and
may be accomplished by a variety of means, dependi
http://users.cs.cf.ac.uk/D.Pizzocaroarc between a bundle and a task indicates
platforms). An D.Pizzocaro@cs.cf.ac.uk

Knowledge representation and reasoning techniques can support sensor-mis
specification of information requirements, to the allocation of assets such as senso
Example
how assets can be matched to mission tasks by formalising the military missions a
using this ontology to drive a matchmaking procdure. The work reported here exten
by providing a richer and more realistic way for a user to specify their information
• Mission process ITA scenario: Peace support operation
semantic matchmakingbased on an to define the search space for efficient asset allocat
! UK and US bases established to surveil the border
he Task-Bundle-Asset modelrely on a Main Supply Route (MSR)
defines the
! Bases
earch space relating what bundles of assets
re required for different types of task.
TASK BUNDLE SENSOR
Higher-level task representations{(UAV, DaylightTV)}
BT1 = allow Predator (UAV)
nowledge base to offer widest range of
STAR solution types."
Reaper (UAV)
Task: detect vehicles
DaylightTV

Bundle 1: UAV
BT2 = {(AcousticArray), (AcousticArray)}
+IMINT Acoustic array 1

Acoustic array 2

http://users.cs.cf.ac.uk/D.Pizzocaro Bundle 2: acoustic D.Pizzocaro@cs.cf.ac.uk

Example
defines the
! Bases
TASK BUNDLE SENSOR
Reaper (UAV)
TASK 1

Detect
DaylightTV
vehicle

Bundle 1: UAV

Acoustic array 2


Example
defines the
! Bases
TASK BUNDLE SENSOR
B1 Reaper (UAV)
TASK 1

Detect B2 DaylightTV
vehicle

Bundle 1: UAV

B3 Acoustic array 2


Previous Work

Ontology-based
matching
matches types of assets to types of tasks,
based on sensing capabilities provided and required.

SENSING ASSET TYPES

TASK

Aerial
Imagery
Intelligence


Methodology

• Semantic matchmaking to evaluate the ﬁtness-for-purpose of collections of asset
types to the information requirements of a task.



Methodology


• We use ontologies for:



Methodology



Specify the information
requirements of a task.
ISR ontologies: tasks, sensors,
etc

Specify the sensing capabilities
provided by assets.


Methodology



Compare the two of them.

MMF ontology
requirements of a task.
etc

provided by assets.


Methodology



Compare the two of them.

MMF ontology requirements of a task.
etc

provided by assets.


MMF ontology

• Based on the pre-existent Missions and Means Framework (MMF)

! MMF is an informal framework to help human planners determine the
capabilities required to accomplish a military mission. (developed by US ARL)

! In the military domain very precise deﬁnitions of capabilities required:
! ISR requirements

Intelligence

Surveillance Reconnaisance


MMF ontology (contd)

• Main concepts and relations in MMF ontology (implemented in OWL DL):

toPerform
entails
Task requires Capability

allocatedTo provides

comprises toAccomplish
Asset

Operation
is-a is-a

comprises toAccomplish Platform mounts System
attachedTo
is-a
Mission
interferesWith
Sensor



• Given a task T, with a set of ISR requirements: RT = {R1, R2, ...}

• Product of matching is a (BT) Bundle Type:
bundle of assets that can satisfy the requirements of a task.





• Steps to build a BT:

1. generate a (PC) Platform Conﬁguration = (P , S)
where P is a single platform type ,
S = {S1, ... , Sm} is a set of sensor types mountable on P (at the same time).







2. generate a (PK) Package Conﬁguration = { PC1 , PC2 , ... }
The aggregate capabilities of PK have to semantically match RT







2. generate a (PK) Package Conﬁguration = { PC1 , PC2 , ... }
The aggregate capabilities of PK have to semantically match RT

3. Create BT by adding cardinality constraints to each PC in the PK:
e.g. : “2 UAVs with a DaylightTV each” ! BT1 = { (UAV, DaylightTV), (UAV, DaylightTV)}


Limitation

• This approach is conceptually simple, extensible and well founded (MMF)

• Limitation: and w
requi
Task requirements have to be speciﬁed at a low level
to dri
! Over-reliant on the ontology of INT types over-
deter
of AC
E.g. RT = { ACINT, IMINT } — th
highe
trivially determines the need next
for acoustic and imagery sensing.

Ou
tasks

• Need for higher-level representation of ISR tasks. •

Fig. 3. Sample concept taxonomies relevant to the ISR domain: ISR tasks
and INT types


Extension 1

Task Representation
specify only WHAT is the information requirement,
and avoid saying HOW it should be obtained.

T = {ACINT, IMINT} T = {“Detect Vechicle”}


NIIRS

• Approach based on National Imagery Interpretability Rating Scale (NIIRS)

! determines the kinds of data that are interpretable
to answer an information requirement: T= {“Detect Vehicles”}
E.g. Visible, Radar, etc.

! deﬁnes ratings on a ten point scale (0–9) Visible NIIRS 4


NIIRS

• Approach based on National Imagery Interpretability Rating Scale (NIIRS)

! determines the kinds of data that are interpretable
to answer an information requirement: T= {“Detect Vehicles”}
E.g. Visible, Radar, etc.

! deﬁnes ratings on a ten point scale (0–9) Visible NIIRS 4

• Platform Conﬁgurations can be rated with NIIRS values

“UAV with DaylightTV-camera” Visible NIIRS 4

• Therefore “UAV with DaylightTVcamera” matches “Detect Vehicles”

! NIIRS can support the task-asset matching


Hybrid reasoning
• To integrate NIIRS in sensor-task matching:

! We formalized it as a set of rules which allow for:

Task speciﬁcation
Detect/Identify/Distinguish <set of detectables>

We adopt an HYBRID REASONING approach: ontology & rule-based reasoning

! Reasoning steps:

1) Specify high-level task:

2) NIIRS rule-based system infers basic capabilities:

3) Ontology-based matchmaking returns the bundle types:


as our approach makes clear, they ca
level capabilities, from which lower-l

Hybrid reasoning inferred. However, we prefer to locate
our ontology as specialisations of the
the Capability class to avoid confusion
current implementation, we omit T as
are required to happen in the same mi



Task speciﬁcation

!0#$20&+$)*+3(4'1$*5$6('(3'0"7(1 !3

Fig. 4. A portion of our ontology of “detectabl

! Reasoning steps: Full details of the representation of t
rules for performing reasoning with i
various “detectable” types of entity are
1) Specify high-level task: drawn from the entities appearing in
the NIIRS documentation. A portion of
2) NIIRS rule-based system infers basic capabilities: in Figure 4. Using this ontology, we
NIIRS interpretation tasks as a set of c
form: N C, DS, F S, C, N T, N R , wh
3) Ontology-based matchmaking returns the bundle types: • N C is a NIIRS “capability” as
distinguish-between, identify);
• DS is a set of detectables, as abo
• F S is a set of more speciﬁc fea
entities (for example, the roads o
the runways of an airport, or the p
http://users.cs.cf.ac.uk/D.Pizzocaro a port)D.Pizzocaro@cs.cf.ac.uk
— these features are also
of “detectables”;





Task speciﬁcation

!0#$20&+$)*+3(4'1$*5$6('(3'0"7(1 !3


1) Specify high-level task: T = {“Detect Vehicles”} drawn from the entities appearing in
in Figure 4. Using this ontology, we
• N C is a NIIRS “capability” as





Task speciﬁcation

!0#$20&+$)*+3(4'1$*5$6('(3'0"7(1 !3


2) NIIRS rule-based system infers basic capabilities: RT = {ACINT-level0, IMINT-level4} we
in Figure 4. Using this ontology,
• N C is a NIIRS “capability” as





Task speciﬁcation

!0#$20&+$)*+3(4'1$*5$6('(3'0"7(1 !3


2) NIIRS rule-based system infers basic capabilities: RT = {ACINT-level0, IMINT-level4} we
in Figure 4. Using this ontology,
3) Ontology-based matchmaking returns the bundle types: • N C is a NIIRS “capability” as
BT1 = {UAV, DaylightTV} BT2 = {AcousticArray, AcousticArray} • DS is a set of detectables, as abo

The Task-Bundle-Asset model defines the
search space relating what bundles of assets
Advantages & Limitations
are required for different types of task.

Higher-level task representations allow
• knowledge base to offer widest range of
Higher-level task representations allow a wider range of assets to satisfy a task.
ISTAR solution types."
TASK 1 “IMINT” Bundle
Detect
vehicle

Bundle 1: UAV
+IMINT “ACINT” Bundle

• Limitation: NIIRS scale is restricted to imagery intelligence (IMINT).

! How did we include acoustic intelligence?acoustic
Bundle 2:
• There is no reason why NIIRS cannot be extended to other types
motes

• We introduced a number of rules for ACINT based on literature
• Still the need for NIIRS to be “ofﬁcially” extended!
Hybrid ontology+rules-based reasoning:
Rule-based representation of (extended)
NIIRS scale allows tasks of form:

Extension 2

Asset Allocation
assign bundles instances to competing tasks,
based on recommended bundle types.

X
X X
Task 4 Task 2

X
Task 1

X
Task 3


Allocation model

• A set of tasks competing for Sensors
(sensing assets)
the exclusive usage of sensing assets.
Tasks Bundles
! For each task:
e11, c11
S1

d = utility demand (p1, d1, w1) T1 B1

w = budget e1
2,
c1
S2

p = priority 2

S3

• A set of sensors to be grouped into (p2, d2, w2) T2 B2

bundles based on bundle types. S4

! For each bundle-task pair:
e = joint utility to a task
c = joint cost to a task

• Goal: an allocation of sensor bundles that maximizes the total proﬁt.


Allocation model

• A set of tasks competing for Sensors
(sensing assets)
the exclusive usage of sensing assets.
Tasks Bundles
! For each task:
e11, c11
S1

d = utility demand (p1, d1, w1) T1 B1

w = budget e1
2,
c1
S2

p = priority 2

S3

• A set of sensors to be grouped into (p2, d2, w2) T2 B2

bundles based on bundle types. S4

! For each bundle-task pair:
e = joint utility to a task
c = joint cost to a task

• Goal: an allocation of sensor bundles that maximizes the total proﬁt.
• The proﬁt is a function of the utility cumulated by the task.


Combinatorial auction

• This can be seen as a combinatorial auction
! bidders: tasks
! items: sensors
! tasks bids for bundles of sensors

• Many algorithms have been proposed:
we use CASS (Combinatorial Auction Structured Search)


Combinatorial auction

• This can be seen as a combinatorial auction
! bidders: tasks
! items: sensors
! tasks bids for bundles of sensors

• Many algorithms have been proposed:
we use CASS (Combinatorial Auction Structured Search)

• Bundle generation issue:
! In the worst case we might have 2N bundles (with N = number of sensors)

! The REASONING step improves the bundle generation:

Generate only bundles Reduce the number of Prune the set of bids
matching bundle types generated bundles placed by tasks


Implementation
Two prototypes of Top-to-bottom applications
to solve the Sensor-Mission assignment.


user to “backtr
Section IV. A screenshot of the new user interface is shown in asset instances o
Figure 5. As before, a user logs-in as a member of a coalition catalogues, obtain
and then and g
SAM - base station
(in our example, we have a US/UK coalition). The user is the package con way. For examp
able to select one or more areas-of-interest on a map (left additional cardinthe vehicle dete
obtained by a U
panel) and, for each, to select multiple ISR tasks (top-right Section III. Bec
• •
panel) using an area-of-interest and selecting tasks using the SAM reason, they cou
Level of use: our NIIRS-based representation. (As we noted in demonstration, t
Fig. 5. Setting
Features:
application IV our approach is currently simpliﬁed to assume that the user, to for e
Section High-level task representation from, mak
data
! Base station "
all tasks are required at the same time; this could easily be is shown in Figu While the cur
! Operational analyst
extended.) " Subscribe bundles manuallyInknowledge-d
of doing this,
these incorporate
yet resources,
shows UK/US w
an inventory ow
and whether the
combinatorial a
assets (shown b
separately. Integ
While simple, th
ber of choices t
point Users need
• to allow th
policies inthat is
way futur
asset has been se
done using
to available data
it ought to
Fabricformalism.
[27]. Add
user to “backtrac
in some ca
and then obtain
(for examp
way. For exampl
normally ta
the vehicle detec
• Further res
obtained by a UA
most appro
reason, they coul
Fig. 5. Setting an area-of-interest and selecting tasks using the SAM
Fig. 6. Selecting an asset bundle manually using the SAM application
It is clear
data from, for ex
http://users.cs.cf.ac.uk/D.Pizzocaro
application D.Pizzocaro@cs.cf.ac.uk

SAM - mobile
• Level of use:

! Mobile user (on the ﬁeld)
! Not an expert
! Time constraint


SAM - mobile
• Level of use:

! Mobile user (on the ﬁeld) • Features:

! Not an expert " High-level task representation

! Time constraint


SAM - mobile
• Level of use:

! Mobile user (on the ﬁeld) • Features:

! Not an expert " High-level task representation

! Time constraint " Automatic asset allocation

Satisfied Unsatisfied


Conclusion & Future

• We extended our original ontology-based approach to provide:

! a richer and more realistic speciﬁcation of task requirements (using NIIRS)

! automatic asset allocation (using combinatorial auctions)

• We also showed these new features in a centralized and a mobile application.

• Future:
- Assets or bundle of assets might be shared between tasks
- Most appropriate “choice points” for user intervention (human-in-the-loop)
- Information delivery to a mobile user (bandwidth constraints)


Thanks for listening!

Questions?


Knowledge-Driven Agile Sensor-Mission Assignment

Recomendados

Recomendados

Más contenido relacionado

Último

Último (20)

Destacado

Destacado (20)

Knowledge-Driven Agile Sensor-Mission Assignment