Un workshop su cosa sono le spatial anchors e come utilizzarlo nelle applicazioni xamarin

1. Realtà aumentata ed
Azure, un binomio
imbattibile

2. Realtà aumentata ed Azure, un binomio imbattibile
Alessio Iafrate
Microsoft MVP, Freelance
developer
Founder DotNetAbruzzo

3. A
G
E
N
D
A
Why is mobile AR relevant?
• By 2020, 100 million consumers will
shop in AR online and in-store,
according to Gartner, Inc.
DatafromSeptember2018by
https://pages.arm.com/arvr-report.html

4. A
G
E
N
D
A
Why does the cloud play a role here?
• The cloud provides the secure
storage and scalable
processing for a digital copy of
the real world which can be
accessed by any user at any
time on any device for shared
AR content.

5. A
G
E
N
D
A
What is a Spatial Anchor?
x
y
z

6. A
G
E
N
D
A
What is a Spatial Anchor?
x
y
z

7. A
G
E
N
D
A
What is a Cloud Spatial Anchor?
x
y
z
x
y
z x
y
z

8. A
G
E
N
D
A
What is a Cloud Spatial Anchor?
x
y
z
x
y
z

9. A
G
E
N
D
A
Collaborative design reviews
Real-time IoT digital twin data on
actual equipment
Training guidance
Persistent multi-user virtual content

10. A
G
E
N
D
A
Wayfinding
• Empowering firstline
workers to navigate large
facilities
• Guiding the way to IoT
sensors or failing equipment
• Navigating warehouses

11. Consumer Scenarios

12. A
G
E
N
D
A
Wayfinding
• Guiding the way through a
store following a spatial
grocery list
• Empowering people who are
blind or low vision with
spatial sound anchors
• Navigating large areas like
theme parks, museums,
festivals

13. A
G
E
N
D
A

14. A
G
E
N
D
A
Persistent multi-user virtual content
• Urban murals
and digital art
• Architecture
visualization

15. A
G
E
N
D
A
Persistent multi-user virtual content
• Urban murals and digital art
• Architecture visualization
• Marketing and ads
• Points of interest
• Games

16. Azure Spatial Anchors

17. A
G
E
N
D
A
What is a Cloud Spatial Anchor?
x
y
z
x
y
z x
y
z

18. A
G
E
N
D
A

19. A
G
E
N
D
A

20. A
G
E
N
D
A
• 1) Collaboration • 2) Wayfinding • 3) UX Considerations

21. A
G
E
N
D
A
HoloLens iOS (ARKit) Android (ARCore)
Unity
C++/WinRT
Xamarin
Unity
Objective-C
Swift
Xamarin
Unity
Java
C++/NDK

22. A
G
E
N
D
A
Demo
http://bit.ly/2VlLcOA

23. A
G
E
N
D
A

24. A
G
E
N
D
A
Mobile
HoloLens
Client
Place notes in the real world
Persistent virtual content
Azure Spatial
Anchors
Azure
App Service
Azure
Cosmos DB
Sharing Service
iOS – Swift, ARKit, SceneKit
Android – Java, ARCore, SceneForm
SOLUTION ARCHITECTURE

25. A
G
E
N
D
A
API Key Concepts
• CloudSpatialAnchorSession
• Provides core services (Ex. Create, locate, update, delete CloudSpatialAnchor’s)
• Captures data about the environment
• CloudSpatialAnchor
• Links to the underlying AR platform Anchor (WorldAnchor on HoloLens, ARAnchor on iOS, Anchor on Android)
• Holds an ID, Expiration, and Properties (Dictionary<String, String>)
• CreateAnchorAsync(CloudSpatialAnchor anchor) { … }
• Save a CloudSpatialAnchor to Azure Spatial Anchors
• Returns the CloudSpatialAnchor with an ID assigned
• CreateWatcher(AnchorLocateCriteria criteria)
• Begins to watch for anchors that meet the specified criteria
• Returns located CloudSpatialAnchor’s through a delegate
• AnchorLocateCriteria
• Assign it an array of CloudSpatialAnchor IDs

26. A
G
E
N
D
A
Mobile
HoloLens
Client
Persistent virtual content
Azure Spatial
Anchors
Azure
App Service
Azure
Cosmos DB
Sharing Service
SOLUTION ARCHITECTURE

27. A
G
E
N
D
A
Mobile
HoloLens
Client
Persistent virtual content
Azure Spatial
Anchors
Sharing
Service
1) Initialize session
2) Create anchor
3) Locate anchors
1. Save anchor
2. Get anchor ID back
3. Save anchor ID in database
4. Get IDs from database
5. Locate anchors by ID
6. Get anchors back
SOLUTION ARCHITECTURE

28. A
G
E
N
D
A
protected void init() {
// Set up ARCore & SceneForm.
this.sceneView = ...
Scene scene = ...
...
// Initialize the CloudSpatialAnchorSession.
this.spatialAnchorsSession = new CloudSpatialAnchorSession();
this.spatialAnchorsSession.Configuration.AccountKey = “xxxxxxxxx”;
this.spatialAnchorsSession.Configuration.AccountId = “xxxxxxxxx”;
...
// Give frames to the CloudSpatialAnchorSession.
scene.Update += (_, args) =>
{
// Pass frames to Spatial Anchors for processing.
this.spatialAnchorsSession?.ProcessFrame(this.sceneView.ArFrame);
};
// Add a callback that tells us how much information about the environment we have.
this.spatialAnchorsSession.OnSessionUpdated += (_, sessionUpdateArgs) =>{
this.sessionUpdated = sessionUpdateArgs.Args.Status.RecommendedForCreateProgress;
}
// Register handleTap() as a callback to be invoked when the user taps to place a sphere.
this.arFragment.TapArPlane += ArFragment_TapArPlane;
...
}
Initialize session

29. A
G
E
N
D
A
Private async void ArFragment_TapArPlane (HitResult hitResult, Plane plane, MotionEvent motionEvent) {
// Create the ARCore Anchor.
Anchor localAnchor = hitResult.createAnchor();
// Render a white sphere at the localAnchor position.
...
// Set the previously created ARCore Anchor as the localAnchor of the CloudSpatialAnchor.
CloudSpatialAnchor cloudAnchor = new CloudSpatialAnchor();
cloudAnchor.LocalAnchor=localAnchor;
// Prompt the user to input a note, set it as a property on the CloudSpatialAnchor.
...
// Wait until we have enough data about the environment
while (this.sessionUpdated < 1) { ... wait ... }
// Save the CloudSpatialAnchor to Azure Spatial Anchors.
this.spatialAnchorsSession.CreateAnchorAsync(cloudAnchor)
.ContinueWith(async cloudAnchorTask =>{
// Save CloudSpatialAnchor ID to the Sharing Service.
PostAnchor(cloudAnchor.getIdentifier());
}
});
Azure Spatial
Anchors
Sharing
Service
Create anchor

30. A
G
E
N
D
A
public void LocateAnchors(String[] identifiers) {
AnchorLocateCriteria criteria = new AnchorLocateCriteria();
criteria.setIdentifiers(identifiers);
this.spatialAnchorsSession.createWatcher(criteria);
}
String[] identifiers = GET https://<foo>.azurewebsites.net/api/anchors
LocateAnchors(identifiers); Azure Spatial
Anchors
Sharing
Service
Locate anchor

31. A
G
E
N
D
A
private void initializeSession() {
// Initialize the CloudSpatialAnchorSession.
this.cloudSession = new CloudSpatialAnchorSession();
...
this.cloudAnchorManager.OnAnchorLocated += CloudAnchorManager_OnAnchorLocated;
}
private void CloudAnchorManager_OnAnchorLocated(object sender, AnchorLocatedEventArgs args)
{
if (args.Args.Status == LocateAnchorStatus.Located) {
// Get the ARCore Anchor from the CloudSpatialAnchor.
Anchor localAnchor = args.Args.Anchor.LocalAnchor;
// Render object at the ARCore Anchor location.
...
}
});
Azure Spatial
Anchors
Locate anchor

32. A
G
E
N
D
A
Azure Spatial Anchors | Wayfinding

33. A
G
E
N
D
A
Wayfinding
public void LocateAnchors(String[] identifiers) {
AnchorLocateCriteria criteria = new AnchorLocateCriteria();
criteria.setIdentifiers(identifiers);
cloudSession.createWatcher(criteria);
}
public void LocateNearbyAnchors(CloudSpatialAnchor anchor) {
NearAnchorCriteria nearAnchorCriteria = new NearAnchorCriteria();
nearAnchorCriteria.setSourceAnchor(anchor);
AnchorLocateCriteria criteria = new AnchorLocateCriteria();
criteria.setNearAnchor(nearAnchorCriteria);
cloudSession.createWatcher(criteria);
}

34. A
G
E
N
D
A
Azure Spatial Anchors | Wayfinding
DCA
B

35. Azure Spatial Anchors | UX Considerations

36. A
G
E
N
D
A

37. A
G
E
N
D
A
Placing anchors
Target interesting visual features
• Avoid blank surfaces or surfaces
without details.
Record from perspectives where
you want the next person to
discover from
• Think about those who will come
after you.

38. A
G
E
N
D
A
Finding anchors

39. A
G
E
N
D
A
References

40. Thank You!!!

41. Thanks to

42. Question Time