深掘りARKit

1. 深掘りARKit with Unity ⾕⼝直嗣

3. では早速起動してみましょう！

4. ゲーム以外も

5. Unity ARKit Plugin

6. ざっくりアプリ構成イメージ

7. UnityARSessionNativeInterface C# on Mono iOS Native

8. UnityARSessionNativeInterface.cs でNativeの関数をラップ

9. namespace UnityEngine.XR.iOS {  ………………………   [DllImport("__Internal")]  private static extern void session_SetPlaneAnchorCallbacks(IntPtr nativeSession, internal_ARAnchorAdded anchorAddedCallback,   internal_ARAnchorUpdated anchorUpdatedCallback,   internal_ARAnchorRemoved anchorRemovedCallback);    Unityのエンジンの中に統合されているっぽいこんな感じでNativeのARKitを呼び出す

10. ざっくり動作イメージ

11. UnityARSessionInterface 3D描画空間の把握

12. Camera関連 Unityのカメラの位置、画⾓、クリッピングをiPhoneのカメラと合わせる

13. ARKit Unity AR空間のカメラの位置の推定 AR空間の平⾯の位置の推定 iPhoneのカメラに合わせた画⾓ AR空間の平⾯の HitTest AR空間の明るさ UnityARSessionInterface Unityのカメラに位置をセット影 Occlusion AR⽤背景明るさに応じたライトヒットテスト

14. UnityARSessionNativeInterface.cs UnityARCameraManager.cs UnityARCameraNearFar.cs

15. void Update () {    if (m_camera != null)  {  // JUST WORKS!  Matrix4x4 matrix = m_session.GetCameraPose(); m_camera.transform.localPosition = UnityARMatrixOps.GetPosition(matrix); m_camera.transform.localRotation = UnityARMatrixOps.GetRotation (matrix);    m_camera.projectionMatrix = m_session.GetCameraProjection ();  }    }    ARKitからカメラの位置情報を取得してUnityのカメラにセット UnityARCameraManager.cs ARKitからカメラのProjectionMatrixを取得してUnityのカメラにセット

16. void UpdateCameraClipPlanes()  {  currentNearZ = attachedCamera.nearClipPlane;  currentFarZ = attachedCamera.farClipPlane;  UnityARSessionNativeInterface.GetARSessionNativeInterface ().SetCameraClipPlanes (currentNearZ, currentFarZ);  }    // Update is called once per frame  void Update () {  if (currentNearZ != attachedCamera.nearClipPlane || currentFarZ != attachedCamera.farClipPlane) {  UpdateCameraClipPlanes ();  }  }    UnityのカメラのNear, Farを取得してARKitのカメラにセット UnityARCameraNearFar.cs

17. これな！

18. 背景 iPhoneのカメラのビデオストリームを背景に

20. ビデオストリームでカラーバッファをクリアそれから3D描画

21. UnityARSessionNativeInterface.cs UnityARVideo.cs

22. void InitializeCommandBuffer()  {  m_VideoCommandBuffer = new CommandBuffer();   m_VideoCommandBuffer.Blit(null, BuiltinRenderTextureType.CurrentActive, m_ClearMaterial);  GetComponent<Camera>().AddCommandBuffer(CameraEvent.BeforeForwardOpaque, m_VideoCommandBuffer);  bCommandBufferInitialized = true;    }  UnityのCommandBuﬀerの機能を使って描画コマンドを挿⼊ UnityARVideo.cs

23. CameraEvent.BeforeForwardOpaque

24. public void OnPreRender()  {  ………  // Texture Y  _videoTextureY = Texture2D.CreateExternalTexture(currentResolution.width, currentResolution.height,  TextureFormat.R8, false, false, (System.IntPtr)handles.textureY);  _videoTextureY.filterMode = FilterMode.Bilinear;  _videoTextureY.wrapMode = TextureWrapMode.Repeat;  _videoTextureY.UpdateExternalTexture(handles.textureY);    // Texture CbCr  _videoTextureCbCr = Texture2D.CreateExternalTexture(currentResolution.width, currentResolution.height,  TextureFormat.RG16, false, false, (System.IntPtr)handles.textureCbCr);  _videoTextureCbCr.filterMode = FilterMode.Bilinear;  _videoTextureCbCr.wrapMode = TextureWrapMode.Repeat;  _videoTextureCbCr.UpdateExternalTexture(handles.textureCbCr);    m_ClearMaterial.SetTexture("_textureY", _videoTextureY);  m_ClearMaterial.SetTexture("_textureCbCr", _videoTextureCbCr);  ……  ビデオストリームをテクスチャーとしてセット UnityARVideo.cs

25. Anchor ARKitで認識した平⾯

27. UnityARSessionNativeInterface.cs UnityARGeneratePlane.cs UnityARAnchorManager.cs

28. namespace UnityEngine.XR.iOS  {  public class UnityARGeneratePlane : MonoBehaviour  {  public GameObject planePrefab;  private UnityARAnchorManager unityARAnchorManager;    // Use this for initialization  void Start () {  unityARAnchorManager = new UnityARAnchorManager();  UnityARUtility.InitializePlanePrefab (planePrefab);  }    void OnDestroy()  {  unityARAnchorManager.Destroy ();  }    void OnGUI()  {  List<ARPlaneAnchorGameObject> arpags = unityARAnchorManager.GetCurrentPlaneAnchors ();  if (arpags.Count >= 1) {  //ARPlaneAnchor ap = arpags [0].planeAnchor;  //GUI.Box (new Rect (100, 100, 800, 60), string.Format ("Center: x:{0}, y:{1}, z: {2}", ap.center.x, ap.center.y, ap.center.z));  //GUI.Box(new Rect(100, 200, 800, 60), string.Format ("Extent: x:{0}, y:{1}, z: {2}", ap.extent.x, ap.extent.y, ap.extent.z));  }  UnityARAnchorManagerを作る UnityARGeneratePlane.cs 位置検出したら返ってくる

29. Hit Test 画⾯タップした位置の平⾯の座標をゲット

30. namespace UnityEngine.XR.iOS  {  public class UnityARHitTestExample : MonoBehaviour  {  public Transform m_HitTransform;    bool HitTestWithResultType (ARPoint point, ARHitTestResultType resultTypes)  {  List<ARHitTestResult> hitResults = UnityARSessionNativeInterface.GetARSessionNativeInterface ().HitTest (point, resultTypes);  if (hitResults.Count > 0) {  foreach (var hitResult in hitResults) {  Debug.Log ("Got hit!");  m_HitTransform.position = UnityARMatrixOps.GetPosition (hitResult.worldTransform);  m_HitTransform.rotation = UnityARMatrixOps.GetRotation (hitResult.worldTransform);  Debug.Log (string.Format ("x:{ 0:0.######} y:{ 1:0.######} z:{ 2:0.######}", m_HitTransform.position.x, m_HitTransform.position.y, m_HitTransform.position.z));  return true;  }  }  return false;  }  iPhoneの画⾯上の点を指定してARKitのヒットテストを呼ぶ UnityARHitTestExample.cs Hit Testの結果をチェック

32. 影 AR空間で影を落とす

33. https://www.youtube.com/watch?v=v3Gym4mlcn0

35. MobileARShadow.shader

36. struct v2f  {  float4 pos : SV_POSITION;    LIGHTING_COORDS(0,1)  };      v2f vert(appdata_base v) {  v2f o;  o.pos = UnityObjectToClipPos (v.vertex);    // 5.) The TRANSFER_VERTEX_TO_FRAGMENT macro populates the chosen LIGHTING_COORDS in the v2f structure  // with appropriate values to sample from the shadow/lighting map  TRANSFER_VERTEX_TO_FRAGMENT(o);    return o;  }    fixed4 frag(v2f i) : COLOR {    // 6.) The LIGHT_ATTENUATION samples the shadowmap (using the coordinates calculated by TRANSFER_VERTEX_TO_FRAGMENT  // and stored in the structure defined by LIGHTING_COORDS), and returns the value as a float.  float attenuation = LIGHT_ATTENUATION(i);  return fixed4(1.0,1.0,1.0,1.0) * attenuation;  }  ShadowMapを取得 MobileARShadow.shader

37. Shadow Map

38. Occlusion 実物の後ろにあるものは描画しない

40. Occlusion 実物の後ろにあるものは描画しない

41. https://www.youtube.com/watch?v=xEmLGoVs7Fs

42. 机の上は描画されている机の下は描画されていない

43. MobileOcclusion.shader

44. ZWrite On  ZTest LEqual  ColorMask 0    CGPROGRAM  #pragma vertex vert  #pragma fragment frag    #include "UnityCG.cginc"    struct appdata  {  float4 vertex : POSITION;  } ;    struct v2f  {  float4 position : SV_POSITION;  } ;    v2f vert (appdata input)  {  v2f output;    output.position = UnityObjectToClipPos(input.vertex);  return output;  }    fixed4 frag (v2f input) : SV_Target  {  return fixed4(0.5, 0.3, 0.0, 1.0);  }  Zは描画 MobileOcclusion.shader Colorは更新しない

45. 現実世界の明るさを反映

46. https://www.youtube.com/watch?v=7Kk6iVr5ULo

48. UnityARAmbient.cs

49. public void Start()  {  l = GetComponent<Light>();  UnityARSessionNativeInterface.ARFrameUpdatedEvent += UpdateLightEstimation;  }    void UpdateLightEstimation(UnityARCamera camera)  {  // Convert ARKit intensity to Unity intensity  // ARKit ambient intensity ranges 0-2000  // Unity ambient intensity ranges 0-8 (for over-bright lights)  float newai = camera.lightEstimation.ambientIntensity;  l.intensity = newai / 1000.0f;    //Unity Light has functionality to filter the light color to correct temperature  //https://docs.unity3d.com/ScriptReference/Light-colorTemperature.html  l.colorTemperature = camera.lightEstimation.ambientColorTemperature;  }    イベントを登録 UnityARKitAmbient.cs ライトの⾊温度を変更

50. 垂直な⾯をUnity上で作って物理シミュレーションと Occlusion

51. https://www.youtube.com/watch?v=akCCwPeHF9k

52. GPSの情報と組み合わせ

53. https://www.youtube.com/watch?v=CGJHELNepAI

深掘りARKit

Estás leyendo una previsualización.

Eche un vistazo a continuación

深掘りARKit

Más Contenido Relacionado

Presentaciones para usted (14)

Similares a 深掘りARKit (20)

Más de Naoji Taniguchi (20)

Más reciente (20)

深掘りARKit

深掘りARKit

Estás leyendo una previsualización.

深掘りARKit

Recomendado

Más Contenido Relacionado

Presentaciones para usted (14)

Similares a 深掘りARKit (20)

Más de Naoji Taniguchi (20)

Más reciente (20)

深掘りARKit