Slides of my presentation about Azure Video Indexer for Video and Audio analysis and metadata extraction at Programa Tu Pandemia 2.0 event organized by MujeresTIC Bolivia
4. Azure Cognitive Services
Percepción Comprensión
Computer Vision
Face API
Custom Vision
Text-to-Speech
Speech-to-Text
Speech Translator
Speaker Recognition
Language Understanding
Text Translator
Text Analytics
QnA Maker
Anomaly Detector
Content Moderator
Personalizer
microsoft.com/cognitive
Visión Voz Lenguaje Decisión
5. Azure Applied AI Services
• Ponga en marcha soluciones de IA en días,
no en meses.
• Genere valor tangible para su organización
rápidamente con servicios de IA para
procesos empresariales comunes.
• Los servicios de IA aplicados combinan
Azure Cognitive Services, IA específica de
tareas y lógica de negocios para ofrecerle
servicios clave de IA.
7. Imagina si pudieras…
• Crear automáticamente metadatos (texto hablado, texto escrito,
rostros, lugares, objetos, etc.) para cualquier archivo de video.
• Buscar clientes frustrados o contar el tiempo que pasan en espera en
un sistema de centro de llamadas.
• Cargar y editar videos en colaboración en tiempo real, encontrar
momentos clave y acelerar el tiempo para publicar contenido nuevo.
• Permitir que los clientes busquen contenido multimedia por
nombre, palabra clave, ubicación, objetos clave…
• Distribuir clips reproducibles de contenido de video en vivo interna o
externamente sin largos tiempos de descarga o codificación.
• Crear resúmenes automáticos o “reels” destacados de tu contenido
de video en función de la detección de escenas, personas específicas
y movimiento dentro del video.
• Traducir su contenido a otros idiomas para llegar a una audiencia
global y diversa.
8. Video AI es la clave para resolver estos desafíos
• Mejorar la visibilidad del contenido
• Aumentar el valor del contenido
• Personalice la experiencia de
visualización
• Descubra información de contenido
oculto
• Reducir el trabajo manual
• Aumentar los ingresos / Impulsar la
visualización
• publicidad dirigida
• Modelado predictivo y
recomendaciones
10. Azure Video Analyzer for Media
• Un servicio que extrae información valiosa.
• Utiliza modelos de machine learning que se pueden personalizar y entrenar
aún más.
• Los detalles de video incluyen identificación de rostros, reconocimiento de
texto, etiquetas de objetos, segmentaciones de escenas...
• Se extraen conocimientos adicionales del audio, como la transcripción y la
detección de emociones.
• Puede usar estos resultados para mejorar la búsqueda, extraer clips, crear
miniaturas y más, mejorando así la participación del usuario.
• Está disponible como servicio y como plataforma.
Video Analyzer
Analice los canales visuales y de audio en un video e
indexe su contenido.
13. Suscríbete al API
• Para poder utilizar Video Analyzer for
Media, necesitas crear una suscripción.
• Puedes registrarte para una prueba:
• Hasta 600 minutos de indexado gratuito
usando el Portal de Video Analyzer.
• Hasta 2400 minutos gratuitos utilizando el
API.
• Si te registras para una prueba, se creará
una suscripción automáticamente.
videoindexer.ai
14. Azure Video Analyzer for Media Portal
• Sube videos:
• Desde una URL
• Desde un Archivo
• Se admiten los formatos de
medios más comunes (MOV,
WMV, MPG, AVI, MP4…)
• Las características se indexan
31. Call to Action
Video Analyzer for Media Portal
https://www.videoindexer.ai/
Video Analyzer for Media Developer Portal
https://api-portal.videoindexer.ai/
Microsoft Learn: Analyze Video
https://docs.microsoft.com/en-us/learn/modules/analyze-video/
33. ¡Gracias por tu atención!
Luis Beltrán
about.me/luis-beltran
https://forms.office.com/r/EyYV40exEe
¿Cómo estuvo mi presentación?
Notas del editor
Azure Cognitive Services is a suite of services and APIs backed by machine learning that enables developers to incorporate intelligent features such as facial recognition in photos and videos, sentiment analysis in text, and language understanding into their applications.
Video Indexer Its purpose is to transform raw video content into content that is searchable, discoverable, and more engaging to the user. Want to generate a video transcript, index words spoken in the video or written on a whiteboard, or create a list of keywords from topics discussed in the video? Video Indexer can do all this and more. It can even find individuals in the video, and sometimes tell who they are.
Video Indexer is both a service and an API. The service is accessed through a Web portal. It allows you to upload videos and examine the information generated from them. The Video Indexer API is a REST API that does everything the portal does, and also allows you to access the information that is generated when videos are indexed.
Video Indexer provides regarding each video that it indexes, and this information is available not only in the portal, but through the Video Indexer API.
Open the Video Indexer portal in your browser and select Get Started. Then select Sign in with your work account (AAD) to sign in with a work or school account, or Sign in with your personal Microsoft account to use a personal Microsoft account. Answer Yes if prompted to let this app access your info.
Select the Upload button. Then select enter a file url to upload a video from a URL.
Paste the following URL into the URL field, and enter "Overview of the Microsoft AI School" as the video name. Then select Upload to begin the upload.
https://topcs.blob.core.windows.net/public/Machine-Learning-in-IoT-solutions_high.mp4
Machine Learning in IoT Solutions
When indexing is complete, you will receive an e-mail notification for each video with a link to the video and a short description of what was found in it, such as people, topics, and keywords. Wait for all three videos to finish indexing, and then proceed to the next exercise.
In Video Indexer, insights are aggregated views of the knowledge extracted from a video, such as faces, keywords, and sentiment. For example, you can see the faces of people appearing in the video, as well as time ranges and percentages for each face shown. Video Indexer cross-references the faces that it finds against a database of thousands of famous people and automatically identifies them. You can see for yourself by opening the "Microsoft in Education" video in the portal. Microsoft CEO Satya Nadella appears in that video, and Video Indexer recognizes him.
Video Indexer automatically generates video transcripts based on its built-in speech and speaker recognition services. It even provides facilities for editing the information that it generated so you can correct errors in transcripts, put names to faces that weren't recognized, and more.
The "Insights" tab shows people featured in the video, keywords generated from the video, topics identified in the video, brands featured in the video, and even emotions found in the video. You can select Play next for any of these items and cycle through the corresponding points in the video.
In this example, Video indexer found two people in the video. It was unable to identify them because they don't appear in its database of famous people. However, you can lend a helping hand by identifying them yourself. Enable editing by selecting the Edit icon in the upper-right corner. Then select the pencil icon next to "Unknown #1" and enter "Sonya Koptyev" as the person's name. Finish up by pressing Enter to save the change.
Repeat this step for the "Unknown #2" in the video. This person's name is "Seth Juarez".
Want to see a full transcript of the video? Select Timeline at the top of the page. Video Indexer uses a deep neural network (DNN) to aid in converting speech to text, but such conversions are rarely perfect. Here, too, you can help out by editing words and phrases that weren't converted properly. To demonstrate, make sure you're still in editing mode and change "High Amsonia captive." to "Hi, I'm Sonya Koptyev.”
Select Insights, and then search for the word "intelligence." This time, the results are conceptual topics that include the search term.
Video Indexer has the ability to translate transcripts into a variety of languages, including German, Dutch, Spanish, French, Czech, Korean, and Japanese. To demonstrate, select Timeline again, select the world icon, and select a language other than English from the drop-down list.
The "Insights" tab shows people featured in the video, keywords generated from the video, topics identified in the video, brands featured in the video, and even emotions found in the video. You can select Play next for any of these items and cycle through the corresponding points in the video.
In this example, Video indexer found two people in the video. It was unable to identify them because they don't appear in its database of famous people. However, you can lend a helping hand by identifying them yourself. Enable editing by selecting the Edit icon in the upper-right corner. Then select the pencil icon next to "Unknown #1" and enter "Sonya Koptyev" as the person's name. Finish up by pressing Enter to save the change.
Repeat this step for the "Unknown #2" in the video. This person's name is "Seth Juarez".
Want to see a full transcript of the video? Select Timeline at the top of the page. Video Indexer uses a deep neural network (DNN) to aid in converting speech to text, but such conversions are rarely perfect. Here, too, you can help out by editing words and phrases that weren't converted properly. To demonstrate, make sure you're still in editing mode and change "High Amsonia captive." to "Hi, I'm Sonya Koptyev.”
Select Insights, and then search for the word "intelligence." This time, the results are conceptual topics that include the search term.
Video Indexer has the ability to translate transcripts into a variety of languages, including German, Dutch, Spanish, French, Czech, Korean, and Japanese. To demonstrate, select Timeline again, select the world icon, and select a language other than English from the drop-down list.
Want to see a full transcript of the video? Select Timeline at the top of the page. Video Indexer uses a deep neural network (DNN) to aid in converting speech to text, but such conversions are rarely perfect. Here, too, you can help out by editing words and phrases that weren't converted properly. To demonstrate, make sure you're still in editing mode and change "High Amsonia captive." to "Hi, I'm Sonya Koptyev.”
Select Insights, and then search for the word "intelligence." This time, the results are conceptual topics that include the search term.
Video Indexer has the ability to translate transcripts into a variety of languages, including German, Dutch, Spanish, French, Czech, Korean, and Japanese. To demonstrate, select Timeline again, select the world icon, and select a language other than English from the drop-down list.
Once a video is indexed, you can search its contents. Type "suggestion" into the search box at the top of the page and press Enter. Confirm that the search results include four instances in which the word "suggestion" was found in the video.
Select Insights, and then search for the word "intelligence." This time, the results are conceptual topics that include the search term.
The Video Indexer portal provides a window into the videos that you index and lets you see a wealth of information extracted from them. But the real power of Video Indexer lies in the Video Indexer API, which lets you submit videos for indexing programmatically and access the results using a REST API. In Exercise 4, you will build an app that uses this API to expose content in the videos you indexed in Exercise 1. But to call the API, you must first subscribe to it and obtain an API key that is transmitted in each request. In this exercise, you will create a Video Indexer API subscription and retrieve the API key created for it.
Open the Video Indexer API portal in your browser and select SIGN IN in the top-right corner. Sign in with your Microsoft account — the same one you used to sign in to the Video Indexer portal. Answer Yes if prompted to let this app access your info.
Select Products, and then select Authorization.
Select the Subscribe button. The subscription will be created.
Select Products again, and then select Authorization. You will now see a list of the subscriptions you have, so select the Product Authorization Subscription.
Select the Subscribe button. The subscription will be created.
Select Products again, and then select Authorization. You will now see a list of the subscriptions you have, so select the Product Authorization Subscription.
Select the Show button next to the Primary Key. Copy the API key to the clipboard, and then select Hide to hide it again.
Now that you have an API key, you can write apps that call the Video Indexer API. The API key travels in an HTTP header in each request. Without a valid API key, the Video Indexer API fails requests placed to it. It is the API's way of ensuring that the caller is authorized.
Open the Video Indexer API portal in your browser and select SIGN IN in the top-right corner. Sign in with your Microsoft account — the same one you used to sign in to the Video Indexer portal. Answer Yes if prompted to let this app access your info.
Select Products, and then select Authorization.
Select the Subscribe button. The subscription will be created.
Select Products again, and then select Authorization. You will now see a list of the subscriptions you have, so select the Product Authorization Subscription.
Select the Subscribe button. The subscription will be created.
Select Products again, and then select Authorization. You will now see a list of the subscriptions you have, so select the Product Authorization Subscription.
Select the Show button next to the Primary Key. Copy the API key to the clipboard, and then select Hide to hide it again.
Now that you have an API key, you can write apps that call the Video Indexer API. The API key travels in an HTTP header in each request. Without a valid API key, the Video Indexer API fails requests placed to it. It is the API's way of ensuring that the caller is authorized.