Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Entity Extraction from Natural Language Text using Apache NiFi and Idyl E3

892 visualizaciones

Publicado el

This presentation describes a dataflow pipeline created in Apache NiFi that uses Idyl E3 Entity Extraction Engine to extract entities from natural language text.

Publicado en: Software
  • Sé el primero en comentar

  • Sé el primero en recomendar esto

Entity Extraction from Natural Language Text using Apache NiFi and Idyl E3

  1. 1. Entity Extraction from Natural Language Text in a Data Flow Pipeline Copyright 2017 Mountain Fog, Inc. All Rights Reserved.
  2. 2. Tools in Use ● Apache NiFi ● Facilitates data flow between disparate sources and services. ● https://nifi.apache.org/ ● Idyl E3 Entity Extraction Engine ● Extracts entities from natural language text via user- generated entity models through a REST API. ● http://www.mtnfog.com/
  3. 3. Goals ● Ingest text files from the file system. ● Extract entities from the text. ● Store entities in a MongoDB database. Text Entities Apache NiFi Dataflow S3
  4. 4. Idyl E3 Entity Extraction Engine Launch via the AWS Marketplace. Comes with an entity model for English-language person entities. REST API for entity extraction.
  5. 5. NiFi Processors A processor executes the dataflow work “of data routing, transformation, or mediation between systems.” [1] We are using: GetFile Processor IdylE3 Processor PutMongoDB Processor [1] https://nifi.apache.org/docs/nifi-docs/html/overview.html
  6. 6. Ingest Text Files Processor’s properties set to read all files from /ingest.
  7. 7. Send Text to Idyl E3 The Idyl E3 endpoint is set in the processor’s properties.
  8. 8. Store Entities in MongoDB The MongoDB URI set in the processor’s properties.
  9. 9. The NiFi Flow
  10. 10. The Result Start the NiFi flow. Files are removed from the ingest directory. Entities appear in the MongoDB collection. Take a well-deserved break.
  11. 11. Going Further Scale Idyl E3 behind a load balancer. Query entities via the Entity Query Language (EQL) processor. Extract other entity types through custom entity models.
  12. 12. support@mtnfog.com www.mtnfog.com

×