SlideShare una empresa de Scribd logo
1 de 8
Descargar para leer sin conexión
Working with Data
Eng Teong Cheah
Microsoft MVP
Working with Datastores
In Azure Machine Learning, datastores are abstractions for cloud data sources. They
encapsulate the information required to connect to data sources. You can access
datastores directly in code by using the the Azure Machine Learning.
Types of Datastore
Azure Machine Learning supports the creation of datastores for multiple kinds of Azure
data source, including:
- Azure Storage (blob and file containers)
- Azure Data Lake stores
- Azure SQL Database
- Azure Databricks file system (DBFS)
Working with Datasets
Datasets are versioned packaged data objects that can be easily consumed in
experiments and pipelines. Datasets are the recommended what to work with data, and re
the primary mechanism for advanced Azure Machine Learning capabilities like data
labeling and data drift monitoring.
Types of Datasets
Datasets are typically on files in a datastores, though they can also be based on URLs and
other sources. You can create the following types of datasets:
- Tabular
The data is read from the dataset as a table. You should use this type of dataset when
your data is consistently structured and you want to work with it in common tabular
data structures, such as Pandas dataframes.
- File
The dataset presents a list of file paths that can be read as though from the file system.
Use this type of dataset when your data is unstructured, or when you need to process
the data at the file level(for example, to train a convolutional neural network from a set
of image files).
Demo
Work with Data
References
Microsoft Docs

Más contenido relacionado

Similar a Working with Data

ADF Demo_ppt.pptx
ADF Demo_ppt.pptxADF Demo_ppt.pptx
ADF Demo_ppt.pptxvamsytaurus
 
Afternoons with Azure - Azure Data Services
Afternoons with Azure - Azure Data ServicesAfternoons with Azure - Azure Data Services
Afternoons with Azure - Azure Data ServicesCCG
 
Data Analytics Meetup: Introduction to Azure Data Lake Storage
Data Analytics Meetup: Introduction to Azure Data Lake Storage Data Analytics Meetup: Introduction to Azure Data Lake Storage
Data Analytics Meetup: Introduction to Azure Data Lake Storage CCG
 
Azure fundamental -Introduction
Azure fundamental -IntroductionAzure fundamental -Introduction
Azure fundamental -IntroductionManishK55
 
DATABASE Lecture 1 and 2.pptx
DATABASE Lecture 1 and 2.pptxDATABASE Lecture 1 and 2.pptx
DATABASE Lecture 1 and 2.pptxRUBAB79
 
Modern Analytics Academy - Data Modeling (1).pptx
Modern Analytics Academy - Data Modeling (1).pptxModern Analytics Academy - Data Modeling (1).pptx
Modern Analytics Academy - Data Modeling (1).pptxssuser290967
 
Transform your data with Azure Data factory
Transform your data with Azure Data factoryTransform your data with Azure Data factory
Transform your data with Azure Data factoryPrometix Pty Ltd
 
Exploring Microsoft Azure Infrastructures
Exploring Microsoft Azure InfrastructuresExploring Microsoft Azure Infrastructures
Exploring Microsoft Azure InfrastructuresCCG
 
2.Introduction to NOSQL (Core concepts).pptx
2.Introduction to NOSQL (Core concepts).pptx2.Introduction to NOSQL (Core concepts).pptx
2.Introduction to NOSQL (Core concepts).pptxRushikeshChikane2
 
Azure Data serices and databricks architecture
Azure Data serices and databricks architectureAzure Data serices and databricks architecture
Azure Data serices and databricks architectureAdventureWorld5
 
Azure data bricks by Eugene Polonichko
Azure data bricks by Eugene PolonichkoAzure data bricks by Eugene Polonichko
Azure data bricks by Eugene PolonichkoAlex Tumanoff
 
Azure DataBricks for Data Engineering by Eugene Polonichko
Azure DataBricks for Data Engineering by Eugene PolonichkoAzure DataBricks for Data Engineering by Eugene Polonichko
Azure DataBricks for Data Engineering by Eugene PolonichkoDimko Zhluktenko
 
Cloud architectural patterns and Microsoft Azure tools
Cloud architectural patterns and Microsoft Azure toolsCloud architectural patterns and Microsoft Azure tools
Cloud architectural patterns and Microsoft Azure toolsPushkar Chivate
 
Integrating with Azure Data Lake
Integrating with Azure Data LakeIntegrating with Azure Data Lake
Integrating with Azure Data LakeRuman Khan
 
Azure Synapse Analytics Overview (r2)
Azure Synapse Analytics Overview (r2)Azure Synapse Analytics Overview (r2)
Azure Synapse Analytics Overview (r2)James Serra
 

Similar a Working with Data (20)

ADF Demo_ppt.pptx
ADF Demo_ppt.pptxADF Demo_ppt.pptx
ADF Demo_ppt.pptx
 
Azure Data Engineering.pptx
Azure Data Engineering.pptxAzure Data Engineering.pptx
Azure Data Engineering.pptx
 
Afternoons with Azure - Azure Data Services
Afternoons with Azure - Azure Data ServicesAfternoons with Azure - Azure Data Services
Afternoons with Azure - Azure Data Services
 
Data Analytics Meetup: Introduction to Azure Data Lake Storage
Data Analytics Meetup: Introduction to Azure Data Lake Storage Data Analytics Meetup: Introduction to Azure Data Lake Storage
Data Analytics Meetup: Introduction to Azure Data Lake Storage
 
Introduction to Azure Data Lake
Introduction to Azure Data LakeIntroduction to Azure Data Lake
Introduction to Azure Data Lake
 
Azure fundamental -Introduction
Azure fundamental -IntroductionAzure fundamental -Introduction
Azure fundamental -Introduction
 
DATABASE Lecture 1 and 2.pptx
DATABASE Lecture 1 and 2.pptxDATABASE Lecture 1 and 2.pptx
DATABASE Lecture 1 and 2.pptx
 
Modern Analytics Academy - Data Modeling (1).pptx
Modern Analytics Academy - Data Modeling (1).pptxModern Analytics Academy - Data Modeling (1).pptx
Modern Analytics Academy - Data Modeling (1).pptx
 
Transform your data with Azure Data factory
Transform your data with Azure Data factoryTransform your data with Azure Data factory
Transform your data with Azure Data factory
 
Exploring Microsoft Azure Infrastructures
Exploring Microsoft Azure InfrastructuresExploring Microsoft Azure Infrastructures
Exploring Microsoft Azure Infrastructures
 
2.Introduction to NOSQL (Core concepts).pptx
2.Introduction to NOSQL (Core concepts).pptx2.Introduction to NOSQL (Core concepts).pptx
2.Introduction to NOSQL (Core concepts).pptx
 
Azure Data serices and databricks architecture
Azure Data serices and databricks architectureAzure Data serices and databricks architecture
Azure Data serices and databricks architecture
 
slides.pptx
slides.pptxslides.pptx
slides.pptx
 
slides.pptx
slides.pptxslides.pptx
slides.pptx
 
Azure data bricks by Eugene Polonichko
Azure data bricks by Eugene PolonichkoAzure data bricks by Eugene Polonichko
Azure data bricks by Eugene Polonichko
 
Azure DataBricks for Data Engineering by Eugene Polonichko
Azure DataBricks for Data Engineering by Eugene PolonichkoAzure DataBricks for Data Engineering by Eugene Polonichko
Azure DataBricks for Data Engineering by Eugene Polonichko
 
Cloud architectural patterns and Microsoft Azure tools
Cloud architectural patterns and Microsoft Azure toolsCloud architectural patterns and Microsoft Azure tools
Cloud architectural patterns and Microsoft Azure tools
 
Integrating with Azure Data Lake
Integrating with Azure Data LakeIntegrating with Azure Data Lake
Integrating with Azure Data Lake
 
Azure Synapse Analytics Overview (r2)
Azure Synapse Analytics Overview (r2)Azure Synapse Analytics Overview (r2)
Azure Synapse Analytics Overview (r2)
 
CC -Unit4.pptx
CC -Unit4.pptxCC -Unit4.pptx
CC -Unit4.pptx
 

Más de Eng Teong Cheah

Responsible Machine Learning
Responsible Machine LearningResponsible Machine Learning
Responsible Machine LearningEng Teong Cheah
 
Machine Learning Workflows
Machine Learning WorkflowsMachine Learning Workflows
Machine Learning WorkflowsEng Teong Cheah
 
Experiments & TrainingModels
Experiments & TrainingModelsExperiments & TrainingModels
Experiments & TrainingModelsEng Teong Cheah
 
Automated Machine Learning
Automated Machine LearningAutomated Machine Learning
Automated Machine LearningEng Teong Cheah
 
Getting Started with Azure Machine Learning
Getting Started with Azure Machine LearningGetting Started with Azure Machine Learning
Getting Started with Azure Machine LearningEng Teong Cheah
 
Hacking Containers - Container Storage
Hacking Containers - Container StorageHacking Containers - Container Storage
Hacking Containers - Container StorageEng Teong Cheah
 
Hacking Containers - Looking at Cgroups
Hacking Containers - Looking at CgroupsHacking Containers - Looking at Cgroups
Hacking Containers - Looking at CgroupsEng Teong Cheah
 
Hacking Containers - Linux Containers
Hacking Containers - Linux ContainersHacking Containers - Linux Containers
Hacking Containers - Linux ContainersEng Teong Cheah
 
Data Security - Storage Security
Data Security - Storage SecurityData Security - Storage Security
Data Security - Storage SecurityEng Teong Cheah
 
Application Security- App security
Application Security- App securityApplication Security- App security
Application Security- App securityEng Teong Cheah
 
Application Security - Key Vault
Application Security - Key VaultApplication Security - Key Vault
Application Security - Key VaultEng Teong Cheah
 
Compute Security - Container Security
Compute Security - Container SecurityCompute Security - Container Security
Compute Security - Container SecurityEng Teong Cheah
 
Compute Security - Host Security
Compute Security - Host SecurityCompute Security - Host Security
Compute Security - Host SecurityEng Teong Cheah
 
Virtual Networking Security - Network Security
Virtual Networking Security - Network SecurityVirtual Networking Security - Network Security
Virtual Networking Security - Network SecurityEng Teong Cheah
 
Virtual Networking Security - Perimeter Security
Virtual Networking Security - Perimeter SecurityVirtual Networking Security - Perimeter Security
Virtual Networking Security - Perimeter SecurityEng Teong Cheah
 
Access Security - Hybrid Identity
Access Security - Hybrid IdentityAccess Security - Hybrid Identity
Access Security - Hybrid IdentityEng Teong Cheah
 

Más de Eng Teong Cheah (20)

Monitoring Models
Monitoring ModelsMonitoring Models
Monitoring Models
 
Responsible Machine Learning
Responsible Machine LearningResponsible Machine Learning
Responsible Machine Learning
 
Training Optimal Models
Training Optimal ModelsTraining Optimal Models
Training Optimal Models
 
Deploying Models
Deploying ModelsDeploying Models
Deploying Models
 
Machine Learning Workflows
Machine Learning WorkflowsMachine Learning Workflows
Machine Learning Workflows
 
Working with Compute
Working with ComputeWorking with Compute
Working with Compute
 
Experiments & TrainingModels
Experiments & TrainingModelsExperiments & TrainingModels
Experiments & TrainingModels
 
Automated Machine Learning
Automated Machine LearningAutomated Machine Learning
Automated Machine Learning
 
Getting Started with Azure Machine Learning
Getting Started with Azure Machine LearningGetting Started with Azure Machine Learning
Getting Started with Azure Machine Learning
 
Hacking Containers - Container Storage
Hacking Containers - Container StorageHacking Containers - Container Storage
Hacking Containers - Container Storage
 
Hacking Containers - Looking at Cgroups
Hacking Containers - Looking at CgroupsHacking Containers - Looking at Cgroups
Hacking Containers - Looking at Cgroups
 
Hacking Containers - Linux Containers
Hacking Containers - Linux ContainersHacking Containers - Linux Containers
Hacking Containers - Linux Containers
 
Data Security - Storage Security
Data Security - Storage SecurityData Security - Storage Security
Data Security - Storage Security
 
Application Security- App security
Application Security- App securityApplication Security- App security
Application Security- App security
 
Application Security - Key Vault
Application Security - Key VaultApplication Security - Key Vault
Application Security - Key Vault
 
Compute Security - Container Security
Compute Security - Container SecurityCompute Security - Container Security
Compute Security - Container Security
 
Compute Security - Host Security
Compute Security - Host SecurityCompute Security - Host Security
Compute Security - Host Security
 
Virtual Networking Security - Network Security
Virtual Networking Security - Network SecurityVirtual Networking Security - Network Security
Virtual Networking Security - Network Security
 
Virtual Networking Security - Perimeter Security
Virtual Networking Security - Perimeter SecurityVirtual Networking Security - Perimeter Security
Virtual Networking Security - Perimeter Security
 
Access Security - Hybrid Identity
Access Security - Hybrid IdentityAccess Security - Hybrid Identity
Access Security - Hybrid Identity
 

Último

Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 
Artificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraArtificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraDeakin University
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 3652toLead Limited
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
Key Features Of Token Development (1).pptx
Key  Features Of Token  Development (1).pptxKey  Features Of Token  Development (1).pptx
Key Features Of Token Development (1).pptxLBM Solutions
 
Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Neo4j
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxnull - The Open Security Community
 
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024BookNet Canada
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 

Último (20)

Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
Artificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraArtificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning era
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
 
The transition to renewables in India.pdf
The transition to renewables in India.pdfThe transition to renewables in India.pdf
The transition to renewables in India.pdf
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptxVulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
Key Features Of Token Development (1).pptx
Key  Features Of Token  Development (1).pptxKey  Features Of Token  Development (1).pptx
Key Features Of Token Development (1).pptx
 
Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
 
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 

Working with Data

  • 3. Working with Datastores In Azure Machine Learning, datastores are abstractions for cloud data sources. They encapsulate the information required to connect to data sources. You can access datastores directly in code by using the the Azure Machine Learning.
  • 4. Types of Datastore Azure Machine Learning supports the creation of datastores for multiple kinds of Azure data source, including: - Azure Storage (blob and file containers) - Azure Data Lake stores - Azure SQL Database - Azure Databricks file system (DBFS)
  • 5. Working with Datasets Datasets are versioned packaged data objects that can be easily consumed in experiments and pipelines. Datasets are the recommended what to work with data, and re the primary mechanism for advanced Azure Machine Learning capabilities like data labeling and data drift monitoring.
  • 6. Types of Datasets Datasets are typically on files in a datastores, though they can also be based on URLs and other sources. You can create the following types of datasets: - Tabular The data is read from the dataset as a table. You should use this type of dataset when your data is consistently structured and you want to work with it in common tabular data structures, such as Pandas dataframes. - File The dataset presents a list of file paths that can be read as though from the file system. Use this type of dataset when your data is unstructured, or when you need to process the data at the file level(for example, to train a convolutional neural network from a set of image files).