Pharo DataFrame: Past, Present, and Future

E
ESUGESUG
Pharo DataFrame:
Past, Present, and Future
Larisa Safina, Oleksandr Zaitsev, Cyril Ferlicot-Delbecque,
Papa Ibrahima Sow
International Workshop on Smalltalk Technologies 2023, Lyon, France
Agenda
What is Data Frames and what they are good for
Evolution of Data Frames in Pharo
Data Frames outside of Pharo
2
Data Frames
Tabular structure
Columns with named headers
Columns/Rows index
Mixed data types in rows
Primitive data types
*The table is courtesy of Zaitsev et al (ESUG22)
3
Data Frames API
Data Import/Export
Grouping and Aggregation
Missing Data Handling
Statistical Operations
Time Series Analysis
Visualization
4
*The table is courtesy of Zaitsev et al (ESUG22)
Pharo DataFrame. Past
Started as a GSoC 2017 project (by Oleks)
Focus of the two followed-up GSoC
Contributions from external developers
5
Pharo DataFrame. Past
Problems:
- No stable maintenance
- Lack of functionality
- Low performance
- Incomplete coherence with Pharo collections
- Lack of detailed documentation
6
Pharo DataFrame. Present
Stable project with a permanent developer (Cyril)
Code optimization
- code quality
- speed and volume improvements
Adding new functionality
7
Awesome
Data Frames
8
Kyle Mitchell (jcmkk3) and
Uwe L. Korn (xhochy)
https://github.com/jcmkk3
/awesome-dataframes
Pharo
DataFrame
vs
Pandas
9
Pharo DataFrame. Future
✨ Better performance ✨
Functionality Enhancements
Better synchronisation with PolyMath and pharo-ai
Big Data Support
Evaluation ❤
Toy Story 1995
10
Do you want to contribute?
https://github.com/PolyMathOrg/DataFrame
11
12
Student project → Mature project with engineers
Better performance
Functionality Enhancements
Evaluation
1 de 12

Recomendados

External Master Data in Alfresco: Integrating and Keeping Metadata Consistent... por
External Master Data in Alfresco: Integrating and Keeping Metadata Consistent...External Master Data in Alfresco: Integrating and Keeping Metadata Consistent...
External Master Data in Alfresco: Integrating and Keeping Metadata Consistent...ITD Systems
2.2K vistas49 diapositivas
A Finnish perspective on FAIRsFAIR outputs por
A Finnish perspective on FAIRsFAIR outputsA Finnish perspective on FAIRsFAIR outputs
A Finnish perspective on FAIRsFAIR outputsJessica Parland-von Essen
249 vistas30 diapositivas
Science and data manipulation in Pharo por
Science and data manipulation in PharoScience and data manipulation in Pharo
Science and data manipulation in PharoESUG
64 vistas21 diapositivas
ALM Search Presentation for the VSS Arch Council por
ALM Search Presentation for the VSS Arch CouncilALM Search Presentation for the VSS Arch Council
ALM Search Presentation for the VSS Arch CouncilSunita Shrivastava
256 vistas25 diapositivas
Flink Apachecon Presentation por
Flink Apachecon PresentationFlink Apachecon Presentation
Flink Apachecon PresentationGyula Fóra
3.6K vistas45 diapositivas
Neo4j Vision and Roadmap por
Neo4j Vision and Roadmap Neo4j Vision and Roadmap
Neo4j Vision and Roadmap Neo4j
39 vistas31 diapositivas

Más contenido relacionado

Similar a Pharo DataFrame: Past, Present, and Future

Dalton Sergio Leonardo Eng Resume 20160803 por
Dalton Sergio Leonardo Eng Resume 20160803Dalton Sergio Leonardo Eng Resume 20160803
Dalton Sergio Leonardo Eng Resume 20160803Dalton Sergio Leonardo
220 vistas4 diapositivas
Overview of Modern Graph Analysis Tools por
Overview of Modern Graph Analysis ToolsOverview of Modern Graph Analysis Tools
Overview of Modern Graph Analysis ToolsKeiichiro Ono
2.4K vistas35 diapositivas
OpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGO por
OpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGOOpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGO
OpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGOOpenNebula Project
246 vistas7 diapositivas
Thinking About Guideline for Data Interoperability - Design concept and workf... por
Thinking About Guideline for Data Interoperability - Design concept and workf...Thinking About Guideline for Data Interoperability - Design concept and workf...
Thinking About Guideline for Data Interoperability - Design concept and workf...Open Cyber University of Korea
595 vistas9 diapositivas
SiLA: Making the standard fit for the future and adapting an open-source coll... por
SiLA: Making the standard fit for the future and adapting an open-source coll...SiLA: Making the standard fit for the future and adapting an open-source coll...
SiLA: Making the standard fit for the future and adapting an open-source coll...Gáspár Incze
225 vistas32 diapositivas
The AmeriFlux Network Data Management System por
The AmeriFlux Network Data Management SystemThe AmeriFlux Network Data Management System
The AmeriFlux Network Data Management SystemIntegrated Carbon Observation System (ICOS)
73 vistas20 diapositivas

Similar a Pharo DataFrame: Past, Present, and Future(20)

Overview of Modern Graph Analysis Tools por Keiichiro Ono
Overview of Modern Graph Analysis ToolsOverview of Modern Graph Analysis Tools
Overview of Modern Graph Analysis Tools
Keiichiro Ono2.4K vistas
OpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGO por OpenNebula Project
OpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGOOpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGO
OpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGO
OpenNebula Project246 vistas
SiLA: Making the standard fit for the future and adapting an open-source coll... por Gáspár Incze
SiLA: Making the standard fit for the future and adapting an open-source coll...SiLA: Making the standard fit for the future and adapting an open-source coll...
SiLA: Making the standard fit for the future and adapting an open-source coll...
Gáspár Incze225 vistas
MuseoTorino, first italian project using a GraphDB, RDFa, Linked Open Data por 21Style
MuseoTorino, first italian project using a GraphDB, RDFa, Linked Open DataMuseoTorino, first italian project using a GraphDB, RDFa, Linked Open Data
MuseoTorino, first italian project using a GraphDB, RDFa, Linked Open Data
21Style1.4K vistas
Data Enthusiasts London: Scalable and Interoperable data services. Applied to... por Andy Petrella
Data Enthusiasts London: Scalable and Interoperable data services. Applied to...Data Enthusiasts London: Scalable and Interoperable data services. Applied to...
Data Enthusiasts London: Scalable and Interoperable data services. Applied to...
Andy Petrella1.3K vistas
Andy Petrella_Med@Scale by Data Fellas: Scalable and Interoperable Genomics d... por Dataconomy Media
Andy Petrella_Med@Scale by Data Fellas: Scalable and Interoperable Genomics d...Andy Petrella_Med@Scale by Data Fellas: Scalable and Interoperable Genomics d...
Andy Petrella_Med@Scale by Data Fellas: Scalable and Interoperable Genomics d...
Dataconomy Media582 vistas
Productive Development with APEX por Simon Boorsma
Productive Development with APEXProductive Development with APEX
Productive Development with APEX
Simon Boorsma856 vistas
2015 Data Science Summit @ dato Review por Hang Li
2015 Data Science Summit @ dato Review2015 Data Science Summit @ dato Review
2015 Data Science Summit @ dato Review
Hang Li976 vistas
System Architecture_BPM TIBCO_Network Inventory por Ela Pamieta
System Architecture_BPM TIBCO_Network InventorySystem Architecture_BPM TIBCO_Network Inventory
System Architecture_BPM TIBCO_Network Inventory
Ela Pamieta202 vistas
How to expand the Galaxy from genes to Earth in six simple steps (and live sm... por Raffaele Montella
How to expand the Galaxy from genes to Earth in six simple steps (and live sm...How to expand the Galaxy from genes to Earth in six simple steps (and live sm...
How to expand the Galaxy from genes to Earth in six simple steps (and live sm...
Raffaele Montella760 vistas
Java EE 7 with Apache Spark for the World’s Largest Credit Card Core Systems ... por Hirofumi Iwasaki
Java EE 7 with Apache Spark for the World’s Largest Credit Card Core Systems ...Java EE 7 with Apache Spark for the World’s Largest Credit Card Core Systems ...
Java EE 7 with Apache Spark for the World’s Largest Credit Card Core Systems ...
Hirofumi Iwasaki1.1K vistas
Apache Flink: Past, Present and Future por Gyula Fóra
Apache Flink: Past, Present and FutureApache Flink: Past, Present and Future
Apache Flink: Past, Present and Future
Gyula Fóra739 vistas
Tripal within the Arabidopsis Information Portal - PAG XXIII por Vivek Krishnakumar
Tripal within the Arabidopsis Information Portal - PAG XXIIITripal within the Arabidopsis Information Portal - PAG XXIII
Tripal within the Arabidopsis Information Portal - PAG XXIII
Vivek Krishnakumar1.2K vistas

Más de ESUG

Workshop: Identifying concept inventories in agile programming por
Workshop: Identifying concept inventories in agile programmingWorkshop: Identifying concept inventories in agile programming
Workshop: Identifying concept inventories in agile programmingESUG
12 vistas16 diapositivas
Technical documentation support in Pharo por
Technical documentation support in PharoTechnical documentation support in Pharo
Technical documentation support in PharoESUG
31 vistas39 diapositivas
The Pharo Debugger and Debugging tools: Advances and Roadmap por
The Pharo Debugger and Debugging tools: Advances and RoadmapThe Pharo Debugger and Debugging tools: Advances and Roadmap
The Pharo Debugger and Debugging tools: Advances and RoadmapESUG
56 vistas44 diapositivas
Sequence: Pipeline modelling in Pharo por
Sequence: Pipeline modelling in PharoSequence: Pipeline modelling in Pharo
Sequence: Pipeline modelling in PharoESUG
86 vistas22 diapositivas
Migration process from monolithic to micro frontend architecture in mobile ap... por
Migration process from monolithic to micro frontend architecture in mobile ap...Migration process from monolithic to micro frontend architecture in mobile ap...
Migration process from monolithic to micro frontend architecture in mobile ap...ESUG
22 vistas35 diapositivas
Analyzing Dart Language with Pharo: Report and early results por
Analyzing Dart Language with Pharo: Report and early resultsAnalyzing Dart Language with Pharo: Report and early results
Analyzing Dart Language with Pharo: Report and early resultsESUG
107 vistas30 diapositivas

Más de ESUG(20)

Workshop: Identifying concept inventories in agile programming por ESUG
Workshop: Identifying concept inventories in agile programmingWorkshop: Identifying concept inventories in agile programming
Workshop: Identifying concept inventories in agile programming
ESUG12 vistas
Technical documentation support in Pharo por ESUG
Technical documentation support in PharoTechnical documentation support in Pharo
Technical documentation support in Pharo
ESUG31 vistas
The Pharo Debugger and Debugging tools: Advances and Roadmap por ESUG
The Pharo Debugger and Debugging tools: Advances and RoadmapThe Pharo Debugger and Debugging tools: Advances and Roadmap
The Pharo Debugger and Debugging tools: Advances and Roadmap
ESUG56 vistas
Sequence: Pipeline modelling in Pharo por ESUG
Sequence: Pipeline modelling in PharoSequence: Pipeline modelling in Pharo
Sequence: Pipeline modelling in Pharo
ESUG86 vistas
Migration process from monolithic to micro frontend architecture in mobile ap... por ESUG
Migration process from monolithic to micro frontend architecture in mobile ap...Migration process from monolithic to micro frontend architecture in mobile ap...
Migration process from monolithic to micro frontend architecture in mobile ap...
ESUG22 vistas
Analyzing Dart Language with Pharo: Report and early results por ESUG
Analyzing Dart Language with Pharo: Report and early resultsAnalyzing Dart Language with Pharo: Report and early results
Analyzing Dart Language with Pharo: Report and early results
ESUG107 vistas
Transpiling Pharo Classes to JS ECMAScript 5 versus ECMAScript 6 por ESUG
Transpiling Pharo Classes to JS ECMAScript 5 versus ECMAScript 6Transpiling Pharo Classes to JS ECMAScript 5 versus ECMAScript 6
Transpiling Pharo Classes to JS ECMAScript 5 versus ECMAScript 6
ESUG37 vistas
A Unit Test Metamodel for Test Generation por ESUG
A Unit Test Metamodel for Test GenerationA Unit Test Metamodel for Test Generation
A Unit Test Metamodel for Test Generation
ESUG53 vistas
Creating Unit Tests Using Genetic Programming por ESUG
Creating Unit Tests Using Genetic ProgrammingCreating Unit Tests Using Genetic Programming
Creating Unit Tests Using Genetic Programming
ESUG46 vistas
Threaded-Execution and CPS Provide Smooth Switching Between Execution Modes por ESUG
Threaded-Execution and CPS Provide Smooth Switching Between Execution ModesThreaded-Execution and CPS Provide Smooth Switching Between Execution Modes
Threaded-Execution and CPS Provide Smooth Switching Between Execution Modes
ESUG52 vistas
Exploring GitHub Actions through EGAD: An Experience Report por ESUG
Exploring GitHub Actions through EGAD: An Experience ReportExploring GitHub Actions through EGAD: An Experience Report
Exploring GitHub Actions through EGAD: An Experience Report
ESUG17 vistas
Pharo: a reflective language A first systematic analysis of reflective APIs por ESUG
Pharo: a reflective language A first systematic analysis of reflective APIsPharo: a reflective language A first systematic analysis of reflective APIs
Pharo: a reflective language A first systematic analysis of reflective APIs
ESUG57 vistas
Garbage Collector Tuning por ESUG
Garbage Collector TuningGarbage Collector Tuning
Garbage Collector Tuning
ESUG20 vistas
Improving Performance Through Object Lifetime Profiling: the DataFrame Case por ESUG
Improving Performance Through Object Lifetime Profiling: the DataFrame CaseImproving Performance Through Object Lifetime Profiling: the DataFrame Case
Improving Performance Through Object Lifetime Profiling: the DataFrame Case
ESUG43 vistas
thisContext in the Debugger por ESUG
thisContext in the DebuggerthisContext in the Debugger
thisContext in the Debugger
ESUG36 vistas
Websockets for Fencing Score por ESUG
Websockets for Fencing ScoreWebsockets for Fencing Score
Websockets for Fencing Score
ESUG18 vistas
ShowUs: PharoJS.org Develop in Pharo, Run on JavaScript por ESUG
ShowUs: PharoJS.org Develop in Pharo, Run on JavaScriptShowUs: PharoJS.org Develop in Pharo, Run on JavaScript
ShowUs: PharoJS.org Develop in Pharo, Run on JavaScript
ESUG46 vistas
Advanced Object- Oriented Design Mooc por ESUG
Advanced Object- Oriented Design MoocAdvanced Object- Oriented Design Mooc
Advanced Object- Oriented Design Mooc
ESUG85 vistas
A New Architecture Reconciling Refactorings and Transformations por ESUG
A New Architecture Reconciling Refactorings and TransformationsA New Architecture Reconciling Refactorings and Transformations
A New Architecture Reconciling Refactorings and Transformations
ESUG28 vistas
BioSmalltalk por ESUG
BioSmalltalkBioSmalltalk
BioSmalltalk
ESUG415 vistas

Último

DSD-INT 2023 European Digital Twin Ocean and Delft3D FM - Dols por
DSD-INT 2023 European Digital Twin Ocean and Delft3D FM - DolsDSD-INT 2023 European Digital Twin Ocean and Delft3D FM - Dols
DSD-INT 2023 European Digital Twin Ocean and Delft3D FM - DolsDeltares
9 vistas23 diapositivas
Headless JS UG Presentation.pptx por
Headless JS UG Presentation.pptxHeadless JS UG Presentation.pptx
Headless JS UG Presentation.pptxJack Spektor
8 vistas24 diapositivas
BushraDBR: An Automatic Approach to Retrieving Duplicate Bug Reports por
BushraDBR: An Automatic Approach to Retrieving Duplicate Bug ReportsBushraDBR: An Automatic Approach to Retrieving Duplicate Bug Reports
BushraDBR: An Automatic Approach to Retrieving Duplicate Bug ReportsRa'Fat Al-Msie'deen
8 vistas49 diapositivas
Unleash The Monkeys por
Unleash The MonkeysUnleash The Monkeys
Unleash The MonkeysJacob Duijzer
8 vistas28 diapositivas
Navigating container technology for enhanced security by Niklas Saari por
Navigating container technology for enhanced security by Niklas SaariNavigating container technology for enhanced security by Niklas Saari
Navigating container technology for enhanced security by Niklas SaariMetosin Oy
14 vistas34 diapositivas
DSD-INT 2023 3D hydrodynamic modelling of microplastic transport in lakes - J... por
DSD-INT 2023 3D hydrodynamic modelling of microplastic transport in lakes - J...DSD-INT 2023 3D hydrodynamic modelling of microplastic transport in lakes - J...
DSD-INT 2023 3D hydrodynamic modelling of microplastic transport in lakes - J...Deltares
12 vistas24 diapositivas

Último(20)

DSD-INT 2023 European Digital Twin Ocean and Delft3D FM - Dols por Deltares
DSD-INT 2023 European Digital Twin Ocean and Delft3D FM - DolsDSD-INT 2023 European Digital Twin Ocean and Delft3D FM - Dols
DSD-INT 2023 European Digital Twin Ocean and Delft3D FM - Dols
Deltares9 vistas
Headless JS UG Presentation.pptx por Jack Spektor
Headless JS UG Presentation.pptxHeadless JS UG Presentation.pptx
Headless JS UG Presentation.pptx
Jack Spektor8 vistas
BushraDBR: An Automatic Approach to Retrieving Duplicate Bug Reports por Ra'Fat Al-Msie'deen
BushraDBR: An Automatic Approach to Retrieving Duplicate Bug ReportsBushraDBR: An Automatic Approach to Retrieving Duplicate Bug Reports
BushraDBR: An Automatic Approach to Retrieving Duplicate Bug Reports
Navigating container technology for enhanced security by Niklas Saari por Metosin Oy
Navigating container technology for enhanced security by Niklas SaariNavigating container technology for enhanced security by Niklas Saari
Navigating container technology for enhanced security by Niklas Saari
Metosin Oy14 vistas
DSD-INT 2023 3D hydrodynamic modelling of microplastic transport in lakes - J... por Deltares
DSD-INT 2023 3D hydrodynamic modelling of microplastic transport in lakes - J...DSD-INT 2023 3D hydrodynamic modelling of microplastic transport in lakes - J...
DSD-INT 2023 3D hydrodynamic modelling of microplastic transport in lakes - J...
Deltares12 vistas
Copilot Prompting Toolkit_All Resources.pdf por Riccardo Zamana
Copilot Prompting Toolkit_All Resources.pdfCopilot Prompting Toolkit_All Resources.pdf
Copilot Prompting Toolkit_All Resources.pdf
Riccardo Zamana10 vistas
Gen Apps on Google Cloud PaLM2 and Codey APIs in Action por Márton Kodok
Gen Apps on Google Cloud PaLM2 and Codey APIs in ActionGen Apps on Google Cloud PaLM2 and Codey APIs in Action
Gen Apps on Google Cloud PaLM2 and Codey APIs in Action
Márton Kodok6 vistas
FOSSLight Community Day 2023-11-30 por Shane Coughlan
FOSSLight Community Day 2023-11-30FOSSLight Community Day 2023-11-30
FOSSLight Community Day 2023-11-30
Shane Coughlan5 vistas
Unlocking the Power of AI in Product Management - A Comprehensive Guide for P... por NimaTorabi2
Unlocking the Power of AI in Product Management - A Comprehensive Guide for P...Unlocking the Power of AI in Product Management - A Comprehensive Guide for P...
Unlocking the Power of AI in Product Management - A Comprehensive Guide for P...
NimaTorabi212 vistas
DSD-INT 2023 Simulation of Coastal Hydrodynamics and Water Quality in Hong Ko... por Deltares
DSD-INT 2023 Simulation of Coastal Hydrodynamics and Water Quality in Hong Ko...DSD-INT 2023 Simulation of Coastal Hydrodynamics and Water Quality in Hong Ko...
DSD-INT 2023 Simulation of Coastal Hydrodynamics and Water Quality in Hong Ko...
Deltares14 vistas
2023-November-Schneider Electric-Meetup-BCN Admin Group.pptx por animuscrm
2023-November-Schneider Electric-Meetup-BCN Admin Group.pptx2023-November-Schneider Electric-Meetup-BCN Admin Group.pptx
2023-November-Schneider Electric-Meetup-BCN Admin Group.pptx
animuscrm15 vistas
DSD-INT 2023 Exploring flash flood hazard reduction in arid regions using a h... por Deltares
DSD-INT 2023 Exploring flash flood hazard reduction in arid regions using a h...DSD-INT 2023 Exploring flash flood hazard reduction in arid regions using a h...
DSD-INT 2023 Exploring flash flood hazard reduction in arid regions using a h...
Deltares9 vistas
Dev-HRE-Ops - Addressing the _Last Mile DevOps Challenge_ in Highly Regulated... por TomHalpin9
Dev-HRE-Ops - Addressing the _Last Mile DevOps Challenge_ in Highly Regulated...Dev-HRE-Ops - Addressing the _Last Mile DevOps Challenge_ in Highly Regulated...
Dev-HRE-Ops - Addressing the _Last Mile DevOps Challenge_ in Highly Regulated...
TomHalpin96 vistas
Ports-and-Adapters Architecture for Embedded HMI por Burkhard Stubert
Ports-and-Adapters Architecture for Embedded HMIPorts-and-Adapters Architecture for Embedded HMI
Ports-and-Adapters Architecture for Embedded HMI
Burkhard Stubert21 vistas

Pharo DataFrame: Past, Present, and Future

  • 1. Pharo DataFrame: Past, Present, and Future Larisa Safina, Oleksandr Zaitsev, Cyril Ferlicot-Delbecque, Papa Ibrahima Sow International Workshop on Smalltalk Technologies 2023, Lyon, France
  • 2. Agenda What is Data Frames and what they are good for Evolution of Data Frames in Pharo Data Frames outside of Pharo 2
  • 3. Data Frames Tabular structure Columns with named headers Columns/Rows index Mixed data types in rows Primitive data types *The table is courtesy of Zaitsev et al (ESUG22) 3
  • 4. Data Frames API Data Import/Export Grouping and Aggregation Missing Data Handling Statistical Operations Time Series Analysis Visualization 4 *The table is courtesy of Zaitsev et al (ESUG22)
  • 5. Pharo DataFrame. Past Started as a GSoC 2017 project (by Oleks) Focus of the two followed-up GSoC Contributions from external developers 5
  • 6. Pharo DataFrame. Past Problems: - No stable maintenance - Lack of functionality - Low performance - Incomplete coherence with Pharo collections - Lack of detailed documentation 6
  • 7. Pharo DataFrame. Present Stable project with a permanent developer (Cyril) Code optimization - code quality - speed and volume improvements Adding new functionality 7
  • 8. Awesome Data Frames 8 Kyle Mitchell (jcmkk3) and Uwe L. Korn (xhochy) https://github.com/jcmkk3 /awesome-dataframes
  • 10. Pharo DataFrame. Future ✨ Better performance ✨ Functionality Enhancements Better synchronisation with PolyMath and pharo-ai Big Data Support Evaluation ❤ Toy Story 1995 10
  • 11. Do you want to contribute? https://github.com/PolyMathOrg/DataFrame 11
  • 12. 12 Student project → Mature project with engineers Better performance Functionality Enhancements Evaluation