Updates on Veda provided by Amit Kanudia from KanORS-EMR

•

0 recomendaciones•59 vistas

Veda online updates - Veda for open-source models TIMES and OSeMOSYSBrowse, Veda Assistant VEDA2.0, VEDAONLINE, VEDA Mr. Amit Kanudia KanORS-EMR 16–17th november 2023, etsap meeting, etsap winter workshop, italy, mr. amit kanudia kanors-emr, november 2023, politecnico di torino lingotto, semi-annual etsap meeting, torino, turin

Software

Veda update
ETSAP meeting Torino
16 November 2023

How it works:
• Identify a license admin for an institution abc@<inst name>.???
• All users who register with email addresses like xyz@<inst name>.???
will be able to access educational version of VO without any further
check.
• Runs will be solved on the NEOS server (3 GB RAM limit)
• Can only make public models out of public GitHub folders

Next steps
• Prepare an email that contracting parties can send to educational
institutes in their countries.
• Prepare demo models with self-learning as the objective.
• Prepare video tutorials around these models.
• Offer webinars.

Different routes from input to results
TIMES
MARKAL
OSeMOSYS
TEMOA
??
Veda Input Files Veda Reports

OSeMOSYS
• Common attributes mapped
• Other attributes supported
• Additional features that improve usability
• TIMES interpolation options (1-5) processed by Veda
• Case management supported
• Last value retained as per scenario order

Make TIMES more accessible
• Ose_CSV folder in the root of Veda model folder would replace the
VT_* files.
• All other Veda files would work as they do.
• Existing OSeMOSYS and TEMOA models will be able to run as TIMES
with little (or very little) effort on the part of their developers.

Make energy system modelling easier
• Create an environment where domain experts can explore the
interplay of key assumptions (via large scenario experiments) using
system models – without much investment in what lies under the
hood.
• Intuitive names for key input parameters and output variables that
will work across multiple modelling frameworks.

Two routes
• Introduce new parameters
• Test existing tables

Future
developments
Create model users around rich and
mature models

Limited sharing of complex models
1. Select a commit
2. Upload a parametric scenario file
3. Synchronize
4. Run
5. View reports

Más contenido relacionado

Similar a Updates on Veda provided by Amit Kanudia from KanORS-EMR

Web Tools for GemStone/S

ESUG

Journey towards serverless infrastructure

Ville Seppänen

Lotus Domino 8.5

Lalit Sharma

Azure Templates for Consistent Deployment

José Maia

Microsoft ASP.NET 5 - The new kid on the block

Christos Matskas

XPages -Beyond the Basics

Ulrich Krause

Migrating Very Large Site Collections (SPSDC)

kiwiboris

Storage Integrations for Container Orchestrators

{code} by Dell EMC

Lessons learned Building Nuxeo EP - Component-based, open source ECM platform

Nuxeo

Keeping consistent environments across your development, test, and production systems can be a complex task. Docker containers offer a way to develop and test your application in the same environment in which it runs in production. You can use tools such as the ECS CLI and Docker Compose for local testing of applications; Jenkins and AWS CodePipeline for building and workflow orchestration; Amazon EC2 Container Registry to store your container images; and Amazon EC2 Container Service to manage and scale containers. In this session, you will learn how to build containers into your development workflow and orchestrate container deployments using Amazon ECS. You will hear how Okta runs 30,000 tests per developer commit and releases 10,000 new lines of code each week to production with a CI system based on 100% AWS services. We'll also discuss how Okta uses ECS for parallelized testing in CI and for production microservices in a multi-region, always on cloud service.

AWS re:Invent 2016: Development Workflow with Docker and Amazon ECS (CON302)

Amazon Web Services

.net Based Component Technologies

prakashk453625

Extension Library - Viagra for XPages

Ulrich Krause

Docker for the enterprise

Bert Poller

What's New in .Net 4.5

Malam Team

Open Source TIMES Excel Reader

IEA-ETSAP

Community day 2013 applied architectures

Panagiotis Kefalidis

DevOps with Elastic Beanstalk - TCCC-2014

scolestock

UKLUG 2012 - XPages, Beyond the basics

Ulrich Krause

AWS Innovate: Moving Microsoft .Net applications one container at a time - Da...

Amazon Web Services Korea

AWS Elastic Beanstalk: Running Multi-Container Docker Applications - DevDay L...

Amazon Web Services

Similar a Updates on Veda provided by Amit Kanudia from KanORS-EMR (20)

Web Tools for GemStone/S

Journey towards serverless infrastructure

Lotus Domino 8.5

Azure Templates for Consistent Deployment

Microsoft ASP.NET 5 - The new kid on the block

XPages -Beyond the Basics

Migrating Very Large Site Collections (SPSDC)

Storage Integrations for Container Orchestrators

Lessons learned Building Nuxeo EP - Component-based, open source ECM platform

AWS re:Invent 2016: Development Workflow with Docker and Amazon ECS (CON302)

.net Based Component Technologies

Extension Library - Viagra for XPages

Docker for the enterprise

What's New in .Net 4.5

Open Source TIMES Excel Reader

Community day 2013 applied architectures

DevOps with Elastic Beanstalk - TCCC-2014

UKLUG 2012 - XPages, Beyond the basics

AWS Innovate: Moving Microsoft .Net applications one container at a time - Da...

AWS Elastic Beanstalk: Running Multi-Container Docker Applications - DevDay L...

Más de IEA-ETSAP

Variable Renewable Energy in China's Transition

IEA-ETSAP

The Nordics as a hub for green electricity and fuels

IEA-ETSAP

The role of Norwegian offshore wind in the energy system transition

IEA-ETSAP

Detail representation of molecule flows and chemical sector in TIMES-BE: prog...

IEA-ETSAP

Green hydrogen trade from North Africa to Europe: optional long-term scenario...

IEA-ETSAP

Optimal development of the Canadian forest sector for both climate change mit...

IEA-ETSAP

Presentation on IEA Net Zero Pathways/Roadmap

IEA-ETSAP

Flexibility with renewable(low-carbon) hydrogen

IEA-ETSAP

Bioenergy in energy system models with flexibility

IEA-ETSAP

Reframing flexibility beyond power - IEA Bioenergy TCP

IEA-ETSAP

Decarbonization of heating in the buildings sector: efficiency first vs low-c...

IEA-ETSAP

The Regionalization Tool: spatial representation of TIMES-BE output data in i...

IEA-ETSAP

Synthetic methane production prospective modelling up to 2050 in the European...

IEA-ETSAP

Energy Transition in global Aviation - ETSAP Workshop Turin

IEA-ETSAP

Integrated Energy and Climate plans: approaches, practices and experiences VO: reduce the distance between modellers and DM, VO: the work process - Making modifications collaboratively, - Running the model, - Reports and collaborative analysis VedaOnline Mr Rocco De Miglio 16–17th november 2023, amit kanudia, etsap meeting, etsap winter workshop, italy, kanors-emr, mr rocco de miglio, mr. amit kanudia kanors-emr, november 2023, politecnico di torino, semi-annual meeting, torino, turin, vedaonline

Integrated Energy and Climate plans: approaches, practices and experiences

IEA-ETSAP

Energy system modeling activities in the MAHTEP Group

IEA-ETSAP

Applying science fiction to approach the future

IEA-ETSAP

Will it leak?: Discussions of leakage risk from subsurface storage of carbon ...

IEA-ETSAP

Long-Term Decarbonization Pathways In Emerging Economies: Insights From 12 Mo...

IEA-ETSAP

A Compatible Energy Trilemma Evaluation Framework for Future Decarbonized, De...

IEA-ETSAP

Más de IEA-ETSAP (20)

Variable Renewable Energy in China's Transition

The Nordics as a hub for green electricity and fuels

The role of Norwegian offshore wind in the energy system transition

Detail representation of molecule flows and chemical sector in TIMES-BE: prog...

Green hydrogen trade from North Africa to Europe: optional long-term scenario...

Optimal development of the Canadian forest sector for both climate change mit...

Presentation on IEA Net Zero Pathways/Roadmap

Flexibility with renewable(low-carbon) hydrogen

Bioenergy in energy system models with flexibility

Reframing flexibility beyond power - IEA Bioenergy TCP

Decarbonization of heating in the buildings sector: efficiency first vs low-c...

The Regionalization Tool: spatial representation of TIMES-BE output data in i...

Synthetic methane production prospective modelling up to 2050 in the European...

Energy Transition in global Aviation - ETSAP Workshop Turin

Integrated Energy and Climate plans: approaches, practices and experiences

Energy system modeling activities in the MAHTEP Group

Applying science fiction to approach the future

Will it leak?: Discussions of leakage risk from subsurface storage of carbon ...

Long-Term Decarbonization Pathways In Emerging Economies: Insights From 12 Mo...

A Compatible Energy Trilemma Evaluation Framework for Future Decarbonized, De...

Último

A Comprehensive Look at Generative AI in Retail App Testing.pdf

kalichargn70th171

Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.

Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...

Globus

CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.

Cyaniclab : Software Development Agency Portfolio.pdf

Cyanic lab

OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam

takuyayamamoto1800

Science gateways allow science and engineering communities to access shared data, software, computing services, and instruments. Science gateways have gained a lot of traction in the last twenty years, as evidenced by projects such as the Science Gateways Community Institute (SGCI) and the Center of Excellence on Science Gateways (SGX3) in the US, The Australian Research Data Commons (ARDC) and its platforms in Australia, and the projects around Virtual Research Environments in Europe. A few mature frameworks have evolved with their different strengths and foci and have been taken up by a larger community such as the Globus Data Portal, Hubzero, Tapis, and Galaxy. However, even when gateways are built on successful frameworks, they continue to face the challenges of ongoing maintenance costs and how to meet the ever-expanding needs of the community they serve with enhanced features. It is not uncommon that gateways with compelling use cases are nonetheless unable to get past the prototype phase and become a full production service, or if they do, they don't survive more than a couple of years. While there is no guaranteed pathway to success, it seems likely that for any gateway there is a need for a strong community and/or solid funding streams to create and sustain its success. With over twenty years of examples to draw from, this presentation goes into detail for ten factors common to successful and enduring gateways that effectively serve as best practices for any new or developing gateway.

How to Position Your Globus Data Portal for Success Ten Good Practices

Globus

Using IESVE for Room Loads Analysis - Australia & New Zealand

IES VE

Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...

rajkumar669520

Even though at surface level ‘java.lang.OutOfMemoryError’ appears as one single error; underlyingly there are 9 types of OutOfMemoryError. Each type of OutOfMemoryError has different causes, diagnosis approaches and solutions. This session equips you with the knowledge, tools, and techniques needed to troubleshoot and conquer OutOfMemoryError in all its forms, ensuring smoother, more efficient Java applications.

TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR

Tier1 app

Data privacy is one of the most critical issues that businesses face. This presentation shares insights on the principles and best practices for ensuring the resilience and security of your workload. Drawing on a real-life project from the HR industry, the various challenges will be demonstrated: data protection, self-healing, business continuity, security, and transparency of data processing. This systematized approach allowed to create a secure AWS cloud infrastructure that not only met strict compliance rules but also exceeded the client's expectations.

Designing for Privacy in Amazon Web Services

KrzysztofKkol1

BoxLang: Review our Visionary Licenses of 2024

Ortus Solutions, Corp

How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?

XfilesPro

A Comprehensive Appium Guide for Hybrid App Automation Testing.pdf

kalichargn70th171

AI/ML Infra Meetup May. 23, 2024 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Lu Qiu (Data & AI Platform Tech Lead, @Alluxio) - Siyuan Sheng (Senior Software Engineer, @Alluxio) Speed and efficiency are two requirements for the underlying infrastructure for machine learning model development. Data access can bottleneck end-to-end machine learning pipelines as training data volume grows and when large model files are more commonly used for serving. For instance, data loading can constitute nearly 80% of the total model training time, resulting in less than 30% GPU utilization. Also, loading large model files for deployment to production can be slow because of slow network or storage read operations. These challenges are prevalent when using popular frameworks like PyTorch, Ray, or HuggingFace, paired with cloud object storage solutions like S3 or GCS, or downloading models from the HuggingFace model hub. In this presentation, Lu and Siyuan will offer comprehensive insights into improving speed and GPU utilization for model training and serving. You will learn: - The data loading challenges hindering GPU utilization - The reference architecture for running PyTorch and Ray jobs while reading data from S3, with benchmark results of training ResNet50 and BERT - Real-world examples of boosting model performance and GPU utilization through optimized data access

AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...

Alluxio, Inc.

The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.

Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...

Globus

Les Buildpacks existent depuis plus de 10 ans ! D’abord, ils étaient utilisés pour détecter et construire une application avant de la déployer sur certains PaaS. Ensuite, nous avons pu créer des images Docker (OCI) avec leur dernière génération, les Cloud Native Buildpacks (CNCF en incubation). Sont-ils une bonne alternative au Dockerfile ? Que sont les buildpacks Paketo ? Quelles communautés les soutiennent et comment ? Venez le découvrir lors de cette session ignite

Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...

Anthony Dahanne

Key takeaways: Challenges of building platforms and the benefits of platformless. Key principles of platformless, including API-first, cloud-native middleware, platform engineering, and developer experience. How Choreo enables the platformless experience. How key concepts like application architecture, domain-driven design, zero trust, and cell-based architecture are inherently a part of Choreo. Demo of an end-to-end app built and deployed on Choreo.

Accelerate Enterprise Software Engineering with Platformless

WSO2

As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.

Globus Compute wth IRI Workflows - GlobusWorld 2024

Globus

In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.

First Steps with Globus Compute Multi-User Endpoints

Globus

Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.

Enhancing Research Orchestration Capabilities at ORNL.pdf

Globus

Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos https://www.amb-review.com/tubetrivia-ai Exclusive Features: AI-Powered Questions, Wide Range of Categories, Adaptive Difficulty, User-Friendly Interface, Multiplayer Mode, Regular Updates. #TubeTriviaAI #QuizVideoMagic #ViralQuizVideos #AIQuizGenerator #EngageExciteExplode #MarketingRevolution #BoostYourTraffic #SocialMediaSuccess #AIContentCreation #UnlimitedTraffic

Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf

AMB-Review

Updates on Veda provided by Amit Kanudia from KanORS-EMR

1. Veda update ETSAP meeting Torino 16 November 2023

2. Free Veda online for students

3. How it works: • Identify a license admin for an institution abc@<inst name>.??? • All users who register with email addresses like xyz@<inst name>.??? will be able to access educational version of VO without any further check. • Runs will be solved on the NEOS server (3 GB RAM limit) • Can only make public models out of public GitHub folders

4. Next steps • Prepare an email that contracting parties can send to educational institutes in their countries. • Prepare demo models with self-learning as the objective. • Prepare video tutorials around these models. • Offer webinars.

5. Veda for open-source models

6. Different routes from input to results TIMES MARKAL OSeMOSYS TEMOA ?? Veda Input Files Veda Reports

7. OSeMOSYS • Common attributes mapped • Other attributes supported • Additional features that improve usability • TIMES interpolation options (1-5) processed by Veda • Case management supported • Last value retained as per scenario order

8. TIMES and OSeMOSYS Browse

9. Make TIMES more accessible • Ose_CSV folder in the root of Veda model folder would replace the VT_* files. • All other Veda files would work as they do. • Existing OSeMOSYS and TEMOA models will be able to run as TIMES with little (or very little) effort on the part of their developers.

10. Make energy system modelling easier • Create an environment where domain experts can explore the interplay of key assumptions (via large scenario experiments) using system models – without much investment in what lies under the hood. • Intuitive names for key input parameters and output variables that will work across multiple modelling frameworks.

11. Veda Assistant

12. Two routes • Introduce new parameters • Test existing tables

13. Future developments Create model users around rich and mature models

14. Limited sharing of complex models 1. Select a commit 2. Upload a parametric scenario file 3. Synchronize 4. Run 5. View reports

Updates on Veda provided by Amit Kanudia from KanORS-EMR

Recomendados

Recomendados

Más contenido relacionado

Similar a Updates on Veda provided by Amit Kanudia from KanORS-EMR

Similar a Updates on Veda provided by Amit Kanudia from KanORS-EMR (20)

Más de IEA-ETSAP

Más de IEA-ETSAP (20)

Último

Último (20)

Updates on Veda provided by Amit Kanudia from KanORS-EMR