Almost every web application at some point needs a way to upload or download files… and no one seems to enjoy building reliable and scalable upload/download servers… and for good reasons too! In fact, you’ll probably need to manage long-running connections and handle files that can be quite large (i.e videos). If you are running a fully serverless backend using API Gateway and Lambda, you probably know that you are limited in terms of payload size and execution time, so things get even more complicated there. In all these cases you should consider offloading this problem to S3 by using S3 pre-signed URLs. Pre-signed URLs are a fantastic tool to handle file download and upload directly in S3 in a managed and scalable fashion. But all that glitters is not gold and S3 pre-signed URLs come with quite a few gotchas… So in this talk, we will explore some use cases, see some potential implementations of S3 pre-signed URLs and uncover some of the gotchas that I discovered while using them. By the end of this talk, you should know exactly when to use pre-signed URLs and how to avoid most of the many mistakes I made with them!
2014년 9월부터 12월까지 3개월 단기 프로젝트로 진행되었던 교내 단기 프로젝트의 개발문서이다.
프로젝트를 진행하다보면 기획자로써 어떠한부분이 부족하고 보완해야하는지 많은 고민을 할 수 있어서 좋다.
많은 부분을 녹여내지는 못했지만 코어시스템부분은 내가 생각한데로 녹여낸것 같아 다행이다..
팀원들이 바쁜일정속에 이렇게 따라와주고 완성되어 매우 흡족!? 하다.
Hibernate is an object/relational mapping tool that maps objects to a relational database. The document provides an overview of key Hibernate concepts like the SessionFactory, Session, persistent and transient objects, and transactions. It also discusses Hibernate tools for mapping files, schema generation, code generation, and configuration via properties files. An example mapping of music tracks, artists, and comments is presented to demonstrate basic Hibernate functionality.
카카오 광고 플랫폼 MSA 적용 사례 및 API Gateway와 인증 구현에 대한 소개if kakao
황민호(robin.hwang) / kakao corp. DSP개발파트
---
최근 Spring Cloud와 Netflix OSS로 MSA를 구성하는 시스템 기반의 서비스들이 많아지는 추세입니다.
카카오에서도 작년에 오픈한 광고 플랫폼 모먼트에 Spring Cloud 기반의 MSA환경을 구성하여, API Gateway도 적용하였는데 1년 반 정도 운영한 경험을 공유할 예정입니다. 더불어 MSA 환경에서는 API Gateway를 통해 인증을 어떻게 처리하는지 알아보고 OAuth2 기반의 JWT Token을 이용한 인증에 대한 이야기도 함께 나눌 예정입니다.
2014년 9월부터 12월까지 3개월 단기 프로젝트로 진행되었던 교내 단기 프로젝트의 개발문서이다.
프로젝트를 진행하다보면 기획자로써 어떠한부분이 부족하고 보완해야하는지 많은 고민을 할 수 있어서 좋다.
많은 부분을 녹여내지는 못했지만 코어시스템부분은 내가 생각한데로 녹여낸것 같아 다행이다..
팀원들이 바쁜일정속에 이렇게 따라와주고 완성되어 매우 흡족!? 하다.
Hibernate is an object/relational mapping tool that maps objects to a relational database. The document provides an overview of key Hibernate concepts like the SessionFactory, Session, persistent and transient objects, and transactions. It also discusses Hibernate tools for mapping files, schema generation, code generation, and configuration via properties files. An example mapping of music tracks, artists, and comments is presented to demonstrate basic Hibernate functionality.
카카오 광고 플랫폼 MSA 적용 사례 및 API Gateway와 인증 구현에 대한 소개if kakao
황민호(robin.hwang) / kakao corp. DSP개발파트
---
최근 Spring Cloud와 Netflix OSS로 MSA를 구성하는 시스템 기반의 서비스들이 많아지는 추세입니다.
카카오에서도 작년에 오픈한 광고 플랫폼 모먼트에 Spring Cloud 기반의 MSA환경을 구성하여, API Gateway도 적용하였는데 1년 반 정도 운영한 경험을 공유할 예정입니다. 더불어 MSA 환경에서는 API Gateway를 통해 인증을 어떻게 처리하는지 알아보고 OAuth2 기반의 JWT Token을 이용한 인증에 대한 이야기도 함께 나눌 예정입니다.
Amazon Cognito와 함께 서버리스를..! - 이재일 (강남비기너모임) :: AWS Community Day 2017AWSKRUG - AWS한국사용자모임
AWS 자원을 효율적으로 이용하기 위해 Cognito를 사용하게 되었습니다. Cognito의 기능과 Cognito를 사용하면 좋은 점, 서비스를 개발하면서 생겼던 이슈 등을 발표합니다. 시연으로 Cognito + S3 + Lambda 를 이용한 서버리스 아키텍처 기반의 포토 갤러리 웹 서비스를 소개합니다.
Webinar: Working with Graph Data in MongoDBMongoDB
With the release of MongoDB 3.4, the number of applications that can take advantage of MongoDB has expanded. In this session we will look at using MongoDB for representing graphs and how graph relationships can be modeled in MongoDB.
We will also look at a new aggregation operation that we recently implemented for graph traversal and computing transitive closure. We will include an overview of the new operator and provide examples of how you can exploit this new feature in your MongoDB applications.
This document discusses API abuse and how to prevent it. It defines API abuse as misusing API functions for malicious activities like server raids or sending excessive requests. There are two main types: remote client impersonation and API flaw exploitation. It provides examples of API abuse at companies like Uber and Voi. To prevent API abuse, the document recommends authenticating that requests come from legitimate apps, checking the app and runtime environment, and using a cloud service to verify authentication rather than checking in the app. App authentication can serve as an additional security factor to prevent API abuse.
AWS Lambda 내부 동작 방식 및 활용 방법 자세히 살펴 보기 - 김일호 솔루션즈 아키텍트 매니저, AWS :: AWS Summit ...Amazon Web Services Korea
AWS Lambda 내부 동작 방식 및 활용 방법 자세히 살펴 보기
김일호 솔루션즈 아키텍트 매니저, AWS
AWS Lambda는 서버리스 아키텍처의 핵심 서비스입니다. 본 세션에서는 AWS Lambda에 내부 동작 방식을 소개하고, Lambda Layer, 맞춤형 런타임 등 신규로 소개된 기능 및 사용시 도움이 되는 성능 및 확장을 위한 다양한 팁들을 소개합니다.
오딘: 발할라 라이징 MMORPG의 성능 최적화 사례 공유 [카카오게임즈 - 레벨 300] - 발표자: 김문권, 팀장, 라이온하트 스튜디오...Amazon Web Services Korea
서비스 런칭을 위해 라이온하트와 카카오게임즈가 어떻게 최적 성능의 인스턴스를 선택하고, Windows 운영 체제를 최적화하며, 왜 Amazon Aurora를 기본 데이터베이스로 채택하였는지를 설명합니다. 또한, 출시부터 운영까지의 과정에서 MMORPG가 어떻게 AWS 상에서 설계되고, 게임 서버 성능을 극대할 수 있었는지에 대해 전달해드립니다.
MongoDB is a document-oriented NoSQL database written in C++. It uses a document data model and stores data in BSON format, which is a binary form of JSON that is lightweight, traversable, and efficient. MongoDB is schema-less, supports replication and high availability, auto-sharding for scaling, and rich queries. It is suitable for big data, content management, mobile and social applications, and user data management.
This document provides an overview and introduction to MongoDB, an open-source, high-performance NoSQL database. It outlines MongoDB's features like document-oriented storage, replication, sharding, and CRUD operations. It also discusses MongoDB's data model, comparisons to relational databases, and common use cases. The document concludes that MongoDB is well-suited for applications like content management, inventory management, game development, social media storage, and sensor data databases due to its flexible schema, distributed deployment, and low latency.
While it is quite common practice to do periodic security assessments of your local network, it is really rare to find a company who puts the same effort for testing the security in their cloud. We have to understand what new threats and risks appeared with the cloud and how should we change our attitude to testing cloud security. The goal of my presentation is to show how security assessment of cloud infrastructure it is different from testing environments in classic architecture. I'll demonstrate a hypothetical attack on a company which is fully deployed in the AWS environment. I’m going to show the whole kill chain starting from presenting cloud-applicable reconnaissance techniques. Then I’ll attack the web application server hosted on EC2 instance to access its metadata. Using the assigned role, I’ll access another AWS EC2 instance to escalate privileges to the administrator and then present how to hide fingerprints in CloudTrail service. Finally, I’ll demonstrate various techniques of silent exfiltrating data from AWS environment, setting up persistent access and describe another potential, cloud-specific threats, e.g. cryptojacking or ransomware in the cloud. The presentation shows practical aspects of attacking cloud services and each step of the kill chain will be presented in a form of an interactive, live demo. On the examples of presented attacks, I’ll show how to use AWS exploitation framework Pacu and other handy scripts.
Building a dev pipeline using GitHub Actions, Node.js, and AWS ECS Fargatedatree
How to build a real pipeline that will deliver features into production in minutes using Node.js web server application, NPM Koa.js, Docker, GitHub Actions as the CI/CD and AWS ECS Fargate.
Continuous Integration and Deployment Best Practices on AWS (ARC307) | AWS re...Amazon Web Services
With AWS, companies now have the ability to develop and run their applications with speed and flexibility like never before. Working with an infrastructure that can be 100 percent API driven enables businesses to use lean methodologies and realize these benefits. This in turn leads to greater success for those who make use of these practices. In this session, we talk about some key concepts and design patterns for continuous deployment and continuous integration, two elements of lean development of applications and infrastructures.
Amazon Cognito와 함께 서버리스를..! - 이재일 (강남비기너모임) :: AWS Community Day 2017AWSKRUG - AWS한국사용자모임
AWS 자원을 효율적으로 이용하기 위해 Cognito를 사용하게 되었습니다. Cognito의 기능과 Cognito를 사용하면 좋은 점, 서비스를 개발하면서 생겼던 이슈 등을 발표합니다. 시연으로 Cognito + S3 + Lambda 를 이용한 서버리스 아키텍처 기반의 포토 갤러리 웹 서비스를 소개합니다.
Webinar: Working with Graph Data in MongoDBMongoDB
With the release of MongoDB 3.4, the number of applications that can take advantage of MongoDB has expanded. In this session we will look at using MongoDB for representing graphs and how graph relationships can be modeled in MongoDB.
We will also look at a new aggregation operation that we recently implemented for graph traversal and computing transitive closure. We will include an overview of the new operator and provide examples of how you can exploit this new feature in your MongoDB applications.
This document discusses API abuse and how to prevent it. It defines API abuse as misusing API functions for malicious activities like server raids or sending excessive requests. There are two main types: remote client impersonation and API flaw exploitation. It provides examples of API abuse at companies like Uber and Voi. To prevent API abuse, the document recommends authenticating that requests come from legitimate apps, checking the app and runtime environment, and using a cloud service to verify authentication rather than checking in the app. App authentication can serve as an additional security factor to prevent API abuse.
AWS Lambda 내부 동작 방식 및 활용 방법 자세히 살펴 보기 - 김일호 솔루션즈 아키텍트 매니저, AWS :: AWS Summit ...Amazon Web Services Korea
AWS Lambda 내부 동작 방식 및 활용 방법 자세히 살펴 보기
김일호 솔루션즈 아키텍트 매니저, AWS
AWS Lambda는 서버리스 아키텍처의 핵심 서비스입니다. 본 세션에서는 AWS Lambda에 내부 동작 방식을 소개하고, Lambda Layer, 맞춤형 런타임 등 신규로 소개된 기능 및 사용시 도움이 되는 성능 및 확장을 위한 다양한 팁들을 소개합니다.
오딘: 발할라 라이징 MMORPG의 성능 최적화 사례 공유 [카카오게임즈 - 레벨 300] - 발표자: 김문권, 팀장, 라이온하트 스튜디오...Amazon Web Services Korea
서비스 런칭을 위해 라이온하트와 카카오게임즈가 어떻게 최적 성능의 인스턴스를 선택하고, Windows 운영 체제를 최적화하며, 왜 Amazon Aurora를 기본 데이터베이스로 채택하였는지를 설명합니다. 또한, 출시부터 운영까지의 과정에서 MMORPG가 어떻게 AWS 상에서 설계되고, 게임 서버 성능을 극대할 수 있었는지에 대해 전달해드립니다.
MongoDB is a document-oriented NoSQL database written in C++. It uses a document data model and stores data in BSON format, which is a binary form of JSON that is lightweight, traversable, and efficient. MongoDB is schema-less, supports replication and high availability, auto-sharding for scaling, and rich queries. It is suitable for big data, content management, mobile and social applications, and user data management.
This document provides an overview and introduction to MongoDB, an open-source, high-performance NoSQL database. It outlines MongoDB's features like document-oriented storage, replication, sharding, and CRUD operations. It also discusses MongoDB's data model, comparisons to relational databases, and common use cases. The document concludes that MongoDB is well-suited for applications like content management, inventory management, game development, social media storage, and sensor data databases due to its flexible schema, distributed deployment, and low latency.
While it is quite common practice to do periodic security assessments of your local network, it is really rare to find a company who puts the same effort for testing the security in their cloud. We have to understand what new threats and risks appeared with the cloud and how should we change our attitude to testing cloud security. The goal of my presentation is to show how security assessment of cloud infrastructure it is different from testing environments in classic architecture. I'll demonstrate a hypothetical attack on a company which is fully deployed in the AWS environment. I’m going to show the whole kill chain starting from presenting cloud-applicable reconnaissance techniques. Then I’ll attack the web application server hosted on EC2 instance to access its metadata. Using the assigned role, I’ll access another AWS EC2 instance to escalate privileges to the administrator and then present how to hide fingerprints in CloudTrail service. Finally, I’ll demonstrate various techniques of silent exfiltrating data from AWS environment, setting up persistent access and describe another potential, cloud-specific threats, e.g. cryptojacking or ransomware in the cloud. The presentation shows practical aspects of attacking cloud services and each step of the kill chain will be presented in a form of an interactive, live demo. On the examples of presented attacks, I’ll show how to use AWS exploitation framework Pacu and other handy scripts.
Building a dev pipeline using GitHub Actions, Node.js, and AWS ECS Fargatedatree
How to build a real pipeline that will deliver features into production in minutes using Node.js web server application, NPM Koa.js, Docker, GitHub Actions as the CI/CD and AWS ECS Fargate.
Continuous Integration and Deployment Best Practices on AWS (ARC307) | AWS re...Amazon Web Services
With AWS, companies now have the ability to develop and run their applications with speed and flexibility like never before. Working with an infrastructure that can be 100 percent API driven enables businesses to use lean methodologies and realize these benefits. This in turn leads to greater success for those who make use of these practices. In this session, we talk about some key concepts and design patterns for continuous deployment and continuous integration, two elements of lean development of applications and infrastructures.
This document provides an overview of continuous integration and deployment best practices on AWS. It discusses what continuous integration is and how it helps with rapid development by making changes and deployments iterative rather than monolithic. This allows bugs to be detected quickly. The document then discusses tools that can be used to implement continuous integration, such as AWS services, configuration management systems like Puppet, deployment frameworks like AWS Elastic Beanstalk, and infrastructure management tools like AWS CloudFormation. It also provides tips for scaling tools like Puppet masters and optimizing continuous integration and deployment workflows.
Say you have an existing app that uses Firebase. But now you want to add payment processing, image processing, send push notifications, or other functionality that really can't be done in the app itself. How can you do these things without spinning up your own servers? Firebase has you covered. In this codelab you learn how to write JavaScript functions that run in response to events that happen in Firebase. You then deploy these functions to Cloud Functions for Firebase, where they run auto-scaled on Google's infrastructure. To get the most value out of attending, be sure to have Node.js and npm installed on your machine along with your favorite text editor.
Serverless computing allows developers to run code without managing servers. It is billed based on usage rather than on servers. Key serverless services include AWS Lambda for compute, S3 for storage, and DynamoDB for databases. While new, serverless offers opportunities to reduce costs and focus on code over infrastructure. Developers must learn serverless best practices for lifecycle management, organization, and hands-off operations. The Serverless Framework helps develop and deploy serverless applications.
Serverless in production, an experience report (FullStack 2018)Yan Cui
This document discusses considerations for making serverless applications production ready. It covers topics like testing, monitoring, logging, deployment pipelines, performance optimization, and security. The document emphasizes principles over specific tools, and recommends focusing on shipping working software through practices like embracing external services for testing instead of mocking.
Serverless is great for web applications and APIs, but this does not mean it cannot be used successfully for other use cases. In this talk, we will discuss a successful application of serverless in the field of High Performance Computing. Specifically we will discuss how Lambda, Fargate, Kinesis and other serverless technologies are being used to run sophisticated financial models at one of the major reinsurance companies in the World. We we learn about the architecture, the tradeoffs, some challenges and some unresolved pain points. Most importantly, we'll find out if serverless can be a great fit for HPC and if we can finally stop managing those boring EC2 instances!
This presentation talks about an Ekino (www.ekino.com) application made for Microsoft Techdays 2012 in Paris.
This application is based on many new technologies : HTML5, CSS3, IndexedDB, File API, D'n'D, SPA, IE10.
This document summarizes a presentation about building a single page image editing application called SnapyX using HTML5 and related technologies. It describes using HTML5 features like canvas and File API for image uploading and processing, CSS3 grids for layout, and Backbone.js with plugins for the client-side architecture including offline storage with IndexedDB. It acknowledges challenges with experimental browsers and technologies but is proud of publishing an app to the Windows Store.
The document discusses AWS Glue, a fully managed ETL service. It provides an overview of Glue's programming environment and data processing model. It then gives several examples of optimizing Glue job performance, including processing many small files, a few large files, optimizing parallelism with JDBC partitions, Python performance, and using the new Python shell job type.
Adopt openjdk and how it impacts you in 2020George Adams
AdoptOpenJDK is rapidly becoming a leading provider of OpenJDK™ binaries. With over 125 million downloads in the last year, it is now a serious contender for your production usage of Java™. AdoptOpenJDK provides prebuilt OpenJDK™ binaries from a fully open-source set of build scripts and infrastructure. This talk will cover how we build on over 15 different platforms, execute over 60,000 tests and distribute OpenJDK™ binaries to millions of users. We will also cover how AdoptOpenJDK binaries compare against the Java™ binaries that you use today. If you’re curious to understand more about our future roadmap, security and supporting platforms like Lego® Mindstorms® then this is the talk for you!
Gojko's 5 rules for super responsive Serverless applicationsDaniel Zivkovic
Gojko Adzic (#AWS Serverless Hero, Trainer, Entrepreneur & Book Author) shares 5 important Architectural ideas to make request processing lightning fast with #Serverless deployments. Video at https://youtu.be/XLLdWYdJ4Vw
P.S. For more interactive lectures like this, go to http://youtube.serverlesstoronto.org/ or sign up for our upcoming live events at https://www.meetup.com/Serverless-Toronto/events/
Serverless in Production, an experience report (AWS UG South Wales)Yan Cui
AWS Lambda has changed the way we deploy and run software, but this new serverless paradigm has created new challenges to old problems - how do you test a cloud-hosted function locally? How do you monitor them? What about logging and config management? And how do we start migrating from existing architectures?
In this talk Yan and Scott will discuss solutions to these challenges by drawing from real-world experience running Lambda in production and migrating from an existing monolithic architecture.
This document provides instructions for a DevSecCon workshop on securing secrets in development pipelines. The workshop aims to help developers and managers address the problem of hardcoded secrets being checked into code repositories by demonstrating how to remove and rotate secrets using a secret management server integrated with Jenkins. Participants will carry out hands-on labs to find hardcoded secrets, replace them with environment variables stored in a secret server, and prevent secrets from being exposed in builds.
IDEALIZE 2023 - NodeJS & Firebase SessionBrion Mario
Node.js Firebase
This document discusses Node.js, RESTful APIs, and Firebase. It provides an overview of Node.js and its event loop model. It also explains what npm and RESTful APIs are. Finally, it demonstrates how to build a basic RESTful API with Node.js and store data in Firestore using Firebase.
This document discusses building a Slack bot using AWS Lambda and the Chalice framework. It describes how FaaS works, options for running functions through AWS Lambda including Python support. It then outlines how to build a Slack bot with Chalice that allows users to query Stack Overflow through natural language requests in Slack. Key steps include setting up the bot to handle requests, retrieving secrets securely, formatting responses, and deploying the code to AWS Lambda to be accessible through Slack. It also briefly discusses additional uses of FaaS beyond a basic bot including cron jobs and handling external events.
Serverless is great for web applications and APIs, but this does not mean it cannot be used successfully for other use cases. In this talk, we will discuss a successful application of serverless in the field of High Performance Computing. Specifically we will discuss how Lambda, Fargate, Kinesis and other serverless technologies are being used to run sophisticated financial models at one of the major reinsurance companies in the World. We we learn about the architecture, the tradeoffs, some challenges and some unresolved pain points. Most importantly, we'll find out if serverless can be a great fit for HPC and if we can finally stop managing those boring EC2 instances!
Serverless in production, an experience reportYan Cui
AWS Lambda has changed the way we deploy and run software, but this new serverless paradigm has created new challenges to old problems - how do you test a cloud-hosted function locally? How do you monitor them? What about logging and config management? And how do we start migrating from existing architectures?
In this talk Yan and Scott will discuss solutions to these challenges by drawing from real-world experience running Lambda in production and migrating from an existing monolithic architecture.
Similar a Everything I know about S3 pre-signed URLs (20)
Did you know JavaScript has iterators? DublinJSLuciano Mammino
How many ways do you know to do iteration with JavaScript and Node.js? While, for loop, for…in, for..of, .map(), .forEach(), streams, iterators, etc! Yes, there are a lot of ways! But did you know that JavaScript has iteration protocols to standardise synchronous and even asynchronous iteration? In this workshop we will learn about these protocols and discover how to build iterators and iterable objects, both synchronous and asynchronous. We will learn about some common use cases for these protocols, explore generators and async generators (great tools for iteration) and finally discuss some hot tips, common pitfalls, and some (more or less successful) wild ideas!
What I learned by solving 50 Advent of Code challenges in Rust - RustNation U...Luciano Mammino
The document discusses using iterator combinators in Rust to solve an Advent of Code puzzle in a more functional style compared to a classic imperative approach. It shows code to parse input data into batches, map lines to values, sum the values, and take the maximum. It then extends this to sort batches descending and take the top 3 sums to solve part 2 of the puzzle in a flexible way using combinators.
Building an invite-only microsite with Next.js & Airtable - ReactJS MilanoLuciano Mammino
The document outlines steps to build an invite-only website using Next.js and Airtable. It discusses:
1) Organizing guest data like name and favorite color in an Airtable database.
2) Setting up a Next.js project and retrieving invite data from Airtable using its API and creating an Invite interface.
3) Creating a Next.js API to fetch invite data based on an invite code parameter.
A design pattern provides a general reusable solution for the common problems that occur in software design. The pattern typically shows relationships and interactions between classes or objects. The idea is to speed up the development process by providing well-tested, proven development/design paradigms. Design patterns come in three different categories. Creational patterns include the generation of object instances. Structure refers to how an object is made and how things behave and interact In this Webinar(Live Meetup) we will be covering - What is node js - When to use node js - Async I/O operations in node js - Advantages of Async/Await - Some interesting - async patterns - Performance comparison
Let's build a 0-cost invite-only website with Next.js and Airtable!Luciano Mammino
The document describes building an invite-only website using Next.js and Airtable. It outlines:
1) Organizing guest data like names and preferences in an Airtable database.
2) Creating a Next.js project and APIs to retrieve invite data from Airtable by invite code.
3) Authenticating access to Next.js pages by checking for a valid invite code parameter.
JavaScript Iteration Protocols - Workshop NodeConf EU 2022Luciano Mammino
The document discusses JavaScript iteration protocols including generators, iterators, iterables, async iterators, and async iterables.
It provides examples of using generator functions to create custom iterables. It explains that generator objects are both iterators and iterables.
The document also explains the iterator and iterable protocols, defining what makes an object an iterator and what makes an object iterable. It shows how to implement these protocols using generator functions, anonymous objects, and classes.
Finally, it covers async iteration protocols, showing how to create async iterators and async iterables, typically using async generator functions. It provides examples of built-in Node.js async iterables like readable streams.
Building an invite-only microsite with Next.js & AirtableLuciano Mammino
Imagine you are hosting a private event and you want to create a website to invite all your guests. Of course you'd like to have an easy way to just share a URL with every guest and they should be able to access all the details of the event. Everyone else should not be allowed to see the page. Even nicer if the website is customized for every guest and if you could use the same website to collect information from the guests (who is coming and who is not). Ok, how do we build all of this? But, most importantly, how do we build it quickly? How do we keep it simple and possibly host it 100% for FREE? I had to do something like this recently so, in this talk, I am going to share my solution, which involves a React SPA (built with Next.js & Vercel) and AirTable as a backend!
How can you take an existing monolith to the cloud with very minimal effort? In this talk we will explore an architecture that can help you to achieve that while focusing on scalability and resilience.
A look inside the European Covid Green Certificate - Rust DublinLuciano Mammino
When I saw how dense the European Covid Green Pass QR code is, I got immediately curious: "WOW, there must be a lot of interesting data in here". So, I started to dig deeper and I found that there's really a great wealth of interesting encoding and verification technologies being used in it! In this talk, I will share what I learned! We will go on a journey where we will explore Base54 encoding, COSE tokens, CBOR serialization, elliptic curve crypto, and much more! Finally, I will also show you how to write a decoder for Green Pass certificates in the most hyped language ever: Rust!
How can you take an existing monolith to the cloud with very minimal effort? In this talk we will explore an architecture that can help you to achieve that while focusing on scalability and resilience.
A presentation by Luciano Mammino(@loige) about what it takes to be recognised as a Senior Software Engineer:
The goal of this slides deck is to:
- Discuss what is expected from a senior software engineer (skills, mindset, duties)
- Packed with opinions (my own and more)
- For devs to be inspired
- For managers to hire, set expectations, support, and evaluate
Node.js: scalability tips - Azure Dev Community VijayawadaLuciano Mammino
You finally built that amazing start-up idea you had in mind for years and you did it using Node.js! That's Great! You just launched it on Hacker News and you are very happy and proud... but now more and more people are using it and you start to have a nasty fear that Node.js won't scale because you now... it's single-threaded! Is your project doomed now? Do you have to invest your time on rewriting it in something like C++ or maybe Rust or even Go? You'd rather invest your time on adding valuable features for your users rather than learning a new language and rewriting everything from scratch, but what if nothing works anymore? And... by the way, what the heck "single-threaded" really means?! Fear no more, dear fellow developer! In this talk, we will discuss the architecture of Node.js going through its strengths and its weaknesses. We will then talk about scalability and I will share some valuable tips and tricks to make your Node.js app scale! Spoiler alert: you probably won't need Go or Rust :)
A look inside the European Covid Green Certificate (Codemotion 2021)Luciano Mammino
When I saw how dense the European Covid Green Pass QR code is, I got immediately curious: "WOW, there must be a lot of interesting data in here". So, I started to dig deeper and I found that there's really a great wealth of interesting encoding and verification technologies being used in it! In this talk, I will share what I learned! We will go on a journey where we will explore Base54 encoding, COSE tokens, CBOR serialization, elliptic curve crypto, and much more! Finally, I will also show you how to write a decoder for Green Pass certificates in the most hyped language ever: Rust!
1. The document discusses observability in AWS and introduces a tool called SLIC Watch that automates the configuration of CloudWatch alarms and dashboards for serverless applications.
2. SLIC Watch generates CloudFormation templates that set up application-specific dashboards and alarms using metrics from services like Lambda, DynamoDB, and API Gateway to help operators more quickly identify issues.
3. The document provides an example of how SLIC Watch could help diagnose issues like DynamoDB throttling and Lambda timeouts by automating the creation of relevant metrics and alarms without requiring manual configuration of CloudWatch.
Semplificare l'observability per progetti ServerlessLuciano Mammino
Hai mai pensato che le tue lambda functions possano fallire senza che tu te ne accorga? Se la risposta é "SI" probabilmente é perché ti sei giá "bruciato" giocando con il cloud, dove errori e fallimenti sono sempre dietro l'angolo. Purtroppo non possiamo prevenire tutti i fallimenti, pero' possiamo essere notificati quando qualcosa va storto cosí da poter reagire tempestivamente. Ma come fare a configurare il nostro ambiente AWS per raggiungere un buon livello di "Observability"? Se hai giá provato ad utilizzare CloudWatch saprai giá quanto possa essere complesso. In questo talk, esploreremo il tema dell'observability per applicazioni Serverless su AWS. Discuteremo problemi e best practices. Infine vi proporró un tool che permette di automatizzare la configurazione di CloudWatch per l'80% delle esigenze in pochi minuti!
Finding a lost song with Node.js and async iterators - NodeConf Remote 2021Luciano Mammino
Did you ever get that feeling when a random song pops into your brain and you can’t get rid of it? Well, that happened to me recently and I couldn’t even remember the title of the damn song! In this talk, I want to share with you the story of how I was able to recover the details of the song by navigating some music-related APIs using JavaScript, Node.js and the magic of async iterators!
Finding a lost song with Node.js and async iterators - EnterJS 2021Luciano Mammino
The document discusses using the Last.fm API and Node.js async iterators to search a user's listening history when only a fragment of a song title is remembered. It describes making requests to retrieve the user's recent tracks paginated over time windows to avoid duplicate results. Code examples show iterating over the pages as a stream to expose the track data for processing.
Most AWS APIs will have limits on the amount of data you can send in one request and sometimes you really need to send a lot of data! To try to maximise the amount of data you can send, while still staying within the limits, some APIs support sending gzip-compressed payloads. But how can you send a gzipped request when using the Python SDK for AWS (boto3)? Well, I needed to answer this question recently and it turned out not to be as easy as I anticipated… Let’s jump into this rabbit hole together and let’s find out the answer!
Finding a lost song with Node.js and async iteratorsLuciano Mammino
Did you ever get that feeling when a random song pops into your brain and you can’t get rid of it? Well, that happened to me recently and I couldn’t even remember the title of the damn song! In this talk, I want to share with you the story of how I was able to recover the details of the song by navigating some music-related APIs using JavaScript, Node.js and the magic of async iterators!
Good observability is a must-have for modern AWS applications. CloudWatch gives you the tools but leaves you with a lot of work to do. To accelerate this, we created a serverless plugin to quickly build best practice alarms and dashboards.
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Speck&Tech
ABSTRACT: A prima vista, un mattoncino Lego e la backdoor XZ potrebbero avere in comune il fatto di essere entrambi blocchi di costruzione, o dipendenze di progetti creativi e software. La realtà è che un mattoncino Lego e il caso della backdoor XZ hanno molto di più di tutto ciò in comune.
Partecipate alla presentazione per immergervi in una storia di interoperabilità, standard e formati aperti, per poi discutere del ruolo importante che i contributori hanno in una comunità open source sostenibile.
BIO: Sostenitrice del software libero e dei formati standard e aperti. È stata un membro attivo dei progetti Fedora e openSUSE e ha co-fondato l'Associazione LibreItalia dove è stata coinvolta in diversi eventi, migrazioni e formazione relativi a LibreOffice. In precedenza ha lavorato a migrazioni e corsi di formazione su LibreOffice per diverse amministrazioni pubbliche e privati. Da gennaio 2020 lavora in SUSE come Software Release Engineer per Uyuni e SUSE Manager e quando non segue la sua passione per i computer e per Geeko coltiva la sua curiosità per l'astronomia (da cui deriva il suo nickname deneb_alpha).
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Project Management Semester Long Project - Acuityjpupo2018
Acuity is an innovative learning app designed to transform the way you engage with knowledge. Powered by AI technology, Acuity takes complex topics and distills them into concise, interactive summaries that are easy to read & understand. Whether you're exploring the depths of quantum mechanics or seeking insight into historical events, Acuity provides the key information you need without the burden of lengthy texts.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
5th LF Energy Power Grid Model Meet-up SlidesDanBrown980551
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
Fueling AI with Great Data with Airbyte WebinarZilliz
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
Webinar: Designing a schema for a Data WarehouseFederico Razzoli
Are you new to data warehouses (DWH)? Do you need to check whether your data warehouse follows the best practices for a good design? In both cases, this webinar is for you.
A data warehouse is a central relational database that contains all measurements about a business or an organisation. This data comes from a variety of heterogeneous data sources, which includes databases of any type that back the applications used by the company, data files exported by some applications, or APIs provided by internal or external services.
But designing a data warehouse correctly is a hard task, which requires gathering information about the business processes that need to be analysed in the first place. These processes must be translated into so-called star schemas, which means, denormalised databases where each table represents a dimension or facts.
We will discuss these topics:
- How to gather information about a business;
- Understanding dictionaries and how to identify business entities;
- Dimensions and facts;
- Setting a table granularity;
- Types of facts;
- Types of dimensions;
- Snowflakes and how to avoid them;
- Expanding existing dimensions and facts.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.
HCL Notes and Domino License Cost Reduction in the World of DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
GraphRAG for Life Science to increase LLM accuracyTomaz Bratanic
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
Digital Marketing Trends in 2024 | Guide for Staying AheadWask
https://www.wask.co/ebooks/digital-marketing-trends-in-2024
Feeling lost in the digital marketing whirlwind of 2024? Technology is changing, consumer habits are evolving, and staying ahead of the curve feels like a never-ending pursuit. This e-book is your compass. Dive into actionable insights to handle the complexities of modern marketing. From hyper-personalization to the power of user-generated content, learn how to build long-term relationships with your audience and unlock the secrets to success in the ever-shifting digital landscape.
11. Always re-imagining
We are a pioneering technology consultancy
focused on AWS and serverless
| |
Accelerated Serverless AI as a Service Platform Modernisation
loige
✉Reach out to us at
😇We are always looking for talent:
hello@fourTheorem.com
fth.link/careers
11
12. We host a weekly podcast about AWS
awsbites.com
loige 12
16. loige
Structure of an HTTP request
POST /profilepic/upload HTTP/1.1
Host: api.meower.com
Content-Type: text/plain
Content-Length: 9
Some data
Method
Path Version
Headers
Body
16
17. loige
What if it's a binary (like a picture)?
PUT /profilepic/upload HTTP/1.1
Host: api.meower.com
Content-Type: image/jpeg
Content-Length: 2097852
����JFIFHH������"��
���Dl��FW�'6N�()H�'p��FD3 [...]
read 2097852
bytes
17
41. loige
⚠VERY important details!
I lied to you a little in those diagrams... 🤥
It's a decent mental model, but it's not accurate 😅
The server never really talks with S3!
The server actually creates the signed URL by itself!
We will see later what's the security model around this idea!
41
42. loige
Is this a good solution? 🙄
✅It's a managed feature (a.k.a. no servers to manage)
✅We can upload and download arbitrarily big files with no practical limits*
✅Reasonably simple and secure
👍Seems good to me!
* objects in S3 are "limited" to 5TB (when using multi-part upload), 5 GB otherwise.
42
43. loige
Generating our first pre-signed URL
$ aws s3 presign
s3://finance-department-bucket/2022/tax-certificate.pdf
https://s3.amazonaws.com/finance-department-bucket/2022/tax-
certificate.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-
Credential=AKIA3SGQVQG7FGA6KKA6%2F20221104%2Fus-east-
1%2Fs3%2Faws4_request&X-Amz-Date=20221104T140227Z&X-Amz-
Expires=3600&X-Amz-SignedHeaders=host&X-Amz-
Signature=b228dbec8c1008c80c162e1210e4503dceead1e4d4751b4d9787
314fd6da4d55
Whoever has this URL can
download the tax certificate!
43
44. loige
What's in a pre-signed URL
https://s3.amazonaws.com/finance-department-bucket/2022/tax-
certificate.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-
Credential=AKIA3SGQVQG7FGA6KKA6%2F20221104%2Fus-east-
1%2Fs3%2Faws4_request&X-Amz-Date=20221104T140227Z&X-Amz-
Expires=3600&X-Amz-SignedHeaders=host&X-Amz-
Signature=b228dbec8c1008c80c162e1210e4503dceead1e4d4751b4d9787
314fd6da4d55
44
45. loige
What's in a pre-signed URL
https://s3.amazonaws.com
/finance-department-bucket
/2022/tax-certificate.pdf
?X-Amz-Algorithm=AWS4-HMAC-SHA256
&X-Amz-Credential=AKIA3SGQXQG7XXXYKKA6%2F20221104...
&X-Amz-Date=20221104T140227Z
&X-Amz-Expires=3600
&X-Amz-SignedHeaders=host
&X-Amz-Signature=b228dbec8c1008c80c162e1210e4503dceead1e4d4...
What if I change this to
/passwords.txt?
45
48. loige
🤓
Once a pre-signed URL is generated you
cannot edit it without breaking it
Photo by on
CHUTTERSNAP Unsplash
⚠Also note that you can use a pre-signed URL as many
times as you want until it expires
48
49. loige
🔐Permissions
Anyone with valid credentials can create a pre-signed URL (client side)
valid credentials = Role, User, or Security Token
The generated URL inherits the permissions of the credentials used to generate it
This means you can generate pre-signed URLs for things you don't have access to 😅
49
50. loige
$ aws s3 presign s3://ireland/i-love-you
https://ireland.s3.eu-west-1.amazonaws.com/i-love-you?X-Amz-
Algorithm=AWS4-HMAC-SHA256&X-Amz-
Credential=AKIA3ABCVQG7FGA6KKA6%2F20221115%2Feu-west-
1%2Fs3%2Faws4_request&X-Amz-Date=20221115T182036Z&X-Amz-
Expires=3600&X-Amz-SignedHeaders=host&X-Amz-
Signature=75749c92d94d03e411e7bbf64419f2af09301d1791b0df54c639
137c715f7888
😱
I swear I don't even know if this
bucket exists or who owns it!
50
56. loige
PUT Method
PUT <preSignedURL> HTTP/1.1
Host: <bucket>.s3.<region>.amazonaws.com
Content-Length: 2097852
����JFIFHH������"��
���Dl��FW�'6N�()H�'p��FD3 [...]
56
57. import { S3Client, GetObjectCommand } from '@aws-sdk/client-s3'
import { getSignedUrl } from '@aws-sdk/s3-request-presigner'
const s3Client = new S3Client()
const command = new PutObjectCommand({
Bucket: "some-bucket",
Key: "some-object"
})
const preSignedUrl = await getSignedUrl(s3Client, command, {
expiresIn: 3600
})
console.log(preSignedUrl)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
loige
Only difference with the
previous example
57
58. loige
PUT Method - Limitations
You cannot set a limit on the upload size (max of 5 GB)! *
You can limit the Content-Type but you can specify exactly one
* Unless you know the exact size in advance
58
59. loige
POST method
It uses the multipart/form-data encoding (form upload)
Gives more freedom to the client to shape the request (Content-Type, file name, etc)
It uses a policy mechanism to define the "rules" of what can be uploaded
E.g. you can limit the supported mime types and provide a maximum file size
You can use it to upload from a web form and even configure the redirect URL
It's not really a URL but more of a pre-signed form!
59
61. loige
POST method Policy
A JSON object (Base64 encoded) that defines the upload rules (conditions) and the
expiration date
This is what gets signed: you cannot alter the policy without breaking the signature
{
"expiration": "2022-11-15T20:46:37Z",
"conditions": [
["content-length-range", 0, 5242880],
["starts-with", "$Content-Type", "image/"],
{"bucket": "somebucket"},
{"X-Amz-Algorithm": "AWS4-HMAC-SHA256"},
{"X-Amz-Credential": "AKIA3SGABCDXXXA6KKA6/20221115/eu-west-1/s3/aws4_request"},
{"X-Amz-Date": "20221115T194637Z"},
{"key": "picture.jpg"}
]
}
61
63. // you can use `url` and `fields` to generate an HTML form
const code = `<h1>Upload an image to S3</h1>
<form action="${url}" method="post" enctype="multipart/form-data">
${Object.entries(fields).map(([key, value]) => {
return `<input type="hidden" name="${key}" value="${value.replace(/"/g, '"')}">`
}).join('n')}
<div><input type="file" name="file" accept="image/png"></div>
<div><input type="submit" value="Upload"></div>
</form>`
1
2
3
4
5
6
7
8
9
10
loige 63
64. loige
Limitations and quirks
It supports only 1 file (cannot upload multiple files in one go)
The file field must be the last entry in the form
(S3 will ignore every other field after the file)
From the browser (AJAX) you need to enable CORS on the bucket
64
65. loige
Should I use PUT or POST? 🧐
PUT is simpler but definitely more limited
POST is slightly more complicated (and less adopted) but it's more flexible
You should probably put some time into learning POST and use that!
65
66. loige
Pre-signed URLs for other operations
S3 pre-signed URLs are not limited to GET, PUT or POST operations
You can literally create pre-signed URLs for any command
(DeleteObject, ListBuckets, MultiPartUpload, etc...)
66
67. loige
Do you need moar examples? 😼
github.com/lmammino/s3-presigned-urls-examples
67
68. loige
... In summary
S3 pre-signed URLs are a great way to authorise operations on S3
They are generally used to implement upload/download features
The signature is created client-side so you can sign anything. Access is validated at
request time
This is not the only solution, you can also use the JavaScript SDK from the frontend
and get limited credentials from Cognito (Amplify makes that process simpler)
For upload you can use PUT and POST, but POST is much more flexible
💬PS: Meower.com doesn't really exist... but... do you want to invest?! It's a great
idea, trust me!
68
69. Cover photo by on
Kelly Sikkema Unsplash
fourtheorem.com
THANKS! 🙌
fth.link/presign
loige
It's a wrap!
69