SlideShare a Scribd company logo
1 of 18
Download to read offline
© 2014 EnterpriseDB Corporation. All rights reserved. 1
Technical Case Studies in PostgreSQL
[ Monitoring Methods ]
Rohit Vyas | 11.04.15
Quality Management Group (PostgreSQL Installer and Tools)
© 2014 EnterpriseDB Corporation. All rights reserved. 2
• Technical cases in PostgreSQL
• Database Monitoring Methods
Agenda
© 2014 EnterpriseDB Corporation. All rights reserved. 3
− Bloats
− Updates/Deletes cause bloats in a table in
PostgreSQL.
− Transaction Wraparound
− No new XID’s if 4 billion mark reached.
− Streaming/WAL Replication Lag
− Number of bytes lag from Master database may
be due to bulk load, network etc.,
− Handling WAL Archiving Files
− Piling up .ready files due to failure of archiver
process
− Catalog Corruption
− Database objects repository(pg_catalog schema)
corruption.
Technical Cases on Production Database
ERROR: database is not accepting commands to avoid
wraparound data loss in database "dvnspatial"
Hint: Stop the postmaster and use a standalone
backend to vacuum that database.
You might also need to commit or roll back old prepared
transactions.
Error executing update (updateorg.matter.security.User)
PANIC: could not write to file
“pg_xlog/xlogtemp.556” :No space left on
device
notice: pg_class row has invalid relnamespace "24580":
no matching entry in pg_namespace
row identity: oid="24581" relname="foo" relkind="r"
© 2014 EnterpriseDB Corporation. All rights reserved. 4
• Using pg_catalog Tables/Views/Functions
• Wrapping pg_catalog in Shell or any scripting
language.
• Using open source tools
− check_postgres.pl, pg_catcheck, pg_cluu, munin, Nagios etc
• Proprietary Tools
− EnterpriseDB Postgres Enterprise Manager
Database Monitoring Methods
© 2014 EnterpriseDB Corporation. All rights reserved. 5
• Tables
− pg_class
− pg_database
• Views
− pg_stat_activity
− pg_stat_all_tables
− pg_stat_archiver
− pg_stat_replication
• Functions
− pg_is_in_recovery()
− pg_last_xlog_replay_location()
− pg_last_xact_replay_timestamp()
− pg_xlog_location_diff()
Using pg_catalogs
© 2014 EnterpriseDB Corporation. All rights reserved. 6
Wrapping pg_catalogs in Shell or Other languages
Max
Connections
Long Running
queries
Idle
Connections
Waiting
Connections
© 2014 EnterpriseDB Corporation. All rights reserved. 7
• check_postgres.pl
• pg_catcheck [ catalog corruption ]
• pg_cluu
Using open source tools
© 2014 EnterpriseDB Corporation. All rights reserved. 8
Sample output of critical probes on Production
Environment
© 2014 EnterpriseDB Corporation. All rights reserved. 9
pg_catcheck
© 2014 EnterpriseDB Corporation. All rights reserved. 10
Monitoring with Proprietary Tools
• Postgres Enterprise
Manager [ EnterpriseDB] PostgreSQL
Instances of
different versions
9.0.x
9.1.x
9.4.x
9.3.x
9.2.x
PEM
Server
© 2014 EnterpriseDB Corporation. All rights reserved. 11
PEM Probes And Alerts
Table Bloat Total table bloat in server
Highest Table Bloat in server
Table Frozen XID Table FrozenXID
Database Frozen XID Database FrozenXID
Number of WAL Files Number of WAL Files
WAL Archive status Number of WAL archiving Pending
Streaming Replication Standby server lag behind the master By WAL
segment
Standby server lag behind the master by WAL
pages
Streaming Replication Lag Number of minutes lag in standby server from
master server
© 2014 EnterpriseDB Corporation. All rights reserved. 12
Dashboard's Charts
© 2014 EnterpriseDB Corporation. All rights reserved. 13
PEM Alerts
© 2014 EnterpriseDB Corporation. All rights reserved. 14
• Complete point and click
management of all database
objects
• 225+ pre-configured database
alerts for immediate global at-a-
glance monitoring complete with
SMTP notifications, SNMP traps
and richly configurable notification
rules (New)
• Remote Monitoring (New)
• A Capacity Planning and Forecast
Manager
• PostgreSQL Expert database
configuration recommendations
and deployment
• Log Management and a Log
Analysis Expert (New)
• Team Management
• Custom probes for gathering
metrics (New)
• SQL Profiler
• Index Advisor
• All optimized for working with
multiple databases consistently
across the enterprise.
Primary Product Features
© 2014 EnterpriseDB Corporation. All rights reserved. 15
• On Premise Server
• Public Cloud [AWS]
• Private Cloud [Open stack ]
Deployment
© 2014 EnterpriseDB Corporation. All rights reserved. 16
References
• http://www.enterprisedb.com/products-services-
training/products/postgres-enterprise-manager
Postgres Enterprise Manager
© 2014 EnterpriseDB Corporation. All rights reserved. 17
Questions?
© 2014 EnterpriseDB Corporation. All rights reserved. 18

More Related Content

What's hot

PostgreSQL Write-Ahead Log (Heikki Linnakangas)
PostgreSQL Write-Ahead Log (Heikki Linnakangas) PostgreSQL Write-Ahead Log (Heikki Linnakangas)
PostgreSQL Write-Ahead Log (Heikki Linnakangas)
Ontico
 
Meet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike Steenbergen
Meet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike SteenbergenMeet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike Steenbergen
Meet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike Steenbergen
distributed matters
 
In-core compression: how to shrink your database size in several times
In-core compression: how to shrink your database size in several timesIn-core compression: how to shrink your database size in several times
In-core compression: how to shrink your database size in several times
Aleksander Alekseev
 
Deploying postgre sql on amazon ec2
Deploying postgre sql on amazon ec2 Deploying postgre sql on amazon ec2
Deploying postgre sql on amazon ec2
Denish Patel
 

What's hot (20)

Patroni - HA PostgreSQL made easy
Patroni - HA PostgreSQL made easyPatroni - HA PostgreSQL made easy
Patroni - HA PostgreSQL made easy
 
Analyze corefile and backtraces with GDB for Mysql/MariaDB on Linux - Nilanda...
Analyze corefile and backtraces with GDB for Mysql/MariaDB on Linux - Nilanda...Analyze corefile and backtraces with GDB for Mysql/MariaDB on Linux - Nilanda...
Analyze corefile and backtraces with GDB for Mysql/MariaDB on Linux - Nilanda...
 
Streaming huge databases using logical decoding
Streaming huge databases using logical decodingStreaming huge databases using logical decoding
Streaming huge databases using logical decoding
 
PostgreSQL and RAM usage
PostgreSQL and RAM usagePostgreSQL and RAM usage
PostgreSQL and RAM usage
 
PostgreSQL Write-Ahead Log (Heikki Linnakangas)
PostgreSQL Write-Ahead Log (Heikki Linnakangas) PostgreSQL Write-Ahead Log (Heikki Linnakangas)
PostgreSQL Write-Ahead Log (Heikki Linnakangas)
 
Streaming replication in PostgreSQL
Streaming replication in PostgreSQLStreaming replication in PostgreSQL
Streaming replication in PostgreSQL
 
Ilya Kosmodemiansky - An ultimate guide to upgrading your PostgreSQL installa...
Ilya Kosmodemiansky - An ultimate guide to upgrading your PostgreSQL installa...Ilya Kosmodemiansky - An ultimate guide to upgrading your PostgreSQL installa...
Ilya Kosmodemiansky - An ultimate guide to upgrading your PostgreSQL installa...
 
Query Parallelism in PostgreSQL: What's coming next?
Query Parallelism in PostgreSQL: What's coming next?Query Parallelism in PostgreSQL: What's coming next?
Query Parallelism in PostgreSQL: What's coming next?
 
PGConf.ASIA 2019 Bali - Building PostgreSQL as a Service with Kubernetes - Ta...
PGConf.ASIA 2019 Bali - Building PostgreSQL as a Service with Kubernetes - Ta...PGConf.ASIA 2019 Bali - Building PostgreSQL as a Service with Kubernetes - Ta...
PGConf.ASIA 2019 Bali - Building PostgreSQL as a Service with Kubernetes - Ta...
 
PostgreSQL Replication High Availability Methods
PostgreSQL Replication High Availability MethodsPostgreSQL Replication High Availability Methods
PostgreSQL Replication High Availability Methods
 
PostgreSQL HA
PostgreSQL   HAPostgreSQL   HA
PostgreSQL HA
 
Streaming replication in practice
Streaming replication in practiceStreaming replication in practice
Streaming replication in practice
 
On The Building Of A PostgreSQL Cluster
On The Building Of A PostgreSQL ClusterOn The Building Of A PostgreSQL Cluster
On The Building Of A PostgreSQL Cluster
 
Meet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike Steenbergen
Meet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike SteenbergenMeet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike Steenbergen
Meet Spilo, Zalando’s HIGH-AVAILABLE POSTGRESQL CLUSTER - Feike Steenbergen
 
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 ViennaAutovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
Autovacuum, explained for engineers, new improved version PGConf.eu 2015 Vienna
 
High Availability PostgreSQL with Zalando Patroni
High Availability PostgreSQL with Zalando PatroniHigh Availability PostgreSQL with Zalando Patroni
High Availability PostgreSQL with Zalando Patroni
 
In-core compression: how to shrink your database size in several times
In-core compression: how to shrink your database size in several timesIn-core compression: how to shrink your database size in several times
In-core compression: how to shrink your database size in several times
 
Deploying postgre sql on amazon ec2
Deploying postgre sql on amazon ec2 Deploying postgre sql on amazon ec2
Deploying postgre sql on amazon ec2
 
Countdown to PostgreSQL v9.5 - Foriegn Tables can be part of Inheritance Tree
Countdown to PostgreSQL v9.5 - Foriegn Tables can be part of Inheritance Tree Countdown to PostgreSQL v9.5 - Foriegn Tables can be part of Inheritance Tree
Countdown to PostgreSQL v9.5 - Foriegn Tables can be part of Inheritance Tree
 
PostgreSQL High Availability in a Containerized World
PostgreSQL High Availability in a Containerized WorldPostgreSQL High Availability in a Containerized World
PostgreSQL High Availability in a Containerized World
 

Viewers also liked

Why Content Marketing Fails
Why Content Marketing FailsWhy Content Marketing Fails
Why Content Marketing Fails
Rand Fishkin
 
A Guide to SlideShare Analytics - Excerpts from Hubspot's Step by Step Guide ...
A Guide to SlideShare Analytics - Excerpts from Hubspot's Step by Step Guide ...A Guide to SlideShare Analytics - Excerpts from Hubspot's Step by Step Guide ...
A Guide to SlideShare Analytics - Excerpts from Hubspot's Step by Step Guide ...
SlideShare
 

Viewers also liked (20)

Optimizer Hints
Optimizer HintsOptimizer Hints
Optimizer Hints
 
Attacking Web Proxies
Attacking Web ProxiesAttacking Web Proxies
Attacking Web Proxies
 
Introduction to cocoa sql mapper
Introduction to cocoa sql mapperIntroduction to cocoa sql mapper
Introduction to cocoa sql mapper
 
Building Machine Learning Pipelines
Building Machine Learning PipelinesBuilding Machine Learning Pipelines
Building Machine Learning Pipelines
 
Cloud Computing (CCSME 2015 talk) - mypapit
Cloud Computing (CCSME 2015 talk) - mypapitCloud Computing (CCSME 2015 talk) - mypapit
Cloud Computing (CCSME 2015 talk) - mypapit
 
8 Ways a Digital Media Platform is More Powerful than “Marketing”
8 Ways a Digital Media Platform is More Powerful than “Marketing”8 Ways a Digital Media Platform is More Powerful than “Marketing”
8 Ways a Digital Media Platform is More Powerful than “Marketing”
 
How Often Should You Post to Facebook and Twitter
How Often Should You Post to Facebook and TwitterHow Often Should You Post to Facebook and Twitter
How Often Should You Post to Facebook and Twitter
 
Slides That Rock
Slides That RockSlides That Rock
Slides That Rock
 
Why Content Marketing Fails
Why Content Marketing FailsWhy Content Marketing Fails
Why Content Marketing Fails
 
What Makes Great Infographics
What Makes Great InfographicsWhat Makes Great Infographics
What Makes Great Infographics
 
Masters of SlideShare
Masters of SlideShareMasters of SlideShare
Masters of SlideShare
 
STOP! VIEW THIS! 10-Step Checklist When Uploading to Slideshare
STOP! VIEW THIS! 10-Step Checklist When Uploading to SlideshareSTOP! VIEW THIS! 10-Step Checklist When Uploading to Slideshare
STOP! VIEW THIS! 10-Step Checklist When Uploading to Slideshare
 
You Suck At PowerPoint!
You Suck At PowerPoint!You Suck At PowerPoint!
You Suck At PowerPoint!
 
10 Ways to Win at SlideShare SEO & Presentation Optimization
10 Ways to Win at SlideShare SEO & Presentation Optimization10 Ways to Win at SlideShare SEO & Presentation Optimization
10 Ways to Win at SlideShare SEO & Presentation Optimization
 
How To Get More From SlideShare - Super-Simple Tips For Content Marketing
How To Get More From SlideShare - Super-Simple Tips For Content MarketingHow To Get More From SlideShare - Super-Simple Tips For Content Marketing
How To Get More From SlideShare - Super-Simple Tips For Content Marketing
 
A Guide to SlideShare Analytics - Excerpts from Hubspot's Step by Step Guide ...
A Guide to SlideShare Analytics - Excerpts from Hubspot's Step by Step Guide ...A Guide to SlideShare Analytics - Excerpts from Hubspot's Step by Step Guide ...
A Guide to SlideShare Analytics - Excerpts from Hubspot's Step by Step Guide ...
 
2015 Upload Campaigns Calendar - SlideShare
2015 Upload Campaigns Calendar - SlideShare2015 Upload Campaigns Calendar - SlideShare
2015 Upload Campaigns Calendar - SlideShare
 
Monitoring pg with_graphite_grafana
Monitoring pg with_graphite_grafanaMonitoring pg with_graphite_grafana
Monitoring pg with_graphite_grafana
 
What to Upload to SlideShare
What to Upload to SlideShareWhat to Upload to SlideShare
What to Upload to SlideShare
 
How to Make Awesome SlideShares: Tips & Tricks
How to Make Awesome SlideShares: Tips & TricksHow to Make Awesome SlideShares: Tips & Tricks
How to Make Awesome SlideShares: Tips & Tricks
 

Similar to Case Studies on PostgreSQL

Multitenant Full Deck Jan 2015 Cloud Team AJ Linkedin
Multitenant Full Deck Jan 2015 Cloud Team AJ LinkedinMultitenant Full Deck Jan 2015 Cloud Team AJ Linkedin
Multitenant Full Deck Jan 2015 Cloud Team AJ Linkedin
Arush Jain
 

Similar to Case Studies on PostgreSQL (20)

SQL PASS Taiwan 七月份聚會-1
SQL PASS Taiwan 七月份聚會-1SQL PASS Taiwan 七月份聚會-1
SQL PASS Taiwan 七月份聚會-1
 
Postgres & Red Hat Cluster Suite
Postgres & Red Hat Cluster SuitePostgres & Red Hat Cluster Suite
Postgres & Red Hat Cluster Suite
 
Best And Worst Practices Deploying IBM Connections
Best And Worst Practices Deploying IBM ConnectionsBest And Worst Practices Deploying IBM Connections
Best And Worst Practices Deploying IBM Connections
 
Best Practices for Becoming an Exceptional Postgres DBA
Best Practices for Becoming an Exceptional Postgres DBA Best Practices for Becoming an Exceptional Postgres DBA
Best Practices for Becoming an Exceptional Postgres DBA
 
Ashnik EnterpriseDB PostgreSQL - A real alternative to Oracle
Ashnik EnterpriseDB PostgreSQL - A real alternative to Oracle Ashnik EnterpriseDB PostgreSQL - A real alternative to Oracle
Ashnik EnterpriseDB PostgreSQL - A real alternative to Oracle
 
9.6_Course Material-Postgresql_002.pdf
9.6_Course Material-Postgresql_002.pdf9.6_Course Material-Postgresql_002.pdf
9.6_Course Material-Postgresql_002.pdf
 
The Central View of your Data with Postgres
The Central View of your Data with PostgresThe Central View of your Data with Postgres
The Central View of your Data with Postgres
 
2007-05-23 Cecchet_PGCon2007.ppt
2007-05-23 Cecchet_PGCon2007.ppt2007-05-23 Cecchet_PGCon2007.ppt
2007-05-23 Cecchet_PGCon2007.ppt
 
Postgres Point-in-Time Recovery
Postgres Point-in-Time RecoveryPostgres Point-in-Time Recovery
Postgres Point-in-Time Recovery
 
The Power of Postgres Plus Cloud Database
The Power of Postgres Plus Cloud DatabaseThe Power of Postgres Plus Cloud Database
The Power of Postgres Plus Cloud Database
 
1049: Best and Worst Practices for Deploying IBM Connections - IBM Connect 2016
1049: Best and Worst Practices for Deploying IBM Connections - IBM Connect 20161049: Best and Worst Practices for Deploying IBM Connections - IBM Connect 2016
1049: Best and Worst Practices for Deploying IBM Connections - IBM Connect 2016
 
Architecting with power vm
Architecting with power vmArchitecting with power vm
Architecting with power vm
 
Multitenant Full Deck Jan 2015 Cloud Team AJ Linkedin
Multitenant Full Deck Jan 2015 Cloud Team AJ LinkedinMultitenant Full Deck Jan 2015 Cloud Team AJ Linkedin
Multitenant Full Deck Jan 2015 Cloud Team AJ Linkedin
 
Still All on One Server: Perforce at Scale
Still All on One Server: Perforce at Scale Still All on One Server: Perforce at Scale
Still All on One Server: Perforce at Scale
 
Simplify IT: Oracle SuperCluster
Simplify IT: Oracle SuperCluster Simplify IT: Oracle SuperCluster
Simplify IT: Oracle SuperCluster
 
Oracle Storage a ochrana dat
Oracle Storage a ochrana datOracle Storage a ochrana dat
Oracle Storage a ochrana dat
 
[db tech showcase Tokyo 2016] E34: Oracle SE - RAC, HA and Standby are Still ...
[db tech showcase Tokyo 2016] E34: Oracle SE - RAC, HA and Standby are Still ...[db tech showcase Tokyo 2016] E34: Oracle SE - RAC, HA and Standby are Still ...
[db tech showcase Tokyo 2016] E34: Oracle SE - RAC, HA and Standby are Still ...
 
CV_avneet
CV_avneetCV_avneet
CV_avneet
 
EDB Database Servers and Tools
EDB Database Servers and Tools EDB Database Servers and Tools
EDB Database Servers and Tools
 
MySQL Scalability and Reliability for Replicated Environment
MySQL Scalability and Reliability for Replicated EnvironmentMySQL Scalability and Reliability for Replicated Environment
MySQL Scalability and Reliability for Replicated Environment
 

More from InMobi Technology

More from InMobi Technology (20)

Ensemble Methods for Algorithmic Trading
Ensemble Methods for Algorithmic TradingEnsemble Methods for Algorithmic Trading
Ensemble Methods for Algorithmic Trading
 
Backbone & Graphs
Backbone & GraphsBackbone & Graphs
Backbone & Graphs
 
24/7 Monitoring and Alerting of PostgreSQL
24/7 Monitoring and Alerting of PostgreSQL24/7 Monitoring and Alerting of PostgreSQL
24/7 Monitoring and Alerting of PostgreSQL
 
Reflective and Stored XSS- Cross Site Scripting
Reflective and Stored XSS- Cross Site ScriptingReflective and Stored XSS- Cross Site Scripting
Reflective and Stored XSS- Cross Site Scripting
 
Introduction to Threat Modeling
Introduction to Threat ModelingIntroduction to Threat Modeling
Introduction to Threat Modeling
 
HTTP Basics Demo
HTTP Basics DemoHTTP Basics Demo
HTTP Basics Demo
 
The Synapse IoT Stack: Technology Trends in IOT and Big Data
The Synapse IoT Stack: Technology Trends in IOT and Big DataThe Synapse IoT Stack: Technology Trends in IOT and Big Data
The Synapse IoT Stack: Technology Trends in IOT and Big Data
 
What's new in Hadoop Yarn- Dec 2014
What's new in Hadoop Yarn- Dec 2014What's new in Hadoop Yarn- Dec 2014
What's new in Hadoop Yarn- Dec 2014
 
Security News Bytes Null Dec Meet Bangalore
Security News Bytes Null Dec Meet BangaloreSecurity News Bytes Null Dec Meet Bangalore
Security News Bytes Null Dec Meet Bangalore
 
Matriux blue
Matriux blueMatriux blue
Matriux blue
 
PCI DSS v3 - Protecting Cardholder data
PCI DSS v3 - Protecting Cardholder dataPCI DSS v3 - Protecting Cardholder data
PCI DSS v3 - Protecting Cardholder data
 
Running Hadoop as Service in AltiScale Platform
Running Hadoop as Service in AltiScale PlatformRunning Hadoop as Service in AltiScale Platform
Running Hadoop as Service in AltiScale Platform
 
Shodan- That Device Search Engine
Shodan- That Device Search EngineShodan- That Device Search Engine
Shodan- That Device Search Engine
 
Big Data BI Simplified
Big Data BI SimplifiedBig Data BI Simplified
Big Data BI Simplified
 
Massively Parallel Processing with Procedural Python - Pivotal HAWQ
Massively Parallel Processing with Procedural Python - Pivotal HAWQMassively Parallel Processing with Procedural Python - Pivotal HAWQ
Massively Parallel Processing with Procedural Python - Pivotal HAWQ
 
Tez Data Processing over Yarn
Tez Data Processing over YarnTez Data Processing over Yarn
Tez Data Processing over Yarn
 
Building Audience Analytics Platform
Building Audience Analytics PlatformBuilding Audience Analytics Platform
Building Audience Analytics Platform
 
Big Data and User Segmentation in Mobile Context
Big Data and User Segmentation in Mobile ContextBig Data and User Segmentation in Mobile Context
Big Data and User Segmentation in Mobile Context
 
Freedom Hack Report 2014
Freedom Hack Report 2014Freedom Hack Report 2014
Freedom Hack Report 2014
 
Hadoop fundamentals
Hadoop fundamentalsHadoop fundamentals
Hadoop fundamentals
 

Recently uploaded

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Recently uploaded (20)

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelMcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 

Case Studies on PostgreSQL

  • 1. © 2014 EnterpriseDB Corporation. All rights reserved. 1 Technical Case Studies in PostgreSQL [ Monitoring Methods ] Rohit Vyas | 11.04.15 Quality Management Group (PostgreSQL Installer and Tools)
  • 2. © 2014 EnterpriseDB Corporation. All rights reserved. 2 • Technical cases in PostgreSQL • Database Monitoring Methods Agenda
  • 3. © 2014 EnterpriseDB Corporation. All rights reserved. 3 − Bloats − Updates/Deletes cause bloats in a table in PostgreSQL. − Transaction Wraparound − No new XID’s if 4 billion mark reached. − Streaming/WAL Replication Lag − Number of bytes lag from Master database may be due to bulk load, network etc., − Handling WAL Archiving Files − Piling up .ready files due to failure of archiver process − Catalog Corruption − Database objects repository(pg_catalog schema) corruption. Technical Cases on Production Database ERROR: database is not accepting commands to avoid wraparound data loss in database "dvnspatial" Hint: Stop the postmaster and use a standalone backend to vacuum that database. You might also need to commit or roll back old prepared transactions. Error executing update (updateorg.matter.security.User) PANIC: could not write to file “pg_xlog/xlogtemp.556” :No space left on device notice: pg_class row has invalid relnamespace "24580": no matching entry in pg_namespace row identity: oid="24581" relname="foo" relkind="r"
  • 4. © 2014 EnterpriseDB Corporation. All rights reserved. 4 • Using pg_catalog Tables/Views/Functions • Wrapping pg_catalog in Shell or any scripting language. • Using open source tools − check_postgres.pl, pg_catcheck, pg_cluu, munin, Nagios etc • Proprietary Tools − EnterpriseDB Postgres Enterprise Manager Database Monitoring Methods
  • 5. © 2014 EnterpriseDB Corporation. All rights reserved. 5 • Tables − pg_class − pg_database • Views − pg_stat_activity − pg_stat_all_tables − pg_stat_archiver − pg_stat_replication • Functions − pg_is_in_recovery() − pg_last_xlog_replay_location() − pg_last_xact_replay_timestamp() − pg_xlog_location_diff() Using pg_catalogs
  • 6. © 2014 EnterpriseDB Corporation. All rights reserved. 6 Wrapping pg_catalogs in Shell or Other languages Max Connections Long Running queries Idle Connections Waiting Connections
  • 7. © 2014 EnterpriseDB Corporation. All rights reserved. 7 • check_postgres.pl • pg_catcheck [ catalog corruption ] • pg_cluu Using open source tools
  • 8. © 2014 EnterpriseDB Corporation. All rights reserved. 8 Sample output of critical probes on Production Environment
  • 9. © 2014 EnterpriseDB Corporation. All rights reserved. 9 pg_catcheck
  • 10. © 2014 EnterpriseDB Corporation. All rights reserved. 10 Monitoring with Proprietary Tools • Postgres Enterprise Manager [ EnterpriseDB] PostgreSQL Instances of different versions 9.0.x 9.1.x 9.4.x 9.3.x 9.2.x PEM Server
  • 11. © 2014 EnterpriseDB Corporation. All rights reserved. 11 PEM Probes And Alerts Table Bloat Total table bloat in server Highest Table Bloat in server Table Frozen XID Table FrozenXID Database Frozen XID Database FrozenXID Number of WAL Files Number of WAL Files WAL Archive status Number of WAL archiving Pending Streaming Replication Standby server lag behind the master By WAL segment Standby server lag behind the master by WAL pages Streaming Replication Lag Number of minutes lag in standby server from master server
  • 12. © 2014 EnterpriseDB Corporation. All rights reserved. 12 Dashboard's Charts
  • 13. © 2014 EnterpriseDB Corporation. All rights reserved. 13 PEM Alerts
  • 14. © 2014 EnterpriseDB Corporation. All rights reserved. 14 • Complete point and click management of all database objects • 225+ pre-configured database alerts for immediate global at-a- glance monitoring complete with SMTP notifications, SNMP traps and richly configurable notification rules (New) • Remote Monitoring (New) • A Capacity Planning and Forecast Manager • PostgreSQL Expert database configuration recommendations and deployment • Log Management and a Log Analysis Expert (New) • Team Management • Custom probes for gathering metrics (New) • SQL Profiler • Index Advisor • All optimized for working with multiple databases consistently across the enterprise. Primary Product Features
  • 15. © 2014 EnterpriseDB Corporation. All rights reserved. 15 • On Premise Server • Public Cloud [AWS] • Private Cloud [Open stack ] Deployment
  • 16. © 2014 EnterpriseDB Corporation. All rights reserved. 16 References • http://www.enterprisedb.com/products-services- training/products/postgres-enterprise-manager Postgres Enterprise Manager
  • 17. © 2014 EnterpriseDB Corporation. All rights reserved. 17 Questions?
  • 18. © 2014 EnterpriseDB Corporation. All rights reserved. 18