SlideShare una empresa de Scribd logo
1 de 35
Descargar para leer sin conexión
July 2021
SPACE MADE SIMPLE™
HOW LOFT LEVERAGES INFLUXDB
CALEBMACLACHLAN
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Agenda
- Loft Overview
- Why Influx
- Implementation Process
- Current State
- Issues Encountered
- Questions
LOFT ORBITAL PROPRIETARY
C O M PA N Y O V E R V I E W
Loft Orbital is a leading provider of Space Infrastructure as a Service,
making space simple
Status: Series A Startup, 20+ Million
Raised
Staff: 60+ mixing tech-savvy and space
people
Facilities:
• San Francisco, California (HQ)
• Boulder, Colorado
• Toulouse, France (EMEA office)
CORPORATE
Rapid access to space
High-performance, heritage spacecraft
End-to-end solution
Onboard our YAM* high-performance and reliable satellites, we
fly the payloads of our customers fast and at low-cost
Reduced cost for the customer
* YAM: Yet Another Mission
A full-service approach
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Satellites for Flexible Payload Accommodation
4
Loft Payload Hub
Customer Payloads
Integrated Hub with Payloads Standard Satellite Bus Spacecraft
Mission Control
(Multiple bus partners)
Modular Payload Hub architecture eliminates bespoke bus engineering and
facilitates multi-payload shared missions
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Example: YAM-2 Mission
5
Cockpit mission operations, data
delivery
Integrated space vehicle
(BCT bus, Loft Payload Hub, customer payloads)
Integrated Payload Hub
Payload & Hub integration
independent from satellite bus
YAM: Yet Another Mission
Customer Payloads
Customer
payloads
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Example: YAM-3 Mission
6
YAM-3 SQM vibration test
Customer Payloads
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Where Are They Now?
7
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Mission Operations
Authorized
Customer User
Cockpit Services &
Storage
Secure
Cloud*
Loft Orbital Mission
Operations
(San Francisco)
Customer End-Point
UI and API access
Telemetry & Mission Data
Cockpit provides:
- Highly-automated operations for both single missions and
distributed constellations
- The capability for short-notice and event-triggered tasking
- Authority over satellite maneuvering within safety envelopes
- Simple external integrations to enable automated tasking from
external sources
- End-to-end encryption (AES-256)
- API and web-based UI access
The Cockpit Mission Control System (MCS) is a cloud-based tool
for automated mission tasking, monitoring, and data routing
*Cockpit can also be deployed on-premise or in private cloud
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Mission Operations
WHY INFLUX?
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Timeseries Database Needs
Spacecraft Operations
- Large volumes of highly critical
spacecraft telemetry
- Scaling for 500 million
measurements/day
- Need near-real-time visualizations
- Easy access for engineers
- Long-term trending is crucial
- Limit Checking & Alerting
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Timeseries Database Needs
Data Sharing
- Loft’s unique multi-tenant model
requires careful data segregation
- Bus suppliers also need easy access
to data
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Timeseries Database Needs
Software Performance Monitoring
- Ingesting 10,000,000+ measurements
per 10-minute pass
- Low latency is critical
- Python
- Django
LOFT ORBITAL
PROPRIETARY
WHAT WERE WE USING BEFORE
▶ Django backend + ORM
▶ PostgreSQL RDB (no
timeseries-specific
optimizations)
▶ GraphQL API layer
▶ Graphene to connect
database layer to API
14
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Why InfluxDB?
- Familiarity
- Large user base & community
- Grafana query builder
- InfluxDB subscriptions and Kapacitor
- Low barrier of entry
IMPLEMENTATION
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Initial Architecture
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Initial Architecture Issues
- Doesn’t fulfill Spacecraft Ops
performance requirements
- Long duration queries very slow
- Bottleneck writing to DB
- Performance degradation on order
of days due to table size
- No customer-specific data filtering
- Not scalable
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Design Architecture
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Design Architecture Challenges
- InfluxDB Cloud 2.0 does not support
subscriptions/streaming (yet)
- Cloud 2.0 InfluxQL compatibility layer
stability issues
- No InfluxDB – GraphQL connector
libraries exist
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Current Architecture
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Software Performance Monitoring Design
Goals
- Capture how long each step of a
process took, every time
- Production-safe
- Low implementation overhead
- Handle 1000s of operations per
second
Design
- Performance Monitoring object that
collects metrics
- Context manager that times actions
- Method to publish discrete metrics
(e.g. queue length)
- Writes to Influx performed in bulk and
only at safe junctures
- All performance monitoring
disabled/enabled with env var
RESULTS
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Example Dashboard - Overview
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Example Dashboard - ADCS
LOFT ORBITAL
PROPRIETARY
ALERTING
Grafana Alerting, because:
▶ Grafana is already a key
part of our Ops flow
▶ Visualizations
▶ Great flexibility in alert
design
▶ Ability to alert based on
1.8 OSS data or Cloud 2.0
data
26
LOFT ORBITAL
PROPRIETARY
FEDERATION
▶ Created adapter for
Graphene to integrate
Influx with existing API
▶ Supports Relay,
including pagination
▶ Measurements & Fields
in InfluxDB relationally
linked to PostgreSQL
▶ 10x speed
improvement over prior
solution
27
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Example Performance Monitoring Code
…
…
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Example Performance Monitoring Dashboard
CHALLENGES & NEXT STEPS
LOFT ORBITAL
PROPRIETARY
LOFT ORBITAL
Challenges & Workarounds
Challenge Workaround
No Subscriptions in Cloud 2.0 Use 1.x locally. Kapacitor publishes to multiple Cloud 2.0
buckets.
Cloud 2.0 InfluxQL stability issues Keep local 1.x as fallback
Auth tokens tied to user account in Cloud 2.0 Use shared admin account to manage tokens
No filtering below Measurement level in Kapacitor Filter using Telegraf, which supports wildcards in
drop_fields (TBI)
Talk to InfluxData team, leverage community
LOFT ORBITAL
PROPRIETARY
WHAT’S NEXT
▶ Scale Up
▶ Move towards initial design
▶ Requires replacement of 1.x
subscription functionality
▶ Advanced filtering for customers
▶ Switch to Telegraf instead of
Kapacitor
▶ Switch to Flux language
▶ Depends on InfluxData, Grafana, or
the community developing Flux
query builder
32
JOIN US!
https://www.loftorbital.com/careers
Email me your resume:
caleb@loftorbital.com
OCTOBER 11-12, 2021
HANDS-ON FLUX TRAINING
October 26-27, 2021
Virtual Experience
We look forward to bringing together our community
of developers to learn, interact and share tips and use
cases.
https://www.influxdays.com/influxdays-north-america-2021-virtual-experience/
QUESTIONS?

Más contenido relacionado

La actualidad más candente

Docker Tutorial For Beginners | What Is Docker And How It Works? | Docker Tut...
Docker Tutorial For Beginners | What Is Docker And How It Works? | Docker Tut...Docker Tutorial For Beginners | What Is Docker And How It Works? | Docker Tut...
Docker Tutorial For Beginners | What Is Docker And How It Works? | Docker Tut...
Simplilearn
 
Introduction to OpenFlow, SDN and NFV
Introduction to OpenFlow, SDN and NFVIntroduction to OpenFlow, SDN and NFV
Introduction to OpenFlow, SDN and NFV
Kingston Smiler
 

La actualidad más candente (20)

KVM High Availability Regardless of Storage - Gabriel Brascher, VP of Apache ...
KVM High Availability Regardless of Storage - Gabriel Brascher, VP of Apache ...KVM High Availability Regardless of Storage - Gabriel Brascher, VP of Apache ...
KVM High Availability Regardless of Storage - Gabriel Brascher, VP of Apache ...
 
Kubernetes
KubernetesKubernetes
Kubernetes
 
Kubernetes
KubernetesKubernetes
Kubernetes
 
Kubernetes PPT.pptx
Kubernetes PPT.pptxKubernetes PPT.pptx
Kubernetes PPT.pptx
 
Orchestration in Action
Orchestration in ActionOrchestration in Action
Orchestration in Action
 
VMware vCloud Director
VMware vCloud DirectorVMware vCloud Director
VMware vCloud Director
 
Kubernetes for Beginners: An Introductory Guide
Kubernetes for Beginners: An Introductory GuideKubernetes for Beginners: An Introductory Guide
Kubernetes for Beginners: An Introductory Guide
 
Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...
Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...
Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...
 
Terraform Tips and Tricks - LAOUC 2022
Terraform Tips and Tricks - LAOUC 2022Terraform Tips and Tricks - LAOUC 2022
Terraform Tips and Tricks - LAOUC 2022
 
Load balancing and Service in Kubernetes
Load balancing and Service in KubernetesLoad balancing and Service in Kubernetes
Load balancing and Service in Kubernetes
 
Docker Tutorial For Beginners | What Is Docker And How It Works? | Docker Tut...
Docker Tutorial For Beginners | What Is Docker And How It Works? | Docker Tut...Docker Tutorial For Beginners | What Is Docker And How It Works? | Docker Tut...
Docker Tutorial For Beginners | What Is Docker And How It Works? | Docker Tut...
 
Kubernetes Networking
Kubernetes NetworkingKubernetes Networking
Kubernetes Networking
 
Monitoring on Kubernetes using prometheus
Monitoring on Kubernetes using prometheusMonitoring on Kubernetes using prometheus
Monitoring on Kubernetes using prometheus
 
Getting Started with Kubernetes
Getting Started with Kubernetes Getting Started with Kubernetes
Getting Started with Kubernetes
 
NETCONF YANG tutorial
NETCONF YANG tutorialNETCONF YANG tutorial
NETCONF YANG tutorial
 
Introduction to OpenFlow, SDN and NFV
Introduction to OpenFlow, SDN and NFVIntroduction to OpenFlow, SDN and NFV
Introduction to OpenFlow, SDN and NFV
 
knolx of KubeCost & Infracost
knolx of KubeCost & Infracostknolx of KubeCost & Infracost
knolx of KubeCost & Infracost
 
Kubernetes Introduction
Kubernetes IntroductionKubernetes Introduction
Kubernetes Introduction
 
Kubernetes networking: Introduction to overlay networks, communication models...
Kubernetes networking: Introduction to overlay networks, communication models...Kubernetes networking: Introduction to overlay networks, communication models...
Kubernetes networking: Introduction to overlay networks, communication models...
 
[오픈소스컨설팅] 쿠버네티스와 쿠버네티스 on 오픈스택 비교 및 구축 방법
[오픈소스컨설팅] 쿠버네티스와 쿠버네티스 on 오픈스택 비교  및 구축 방법[오픈소스컨설팅] 쿠버네티스와 쿠버네티스 on 오픈스택 비교  및 구축 방법
[오픈소스컨설팅] 쿠버네티스와 쿠버네티스 on 오픈스택 비교 및 구축 방법
 

Similar a Spacecrafts Made Simple: How Loft Orbital Delivers Unparalleled Speed-to-Space and Reliability Using Telegraf, InfluxDB Cloud and Google Cloud

Intro to Project Calico: a pure layer 3 approach to scale-out networking
Intro to Project Calico: a pure layer 3 approach to scale-out networkingIntro to Project Calico: a pure layer 3 approach to scale-out networking
Intro to Project Calico: a pure layer 3 approach to scale-out networking
Packet
 
Living objects network performance_management_v2
Living objects network performance_management_v2Living objects network performance_management_v2
Living objects network performance_management_v2
Yoan SMADJA
 

Similar a Spacecrafts Made Simple: How Loft Orbital Delivers Unparalleled Speed-to-Space and Reliability Using Telegraf, InfluxDB Cloud and Google Cloud (20)

Activeeon technology for Big Compute and cloud migration
Activeeon technology for Big Compute and cloud migrationActiveeon technology for Big Compute and cloud migration
Activeeon technology for Big Compute and cloud migration
 
Gateway Design with Eclipse Kura - Taking Kura to heights
Gateway Design with Eclipse Kura - Taking Kura to heightsGateway Design with Eclipse Kura - Taking Kura to heights
Gateway Design with Eclipse Kura - Taking Kura to heights
 
Red hat's updates on the cloud & infrastructure strategy
Red hat's updates on the cloud & infrastructure strategyRed hat's updates on the cloud & infrastructure strategy
Red hat's updates on the cloud & infrastructure strategy
 
Workload Automation for Cloud Migration and Machine Learning Platform
Workload Automation for Cloud Migration and Machine Learning PlatformWorkload Automation for Cloud Migration and Machine Learning Platform
Workload Automation for Cloud Migration and Machine Learning Platform
 
End-to-end IoT solutions with Java and Eclipse IoT
End-to-end IoT solutions with Java and Eclipse IoTEnd-to-end IoT solutions with Java and Eclipse IoT
End-to-end IoT solutions with Java and Eclipse IoT
 
Cloud Native Applications on OpenShift
Cloud Native Applications on OpenShiftCloud Native Applications on OpenShift
Cloud Native Applications on OpenShift
 
Openstack Summit Tokyo 2015 - Building a private cloud to efficiently handle ...
Openstack Summit Tokyo 2015 - Building a private cloud to efficiently handle ...Openstack Summit Tokyo 2015 - Building a private cloud to efficiently handle ...
Openstack Summit Tokyo 2015 - Building a private cloud to efficiently handle ...
 
How we scale up our architecture and organization at Dailymotion
How we scale up our architecture and organization at DailymotionHow we scale up our architecture and organization at Dailymotion
How we scale up our architecture and organization at Dailymotion
 
Functioning incessantly of Data Science Platform with Kubeflow - Albert Lewan...
Functioning incessantly of Data Science Platform with Kubeflow - Albert Lewan...Functioning incessantly of Data Science Platform with Kubeflow - Albert Lewan...
Functioning incessantly of Data Science Platform with Kubeflow - Albert Lewan...
 
Intro to Project Calico: a pure layer 3 approach to scale-out networking
Intro to Project Calico: a pure layer 3 approach to scale-out networkingIntro to Project Calico: a pure layer 3 approach to scale-out networking
Intro to Project Calico: a pure layer 3 approach to scale-out networking
 
Living objects network performance_management_v2
Living objects network performance_management_v2Living objects network performance_management_v2
Living objects network performance_management_v2
 
Safer Commutes & Streaming Data | George Padavick, Ohio Department of Transpo...
Safer Commutes & Streaming Data | George Padavick, Ohio Department of Transpo...Safer Commutes & Streaming Data | George Padavick, Ohio Department of Transpo...
Safer Commutes & Streaming Data | George Padavick, Ohio Department of Transpo...
 
Integrating Active Networking and Commercial-Grade Routing Platforms
Integrating Active Networking and Commercial-Grade Routing PlatformsIntegrating Active Networking and Commercial-Grade Routing Platforms
Integrating Active Networking and Commercial-Grade Routing Platforms
 
How eStruxture Data Centers is Using ECE to Rapidly Scale Their Business
How eStruxture Data Centers is Using ECE to Rapidly Scale Their BusinessHow eStruxture Data Centers is Using ECE to Rapidly Scale Their Business
How eStruxture Data Centers is Using ECE to Rapidly Scale Their Business
 
Service Engineering, ZHAW for CeBIT
Service Engineering, ZHAW for CeBITService Engineering, ZHAW for CeBIT
Service Engineering, ZHAW for CeBIT
 
Turbocharge the NFV Data Plane in the SDN Era - a Radisys presentation
Turbocharge the NFV Data Plane in the SDN Era - a Radisys presentationTurbocharge the NFV Data Plane in the SDN Era - a Radisys presentation
Turbocharge the NFV Data Plane in the SDN Era - a Radisys presentation
 
Upgrading Openstack from Kilo to Mitaka
Upgrading Openstack from Kilo to MitakaUpgrading Openstack from Kilo to Mitaka
Upgrading Openstack from Kilo to Mitaka
 
What's new in OpenStack Liberty
What's new in OpenStack LibertyWhat's new in OpenStack Liberty
What's new in OpenStack Liberty
 
Overview xs en
Overview xs enOverview xs en
Overview xs en
 
Faster, more Secure Application Modernization and Replatforming with PKS - Ku...
Faster, more Secure Application Modernization and Replatforming with PKS - Ku...Faster, more Secure Application Modernization and Replatforming with PKS - Ku...
Faster, more Secure Application Modernization and Replatforming with PKS - Ku...
 

Más de InfluxData

How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
InfluxData
 
How Delft University's Engineering Students Make Their EV Formula-Style Race ...
How Delft University's Engineering Students Make Their EV Formula-Style Race ...How Delft University's Engineering Students Make Their EV Formula-Style Race ...
How Delft University's Engineering Students Make Their EV Formula-Style Race ...
InfluxData
 
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
InfluxData
 
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
InfluxData
 

Más de InfluxData (20)

Announcing InfluxDB Clustered
Announcing InfluxDB ClusteredAnnouncing InfluxDB Clustered
Announcing InfluxDB Clustered
 
Best Practices for Leveraging the Apache Arrow Ecosystem
Best Practices for Leveraging the Apache Arrow EcosystemBest Practices for Leveraging the Apache Arrow Ecosystem
Best Practices for Leveraging the Apache Arrow Ecosystem
 
How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...
How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...
How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...
 
Power Your Predictive Analytics with InfluxDB
Power Your Predictive Analytics with InfluxDBPower Your Predictive Analytics with InfluxDB
Power Your Predictive Analytics with InfluxDB
 
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
 
Build an Edge-to-Cloud Solution with the MING Stack
Build an Edge-to-Cloud Solution with the MING StackBuild an Edge-to-Cloud Solution with the MING Stack
Build an Edge-to-Cloud Solution with the MING Stack
 
Meet the Founders: An Open Discussion About Rewriting Using Rust
Meet the Founders: An Open Discussion About Rewriting Using RustMeet the Founders: An Open Discussion About Rewriting Using Rust
Meet the Founders: An Open Discussion About Rewriting Using Rust
 
Introducing InfluxDB Cloud Dedicated
Introducing InfluxDB Cloud DedicatedIntroducing InfluxDB Cloud Dedicated
Introducing InfluxDB Cloud Dedicated
 
Gain Better Observability with OpenTelemetry and InfluxDB
Gain Better Observability with OpenTelemetry and InfluxDB Gain Better Observability with OpenTelemetry and InfluxDB
Gain Better Observability with OpenTelemetry and InfluxDB
 
How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...
How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...
How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...
 
How Delft University's Engineering Students Make Their EV Formula-Style Race ...
How Delft University's Engineering Students Make Their EV Formula-Style Race ...How Delft University's Engineering Students Make Their EV Formula-Style Race ...
How Delft University's Engineering Students Make Their EV Formula-Style Race ...
 
Introducing InfluxDB’s New Time Series Database Storage Engine
Introducing InfluxDB’s New Time Series Database Storage EngineIntroducing InfluxDB’s New Time Series Database Storage Engine
Introducing InfluxDB’s New Time Series Database Storage Engine
 
Start Automating InfluxDB Deployments at the Edge with balena
Start Automating InfluxDB Deployments at the Edge with balena Start Automating InfluxDB Deployments at the Edge with balena
Start Automating InfluxDB Deployments at the Edge with balena
 
Understanding InfluxDB’s New Storage Engine
Understanding InfluxDB’s New Storage EngineUnderstanding InfluxDB’s New Storage Engine
Understanding InfluxDB’s New Storage Engine
 
Streamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDB
Streamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDBStreamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDB
Streamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDB
 
Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...
Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...
Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...
 
Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022
Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022
Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022
 
Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022
 
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
 
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
 

Último

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Último (20)

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUKSpring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 

Spacecrafts Made Simple: How Loft Orbital Delivers Unparalleled Speed-to-Space and Reliability Using Telegraf, InfluxDB Cloud and Google Cloud

  • 1. July 2021 SPACE MADE SIMPLE™ HOW LOFT LEVERAGES INFLUXDB CALEBMACLACHLAN
  • 2. LOFT ORBITAL PROPRIETARY LOFT ORBITAL Agenda - Loft Overview - Why Influx - Implementation Process - Current State - Issues Encountered - Questions
  • 3. LOFT ORBITAL PROPRIETARY C O M PA N Y O V E R V I E W Loft Orbital is a leading provider of Space Infrastructure as a Service, making space simple Status: Series A Startup, 20+ Million Raised Staff: 60+ mixing tech-savvy and space people Facilities: • San Francisco, California (HQ) • Boulder, Colorado • Toulouse, France (EMEA office) CORPORATE Rapid access to space High-performance, heritage spacecraft End-to-end solution Onboard our YAM* high-performance and reliable satellites, we fly the payloads of our customers fast and at low-cost Reduced cost for the customer * YAM: Yet Another Mission A full-service approach
  • 4. LOFT ORBITAL PROPRIETARY LOFT ORBITAL Satellites for Flexible Payload Accommodation 4 Loft Payload Hub Customer Payloads Integrated Hub with Payloads Standard Satellite Bus Spacecraft Mission Control (Multiple bus partners) Modular Payload Hub architecture eliminates bespoke bus engineering and facilitates multi-payload shared missions
  • 5. LOFT ORBITAL PROPRIETARY LOFT ORBITAL Example: YAM-2 Mission 5 Cockpit mission operations, data delivery Integrated space vehicle (BCT bus, Loft Payload Hub, customer payloads) Integrated Payload Hub Payload & Hub integration independent from satellite bus YAM: Yet Another Mission Customer Payloads Customer payloads
  • 6. LOFT ORBITAL PROPRIETARY LOFT ORBITAL Example: YAM-3 Mission 6 YAM-3 SQM vibration test Customer Payloads
  • 8. LOFT ORBITAL PROPRIETARY LOFT ORBITAL Mission Operations Authorized Customer User Cockpit Services & Storage Secure Cloud* Loft Orbital Mission Operations (San Francisco) Customer End-Point UI and API access Telemetry & Mission Data Cockpit provides: - Highly-automated operations for both single missions and distributed constellations - The capability for short-notice and event-triggered tasking - Authority over satellite maneuvering within safety envelopes - Simple external integrations to enable automated tasking from external sources - End-to-end encryption (AES-256) - API and web-based UI access The Cockpit Mission Control System (MCS) is a cloud-based tool for automated mission tasking, monitoring, and data routing *Cockpit can also be deployed on-premise or in private cloud
  • 11. LOFT ORBITAL PROPRIETARY LOFT ORBITAL Timeseries Database Needs Spacecraft Operations - Large volumes of highly critical spacecraft telemetry - Scaling for 500 million measurements/day - Need near-real-time visualizations - Easy access for engineers - Long-term trending is crucial - Limit Checking & Alerting
  • 12. LOFT ORBITAL PROPRIETARY LOFT ORBITAL Timeseries Database Needs Data Sharing - Loft’s unique multi-tenant model requires careful data segregation - Bus suppliers also need easy access to data
  • 13. LOFT ORBITAL PROPRIETARY LOFT ORBITAL Timeseries Database Needs Software Performance Monitoring - Ingesting 10,000,000+ measurements per 10-minute pass - Low latency is critical - Python - Django
  • 14. LOFT ORBITAL PROPRIETARY WHAT WERE WE USING BEFORE ▶ Django backend + ORM ▶ PostgreSQL RDB (no timeseries-specific optimizations) ▶ GraphQL API layer ▶ Graphene to connect database layer to API 14
  • 15. LOFT ORBITAL PROPRIETARY LOFT ORBITAL Why InfluxDB? - Familiarity - Large user base & community - Grafana query builder - InfluxDB subscriptions and Kapacitor - Low barrier of entry
  • 18. LOFT ORBITAL PROPRIETARY LOFT ORBITAL Initial Architecture Issues - Doesn’t fulfill Spacecraft Ops performance requirements - Long duration queries very slow - Bottleneck writing to DB - Performance degradation on order of days due to table size - No customer-specific data filtering - Not scalable
  • 20. LOFT ORBITAL PROPRIETARY LOFT ORBITAL Design Architecture Challenges - InfluxDB Cloud 2.0 does not support subscriptions/streaming (yet) - Cloud 2.0 InfluxQL compatibility layer stability issues - No InfluxDB – GraphQL connector libraries exist
  • 22. LOFT ORBITAL PROPRIETARY LOFT ORBITAL Software Performance Monitoring Design Goals - Capture how long each step of a process took, every time - Production-safe - Low implementation overhead - Handle 1000s of operations per second Design - Performance Monitoring object that collects metrics - Context manager that times actions - Method to publish discrete metrics (e.g. queue length) - Writes to Influx performed in bulk and only at safe junctures - All performance monitoring disabled/enabled with env var
  • 26. LOFT ORBITAL PROPRIETARY ALERTING Grafana Alerting, because: ▶ Grafana is already a key part of our Ops flow ▶ Visualizations ▶ Great flexibility in alert design ▶ Ability to alert based on 1.8 OSS data or Cloud 2.0 data 26
  • 27. LOFT ORBITAL PROPRIETARY FEDERATION ▶ Created adapter for Graphene to integrate Influx with existing API ▶ Supports Relay, including pagination ▶ Measurements & Fields in InfluxDB relationally linked to PostgreSQL ▶ 10x speed improvement over prior solution 27
  • 28. LOFT ORBITAL PROPRIETARY LOFT ORBITAL Example Performance Monitoring Code … …
  • 29. LOFT ORBITAL PROPRIETARY LOFT ORBITAL Example Performance Monitoring Dashboard
  • 31. LOFT ORBITAL PROPRIETARY LOFT ORBITAL Challenges & Workarounds Challenge Workaround No Subscriptions in Cloud 2.0 Use 1.x locally. Kapacitor publishes to multiple Cloud 2.0 buckets. Cloud 2.0 InfluxQL stability issues Keep local 1.x as fallback Auth tokens tied to user account in Cloud 2.0 Use shared admin account to manage tokens No filtering below Measurement level in Kapacitor Filter using Telegraf, which supports wildcards in drop_fields (TBI) Talk to InfluxData team, leverage community
  • 32. LOFT ORBITAL PROPRIETARY WHAT’S NEXT ▶ Scale Up ▶ Move towards initial design ▶ Requires replacement of 1.x subscription functionality ▶ Advanced filtering for customers ▶ Switch to Telegraf instead of Kapacitor ▶ Switch to Flux language ▶ Depends on InfluxData, Grafana, or the community developing Flux query builder 32
  • 33. JOIN US! https://www.loftorbital.com/careers Email me your resume: caleb@loftorbital.com
  • 34. OCTOBER 11-12, 2021 HANDS-ON FLUX TRAINING October 26-27, 2021 Virtual Experience We look forward to bringing together our community of developers to learn, interact and share tips and use cases. https://www.influxdays.com/influxdays-north-america-2021-virtual-experience/