Migrating IT to the Cloud - Zenoss in Amazon Web Services
•Descargar como PPTX, PDF•
1 recomendación•694 vistas
Nick Turner from Zenoss' presentation on hybrid IT at GalaxZ 16.
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Destacado
Destacado (18)
Similar a Migrating IT to the Cloud - Zenoss in Amazon Web Services
Similar a Migrating IT to the Cloud - Zenoss in Amazon Web Services (20)
Más de Zenoss
Más de Zenoss (17)
Último
Último (20)
Migrating IT to the Cloud - Zenoss in Amazon Web Services
- 1. © 2016 All Rights Reserved CONFIDENTIAL#GALAXZ16 @nickclarkturner #GALAXZ16 Migrating IT to the Cloud Nick Turner @nickclarkturner Zenoss in Amazon Web Services
- 2. © 2016 All Rights Reserved CONFIDENTIAL#GALAXZ16 @nickclarkturner Topics we’ll be covering… • Benefits of the Cloud • Benefits, Feasibility, and Objections • Zenoss in the Cloud • Internal use • Zenoss monitoring Zenoss • Deploying Zenoss in AWS • Challenges of Migrating to the Cloud • Overall and Zenoss-Specific • Cloud Governance and Cost Controls • Recommended Reading
- 3. © 2016 All Rights Reserved CONFIDENTIAL#GALAXZ16 @nickclarkturner Benefits of the Cloud
- 4. © 2016 All Rights Reserved CONFIDENTIAL#GALAXZ16 @nickclarkturner Benefits of the Cloud • Amazon Elastic Compute Cloud was introduced in August 2006 • Does it make sense to move X to the cloud? • Does your compute load burst? How scalable is your infrastructure? • Elastic Infrastructure (Availability Zones, Regions, Virtual Private Clouds, Route53) • Storage (S3, EBS, Snapshots, AMI) • How critical is uptime? Do you have SPOFs? How redundant is my infrastructure? • Available Zones, S3 Cross-Region replication, Elastic Load Balancing • How truly international is your co-location or managed datacenter partner? • Does your datacenter meet regulatory/compliance needs? • SOC, PCI, HIPAA, ISO, etc…
- 5. © 2016 All Rights Reserved CONFIDENTIAL#GALAXZ16 @nickclarkturner Benefits of the Cloud • But I don’t have the expertise… • A lot of transferable concepts from Physical to Cloud or Cloud to Cloud • Managed AWS Services (Rackspace, Datapipe, Connectria, etc...) • What are the knowledge retention realities of your current infrastructure? • Documentation, cross training, and wikis vs. Tribal • But CapEx vs. OpEx... Short-term vs. Long-term • What are the financial needs of the business? • Available capital? EBITDA obsession? Time value of money? • Try and level the financial playing ground with a discounted cash flow analysis or other tools • The realities of your CapEx investment reaching EOL or maximum capacity • Will needed capital be there in 5 years when your compute ot storage needs it?
- 6. © 2016 All Rights Reserved CONFIDENTIAL#GALAXZ16 @nickclarkturner Zenoss in the Cloud
- 7. © 2016 All Rights Reserved CONFIDENTIAL#GALAXZ16 @nickclarkturner Zenoss in the Cloud • We followed the department by department lift and shift model • Different Accounts with Consolidated Billing • Automation isn’t easy, but it is your friend. • SaltStack, Ansible, Chef, Cloud Formation, YAML, Boto Python • Architected each solution around use-case: • QA/Engineering • Short term use, frequent version changes, build and destroy, contractor access • Training • Short term use and destroy, components differ based on curriculum, multi-region, externally accessible • Demo • Short term and random simultaneous usage, need to showcase all product features with historical data • ZaaS • Long term use, highly available, geographically redundant data retention, minimize SPOFs where arch allows
- 8. © 2016 All Rights Reserved CONFIDENTIAL#GALAXZ16 @nickclarkturner Zenoss in the Cloud • Zenoss-as-a-Service (Zenoss Monitoring Zenoss) • Visibility, Uniformity, Quick Recovery • Critical ZaaS ZenPacks • AWS ZenPack • Linux Monitor ZenPack • AWS RDS Monitoring • ControlCenter ZenPack • RabbitMQ ZenPack • PagerDuty ZenPack • Slack Notifications • Web-Based Synthetic Transactions
- 9. © 2016 All Rights Reserved CONFIDENTIAL#GALAXZ16 @nickclarkturner Zenoss in the Cloud • Deploying Zenoss in AWS • Requirements have changed from 4x to 5x, but on par costs of running Zenoss Service Dynamics (Resource Manager, Impact, and Analytics) are similar. • For 4x ZSD depending on the number of Managed Resources would vary from: • RM = m4.xlarge -> m4.2xlarge -> m4.4xlarge, Impact = c4.2xlarge, Analytics = m4.2xlarge with RDS on db.m4.large all backed with EBS volumes. • For 5x the perspective shifts to resource pools • CC Resource Pool = m4.xlarge, Host Pool = m4.2xlarge xN, Analytics = m4.2xlarge, all with EBS • Nice instance comparative resource - http://www.ec2instances.info • Zenoss working on deploying an AMI to the Marketplace Image courtesy of https://www.botmetric.com/blog/how-to-choose-aws-instance-family-for-your-business-workloads/
- 10. © 2016 All Rights Reserved CONFIDENTIAL#GALAXZ16 @nickclarkturner Challenges Migrating to the Cloud
- 11. © 2016 All Rights Reserved CONFIDENTIAL#GALAXZ16 @nickclarkturner Challenges of Migrating to the Cloud • Building out an infrastructure • Cloud Service Accounts/Architecture • Too small (Account per service/customer – Automation/Management Limitations) • One Size Fits All (Security concerns, Account Limitations) • Seek guidance from vendor. • Migration Strategy • All at once or re-architecture • Complexity may cause analysis paralysis • Lift and Shift sensible workloads • Re-inventing the wheel? Are processes similar are efforts duplicated?
- 12. © 2016 All Rights Reserved CONFIDENTIAL#GALAXZ16 @nickclarkturner Challenges of Migrating to the Cloud • Zenoss in AWS Considerations • Networking (How Will Zenoss communicate with collectors or what it is monitoring?) • Connecting VPCs, L2TPw/IPSEC VPN tunnels, or OpenVPN tunnels • In ZaaS we define a pool per remote collector and utilize OpenVPN for connectivity • Are your services communicating internally or externally? • The route traffic takes when talking with other systems can impact security and performance • Should you offload processes like mail, database (RDS/HBASE), log aggregation, etc… • Do you need multiple environments ”Test” and “Prod”? • Plan for growth! • Zenoss in Azure? • On “paper” everything looked good, but the distributed infrastructure was not as seamless as stated and resulted in the application crashing.
- 13. © 2016 All Rights Reserved CONFIDENTIAL#GALAXZ16 @nickclarkturner Cloud Governance and Cost Controls
- 14. © 2016 All Rights Reserved CONFIDENTIAL#GALAXZ16 @nickclarkturner Cloud Governance and Cost Controls • Considerations for Cost Control Tool Selection • Purchasing On-Demand versus Reservations • Complexity (No-Upfront, Partial Upfront, Full-Upfront) • Dynamic Environment (m1 paravirtual (PV) today m4 hardware virtual machine (HVM) tomorrow) • Multiple Accounts / Different AZs • What is the best buy for my budget? What if I choose poorly? • Are they assisting you or is it up to you? • Tagging is your friend • The more you tag environments upon build for Environment, Department, Customer, etc… the easier it is to manage costs and the more governance tools can do for you. • Helps you determine what to shut off and what to keep on. What can be deleted and what should be retained.
- 15. © 2016 All Rights Reserved CONFIDENTIAL#GALAXZ16 @nickclarkturner Cloud Governance and Cost Controls • After evaluating others went with Cloud Health Tech • Reserved Instance Recommendations • Cost Tracking and Projections • Cleaning out the noise (RI Prepay, Unused RI) • Detailed Historical Service Data • Multiple Accounts, Different Perspectives (Tagging for Environment, Department, Customer) • Automated Reserved Instances/Health Checks • Automated Modifications (Dynamic AZ RI changes) • Unattached volume clean up • Stopping instances • Securing Environment (Open Ports) • Instance Utilization Recommendations
- 16. © 2016 All Rights Reserved CONFIDENTIAL#GALAXZ16 @nickclarkturner Recommended Reading
- 17. © 2016 All Rights Reserved CONFIDENTIAL#GALAXZ16 @nickclarkturner Recommended Reading • Six Things We Learned While Migrating Our Training Infrastructure to Amazon AWS • 3 Reasons Your Cloud Migration Project Will Fail • Capex vs. Opex: Most People Miss the Point About Cloud Economics • How to Break Down the OpEx vs. CapEx Cloud Computing Debate • Cloud Benefits: The Opex / Capex Benefit Debate • 13 Tips to Achieve Cloud Success • How To Choose Aws Instance Family For Your Business Workloads ? • Gartner: Cloud management tools are lagging • CloudHealth Technologies Named a ‘Cool Vendor’ by Gartner
- 18. © 2016 All Rights Reserved CONFIDENTIAL#GALAXZ16 @nickclarkturner Q&A • Nick Turner @nickclarkturner • Director, IT Operations @Zenoss
Notas del editor
- My name is Nick Turner and I’m the Director of IT Operations at Zenoss. Feel free to follow me on Twitter to converse during or after the conference, I’m @nickclarkturner Today we’ll be talking about Migrating IT to the Cloud, specifically Zenoss’ experience partnering with Amazon Web Services.
- Just to be clear my usage of “the cloud” is in reference to IaaS providers like AWS, Azure, and Google Cloud but primarily this presentation is focused on AWS terminology and concepts. Today the cloud is almost 10 years old as Amazon EC2 was introduced in August of 2006 so the concept and benefits of cloud computing are no longer new. Azure is becoming more competitive and Google Cloud has recently entered the market. However there are still companies out there that are sitting on the sidelines and choosing to avoid the cloud. The question that companies need to ask themselves is does it make sense to move X to the cloud? X being their total infrastructure, a specific workload, initiative, product, departmental infrastructure, etc… And some major driving concepts or questions are: Is running items with variable use in my infrastructure the most efficient use of resources, what happens when those resources need to temporarily or permanently scale, and is my infrastructure as resilient, ubiquitous, secure, and compliant as I would like? The cloud provides unprecedented scalability of compute and storage across different availability zones or cluster of datacenters, across different regions throughout the globe, and with integrated and scalable networking and supportive systems for managing networks, name resolution, load balancing, storage tiers, recovery, and automation. Of course while some cloud redundancy is included, some is a-la-carte and has to be architected into your cloud solution. You can still put all of your eggs in one basket in the cloud if you don’t architect in utilization of availability zones, cross-region replication of data, or elastic load balancing to name a few. Additionally the international footprint or availability of services in international offerings may be a hinderence to global expansion, especially if you want the same ability to dynamically scale internationally as it makes sense to do so. Finally, does your datacenter meet regulatory and compliance needs? What Tier is your datacenter, meaning how redundant is their infrastructure?
- The most common objections to cloud adoption I encounter have to do with a lack of cloud experience or financial objections In regards to lack of experience… a lot of the concepts that exist with physical datcenters are transferable to their cloud counterparts. Additionally the same is true from cloud partner to partner. Different terminology but essentially the same meaning. Getting up-to-speed is a small up-front investment hurdle to overcome. Otherwise you can always pay others to manage your AWS for you. Partners such as... You may also have a false sense of security in your existing operations but have far more risk than you realize with a few key individuals holding the keys to the kingdom. What happens when key individuals leave? What happens if you buy a new company or your company is bought? How is your... When it comes to assessing financial feasibility it really comes down to the needs of the business. Is there available capital? Does the company care more about EBITDA today than the long term viability of the company? Is the time value of money important and you want to have your cloud cake and eat it too? A great strategy is to level the playing field as much as possible if both options are on the table and feasible with analyses like a discounted cash flow for example If you go the CapEx route, will the realities of reaching max capacity or equipment end of life survive 5 years of financial and budget plans? <Experience with company acquisition...> Expesite vs. 360Facility
- As each department hit an inflection point where feasibility and benefits of the cloud migration became apparent they were moved. We went with a model of isolating each departmental use case to their own account for logical segmentation, “blast radius”, and architectural freedom. Even with different accounts we could still have consolidated billing with the segmentation helping departmental chargebacks. However not enough work was done up front to centralize tool utilization or adjust as our product architecture incorporated docker and control center, so depending on the moment and team performing our cloud adoption we are using a cocktail of different solutions that were compatible with our supported OS. Tools such as… When architecting each solution the level of automation varied based on the needs of the environment…
- Of course, in addition to using the cloud to host our ZaaS infrastructure we are able to use Zenoss to monitor the health of that infrastructure. Which in my obviously biased opinion is the best choice to use ZaaS. As we have complete visibility into the health of the infrastructure, the application, and the ZenPacks, there is uniformity in environment deployments which speeds up the mean time to resolution as we can leverage comparative analysis and not waste time blaming the infrastructure, and little mistakes which could lead to hours or days of downtime can be resolved quickly by our team of experts who can interface directly with the engineers. Here are some NOC dashboards we use to display global deployed collectors, and track critical events for our AWS or distributed collector infrastructure. Other graphs give us comparative visibility into CPU, RAM, and Disk health on all customer environments. We rely on a few ZenPacks to monitor the health of our Zenoss environments and they are the following…
- When it comes to deploying Zenoss in AWS, Choosing the right instance type is important for performance. With 4x it was a matter of aligning instance resources with amount of managed resources being monitored With 5x on the other hand it is a matter of assigning instance resources to resource pools. Even with the architectural change I’ve seen that costs between running 4x ZSD and 5x ZSD are very similar. I’ve listed some recommendations on what we’ve had success selecting from AWS in the past… Some customers with 5x are choosing to offload HBASE like we’d previously offloaded MySQL with RDS which I’ll touch on a bit more later. For an easy tool on comparing EC2 instance types, I recommend… Additionally to simplify AWS deployment for customers, Zenoss is currently working on deploying an AMI to the marketplace to automate a lot of the process for standing up Zenoss successfully in the cloud. For more details please reach out to me and I can put you in touch with our Product Manager driving that initiative.
- When it comes to determining how many cloud service accounts to establish, my recommendation is the goldilocks approach… not too big or too small, but somewhere in the middle. Maybe base it on product, platform, initiative, or department. - When too small you can have too many cloud accounts to manage that may push the bounds of what cloud governance tools can accomodate, you will likely have automation issues if you want to dynamically create accounts and tie them to other account networks or billing. - When too big you might be putting items that don’t require or are severely inhibited by security with systems that need to be locked down. You will likely start to hit your head against account soft limits for amount of running instances, storage buckets, networks, etc… While those can be easily lifted at times, you could potentially hit the upper bounds of what is allowed. You might also be putting all your eggs in one basket and have reliability issues by being bound to a single AZ or Region. - If you can’t decide on your own then generally cloud services will happily assist you with the architecture that makes the most sense for you and will try to steer you to something more managable if possible. Your Migration Strategy could result in success or failure of cloud adoption. Planning a mass exodus or migration can overly complicate cloud adoption, scare away the risk averse or cause analysis paralysis that prevents the initiative from ever moving past the design phase. Similarly if cloud adoption is put on hold until product or platform re-architecture can occur then it may be shelved indefinitely due to complexity or potential level of investment required to accomplish. Meanwhile existing architectures could get a performance boost from running as architected in a cloud environment. A major challenge as mentioned earlier with the lift and shift model could result in duplication of effort or processes used by disparate teams varying wildly or becoming antiquated as technology evolves. One team may choose to use Chef and Ruby and another comes along and chooses Ansible and Python. Centralization of these efforts by team makes sense for knowledge retention and tool or process standardization. Conversely to the all-at-once strategy some efficiencies that are discovered analyzing the migration of one workload might not get applied to a workload that was previously migrated. If there is willingness and capability to re-architect to be cloud optimized via microservices or othter best practices then avoid lift and shift if possible.
- Some challenges we’ve experienced running Zenoss is AWS that need to be thought through via certain considerations when planning out your deployment. - Networking… - Security Groups/DNS/Traffic flow… Offloading processes for greater availability, historical record keeping, or managing multiple environments. Do you need to manage changes by running them through a test environment prior to hitting a production one. Should those environments be in different AZs? Different Regions? Make sure you allocate enough resources to allow for performance growth (CPU, RAM, Disk I/O), as well as storage consumption. Isolating backup processes to their own volume. So why didn’t we choose Azure? Well mostly it was timing, but in the last year when we began testing with Azure and a customer chose Azure to deploy they ran into stability challenges. On paper the environment looked like it was architected with more than enough resources, but the distributed infrastructure of Azure was not as seamless as advertised and occasional breakdowns in communication would cause havoc with our application. Issues we’d never experienced with high or lower classes of service with AWS.
- Cloud administration is not simple or easy and at scale can become unworkable if you don’t plan in advance. Some considerations for cost control tool selection really center around the concept of purchasing reserved instances… - Complexity… - What if your users switch from one instance type to another? One AZ to another? Sometimes there are challenges migrating away from 1 instance type to another when factoring in changes like PV to HVM. Different Accounts/Azs What RI purchase cocktail makes the most sense based on my usage and my available budget. What if you choose poorly against a certain AZ or a certain instance type. The marketplace. How am I performing against my purchase once I make one How much are they providing assistance in making this decision, getting visibility into the health of that decision, or helping you manage changes to your decision after the fact. The best thing to do to your environments is to tag everything that you can as part of your automation. Cloud Governance tools are so much more powerful and insightful when they can help reduce noise and add clarity to the reporting and guide you in any business decisions that have to be made. Additionally any automation to stop instances that are not in use, or remove storage, snapshots, etc… that are no longer in use.
- After extensive evaluation we found Cloud Health Tech to be a superior offering far more capable and showing more promise than others. Reserved Instance Recommendations had a budget based modifier. The system proactively sends reports on health and utilization of previous RI purchases. The reporting feature is very strong, and gives you the capacity to automate reports and custom configure them to clear out the noise that Reserved Instances can have in reviewing your data. Such as… Not only does it provide historical data but can view it from different perspectives of use be it service or a custom tagged perspectives like environment, customer, and department. The consistent value it provides is around automated RI modifications, and health checks, Etc… on slide.