SlideShare una empresa de Scribd logo
1 de 22
Descargar para leer sin conexión
the NML project
   <yanglei@snda.com>
Before we start...

  This is a purely technical discussion,
  don’t bring politics in. That is:
• Which dept. should in charge?

• Why not develop in PHP/Java because nobody else in the
  company can program in Ruby.

• How to integrate NML into XX system?
Goal

Out-of-band Server Management

Extremely configurable OS install via
SOL(Serial Over Lan)

An intelligent system to control the whole
process, minimum human intervention

Build an open-source matrix for Server/OS
distro combinations
Status
Member : me      wangjunyan (docs)

Subproject Member: lijiehui (LXC:
Linux container environment)

github:   https://github.com/op-sdo-com/nml


                    Fork us!
Status


Special thanks to wangjunyan
      and dinghaifeng!
HP is a coward(think
    their WebOS)
IBM, Dell, HP

HP closed ipmi port(udp 623), force
customers to use iLO.

Practically, iLO is okay. But you need
to buy a license before using Remote
console redirection while IBM & Dell
let you do anything!
Work through

10.132.17.100-150 (prod. IP range)

10.132.17.200-250 (IPMI IP range)

One-to-One mapping (dynamic IP
allocation is just impossible for now,
but this can be improved)

The current solution is neither secure nor
sufficiently isolated.
Work through
1.    Set to boot from PXE then restart:

     ipmitool -I lanplus -U ibm3550 -H 10.132.17.200 -P XX chassis bootdev
     pxe

     ipmitool -I lanplus -U ibm3550 -H 10.132.17.200 -P XX chassis power
     cycle


2.    Configure DHCP sever to reply by MAC and refuse any
     other DHCP request(!!)

     PS   dhcp3 supports dynamic configuration update via
     OMAPI. see man dhcpd.conf
Architecture

NML’s encapsulates all the
intelligence in HTTP.

DHCP and iPXE configurations are kept
to a minimum.

Centralized configuration is easy to
maintain.
Work through
host aoti_200 {
    # eth0, eth1
    hardware ethernet 00:1A:64:99:E7:50;
    # hardware ethernet 00:1A:64:99:E7:52;
    fixed-address 10.132.17.109;
    server-name "10.132.17.108";
    if exists user-class and option user-class = "iPXE" {
        filename "http://10.132.17.108/nml/ipxe";
    } else {
        filename "undionly.kpxe";
    }
}
Work through

            iPXE V.S. PXE

iPXE liberate us from TFTP(stupid
UDP). iPXE supports HTTP(even iSCSI),
so the system scales.

iPXE lays the foundation to an
automatic assessment management
platform.
Work through
#!ipxe

chain http://nml.snda.com/nml/chain/${manufacturer}/$
{product}/${uuid}?mac=${net0/mac}

${manufacturer}, ${product}, ${uuid}, ${net0/mac} are
variables exposed by BIOS.

Human make mistakes but BIOS are not.

PS: This is probably the earliest stage to obtain
hardware info. Early == Accurate
Work through
From now on, all the network
communication is done through HTTP.

Also, the intelligence comes in:
 get '/nml/pxelinux.cfg/:uuid' do

   uuid = params[:uuid]

    install(uuid, get_ipaddr(uuid), get_gateway(uuid),
get_hostname(uuid), get_iface(uuid), get_baudrate(uuid),
get_release(uuid))


 end
Work through
def install(uuid, ipaddr, gateway, hostname, iface, baudrate, release)
  indent = ' ' * 4
  head = "serial 0 #{baudrate}ntimeout 50nlabel pxeboot"
  tail = "default ubuntu-installer/amd64/boot-screens/vesamenu.c32"
  kernel = indent + "kernel %s/linux" % [release]
  # static ip configuration, avoid dhcp in the preseeding stage
  configs = [
      "console-tools/archs=skip-config",   "console-keymaps-at/keymap=us",
      "vga=normal", "netcfg/confirm_static=true", "netcfg/disable_dhcp=true",
      "netcfg/get_hostname=#{hostname}", "netcfg/get_domain=.nml",
      "netcfg/get_nameservers=%s" % [@@dns],
      "netcfg/get_ipaddress=#{ipaddr}", "netcfg/get_netmask=255.255.255.0",
      "netcfg/get_gateway=#{gateway}",   "console=ttyS0,#{baudrate}n8",
      "interface=#{iface}", "initrd=#{release}/initrd.gz",
       "auto url=http://%s/%s/preseed/#{uuid}" % [@@master, @@base]
  ]


  append = indent + 'append ' + configs.join(' ') + ' -- quiet'
  [head, kernel, append, tail].join("n") + "n"
end
Architecture

What’s is preseed?


Preseed is kickstart for Debian.


Kickstart is answers to questions when
you manually install a system.
Architecture

NML tries to provide maximum flexibility from the bottom.


Policy makers decided how to utilize it.


Maximum flexibility == Each machine can pull its own
configuration set.


NML tries hard to be OS/Hardware independent. (Goal 3:
build a matrix)
Architecture
     I know real world op desperately want consistency, but
     this is policy.


     NML focus on Mechanism.


     Why flexibility matters? Any real world examples?


1.   Let the system generate distinct password for every machine. I love elegant
     solution to security.
2.   Gain access to partition manager. (ext3, ext4, btrfs and LVM!)

3.   Move prelinux script to the preseeding stage ensure a continuous integration
     of company policy (Lessons: Polices can never be applied without powerful
     infra.)
4.   Automatic network interfaces configuration. Ubuntu installer smartly apply
     network configuration to /etc/network/interfaces, so does CentOS’s anaconda.
Architecture
    Preseed/Kickstart V.S. Image clone


•   Preseeding is slow. Although installer could utilize yum/apt mirror to speed
    up package downloading, the entire retrieve-prepare-configure cycle can’t be
    optimized further.

•   Image clone is suitable for creating VM.(Xen, LXC, etc) But it is too dump

    to do anything intelligence.




    But we want the best of both world!
    Solution:

        n_preseed = normailize(uuid.preseed, uuid.hardware)
       n_preseed.exists?
         n_preseed.clone(server_ip, uuid)
       else
         install(uuid)
Architecture
1. Yum/Apt mirror ensure 99% cache hit, all the packages are
   pulled from LAN. Local master only maintain cache.

2. Why not directly mirror upstream repo.?
     1.   The bandwidth of upstream mirror is likely to fluctuate(e.g.,
          us.archieve.ubuntu.com)
     2.   Most packages will never be downloaded. In fact, the standard
          installation of CentOS 6.0 only needs less than 380 packages where a
          full fledged repo contains 15K. (2.5%)



3. Repo. implementations
     1.   Yum: nginx error_page + proxy_pass + ppull.rb

          upstream mirror: mirrors.sdo.com (Why not proxy_cache? Because nginx
          has some issue with range-request when proxy_cache is enabled.)

     2.   Apt: apt-cacher-ng

          upstream mirror: mirror.lupaworld.com
The Matrix
          Ubuntu   Ubuntu   CentOS   CentOS   RHEL   RHEL                                         Arch
                                                            FreeBSD   Gentoo   Fedora   Debian
           10.04    11.04     5.6      6.0     5.6    6.0                                        Linux


  IBM
 x3550      Y        Y        Y        Y
   HP
Prolian
t DL360
   G5

  IBM
 x3550
   M2

  Dell
PowerEd
ge R610

   HP
Prolian
t DL385
   G2

   IBM
BladeCe
  nter
  LS22

• Y means both i386 and amd64 is passed
• Y* means M[ij] needs extra configuration
Architecture
1. Why hardware has dependency on OS distro.?

Every OS distro. may bring surprise. e.g.
Ubuntu-11.04(codename natty)’s radeon card drivers is
incompatible with IBM x3550. You got kernel panic after
installation.

2. What’s the purpose to support all Linux distro.?

 •   We want Total World Domination
 •   NML is about mechanism not policy
 •   Linode supports all distro. on Xen! Our task is easier.

3. Is it time-consuming to support all linux distro.?

Just do it.
Questions?



One obvious question: What is NML?

Más contenido relacionado

La actualidad más candente

NFD9 - Matt Peterson, Data Center Operations
NFD9 - Matt Peterson, Data Center OperationsNFD9 - Matt Peterson, Data Center Operations
NFD9 - Matt Peterson, Data Center OperationsCumulus Networks
 
NVDIMM block drivers with NFIT
NVDIMM block drivers with NFITNVDIMM block drivers with NFIT
NVDIMM block drivers with NFITjoeylikernel
 
OSDC 2014: Nat Morris - Open Network Install Environment
OSDC 2014: Nat Morris - Open Network Install EnvironmentOSDC 2014: Nat Morris - Open Network Install Environment
OSDC 2014: Nat Morris - Open Network Install EnvironmentNETWAYS
 
[OpenStack Day in Korea 2015] Track 1-6 - 갈라파고스의 이구아나, 인프라에 오픈소스를 올리다. 그래서 보이...
[OpenStack Day in Korea 2015] Track 1-6 - 갈라파고스의 이구아나, 인프라에 오픈소스를 올리다. 그래서 보이...[OpenStack Day in Korea 2015] Track 1-6 - 갈라파고스의 이구아나, 인프라에 오픈소스를 올리다. 그래서 보이...
[OpenStack Day in Korea 2015] Track 1-6 - 갈라파고스의 이구아나, 인프라에 오픈소스를 올리다. 그래서 보이...OpenStack Korea Community
 
High performance content hosting
High performance content hosting High performance content hosting
High performance content hosting Aleksey Korzun
 
XPDS16: Xenbedded: Xen-based client virtualization for phones and tablets - ...
XPDS16:  Xenbedded: Xen-based client virtualization for phones and tablets - ...XPDS16:  Xenbedded: Xen-based client virtualization for phones and tablets - ...
XPDS16: Xenbedded: Xen-based client virtualization for phones and tablets - ...The Linux Foundation
 
System Device Tree update: Bus Firewalls and Lopper
System Device Tree update: Bus Firewalls and LopperSystem Device Tree update: Bus Firewalls and Lopper
System Device Tree update: Bus Firewalls and LopperStefano Stabellini
 
XPDDS19: [ARM] OP-TEE Mediator in Xen - Volodymyr Babchuk, EPAM Systems
XPDDS19: [ARM] OP-TEE Mediator in Xen - Volodymyr Babchuk, EPAM SystemsXPDDS19: [ARM] OP-TEE Mediator in Xen - Volodymyr Babchuk, EPAM Systems
XPDDS19: [ARM] OP-TEE Mediator in Xen - Volodymyr Babchuk, EPAM SystemsThe Linux Foundation
 
4. open mano set up and usage
4. open mano set up and usage4. open mano set up and usage
4. open mano set up and usagevideos
 
Building a Two Node SLES 11 SP2 Linux Cluster with VMware
Building a Two Node SLES 11 SP2 Linux Cluster with VMwareBuilding a Two Node SLES 11 SP2 Linux Cluster with VMware
Building a Two Node SLES 11 SP2 Linux Cluster with VMwaregeekswing
 
[2015-11월 정기 세미나]K8s on openstack
[2015-11월 정기 세미나]K8s on openstack[2015-11월 정기 세미나]K8s on openstack
[2015-11월 정기 세미나]K8s on openstackOpenStack Korea Community
 
XPDS16: libvirt and Tools: What's New and What's Next - James Fehlig, SUSE
XPDS16: libvirt and Tools: What's New and What's Next - James Fehlig, SUSEXPDS16: libvirt and Tools: What's New and What's Next - James Fehlig, SUSE
XPDS16: libvirt and Tools: What's New and What's Next - James Fehlig, SUSEThe Linux Foundation
 
Securing Your Linux System
Securing Your Linux SystemSecuring Your Linux System
Securing Your Linux SystemNovell
 
Hardening Linux and introducing Securix Linux
Hardening Linux and introducing Securix LinuxHardening Linux and introducing Securix Linux
Hardening Linux and introducing Securix LinuxSecurity Session
 
OffensiveCon2022: Case Studies of Fuzzing with Xen
OffensiveCon2022: Case Studies of Fuzzing with XenOffensiveCon2022: Case Studies of Fuzzing with Xen
OffensiveCon2022: Case Studies of Fuzzing with XenTamas K Lengyel
 
ONIE / Cumulus Networks Webinar
ONIE / Cumulus Networks WebinarONIE / Cumulus Networks Webinar
ONIE / Cumulus Networks WebinarCumulus Networks
 
XPDS16: The OpenXT Project in 2016 - Christopher Clark, BAE Systems
XPDS16: The OpenXT Project in 2016 - Christopher Clark, BAE SystemsXPDS16: The OpenXT Project in 2016 - Christopher Clark, BAE Systems
XPDS16: The OpenXT Project in 2016 - Christopher Clark, BAE SystemsThe Linux Foundation
 
VM Forking and Hypervisor-based fuzzing
VM Forking and Hypervisor-based fuzzingVM Forking and Hypervisor-based fuzzing
VM Forking and Hypervisor-based fuzzingTamas K Lengyel
 

La actualidad más candente (20)

NFD9 - Matt Peterson, Data Center Operations
NFD9 - Matt Peterson, Data Center OperationsNFD9 - Matt Peterson, Data Center Operations
NFD9 - Matt Peterson, Data Center Operations
 
NVDIMM block drivers with NFIT
NVDIMM block drivers with NFITNVDIMM block drivers with NFIT
NVDIMM block drivers with NFIT
 
OSDC 2014: Nat Morris - Open Network Install Environment
OSDC 2014: Nat Morris - Open Network Install EnvironmentOSDC 2014: Nat Morris - Open Network Install Environment
OSDC 2014: Nat Morris - Open Network Install Environment
 
[OpenStack Day in Korea 2015] Track 1-6 - 갈라파고스의 이구아나, 인프라에 오픈소스를 올리다. 그래서 보이...
[OpenStack Day in Korea 2015] Track 1-6 - 갈라파고스의 이구아나, 인프라에 오픈소스를 올리다. 그래서 보이...[OpenStack Day in Korea 2015] Track 1-6 - 갈라파고스의 이구아나, 인프라에 오픈소스를 올리다. 그래서 보이...
[OpenStack Day in Korea 2015] Track 1-6 - 갈라파고스의 이구아나, 인프라에 오픈소스를 올리다. 그래서 보이...
 
High performance content hosting
High performance content hosting High performance content hosting
High performance content hosting
 
XPDS16: Xenbedded: Xen-based client virtualization for phones and tablets - ...
XPDS16:  Xenbedded: Xen-based client virtualization for phones and tablets - ...XPDS16:  Xenbedded: Xen-based client virtualization for phones and tablets - ...
XPDS16: Xenbedded: Xen-based client virtualization for phones and tablets - ...
 
System Device Tree update: Bus Firewalls and Lopper
System Device Tree update: Bus Firewalls and LopperSystem Device Tree update: Bus Firewalls and Lopper
System Device Tree update: Bus Firewalls and Lopper
 
XPDDS19: [ARM] OP-TEE Mediator in Xen - Volodymyr Babchuk, EPAM Systems
XPDDS19: [ARM] OP-TEE Mediator in Xen - Volodymyr Babchuk, EPAM SystemsXPDDS19: [ARM] OP-TEE Mediator in Xen - Volodymyr Babchuk, EPAM Systems
XPDDS19: [ARM] OP-TEE Mediator in Xen - Volodymyr Babchuk, EPAM Systems
 
4. open mano set up and usage
4. open mano set up and usage4. open mano set up and usage
4. open mano set up and usage
 
Building a Two Node SLES 11 SP2 Linux Cluster with VMware
Building a Two Node SLES 11 SP2 Linux Cluster with VMwareBuilding a Two Node SLES 11 SP2 Linux Cluster with VMware
Building a Two Node SLES 11 SP2 Linux Cluster with VMware
 
[2015-11월 정기 세미나]K8s on openstack
[2015-11월 정기 세미나]K8s on openstack[2015-11월 정기 세미나]K8s on openstack
[2015-11월 정기 세미나]K8s on openstack
 
XPDS16: libvirt and Tools: What's New and What's Next - James Fehlig, SUSE
XPDS16: libvirt and Tools: What's New and What's Next - James Fehlig, SUSEXPDS16: libvirt and Tools: What's New and What's Next - James Fehlig, SUSE
XPDS16: libvirt and Tools: What's New and What's Next - James Fehlig, SUSE
 
Securing Your Linux System
Securing Your Linux SystemSecuring Your Linux System
Securing Your Linux System
 
Hardening Linux and introducing Securix Linux
Hardening Linux and introducing Securix LinuxHardening Linux and introducing Securix Linux
Hardening Linux and introducing Securix Linux
 
OffensiveCon2022: Case Studies of Fuzzing with Xen
OffensiveCon2022: Case Studies of Fuzzing with XenOffensiveCon2022: Case Studies of Fuzzing with Xen
OffensiveCon2022: Case Studies of Fuzzing with Xen
 
ONIE / Cumulus Networks Webinar
ONIE / Cumulus Networks WebinarONIE / Cumulus Networks Webinar
ONIE / Cumulus Networks Webinar
 
XPDS16: The OpenXT Project in 2016 - Christopher Clark, BAE Systems
XPDS16: The OpenXT Project in 2016 - Christopher Clark, BAE SystemsXPDS16: The OpenXT Project in 2016 - Christopher Clark, BAE Systems
XPDS16: The OpenXT Project in 2016 - Christopher Clark, BAE Systems
 
VM Forking and Hypervisor-based fuzzing
VM Forking and Hypervisor-based fuzzingVM Forking and Hypervisor-based fuzzing
VM Forking and Hypervisor-based fuzzing
 
kdump: usage and_internals
kdump: usage and_internalskdump: usage and_internals
kdump: usage and_internals
 
Xen in Linux 3.x (or PVOPS)
Xen in Linux 3.x (or PVOPS)Xen in Linux 3.x (or PVOPS)
Xen in Linux 3.x (or PVOPS)
 

Similar a the NML project

Instalando Cacti no CentOS 5
Instalando Cacti no CentOS 5Instalando Cacti no CentOS 5
Instalando Cacti no CentOS 5Carlos Eduardo
 
Tuning systemd for embedded
Tuning systemd for embeddedTuning systemd for embedded
Tuning systemd for embeddedAlison Chaiken
 
Minimal OpenStack LinuxCon NA 2015
Minimal OpenStack LinuxCon NA 2015Minimal OpenStack LinuxCon NA 2015
Minimal OpenStack LinuxCon NA 2015Sean Dague
 
System Imager.20051215
System Imager.20051215System Imager.20051215
System Imager.20051215guest95b42b
 
Network Stack in Userspace (NUSE)
Network Stack in Userspace (NUSE)Network Stack in Userspace (NUSE)
Network Stack in Userspace (NUSE)Hajime Tazaki
 
Rmll Virtualization As Is Tool 20090707 V1.0
Rmll Virtualization As Is Tool 20090707 V1.0Rmll Virtualization As Is Tool 20090707 V1.0
Rmll Virtualization As Is Tool 20090707 V1.0guest72e8c1
 
2015.10.05 Updated > Network Device Development - Part 1: Switch
2015.10.05 Updated > Network Device Development - Part 1: Switch2015.10.05 Updated > Network Device Development - Part 1: Switch
2015.10.05 Updated > Network Device Development - Part 1: SwitchCheng-Yi Yu
 
Practical Tips for Novell Cluster Services
Practical Tips for Novell Cluster ServicesPractical Tips for Novell Cluster Services
Practical Tips for Novell Cluster ServicesNovell
 
A Fabric/Puppet Build/Deploy System
A Fabric/Puppet Build/Deploy SystemA Fabric/Puppet Build/Deploy System
A Fabric/Puppet Build/Deploy Systemadrian_nye
 

Similar a the NML project (20)

Instalando Cacti no CentOS 5
Instalando Cacti no CentOS 5Instalando Cacti no CentOS 5
Instalando Cacti no CentOS 5
 
.ppt
.ppt.ppt
.ppt
 
Xen time machine
Xen time machineXen time machine
Xen time machine
 
Slim Server Theory
Slim Server TheorySlim Server Theory
Slim Server Theory
 
LSA2 - 02 Namespaces
LSA2 - 02  NamespacesLSA2 - 02  Namespaces
LSA2 - 02 Namespaces
 
Tuning systemd for embedded
Tuning systemd for embeddedTuning systemd for embedded
Tuning systemd for embedded
 
Minimal OpenStack LinuxCon NA 2015
Minimal OpenStack LinuxCon NA 2015Minimal OpenStack LinuxCon NA 2015
Minimal OpenStack LinuxCon NA 2015
 
Howto Pxeboot
Howto PxebootHowto Pxeboot
Howto Pxeboot
 
System Imager.20051215
System Imager.20051215System Imager.20051215
System Imager.20051215
 
Network Stack in Userspace (NUSE)
Network Stack in Userspace (NUSE)Network Stack in Userspace (NUSE)
Network Stack in Userspace (NUSE)
 
Rmll Virtualization As Is Tool 20090707 V1.0
Rmll Virtualization As Is Tool 20090707 V1.0Rmll Virtualization As Is Tool 20090707 V1.0
Rmll Virtualization As Is Tool 20090707 V1.0
 
RMLL / LSM 2009
RMLL / LSM 2009RMLL / LSM 2009
RMLL / LSM 2009
 
Lab Network+
Lab Network+Lab Network+
Lab Network+
 
Interview Questions
Interview QuestionsInterview Questions
Interview Questions
 
Rac on NFS
Rac on NFSRac on NFS
Rac on NFS
 
2015.10.05 Updated > Network Device Development - Part 1: Switch
2015.10.05 Updated > Network Device Development - Part 1: Switch2015.10.05 Updated > Network Device Development - Part 1: Switch
2015.10.05 Updated > Network Device Development - Part 1: Switch
 
Linux scheduler
Linux schedulerLinux scheduler
Linux scheduler
 
Fuzzing_with_Xen.pdf
Fuzzing_with_Xen.pdfFuzzing_with_Xen.pdf
Fuzzing_with_Xen.pdf
 
Practical Tips for Novell Cluster Services
Practical Tips for Novell Cluster ServicesPractical Tips for Novell Cluster Services
Practical Tips for Novell Cluster Services
 
A Fabric/Puppet Build/Deploy System
A Fabric/Puppet Build/Deploy SystemA Fabric/Puppet Build/Deploy System
A Fabric/Puppet Build/Deploy System
 

Último

Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 

Último (20)

Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 

the NML project

  • 1. the NML project <yanglei@snda.com>
  • 2. Before we start... This is a purely technical discussion, don’t bring politics in. That is: • Which dept. should in charge? • Why not develop in PHP/Java because nobody else in the company can program in Ruby. • How to integrate NML into XX system?
  • 3. Goal Out-of-band Server Management Extremely configurable OS install via SOL(Serial Over Lan) An intelligent system to control the whole process, minimum human intervention Build an open-source matrix for Server/OS distro combinations
  • 4. Status Member : me wangjunyan (docs) Subproject Member: lijiehui (LXC: Linux container environment) github: https://github.com/op-sdo-com/nml Fork us!
  • 5. Status Special thanks to wangjunyan and dinghaifeng!
  • 6. HP is a coward(think their WebOS) IBM, Dell, HP HP closed ipmi port(udp 623), force customers to use iLO. Practically, iLO is okay. But you need to buy a license before using Remote console redirection while IBM & Dell let you do anything!
  • 7. Work through 10.132.17.100-150 (prod. IP range) 10.132.17.200-250 (IPMI IP range) One-to-One mapping (dynamic IP allocation is just impossible for now, but this can be improved) The current solution is neither secure nor sufficiently isolated.
  • 8. Work through 1. Set to boot from PXE then restart: ipmitool -I lanplus -U ibm3550 -H 10.132.17.200 -P XX chassis bootdev pxe ipmitool -I lanplus -U ibm3550 -H 10.132.17.200 -P XX chassis power cycle 2. Configure DHCP sever to reply by MAC and refuse any other DHCP request(!!) PS dhcp3 supports dynamic configuration update via OMAPI. see man dhcpd.conf
  • 9. Architecture NML’s encapsulates all the intelligence in HTTP. DHCP and iPXE configurations are kept to a minimum. Centralized configuration is easy to maintain.
  • 10. Work through host aoti_200 { # eth0, eth1 hardware ethernet 00:1A:64:99:E7:50; # hardware ethernet 00:1A:64:99:E7:52; fixed-address 10.132.17.109; server-name "10.132.17.108"; if exists user-class and option user-class = "iPXE" { filename "http://10.132.17.108/nml/ipxe"; } else { filename "undionly.kpxe"; } }
  • 11. Work through iPXE V.S. PXE iPXE liberate us from TFTP(stupid UDP). iPXE supports HTTP(even iSCSI), so the system scales. iPXE lays the foundation to an automatic assessment management platform.
  • 12. Work through #!ipxe chain http://nml.snda.com/nml/chain/${manufacturer}/$ {product}/${uuid}?mac=${net0/mac} ${manufacturer}, ${product}, ${uuid}, ${net0/mac} are variables exposed by BIOS. Human make mistakes but BIOS are not. PS: This is probably the earliest stage to obtain hardware info. Early == Accurate
  • 13. Work through From now on, all the network communication is done through HTTP. Also, the intelligence comes in: get '/nml/pxelinux.cfg/:uuid' do uuid = params[:uuid] install(uuid, get_ipaddr(uuid), get_gateway(uuid), get_hostname(uuid), get_iface(uuid), get_baudrate(uuid), get_release(uuid)) end
  • 14. Work through def install(uuid, ipaddr, gateway, hostname, iface, baudrate, release) indent = ' ' * 4 head = "serial 0 #{baudrate}ntimeout 50nlabel pxeboot" tail = "default ubuntu-installer/amd64/boot-screens/vesamenu.c32" kernel = indent + "kernel %s/linux" % [release] # static ip configuration, avoid dhcp in the preseeding stage configs = [ "console-tools/archs=skip-config", "console-keymaps-at/keymap=us", "vga=normal", "netcfg/confirm_static=true", "netcfg/disable_dhcp=true", "netcfg/get_hostname=#{hostname}", "netcfg/get_domain=.nml", "netcfg/get_nameservers=%s" % [@@dns], "netcfg/get_ipaddress=#{ipaddr}", "netcfg/get_netmask=255.255.255.0", "netcfg/get_gateway=#{gateway}", "console=ttyS0,#{baudrate}n8", "interface=#{iface}", "initrd=#{release}/initrd.gz", "auto url=http://%s/%s/preseed/#{uuid}" % [@@master, @@base] ] append = indent + 'append ' + configs.join(' ') + ' -- quiet' [head, kernel, append, tail].join("n") + "n" end
  • 15. Architecture What’s is preseed? Preseed is kickstart for Debian. Kickstart is answers to questions when you manually install a system.
  • 16. Architecture NML tries to provide maximum flexibility from the bottom. Policy makers decided how to utilize it. Maximum flexibility == Each machine can pull its own configuration set. NML tries hard to be OS/Hardware independent. (Goal 3: build a matrix)
  • 17. Architecture I know real world op desperately want consistency, but this is policy. NML focus on Mechanism. Why flexibility matters? Any real world examples? 1. Let the system generate distinct password for every machine. I love elegant solution to security. 2. Gain access to partition manager. (ext3, ext4, btrfs and LVM!) 3. Move prelinux script to the preseeding stage ensure a continuous integration of company policy (Lessons: Polices can never be applied without powerful infra.) 4. Automatic network interfaces configuration. Ubuntu installer smartly apply network configuration to /etc/network/interfaces, so does CentOS’s anaconda.
  • 18. Architecture Preseed/Kickstart V.S. Image clone • Preseeding is slow. Although installer could utilize yum/apt mirror to speed up package downloading, the entire retrieve-prepare-configure cycle can’t be optimized further. • Image clone is suitable for creating VM.(Xen, LXC, etc) But it is too dump to do anything intelligence. But we want the best of both world! Solution: n_preseed = normailize(uuid.preseed, uuid.hardware) n_preseed.exists? n_preseed.clone(server_ip, uuid) else install(uuid)
  • 19. Architecture 1. Yum/Apt mirror ensure 99% cache hit, all the packages are pulled from LAN. Local master only maintain cache. 2. Why not directly mirror upstream repo.? 1. The bandwidth of upstream mirror is likely to fluctuate(e.g., us.archieve.ubuntu.com) 2. Most packages will never be downloaded. In fact, the standard installation of CentOS 6.0 only needs less than 380 packages where a full fledged repo contains 15K. (2.5%) 3. Repo. implementations 1. Yum: nginx error_page + proxy_pass + ppull.rb upstream mirror: mirrors.sdo.com (Why not proxy_cache? Because nginx has some issue with range-request when proxy_cache is enabled.) 2. Apt: apt-cacher-ng upstream mirror: mirror.lupaworld.com
  • 20. The Matrix Ubuntu Ubuntu CentOS CentOS RHEL RHEL Arch FreeBSD Gentoo Fedora Debian 10.04 11.04 5.6 6.0 5.6 6.0 Linux IBM x3550 Y Y Y Y HP Prolian t DL360 G5 IBM x3550 M2 Dell PowerEd ge R610 HP Prolian t DL385 G2 IBM BladeCe nter LS22 • Y means both i386 and amd64 is passed • Y* means M[ij] needs extra configuration
  • 21. Architecture 1. Why hardware has dependency on OS distro.? Every OS distro. may bring surprise. e.g. Ubuntu-11.04(codename natty)’s radeon card drivers is incompatible with IBM x3550. You got kernel panic after installation. 2. What’s the purpose to support all Linux distro.? • We want Total World Domination • NML is about mechanism not policy • Linode supports all distro. on Xen! Our task is easier. 3. Is it time-consuming to support all linux distro.? Just do it.

Notas del editor

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n