a slide researched Project meniscus

1. Project Meniscus
Simplex Internet / Research Institute / Jeong, Ki-Chul
2013. 4. 29

2. Meniscus?
• LAAS(Logging-as-a-Service) Backend API
• Python based system
• Event - collection, transit, processing
• Large-scale
• Cloud Logging

3. SEE THE ANIMATED OVERVIEW

4. APIS

5. Tenant API
• Tenant = {Host, Event Producer, Profile}
• A Tenant = A Consumer
• Event Producer = APP or Service
• Profile = [Event Producer, …]
• Host = Node , link to a profile(1:1)

6. Tenant API
• Requests to coordinator worker
• Tenant, Host, Event Producer, Profile
=> CRUD Available
• has a TOKEN
=> validate, reset a token

7. Correlator API
• Requests
To a Correlator worker server
From a Consumer server
via http(s)
with a Message token(Tenant token)
• Stores a collection named “logs” in Mongo

8. Coordinator API
• Pairing
- registers a worker
• Configuration
– requests a worker’s configuration
• Status
– updates a workers status

9. Status API
• Worker Status API (PUT, Manually)
- System Load Average
- Disk Usage
• Coordinator Status API
- grid status
- individual worker status

10. Broadcaster API
• What’s Brodcaster API?
I don’t know well!
• Communicates with coordinator worker
server.
• New Feature!

11. GETTING STARTED

12. Server deployment example
Host1
• Coordinator
• MongoDB
Host2
• Correlator
Host3
• Storage

13. HOST1
• Creating a development environment
[LINK]
• Install MongoDB
[LINK]

14. uWSGI.ini

15. meniscus/etc/meniscus/meniscus.conf

16. Run uwsgi daemon!!
# uwsgi uwsgi.ini

17. HOST2
• Creating a development environment
[LINK]
• Run uwsgi daemon!!
Default Persona : pairing

18. CURL Request
• Attempts pairing with coordinator
• Becomes correlation worker after automatically restart

19. HOST3
• Creating a development environment
[LINK]
• Run uwsgi daemon!!
Default Persona : pairing

20. CURL Request
• Attempts pairing with coordinator
• Becomes store worker after automatically restart

21. COMPLETED FOR SETTING UP

22. Logging a data
• Already MUST register a tenant
• Request to correlator server(host2)

23. Flow
Correlator
(host2)
Storage
(host3)
MongoDB
(host1)

24. Conclusion
• Well-made backend APIs for logging
(Good architecture)
• Distributed workers processing
• Diverse Persona(worker)
• In early development
• Nobody in its community
(Stack Overflow, Google Groups)

25. Not yet
• Data Store – Only MongoDB
Hadoop / HDFS / Hbase
Elastic Search
• Perfect documentation
need more examples
need exact examples

26. The thing we can now do with meniscus is
JUST to create log data into mongodb via
restful api and to fetch log data from
mongodb.

27. THANKS