Ampd is a proposed music streaming platform that allows users to upload, listen to, and discuss music. It aims to give smaller artists exposure by focusing on hosting a wide range of music rather than just popular songs. The platform would allow live streaming of new music from artists, as well as social media integration and data analysis features to recommend similar songs and provide analytics to help artists. The proposed software would be built using Node.js for the backend server, React for the frontend client, and MongoDB for the database. Key features would include the ability to upload and stream songs, an online chat for discussing music, and analytics dashboards for artists.
The Ultimate Test Automation Guide_ Best Practices and Tips.pdf
Ampd (Technical Report)
1. Ampd: A music platform for users to share, listen and
discuss music
Afnan Ur Rehman (104726426), Andrew Martinho (104183524),
Jeremiah Gilbert (104191769), Vince Zhang (104431047)
Abstract: Ampd is a proposed solution to the problem of current music streaming services catering only well known
artists. It allows for users to live stream music along with a chat system for users to discuss their favorite songs. Proposed
software will include the ability to have the artist stream new music live without having to upload, similar to current
video streaming services like Twitch, along with social media integration. We also propose a data analysis system to
further help connect small artists to their potential fan base. The data analysis would recommend songs to users that are
similar to other songs they have played recently, along with songs on which they have commented or reviewed positively.
Furthermore, data analysis will be used to provide a dashboard to artists, visualizing the feedback on their work so that
they can use it to influence their future music for better outreach and appraisal.
Index Terms— Bcrypt, React, Sockets, Multer, NoSQL, MongoDB, Express.js, Node.js, ORM, Mongoose and Cloudinary
I. INTRODUCTION1
Ampd is a music streaming service that allows users and
artists to upload their music and listen to it wherever they
have access to an internet connection. Artists also have the
option of letting other people listen to their music to help
them gain access to larger platforms they otherwise
wouldn’t be qualified for.
This project came as a way to fulfill the desire to have a
user’s entire music collection in one place without the issue
of some songs not being available on other services due to
mutually exclusive licensing agreements or labels that are
too small to be noticed by large streaming services.
The widespread use of music streaming services has lead to
a necessity to have songs on as many platforms as possible,
however some artists who are unable to sign onto labels
with agreements with these services cannot get their music
onto the platforms that will get the most attention.
Ampd’s focus on hosting the music users want to hear and
de-emphasis on providing the songs that are on average the
most popular allows for more niche and unheard of artists
having a higher chance of being recognized for their work.
II. LITERATURE REVIEW
Since the Ampd project is designed to be a real-time service
for music streaming, the technology used in its design needs
to be suitably fast. In a comparison between six web servers
running on instanced machines, Node.js was within
milliseconds of average running time with a C#/F# based
server system. Although the speed wasn’t the best out of
those compared, the speed measured was more than enough
to satisfy real-time performance concerns. In addition, the
process of development would be made easier due to
Node.js massive library of modules to extend functionality
to whatever is required. [1]
The performance of React must also be considered, as the
frontend is tied deeply to the user interface, and thus if the
UI doesn’t feel responsive, it will hurt user experience.
Caution must be used when using React, as the framework
makes development easy, but also has a side effect of
making poor performance easy to fall into. Design of the
React components must be done very cautiously to avoid as
many unnecessary re-renders as possible. If this can be
achieved, then React will be useful for its ease of
development and its acceptable speed for the purposes
required.
The choice of database engine is a very important
consideration for the application. Since the application
relies on a large amount of data per user, mostly containing
the metadata of hundreds to thousands of songs per user,
fast read and write speeds of the database are a high
priority. To this end, a NoSQL database is the optimal
choice over an SQL database. SQL databases use strict
schemas that ensure data integrity with each operation
performed, however this slows down the operation of the
database considerably. NoSQL databases forego this
schema adherence and instead uses a dynamic schema that
can change with each addition of data. This improvement in
speed is much better for scalability of large data
infrastructure such as what would be required by this
project. [2]
2. III. PROJECT DETAILS AND METHODOLOGY
A. Methodology
The project was completed using an iterative and
incremental approach to development. The design of the
system lead to minimal impact from changes between
systems. For instance, the login system could be completely
redone as long as the end result is a valid authentication
token on the user’s machine. This was done to allow for
rapid priority shifts as the project went forward.
B. Definitions
ORM: Object-Relational Mapping is a paradigm which
maps the attributes in object oriented language to attributes
and in database. This makes querying and persisting data in
database easier.
Middleware: Middleware is kind of glue which connects
two components together. E.g. connecting Client side of the
application to a Cloud web server.
Cloud Computing: In simple words: ‘Instead of using local
servers and resources, using servers, storage, databases,
network, software and analytics over the internet is called
Cloud Computing’.
NoSQL DB: A NoSQL database is non-relational database
which means it is schema-less. There are four kinds of
NoSQL databases, which are; Document databases, Key-
value stores, Wide column stores and Graph stores.
Web Socket: It is a communication protocol used over a
single TCP connection It provides two-way communication
between client and server.
C. Specification
The application is divided into three main components: The
front end client application which is used to interface
between the user and the available data; The server, which
acts to manipulate the data in storage based on the requests
received from the client interface; And the database, a
collection of all information required for the application to
function. This can be further subdivided into the local
information, stored on the same network as the server, and
the cloud storage, a much larger reserve stored at a remote
location.
There are two types of users to consider in the application.
General users have the ability to create accounts, log in,
upload music, and stream it later. Administrators have
enhanced privileges, including reading and acting upon
reports of misuse of the service.
Fig 1.0
Users should not be able to access any part of the
application except the sign in/sign up forms without a valid
authentication token. Administrators should not be able to
access administrative privileges without an authentication
token associated with an administrator account.
Fig 2.0
After a user has been authenticated, they should be allowed
to browse music that they have uploaded and any music that
an artist has uploaded and made public. Upon playback of a
song, an online chat client specific to that song/artist should
be displayed, in addition to playback of the specified song.
3. D. Architecture and Design
Fig 3.0
The Node server is the central point of the design of the
system. It contains two major subcomponents. One to
manage stream requests and playback of audio streams, and
another to handle the peer-to-peer connection of websockets
from one user to another. Any information sent by the
server to the user should be bundled in Webpack collections
to increase the data-overhead ratio as much as possible. The
bundled data will be sent to the user and displayed on a
React based frontend.
The following DFD diagrams highlight the flow of data
through various phases in our application for different
functionalities.
Fig 4.0
The information for user information and song metadata
will be stored in a MongoDB database. The Node server
will query this database to authenticate users and to
populate the song browser the user sees. A separate cloud
storage provider will hold the song data, and be retrieved
when a user requests playback of a song. The database and
cloud storage will communicate whenever a song is
uploaded or requested for playback by sending the
requested song identifier to the cloud storage, which will
respond by sending the audio to be streamed to the server’s
stream provider.
The following Sequence diagram shows the behavior of our
application:
Fig 5.0
Figure 6.0 further elaborates the behavior of our application
using activity diagrams.
Fig 6.0
E. Platform
The project relies on a number of web platform
technologies, including Node.js, React, Webpack,
MongoDB, and Cloudinary.
4. Node.js is a JavaScript based web server notable for its
module system which allows for additional functionality to
be added quickly as necessary. Libraries include data
parsing and encryption algorithms which have been tested
hundreds of thousands of times. This extensibility made
node a good choice for our purposes as it allowed for
additional functionality to be added quickly.
The client to the application was designed with React, a
front end framework capable of creating single page
applications and which utilizes a model-view-controller
model of application design, which is particularly well
suited to database applications. As an addition to React,
Webpack comes as an efficient method of sending a large
number of files to a web browser by making fewer GET
requests to retrieve the required data to display a page.
The database driver that was decided was the MongoDB
driver to allow for fast changes in the data that was stored,
and to allow for a large number of concurrent accesses
which is a necessary consideration for a streaming platform.
Finally, the uploaded music is stored in cloud storage
through the provider Cloudinary. This was done as the
potential number of songs to be stored along with the
average size of high quality audio files could result in on
site storage being exceeded with relatively few active users.
By using a cloud storage host, the need for excess storage is
mitigated by simply requesting more storage as needed
instead of being responsible for a large storage array.
IV. IMPLEMENTATION
A. Client Side
Ampd is a Single page application, where only a certain
section of the application is updated depending on the flow
directed by the client, rather than having separate pages for
each functionality linked together.
The landing page of the application prompts the user to
either “Sign in” if he/she already have credentials, or “Sign
up” otherwise. For Sign in into the application, the user has
to enter their email address and password, and for the Sign
Up, user has to enter his/her email address, desired
password, first name, last name and agree to the terms of
service. We have applied some preliminary client side
checks such as validating the length, format of input, and
required fields.
Fig 7.0
Fig 8.0
Since our application is a single page application, we have
made use of React and React router frameworks to allow
the users to switch between the options of Sign in and Sign
Up. Upon clicking either one of the two, only the
component (content area) on the right gets switched to the
other one among the two. This increases the performance,
as we don’t have to re-render the whole page.
When a new user creates account from Sign up page, his/her
details are sent to the server in a HTTP POST request and
there the data is stored in the database in encrypted form
using the Bcrypt library. After the successful completion of
the account creation, user is redirected to the Sign in page
and when the user enters his/her credentials, a HTTP POST
request is sent again to the server with the user’s entered
credentials for his/her new account. Upon successful
verification of the credentials a success message is returned
to the client with a token, which the client stores in the local
storage for future authentication of the user and maintaining
the session for that particular user.
After successfully logging in the application, user sees the
new panel on the right which shows a music player with
various buttons traverse through a playlist or shuffle music.
The user also gets a logout and song upload button. If user
clicks the logout button, a POST request is sent to the server
informing the server that user has logged out, so his/her
session status is set to logged out in the DB table called
UserSession. The client also deletes the token for that user
from local storage, in order to make sure that no other user
could use the system without logging in. To perform this
authentication, application sends a GET request every time
the music player page URL is hit, which verifies from
database if there exists a valid session for the user and then
compares it to the local storage token. We are doing this
verification in the ComponentDidMount method on client
side, which is the first method to get executed in the life
cycle of React components.
5. Fig 9.0
The Browse button when clicked opens up a window
allowing users to locate the local .mp3 file which the user
wants to upload and share with friends. We have added
certain checks which insure that only .mp3 files get
uploaded. Once the file is selected the client sends a
multipart HTTP POST request to Node server, which
contains the song.
Fig 10.0
The client side also has a group chat option in the right
bottom corner, where all the users currently using the
application can chat, share and discuss music. Songs
uploaded by each user are accessible to all the other users
connected. For implementing this group chat feature, we
used a JavaScript library called Socket.io. It opens up a web
socket to communicate with server on a given port and IP
address.
Our client side has a component named MusicPlayer.jsx.
This component is used in the parent component called
Home.js. The Home component passes the playlist of all the
songs on cloud whose URLs and information is stored in
the database to MusicPlayer.jsx component. In this
component we just extract the URLs from the playlist array
passed and get those songs from the cloud along with their
respective cover photos and map the artist names and title
for that song on the music player screen.
Tools and Technologies: We implemented the
application’s client side using JavaScript’s new library
called React.js. The reason we choose React is its modular
nature towards designing user interface. It breaks up the
user interface into components and each component has its
own state. This makes the application development painless
and at the same time implementing the concept of
separation of concern under the hood. Each component has
to take care of its own state without worrying about what is
going on in other parts of the application. The state
elements common among multiple components are
specified in parent components and this concept in the
world of React is called lifting the state up. This is usually
done, if the users don’t want to implement Flux architecture
(Redux). Another reason behind our decision to use React.js
was the fact that React uses the concept of Virtual DOM.
This Virtual DOM is a tree with nodes containing the
attributes, objects and properties. If at any instance the data
model of the application gets changed, a new Virtual DOM
is created reflecting the changes in state that have occurred.
The new Virtual DOM is compared with the old one and the
changes between both of them are calculated. This is
particularly useful in React development because instead of
re-rendering the whole page again on the browser, only the
changes calculated are re-rendered. This significantly
increased the applications response and load time.
For bundling and packing our application we have used
Webpack, which makes the modern web application
development easier. Webpack also supports advanced
functionality such as hot module reloading (instant updates
of React components without refresh) and lazy loading
(load bundles as you need).
For Styling we have used Sass styling language. Which is
compatible with CSS and has a lot more features than
Bootstrap and other CSS frameworks.
For communication between our application’s frontend and
backend, we utilized web sockets using the library of
Socket.io and Axios (HTTP POST and GET requests).
For routing between pages and making single page
application possible we relied on React Router framework.
In this framework we can define routes and navigation links
between different components.
B. Server Side and Cloud Server
For server side programming we have used the Node.js
environment with Express.js (APIs creation framework).
We have created our Express APIs for sign in, sign up,
logout, verification, get playlist and upload songs. These
allow our client side to communicate with server using
HTTP GET and POST requests.
In case of Sign in, the client side sends HTTP POST request
with user’s credentials to the server. The server extracts the
credentials and compare the credentials sent with the
encrypted credentials stored in our DB. For storing user’s
information, we have created a document called Users in
DB. If the credentials sent by the users are successfully
matched and verified with the credentials stored in the DB,
the server sends a unique token for that specific client to the
client side, so the client can maintain a token based session
6. for that user as he is authenticated user. The server also
tracks the user’s login and logout activity in a document in
our DB called UserSession (For keeping session
information).
In case of Sign up, the client sends a HTTP POST request
with the data of user needed to create the account and that
data is inserted in the Users document in DB. Upon
successful insertion of new user, a success message is sent
to the client indicating that the new user has been created
and user can proceed with the Sign in.
In case of verify users a HTTP GET request is sent to the
server with the token of current user logged in to see if
his/her session is active or did the user logged out. This
verify request is sent to the server every time a page reload
happens and for this purpose we have used React’s lifecycle
hook method called ComponentDidMount at client’s end.
In case of logout, essentially the same thing happens, but
additionally at backend the session is marked as inactive in
the UserSession document. Once the user is marked as
logged out and session is set to inactive, the user would
have to sign in again in order to access the music player
page.
Our server also has an Express API for uploading songs
from client to Node server in a multipart POST request and
then from Node server sending the song to Cloudinary
Cloud using another POST request of its own.
The last API which we have created on our server is for
requesting the playlist of all the songs currently in the store.
This request is made by the client at the start of each page
reload to keep all the clients in sync and updated to the
shared music repository. When this get playlist request is
made, the server runs a find query on the document named
Songs (Has all the information for each song) in DB. The
resultant array containing objects in Songs document are
sent to client side in response.
If a new song is uploaded to the server, the details like
Cloud URL for the song, title, artists and cover photo URL
are stored in our database document named Songs. These
details are also sent back to all the clients connected to the
server via POST request response and socket messages, so
the clients update the playlist in order to access songs
shared by fellow users.
On the server-end we initiate a socket connection with the
client on port 8080. This socket listens for messages, once
any client sends a message, it broadcasts that message to all
the clients connected to the server. Following code snippet
implements this functionality.
Fig 11.0
Tools and Technologies: We have used Node.js
environment for our server side development. Node.js is an
environment which allows developers to execute JavaScript
code outside the browser.
In the environment of Node.js we have used a very
comprehensive framework called Express.js. This is used
for developing APIs, which other applications can call and
have access to the server.
For the database we have used MongoDB, which is a
NoSQL, document oriented database. Data in MongoDB is
stored in documents instead of tables, and the structure is
that of JSON. MongoDB is highly scalable, fast and
dynamic that’s why we choose MongoDB.
For our server to communicate with the DB, we have used
Mongoose ORM. We have created three models of Users,
UserSession and Songs. Objects of these models are used to
insert and retrieve data to and from documents in db.
For storing the songs, we have utilized cloud web services
offered by Cloudinary. Making use of cloud increases the
availability of songs, as the client can easily get the songs
and images from the URL it has for cloud storage.
We have also made use of Socket.io, which is JS library for
building real time web applications. Socket.io allows 2
ways communication, which we are using to allow users to
chat with each other.
V. SETUP
For setting up the project, users need to create their cloud
account on Cloudinary, a text editor or IDE like Atom or
Visual Studio, a browser and NPM (node) installed on
machine.
After getting all the tools mentioned above, the user can
open the project in their IDE, if they want to make changes
to the code, start the MongoDB using the command
“mongod” on Terminal if using Mac or CMD in using
Windows machine. This command with start the MongoDB
server. After this users can traverse to the project’s folder
using terminal and execute the command “npm run
start:dev”, which will execute both server and client.
7. Once all the above mentioned instructions have been
followed, users will be able to access the website on
http://localhost:8080/.
VI. CHALLENGES
The most difficult challenge faced during the creation of the
project was managing multiple live feeds of data
simultaneously, such as multiple users playing songs, or a
text feed alongside a music stream being sent to the same
user. Without proper synchronization between these two
sources of data, one stream would overshadow the other,
either reducing the quality of audio being streamed, or
delaying messages being sent by several seconds. This was
alleviated by sending data across a spectrum of available
ports, so as to not interfere with simultaneous connections.
VII. CONCLUSION
The Ampd project successfully demonstrated that a personal
music streaming service was possible on a technical level
using modern web streaming technologies. Utilizing Node
for an extensible web server capable of the functionality
required by the project allowed for fast addition of features.
MongoDB was considerable fast in terms of performance,
although development around the MongoDB driver was
difficult given the design of Mongo makes manipulating
data manually more difficult than SQL based drivers. Using
a cloud service API did increase the complexity of the
application, however the potential trade-off of storage to be
maintained made the added complexity worthwhile.
VIII. FUTURE WORK
Future goals for the project include adding VoIP rooms for
users to talk to one another while listening to similar songs
or artists. Discussion of music is good for discovering new
interests and tastes, and for accelerating the growth of small
artists. Along with this, another potential feature to
implement would be a data analysis system which can first
of all recommend songs to users based on other music they
frequently listen to, along with what music they post
positive comments or reviews on. The data analysis system
can also provide a dashboard to artists, showing graphs that
help visualize the feedback on their work, allowing them to
use that feedback to influence their future work to cater
more to the fan base they have, and the fan base they wish
to impress.
IX. REFERENCES
[1] Lin, Y. (2018). Comparing AWS Lambda performance of Node.js,
Python, Java, C# and Go. [online] Available at:
https://read.acloud.guru/comparing-aws-lambda-performance-of-
node-js-python-java-c-and-go-29c1163c2581 [Accessed 30 Nov.
2018]. https://marmelab.com/blog/2017/02/06/react-is-slow-react-is-
fast.html
[2] Shah, M. (2018). MongoDB vs MySQL: A Comparative Study on
Databases. [online] Enterprise Mobility, IoT and Business analytics
services | Simform, LLC. Available at:
https://www.simform.com/mongodb-vs-mysql-databases/ [Accessed
30 Nov. 2018].
[3] Wei-Ping, Z., Ming-Xin, L. I., & Huan, C. (2011, May). Using
MongoDB to implement textbook management system instead of
MySQL. In Communication Software and Networks (ICCSN), 2011
IEEE 3rd International Conference on (pp. 303-305). IEEE.
[4] Yu, S., Wang, C., Ren, K., & Lou, W. (2010, March). Achieving
secure, scalable, and fine-grained data access control in cloud
computing. In Infocom, 2010 proceedings IEEE (pp. 1-9). Ieee.
[5] Arora, R., & Aggarwal, R. R. (2013). Modeling and querying data in
mongodb. International Journal of Scientific and Engineering
Research, 4(7), 141-144.
[6] Huang, Y. W., Huang, S. K., Lin, T. P., & Tsai, C. H. (2003, May).
Web application security assessment by fault injection and behavior
monitoring. In Proceedings of the 12th international conference on
World Wide Web (pp. 148-159). ACM.
[7] Huang, Y. W., Tsai, C. H., Lin, T. P., Huang, S. K., Lee, D. T., &
Kuo, S. Y. (2005). A testing framework for Web application security
assessment. Computer Networks, 48(5), 739-761.
[8] Huang, Y. W., Yu, F., Hang, C., Tsai, C. H., Lee, D. T., & Kuo, S. Y.
(2004, May). Securing web application code by static analysis and
runtime protection. In Proceedings of the 13th international
conference on World Wide Web (pp. 40-52). ACM.
[9] Nanda, S., Lam, L. C., & Chiueh, T. C. (2007, November). Dynamic
multi-process information flow tracking for web application security.
In Proceedings of the 2007 ACM/IFIP/USENIX international
conference on Middleware companion (p. 19). ACM.
[10] Huang, Y. W., Tsai, C. H., Lee, D. T., & Kuo, S. Y. (2004,
November). Non-detrimental web application security scanning. In
Software Reliability Engineering, 2004. ISSRE 2004. 15th
International Symposium on (pp. 219-230). IEEE.
[11] Leff, A., & Rayfield, J. T. (2001). Web-application development
using the model/view/controller design pattern. In Enterprise
Distributed Object Computing Conference, 2001. EDOC'01.
Proceedings. Fifth IEEE International (pp. 118-127). IEEE.
[12] Sen, A., & Larson, M. (2015, September). From Sensors to Songs: A
Learning-Free Novel Music Recommendation System using
Contextual Sensor Data. In LocalRec@ RecSys (pp. 40-43).
[13] Goto, M., & Goto, T. (2005, September). Musicream: New Music
Playback Interface for Streaming, Sticking, Sorting, and Recalling
Musical Pieces. In ISMIR (pp. 404-411).
[14] Kontaki, M., Karydis, I., & Manolopoulos, Y. (2007). Content-based
information retrieval in streaming music. In Proc. Pan-Hellenic
Conference in Informatics (pp. 249-259).
[15] Van Zyl, P., Kourie, D. G., & Boake, A. (2006, October). Comparing
the performance of object databases and ORM tools. In Proceedings
of the 2006 annual research conference of the South African institute
of computer scientists and information technologists on IT research
in developing countries (pp. 1-11). South African Institute for
Computer Scientists and Information Technologists.
[16] Cholia, S., Skinner, D., & Boverhof, J. (2010, November). NEWT: A
RESTful service for building High Performance Computing web
applications. In 2010 Gateway Computing Environments Workshop
(GCE 2010) (pp. 1-11). IEEE.
[17] Chieu, T. C., Mohindra, A., Karve, A. A., & Segal, A. (2009,
October). Dynamic scaling of web applications in a virtualized cloud
computing environment. In E-Business Engineering, 2009.
ICEBE'09. IEEE International Conference on (pp. 281-286). IEEE.
[18] Jackson, K. R., Ramakrishnan, L., Muriki, K., Canon, S., Cholia, S.,
Shalf, J., ... & Wright, N. J. (2010, November). Performance analysis
of high performance computing applications on the amazon web
services cloud. In 2nd IEEE international conference on cloud
computing technology and science (pp. 159-168). IEEE.