Performed Data Analytics on the massive yelp academic dataset having almost 2.2 million reviews for 77k businesses by making using various technologies like Hadoop Pig, Hive, SQL, R, Tableau.

2. [AUTHOR NAME] 2
YELP DATASET ANALYSIS REPORT
1. Summary of number of reviews by US City, by Categories
In order to analyze according to these given conditions, I have made use of two of the given Datasets from the Yelp Academic
Challenge Dataset i.e. the Business and Reviews Datasets.
Both the datasets were loaded in Pig using twitter’s elephant-bird JsonLoader as the schema of the datasets is highly nested
with mixed data types. The .jar files for the various components of elephant bird were loaded through the properties tab on
the Pig Editor in Hue web UI. The 3 .jar files were: /user/cloudera/elephant-bird-core-4.13.jar, /user/cloudera/elephant-bird-
hadoop-compat-4.13.jar & /user/cloudera/elephant-bird-pig-4.13.jar respectively available at:
I. http://mvnrepository.com/artifact/com.twitter.elephantbird/elephant-bird-core/4.13
II. http://mvnrepository.com/artifact/com.twitter.elephantbird/elephant-bird-hadoop-compat/4.13
III. http://mvnrepository.com/artifact/com.twitter.elephantbird/elephant-bird-pig/4.13
NOTE: The above 3 .json files have been used in all of the questions
Once the .json file for the dataset has been uploaded as maps, it is stored in a generic variable. Then from that variable we
generate the fields we require for analysis using the format name_of_map#’field_name’ as ‘field_name’. As an example if we
loaded the business dataset with the map name as business, and we wish to generate the business_id field, then our syntax
will look like business#’business_id’ as business_id.
In this case I have generated the fields categories, city, business_id, state, latitude, longitude from the business dataset and
the fields business_id and review_id from the reviews dataset. To obtain US cities we filter them first based on the edge
coordinates of USA Mainland. However, as cities like Waterloo are present, we then filter it by State to remove the Canadian
states of Ontario and Quebec. They are then joined together on their common field business_id in a new variable joined. Once
they have been joined together, I generated the city and categories for each of the records in joined. As the categories given
in the business dataset are nested and each business can be classified under various different categories, I flattened the
categories so that we can identify each category associated with the business individually. Once the categories have been
flattened, I then grouped the variable flattened by city and categories, so that we can see the results grouped respectively.
However, once we group any field, it’s schema changes. So in order to extract the desired result, for each of the records in the
grouped variable, I have flattened the grouping done previously, as city and categories and then generated the count of reviews
associated with it.
Finally, I have ordered the results by city, so that I can arrange the final output by showing the number of reviews for each
business category within each city in the dataset. I then stored the final variable into a folder in HDFS using the PigStorage
method making it a Tab Separated Variable File.
A few exceptions which I noted while analyzing the output of the operation is that few records do not have any city mentioned
in their city field, while in the case of some records the same city has been specified differently, like 110.Las Vegas and Las
Vegas. Such discrepancies can cause minor fluctuations while analyzing the output dataset.
Basic Analysis of the Number of Reviews in Tableau, suggests that the most number of reviews have come from the city of Las
Vegas as shown by Figure 1 and the most number of reviews for any individual category are for the Restaurant’s category as
shown by Figure 2.
Figure 1

3. [AUTHOR NAME] 3
YELP DATASET ANALYSIS REPORT
Figure 2
PIG SCRIPT:
A = LOAD './yelp_academic_dataset_business.json' USING com.twitter.elephantbird.pig.load.JsonLoader('-nestedLoad=true ') AS
(yelp: map[]);
business = FOREACH A GENERATE yelp#'categories' as categories, yelp#'business_id' as business_id, yelp#'city' as city, yelp#'state' as
state,(float)yelp#'latitude' as latitude, (float)yelp#'longitude' as longitude ;
coordinates_business = FILTER business BY (latitude<49.384472) AND (latitude>24.520833) AND (longitude<-66.950) AND
(longitude>-124.766667);
us_business = FILTER coordinates_business BY NOT ( (state matches '.*ON.*') OR (state matches '.*QC.*') );
businesses = FOREACH us_business GENERATE categories, business_id, city ;
B = LOAD './yelp_academic_dataset_review.json' USING com.twitter.elephantbird.pig.load.JsonLoader('-nestedLoad=true ') AS
(review: map[]);
revie = FOREACH B GENERATE review#'business_id' as business_id, review#'review_id' as review_id;
joined = JOIN businesses by business_id, revie by business_id;
flatting = FOREACH joined GENERATE city, FLATTEN(categories);
grouped = GROUP flatting by (city, categories);
results = FOREACH grouped GENERATE FLATTEN(group) AS (city,categories), COUNT(flatting);
finals = ORDER results by city;
STORE finals INTO './Q1' USING PigStorage('t');
TRUNCATED OUTPUT:
Magicians 3
Event Planning & Services 3
110. Las Vegas Automotive 12
110. Las Vegas Auto Repair 12
Ahwatukee Pet Boarding/Pet Sitting 10
Ahwatukee Fitness & Instruction 20
Ahwatukee Sewing & Alterations 20
Ahwatukee Health & Medical 13
Ahwatukee Hotels & Travel 14
Ahwatukee Eyelash Service 4
Ahwatukee Carpet Cleaning 4
Ahwatukee Specialty Food 3
Ahwatukee Local Services 30
Ahwatukee Health Markets 3
Ahwatukee Pediatricians 13
Ahwatukee Home Services 6
Ahwatukee Beauty & Spas 4
Ahwatukee Truck Rental 6
Ahwatukee Self Storage 6
Figure 2

4. [AUTHOR NAME] 4
YELP DATASET ANALYSIS REPORT
2. Ranking of cities on the basis of stars in each category
In order to analyze according to these given conditions, I have made use of two of the given Datasets from the Yelp Academic
Challenge Dataset i.e. the Business and Reviews Datasets as in the last example.
Both the datasets were loaded in Pig using twitter’s elephant-bird JsonLoader as the schema of the datasets is highly nested
with mixed data types. The .jar files for the various components of elephant bird were loaded through the properties tab on
the Pig Editor in Hue web UI. Once the .json file for the dataset has been uploaded as maps, it is stored in a generic variable.
Then from that variable we generate the fields we require for analysis using the format name_of_map#’field_name’ as
‘field_name’. As an example if we loaded the business dataset with the map name as business, and we wish to generate the
categories field, then our syntax will look like business#’categories’ as categories.
In this case I have generated the fields categories, city, business_id from the business dataset and the fields business_id and
stars from the reviews dataset. As when we stored all the data in the .json file in terms of a map in key value pairs, we have
to make sure that whenever we are extracting any number we have to typecast it by specifying the data type like int or float
before we generate the field from the data loaded using the twitter elephant bird API. The two are then joined using their
common field i.e. business_id. Once they have been joined together, I generated the city, stars, categories for each of the
records in the joined variable. As the categories given in the business dataset are nested and each business can be classified
under various different categories, I flattened the categories so that we can identify each category associated with the business
individually.
Once the categories have been flattened, I then grouped the variable flattened by city and categories, so that we can see the
results grouped respectively. However, once we group any field, it’s schema changes. So in order to extract the desired result,
for each of the records in the grouped variable, I have flattened the grouping done previously, as city and categories and
generated the average value of the stars within that group and renamed the calculated field as rankings. It should be noted,
that in order to access the stars field we have to mention the variable name in which the field stars exist. In this case we called
the stars field using the syntax flattened_join.stars. Just as noted in the last part, in this part also the problem with the same
city with different names exists like Las Vegas and 110.Las Vegas
The Mean Rating has been found as 3.747, with the minimum and maximum rating values as 1.0 and 5.0. The Median average
rating is 3.758.
Figure 3 shows the number of cities in which the average rating for businesses of all categories have been grouped according
to the categories: Less than 1.5 Stars, 1.5-3 Stars, 3-4.5 Stars, Greater than 4.5 Stars. We can see that most of the cities have
businesses in the 3-4.5 range.
Figure 4 similarly illustrates the number of categories grouped according to their average ratings placed in categories as: Not
Good(Lesser than 1.5 stars), Fair(1.5 – 3 stars), Good(3 – 4.5 stars), Excellent(Above 4.5 stars). We can see that almost 40% of
the categories a Good Rating i.e. 3 – 4.5 stars.
Figure 3
Figure 4

5. [AUTHOR NAME] 5
YELP DATASET ANALYSIS REPORT
3. Average Rank for Businesses within 5 miles of Carnegie Melllon University, Pittsburgh, PA
In order to analyze according to these given conditions, I have made use of two of the given Datasets from the Yelp Academic
Challenge Dataset i.e. the Business and Reviews Datasets as in the last example.Both the datasets were loaded in Pig using
twitter’s elephant-bird JsonLoader as the schema of the datasets is highly nested with mixed data types. The .jar files for the
various components of elephant bird were loaded through the properties tab on the Pig Editor in Hue web UI. Once the .json
file for the dataset has been uploaded as maps, it is stored in a generic variable. Then from that variable we generate the fields
we require for analysis using the format name_of_map#’field_name’ as ‘field_name’. As an example if we loaded the business
dataset with the map name as business, and we wish to generate the categories field, then our syntax will look like
business#’categories’ as categories.
In this case I have generated the fields categories, latitude, longitude, business_id from the business dataset and the fields
business_id and stars from the reviews dataset. As when we stored all the data in the .json file in terms of a map in key value
pairs, we have to make sure that whenever we are extracting any number we have to typecast it by specifying the data type
like int or float before we generate the field from the data loaded using the twitter elephant bird API. In order to then obtain
PIG SCRIPT:
A = LOAD './yelp_academic_dataset_business.json' USING com.twitter.elephantbird.pig.load.JsonLoader('-
nestedLoad=true ') AS (yelp: map[]);
business = FOREACH A GENERATE yelp#'categories' as categories, yelp#'city' as city, yelp#'business_id' as business_id ;
B = LOAD './yelp_academic_dataset_review.json' USING com.twitter.elephantbird.pig.load.JsonLoader('-
nestedLoad=true ') AS (review: map[]);
revie = FOREACH B GENERATE review#'business_id' as business_id, (int)review#'stars' as stars;
joined = JOIN business by business_id, revie by business_id;
flatting = FOREACH joined GENERATE city, stars, FLATTEN(categories);
grouped = GROUP flatting by (categories, city);
results = FOREACH grouped GENERATE FLATTEN(group) AS (categories, city), AVG(flatting.stars) AS rankings;
outputer = ORDER results BY categories, rankings DESC;
STORE outputer INTO './Q2' USING PigStorage('t');
TRUNCATED OUTPUT:
ATV Rentals/Tours Sun City 5.0
ATV Rentals/Tours Wickenburg 5.0
ATV Rentals/Tours Las Vegas 4.829787234042553
ATV Rentals/Tours Henderson 4.615384615384615
ATV Rentals/Tours Phoenix 4.514285714285714
ATV Rentals/Tours North Las Vegas 3.825
Accessories North Las Vegas 5.0
Accessories Cave Creek 5.0
Accessories Middleton 4.5
Accessories Verdun 4.333333333333333
Accessories Madison 4.016666666666667
Accessories Gilbert 4.0
Accessories Pineville 4.0
Accessories Phoenix 3.969924812030075
Accessories Pittsburgh 3.9586206896551723
Accessories Las Vegas 3.936688311688312
Accessories Westmount 3.9166666666666665
Accessories Fort Mill 3.888888888888889
Accessories Champaign 3.8636363636363638
Accessories Queen Creek 3.857142857142857
Accessories Charlotte 3.8114285714285714
Accessories Surprise 3.8
Accessories Scottsdale 3.757798165137615
Accessories Karlsruhe 3.75
Accessories Peoria 3.7419354838709675
Accessories Paradise Valley 3.7333333333333334
Accessories Goodyear 3.6666666666666665
Accessories Edinburgh 3.6125

6. [AUTHOR NAME] 6
YELP DATASET ANALYSIS REPORT
the businesses within 5 miles from the specified location of Carnegie Mellon University, I specified the parameters for the limits
of latitude and longitude which have to be satisfied if these businesses are to located within the given area specifications.
These limits were then used with the FILTER BY command which gave the desired results. The two are then joined using their
common field i.e. business_id. Once they have been joined together, I generated the stars, categories for each of the records
in the joined variable. As the categories given in the business dataset are nested and each business can be classified under
various different categories, I flattened the categories so that we can identify each category associated with the business
individually. Once the categories have been flattened, I then grouped the variable flattened by categories, so that we can see
the results grouped accordingly. However, once we group any field, it’s schema changes. So in order to extract the desired
result, for each of the records in the grouped variable, I have flattened the grouping done previously, as categories and
generated the average value of the stars within that group and renamed the calculated field as rankings. It should be noted,
that in order to access the stars field we have to mention the variable name in which the field stars exist. In this case we called
the stars field using the syntax flattened.stars.
The Mean Rating from the businesses in range has been found as 3.901, with the minimum and maximum rating values as 1.0
and 5.0. The Median average rating is 3.971.
Figure 5
Figure 6

7. [AUTHOR NAME] 7
YELP DATASET ANALYSIS REPORT
Figure 5 above, I’ve represented only the starting portion of a figure which shows the rating for each category relative to the
average rating for all categories within the region. The straight line in the figure represents the average value of ratings across
all categories. In Figure 6, I have selected a few of the categories which would be suiting the area around a college campus. As
expected we can notice that the ratings for businesses like boxing, educational stores, guitar stores, books, bike rentals have
their ratings on the higher side, all being above 4. Such ratings are expected as the area is around a college and their majority
crowd would be students. Also as we selected, businesses like rehabilitation, engraving, retirement homes do not have that
great ratings as compared to the other selected ratings.
4. Reviewers Ranked by number of reviews & Category wise analysis of Top 10 Reviewers
In order to analyze according to these given conditions, I have made use of three Datasets from the Yelp Academic Challenge
Dataset i.e. the Business, Users and Reviews Datasets. All the datasets were loaded in Pig using twitter’s elephant-bird
JsonLoader as the schema of the datasets is highly nested with mixed data types. The .jar files for the various components of
elephant bird were loaded through the properties tab on the Pig Editor in Hue web UI. I have generated the fields review_count,
name, user_id from the users dataset. From the business dataset: categories & business_id and the fields business_id, user_id
and stars from the reviews dataset. As when we stored all the data in the .json file in terms of a map in key value pairs, we
PIG SCRIPT:
A = LOAD './yelp_academic_dataset_business.json' USING com.twitter.elephantbird.pig.load.JsonLoader('-nestedLoad=true ') AS
(yelp: map[]);
business = FOREACH A GENERATE yelp#'categories' as categories, yelp#'business_id' as business_id,(float)yelp#'latitude' as latitude,
(float)yelp#'longitude' as longitude ;
business_in_range = FILTER business BY (latitude<40.5245131) AND (latitude>40.3578471) AND (longitude>-80.0261624) AND
(longitude<-79.8594964);
B = LOAD './yelp_academic_dataset_review.json' USING com.twitter.elephantbird.pig.load.JsonLoader('-nestedLoad=true ') AS
(review: map[]);
revie = FOREACH B GENERATE review#'business_id' as business_id, (int)review#'stars' as stars;
joined = JOIN business_in_range by business_id, revie by business_id;
flattened = FOREACH joined GENERATE stars, FLATTEN(categories);
grouped = GROUP flattened by categories;
results = FOREACH grouped GENERATE FLATTEN(group) AS categories, AVG(flattened.stars) AS rankings;
outputer = ORDER results BY categories;
STORE outputer INTO './Q3' USING PigStorage('t');
TRUNCATED OUTPUT:
Accessories 3.9863013698630136
Accountants 5.0
Active Life 4.100411039342337
Acupuncture 4.241379310344827
Adult Education 5.0
Adult Entertainment 3.55
Advertising 4.125
African 4.625
Airport Shuttles 4.717948717948718
American (New) 3.761559037065342
American (Traditional) 3.5828891622249555
Amusement Parks 4.169014084507042
Animal Shelters 4.132075471698113
Antiques 4.291666666666667
Apartments 2.8396946564885495
Appliances 3.087719298245614
Appliances & Repair 3.533333333333333
Aquariums 3.9019607843137254
Arcades 3.3969465648854964
Argentine 4.697594501718213
Art Classes 4.627450980392157
Art Galleries 4.11344537815126
Art Schools 4.291666666666667
Art Supplies 4.146341463414634
Arts & Crafts 4.328638497652582
Arts & Entertainment 4.093375214163335
Asian Fusion 3.693535514764565

8. [AUTHOR NAME] 8
YELP DATASET ANALYSIS REPORT
have to make sure that whenever we are extracting any number we have to typecast it by specifying the data type like int or
float before we generate the field from the data loaded using the twitter elephant bird API.
The first part of the question involves finding out the reviewers and sorting them by their number of reviews in the descending
order, keeping the user with most reviews at the top. For this we simply have to use only one dataset, the users dataset. We
load the data, and generate the user_id, review_count. Further we can simple use the ORDER BY command of pig to sort them
according to the field we wish. To sort them in a descending fashion, we use the DESC option along with order by.
For the second part I’ve ordered the users information by their review count and limited the set to only the users with the 10
highest number of reviews. I’ve then joined this with the reviews dataset generated previously. This is then further joined with
the business dataset. And then flattened according to their names and the categories they belong to along with the average
ratings for each category they have reviewed.
The Average Rating for the top 10 reviewers has been found as 3.678, with the minimum and maximum rating values as 1.0
and 5.0. The Median average rating is 3.692. The Average number of reviews has been found as 28, with the minimum and
maximum number of reviews by a single user being 0 and 10,320. The total number of reviews in the dataset was 15,261,802.
Figure 7 is showing us for average rating which is given by each of the reviews with the top 10 number of reviews. From the
figure we can conclude that the user with the highest average rating is Shila.
Figure 7
Figure 8

9. [AUTHOR NAME] 9
YELP DATASET ANALYSIS REPORT
The Figure 8 is showing us for number of reviews from the given dataset for users. In this figure I have limited the number of
users to 10, to achieve greater clarity. From this we infer that though the highest average rating is for Shila, Victor has the
greatest number of reviews.
5. Ratings of the Top 10 & Bottom 10 Food Businesses around CMU by month
In this I have firstly loaded the business dataset and generated the name, categories, business_id, latitude, longitude, stars
fields from it. Then I have filtered the data according to the location specifications of CMU. After this I have generated the
columns other than the latitude and longitude.
I have in this same step converted the categories field to a bag by using the TOBAG operator. This bag was then converted to
a string in the same step using the BagToString function. This was done so that I could filter the categories easily depending on
whether they had food in their categories or not. For this I used the matches operator which is used to find a string within
another string and returns TRUE if it is found and FALSE otherwise. After this I ordered the remaining businesses by their stars
and then limited them to the top 10. For the bottom 10 I ordered the businesses in the ascending order and for the top 10 in
the descending order.
PIG SCRIPT:
A = LOAD './yelp_academic_dataset_user.json' USING com.twitter.elephantbird.pig.load.JsonLoader('-nestedLoad=true ') AS (yelp: map[]);
users = FOREACH A GENERATE yelp#'user_id' as user_id, yelp#'name' as name, (int)yelp#'review_count' as review_count;
B = LOAD './yelp_academic_dataset_review.json' USING com.twitter.elephantbird.pig.load.JsonLoader('-nestedLoad=true ') AS (review: map[]);
revie = FOREACH B GENERATE review#'business_id' as business_id, (int)review#'stars' as stars, review#'user_id' as user_id;
C = LOAD './yelp_academic_dataset_business.json' USING com.twitter.elephantbird.pig.load.JsonLoader('-nestedLoad=true ') AS (busines: map[]);
business = FOREACH C GENERATE busines#'business_id' as business_id, busines#'categories' as categories;
ordered_users = ORDER users by review_count DESC;
STORE ordered_users INTO './Q4a' using PigStorage('t');
ordered = ORDER users by review_count DESC ;
top_10 = LIMIT ordered 10;
joined = JOIN top_10 by user_id, revie by user_id ;
generator = FOREACH joined GENERATE name as Name, stars as Stars, business_id as Business_ID;
join_again = JOIN generator by Business_ID, business by business_id;
flattened = FOREACH join_again GENERATE Stars, Name, FLATTEN(categories);
grouped = GROUP flattened by (Name,categories);
results = FOREACH grouped GENERATE FLATTEN(group) AS (Name, categories) , AVG(flattened.Stars) AS rankings;
STORE results INTO './Q4b' USING PigStorage('t');
TRUNCATED OUTPUT:
Neal Bars 4.25
Neal Food 3.4615384615384617
Neal Thai 2.0
Neal Cafes 3.5
Neal Greek 3.5
Neal Pizza 2.0
Neal Taxis 3.0
Neal Hotels 4.0
Neal Indian 3.0
Neal Burgers 3.75
Neal Italian 4.0
Neal Lounges 4.333333333333333
Neal Mexican 3.0
Neal Resorts 5.0
Neal Airports 3.3333333333333335
Neal Bakeries 3.0
Neal Caterers 5.0
Neal Day Spas 5.0
Neal Desserts 3.0
Neal Shopping 4.2
Neal Fast Food 3.5714285714285716
Neal Nightlife 4.25
Neal Automotive 4.0
Neal Bookstores 5.0
Neal Car Rental 4.0
Neal Drugstores 4.0
Neal Sushi Bars 5.0

10. [AUTHOR NAME] 10
YELP DATASET ANALYSIS REPORT
After joining it with the reviews dataset, I have then generated the columns by making use of their location in the joined table
and then naming them. For extracting the month from the date, I have used the SUBSTRING operator. As the date was aleaady
in chararray format, and specified as ‘yyyy-mm-dd’ we can simply denote the starting and ending location and extract the date
from the given dates. Then I have grouped it according to business_id, name, month. I have included the business_id also as
there are cases when there are many businesses with the same name, which can cause ambiguity. This was seen in the bottom
10 businesses where McDonalds was present 3 times. Therefore, using business_id along with the name helps us to identify
them individually.
We obtained the two sequence files separately, and then used the Hadoop HDFS commands to concatenate them into a single
file, then copying them from local file system to HDFS. Q5.txt is the final output file. For this we use the commands:
v hadoop dfs –mv /user/cloudera/BOTTOM_10_FOODS/part-r-00000 /user/cloudera/TOP_10_FOODS/part-r-00001
v hadoop fs –rm /user/cloudera/TOP_10_FOODS/_SUCCESS
v hadoop fs –getmerge /user/cloudera/TOP_10_FOODS/ /user/cloudera/Q5.txt
v hadoop fs –copyFromLocal /user/cloudera/Q5.txt /user/cloudera/
In the truncated output below, I have shown both the output files before concatenation.
In Figure 9 we can see the Top 10 Food Businesses as located on the map
Below, in Figure 10 we can see the Bottom 10 Food Businesses as located on the map
It can be seen in both of the maps shown above that all the businesses are located near Carnegie Mellon University
Figure 9
Figure 10

11. [AUTHOR NAME] 11
YELP DATASET ANALYSIS REPORT
PIG SCRIPT:
A = LOAD './yelp_academic_dataset_business.json' USING com.twitter.elephantbird.pig.load.JsonLoader('-nestedLoad=true ') AS (yelp: map[]);
business = FOREACH A GENERATE yelp#'categories' as categories, (float)yelp#'stars' as stars, yelp#'name' as name, yelp#'business_id' as
business_id,(float)yelp#'latitude' as latitude, (float)yelp#'longitude' as longitude ;
business_in_range = FILTER business BY (latitude<40.5245131) AND (latitude>40.3578471) AND (longitude>-80.0261624) AND (longitude<-
79.8594964);
binrange = FOREACH business_in_range GENERATE name, stars, business_id, org.apache.pig.builtin.BagToString(TOBAG(categories)) as category;
filters = FILTER binrange BY category matches '.*Food.*';
ordered = ORDER filters by stars DESC;
top_10 = limit ordered 10;
B = LOAD './yelp_academic_dataset_review.json' USING com.twitter.elephantbird.pig.load.JsonLoader('-nestedLoad=true ') AS (review: map[]);
rev = FOREACH B GENERATE (float)review#'rating' as ratings, review#'business_id' as business_id, review#'date' as date;
joining = JOIN top_10 by business_id, rev by business_id;
top_join = FOREACH joining GENERATE $0 as name, $1 as ratings, $5 as business_id, (int)SUBSTRING($6, 5, 7) as month;
grouped = GROUP top_join by (business_id, name, month);
flatting = FOREACH grouped GENERATE FLATTEN (group) as (business_id, name, month), AVG(top_join.ratings);
STORE flatting INTO './TOP_10_FOODS' using PigStorage('t');
orderedbottom = ORDER filters by stars;
bottom_10 = limit orderedbottom 10;
joiningb = JOIN bottom_10 by business_id, rev by business_id;
bottom_join = FOREACH joiningb GENERATE $0 as name, $1 as ratings, $5 as business_id, (int)SUBSTRING($6, 5, 7) as month;
groupedb = GROUP bottom_join by (business_id, name, month);
flattingb = FOREACH groupedb GENERATE FLATTEN (group) as (business_id, name, month), AVG(bottom_join.ratings);
STORE flattingb INTO './BOTTOM_10_FOODS' using PigStorage('t');
TRUNCATED OUTPUT FOR TOP 10:
08eRFhpedodAf6atSRK09g The Colombian Spot 12 5.0
0h2My97xjAjc1pNrMM266Q Five Points Artisan Bakeshop 1 5.0
0h2My97xjAjc1pNrMM266Q Five Points Artisan Bakeshop 2 5.0
0h2My97xjAjc1pNrMM266Q Five Points Artisan Bakeshop 3 5.0
0h2My97xjAjc1pNrMM266Q Five Points Artisan Bakeshop 5 5.0
0h2My97xjAjc1pNrMM266Q Five Points Artisan Bakeshop 6 5.0
0h2My97xjAjc1pNrMM266Q Five Points Artisan Bakeshop 7 5.0
0h2My97xjAjc1pNrMM266Q Five Points Artisan Bakeshop 8 5.0
0h2My97xjAjc1pNrMM266Q Five Points Artisan Bakeshop 9 5.0
0h2My97xjAjc1pNrMM266Q Five Points Artisan Bakeshop 10 5.0
0h2My97xjAjc1pNrMM266Q Five Points Artisan Bakeshop 11 5.0
0h2My97xjAjc1pNrMM266Q Five Points Artisan Bakeshop 12 5.0
3EMEYlCxPiygL4Tu_z9beQ Fine Wine & Good Spirits 4 5.0
3EMEYlCxPiygL4Tu_z9beQ Fine Wine & Good Spirits 5 5.0
3EMEYlCxPiygL4Tu_z9beQ Fine Wine & Good Spirits 11 5.0
TRUNCATED OUTPUT FOR BOTTOM 10:
6C1Igw4BzRmg5Et8GSVfpA Seven Eleven Penn Avenue 5 1.5
6C1Igw4BzRmg5Et8GSVfpA Seven Eleven Penn Avenue 6 1.5
9KsHPdF1-P_CiXnvugdQvQ Foodland 1 1.5
9KsHPdF1-P_CiXnvugdQvQ Foodland 3 1.5
9KsHPdF1-P_CiXnvugdQvQ Foodland 4 1.5
9KsHPdF1-P_CiXnvugdQvQ Foodland 7 1.5
9KsHPdF1-P_CiXnvugdQvQ Foodland 8 1.5
9KsHPdF1-P_CiXnvugdQvQ Foodland 9 1.5
9KsHPdF1-P_CiXnvugdQvQ Foodland 11 1.5
BbIh5NTizhV4Fq_mLmNkpg Long John Silver's 1 1.5
BbIh5NTizhV4Fq_mLmNkpg Long John Silver's 7 1.5
BbIh5NTizhV4Fq_mLmNkpg Long John Silver's 8 1.5
CL3tZqbYT7B5zgewKCS6-Q McDonald's 8 1.0
CL3tZqbYT7B5zgewKCS6-Q McDonald's 12 1.0
KT8KJ4zt-IPqpLzACdpEZg Wendy's 2 1.5
KT8KJ4zt-IPqpLzACdpEZg Wendy's 3 1.5