Twet

Twet

Alexandru Burada, Gheorghe Prelipcean

Faculty of Computer Science, Al. I. Cuza University Iasi, Romania
alexandru.burada@info.uaic.ro, gheorghe.prelipcean.info.uaic.ro

Abstract. Twet is a mashup that uses WordNet to search terms the user filled in
and using the results from the WordNet thesaurus, search popular social sites
like Twitter, Flickr or Youtube for content related to those terms. Also, using
Bing Maps, it searches for the location of the photos.

Keywords: mashup, wordnet, synset, twitter, flickr, geo-location

1 Introduction

A mashup is an application that combines multiple external services to create a
content aggregation useful to the user. Twet is a mashup because it uses services
offered by popular social sites such as Twitter, Flickr or Youtube, together with Bing
Maps and WordNet. This application allows users to search on popular social sites not
only by keywords, but by trying to find similar terms to the ones they requested. This
way, the results of the search are not limited by the actual words they used, but are
enhanced by using the synsets provided by WordNet. In this context the application
gives the possibility they could create an overview of how the idea or information is
perceived globally, can locate images or ideas of people who were interested in this
subject.
A usage scenarios is when a user has vague knowledge about a certain idea, not
necessarily something very concrete. The best way he can get introduced to the term
is a collection of images, texts and videos related to this concept, and this is where
Twet comes into play.
This ensures the results are, in most cases, closer to what the user intended to
search. The risk is that by searching on a greater number of terms, the results are
broader and thus less exact, but this can be controlled by the number of terms used in
the search.

2 Architecture

Twet was designed using the .NET Platform, as a Silverlight 4 application. Silverlight
4 is currently in beta status and it can only be used together with Visual Studio 2010,

2 Alexandru Burada, Gheorghe Prelipcean

also in beta. Because of the size of the WordNet database, we decided to add a server
behind the application, that will be called for requesting synsets. All other calls will
be made by the Silverlight client, using Microsoft's new web platform at its full
potential.
The server is stateless and it is exposed by a single, general purpose request
handler. The Request Handler is something similar to Java Servlets, so it is a basic
communication method. We have managed to use this simple method at its full
power, by designing a general method of calling server methods, albeit we only used
it for the single purpose of searching the WordNet thesaurus.
The fallowing diagram illustrates the design of the client - server call mechanism.

Fig. 1. Class flow diagram for client - server communication

The key to making a generic call mechanism is to use reflection and invoke
dynamic methods on the server. The client has to know the name of method, the name
of the class, assembly and the parameters need for invoking the server method. This
information is contained in the AsyncContext class, that will be serialized to the
Request Handler.

AsyncCallContext c = new AsyncCallContext()
{
AssemblyName = "Twet.Web",
ClassName = "Twet.Web.WordNet.TwetWordNet",
MethodName = "FindSynonyms",
Request = request,
ResultCallBack = target,

Twet 3

DestinationThread = destinationDispatcher,
RequestData = word
};

Serialization is the responsibility of the Serializer class. For now, it can only
serialize primitive types and collections, but it can be enhanced to support serializing
any kind of objects (section 3).
Once the request gets to the server, a general purpose RequestHandler invokes
the requested method via reflection.

string assemblyName = reader.ReadString();
string className = reader.ReadString();
string methodName = reader.ReadString();

object o = Serializer.DeserializeObject(reader);
Type classType = Type.GetType(className + "," +
assemblyName);
object cls =
Activator.CreateInstance(classType);

response = classType.InvokeMember(methodName,
BindingFlags.InvokeMethod | BindingFlags.Public |
BindingFlags.Instance, null, cls, new object[] {o});

BinaryWriter w = new
BinaryWriter(context.Response.OutputStream);
Serializer.SerializeObject(w, response);
w.Close();

Since all external communication in Silverlight is asynchronous, we have a
problem with executing the response on the same thread as the thread of the controls.
This problem is solved by passing the Dispatcher object when calling a server
method, and also passing a AsyncDataReceiver, a handler to be executed when the
response is received. The fallowing code snippet show how we use the Dispatcher
passed by the caller to invoke the response handler.

private static void CallReceiver(Dispatcher disp,
AsyncDataReceiver
receiver, ReceivedData data)
{
if (receiver == null) return;
if (disp != null)
{
disp.BeginInvoke(() => receiver(data));
}
else
{
receiver(data);
}


}

2.1 Silverlight Application

The entry point to the web interface is the represented by the MainPage.xaml.

Fig. 2. Twet welcome screen

The main canvas where the text box and search button are present overlaps the
map control provided by Bing Maps Silverlight Control [2]. Bing Maps SDK requires
a key to be provided.
The words the user has filled in are sent to the server, to be searched on the
WordNet synsets. The twitter messages returned by the server are displayed using a
3D-like animation, that we are going to discuss next. Because the total number of
results is big and adding too many messages to the interface will make it very hard to
read (see Fig. 3.), we only added at most 3 messages from each synonym in the
synset.
For the 3D text animation we have used the an implementation done by Terence
Tsang [3] distributed under GNU GPL. This implementation has been modified to
support displaying a control made of an image representing the avatar of the user who
posted that message, his username on twitter and the actual message. Also, we needed
a way to stop the animation when moving away from the page.

Twet 5

The animation consists of a 3D space where the twitter message comes from
somewhere far away from the viewpoint until it passes the camera viewpoint,
something similar to space movement. Using Silverlight's ScaleTransform, this can be
easily achieved. Another feature of the 3D space is that, when you click on one of the
messages, it will move the camera in the direction of that message, faster than it
would normally take for that text to come right in front of the viewpoint. As the text
moves closer, it's opacity is reduced to become more easily readable.
All this time, the map is shown behind the canvas of the 3D space and the
readability of the text is not affected by the transparency of the canvas.

Fig. 3. 3D Text Space view

We decided not to geo-locate twitter messages because we noticed that the percent
of messages that contain the geo data is very small, and the user could be confused as
to why some messages have been located on the map and why the others have not.
Also, twitter does not offer a way to make a query that will return messages that
satisfy a search and contain geo information.

2.2 Flickr search

Searching photos containing the words returned by the WordNet thesaurus was made
using the Flickr API. All request were initiated by the Silverlight application, without
the need of calling the back-end server.
Unlike Twitter, Flickr has a parameter that limits the results to only those
containing information for their geo-location. Using the RESTfull API offered by
Flickr, Twet makes a request for all photos containing the specified tags and having


geo-data. The returned XML needs to be parsed and interpreted. After finding the
URI of the photos, we can instantiate the Image objects that will be the placeholder
for displaying the photos.
An interesting security problem was raised by this feature of Twet. The problem
was making cross-domain calls. Silverlight applications are restricted to making such
requested unless the target domain has a crossdomain.xml or clientaccesspolicy.xml.
Since photos are saved on different flickr servers, all need to have this file at the root
of their domain for Silverlight to allow such requests. Loading photos from a cross-
domain site is, this way, possible, but with other security limitations. One such
limitation is that we cannot access the actual pixel of the photo when loaded into a
WritableBitmap object. This object is needed to make a scale a rotate transformation,
while the actual Image object that is loading the image can be recycled if we make a
copy of the WritableBitmap; since we cannot access the pixels and cannot copy the
image, we cannot free the memory of the Image class, so our application will require
a little more memory.
Scaling the image can only be done once the image was loaded (we do not have
information about the photos size before this), so we attached an event handler for the
loaded event of each photo. In this event handler we compute the display width and
height of the photo, in the fallowing way:

BitmapImage bi = sender as BitmapImage;
if (cx >= cy)
{
// Set width to 300 and compute height
cy *= 300.0 / cx;
cx = 300.0;
}
else
{
// Set height to 300 and compute height
cx *= 300.0 / cy;
cy = 300.0;
}

WriteableBitmap wb1 = new WriteableBitmap((int)cx,
(int)cy);
ScaleTransform transform = new ScaleTransform();
transform.ScaleX = cx / bi.PixelWidth;
transform.ScaleY = cy / bi.PixelHeight;
wb1.Render(image, transform);
wb1.Invalidate();

Geo-locating an image is postponed until the user makes the request of locating the
image. Since the request and responses are very small, this operation is very fast and
the user does not even notice the request.
Using the API offered by Bing Maps Silverlight SDK, we can zoom on the map
and add a pushpin on the specified coordinates. The map is integrated in the

Twet 7

Silverlight Application. In fact, this is the direction Bing Maps is taking, to using
Silverlight for displaying maps. The advantages of this are a better user experience
since images are no longer loaded by the browser, with the usual lag. Also, by using
Silverlight, some animations can be used for zooming to an area or displaying the
Street View. For a live demo of the new Bing Maps, go to [5]

Fig. 4. The photo canvas

Images can be dragged across the canvas and clicking on it will bring in front of
the others, so it can be viewed better. When the mouse is over the image and it is not
being dragged, a small icon is displayed in the top left corner. If the users clicks on
this icon, he will see the location on the map where that photo was taken.

double x = e.GetPosition(null).X;
double y = e.GetPosition(null).Y;

double dx = x - _lastx;
double dy = y - _lasty;

//save the new position for future delta calculation
_lastx = x;
_lasty = y;


Fig. 5. Pin added in the location where the photo was localized. The zoom level is set
accroding to the the accurecy reported by Flickr. The photo canvas is minimez to the
left of the screen, and a button is displayed allowing the photo canvas to return to
normal size

Dragging images is a simple operation because the parent of all rendered photos is
a Silverlight Canvas element. The Canvas has a collection of children, but it does not
keep the position for any one of them; instead, each child keeps his own position
relative to the canvas. When dragging an image, we move it by setting the
Canvas.Left and Canvas.Top properties. They are called attached properties because
they are not regular dependency properties of the object. They are attached to that
object and interpreted only by the object's parent.

2.3 YouTube search

Integrating the YouTube video presented a challenge. The first option was to get a
hold on a H.264 stream from YouTube and, since Silverlight supports this format, we
could display it our application. The problem, also described in [4], is that it can be
hard to get a hold on the source of the h.264 encoding stream. One solution was using
a Greasemonkey script on a video that was uploaded in HD. Since this was very hard
to implement in a programmatic manner, we looked for another solution.
Silverlight allows html code to be displayed on top or behind the Silverlight
application if we use it in the windowless mode. This is a initialization parameter of
the Silverlight plugin and it allows html and silverlight to be on top of each other. The
main problem is that this has significant impact on the performance of the application.

Twet 9

Our 3D text space animation can be the main feature of Twet affected by this option,
but it is the only solution of embedding the Youtube video.
One of the new features introduced in Silverlight 3 was out-of-browser experience.
This meant installing the application on the client machine, where he can run it as a
usual desktop application. In this case, there is no page to host the application so we
cannot include the html object of the YouTube video. Solving this was done by using
a new feature introduced in Silverlight 4, which is the WebBrowser element. This
feature is only available for out-of-browser Silverlight applications and it allows
embedding html code in the application. One advantage is that it does not have any
performance impact, since this is a regular UI Element.
Another feature introduced in Silverlight 4 was native support for mouse wheel
event. This way, the scroll bar that appears on the ListBox holding the list of
YouTube videos matching a search can respond correctly to the wheel event. In
Silverlight 3 this could have been implemented by the developer, while in previous
versions it was unsupported.

Fig. 6. YouTube video embedded in the out-of-browser Twet application. On the
left of the screen, the ListBox containing the list of videos matching the query, each
having a thumbnail displayed.


2.4 WordNet .Net

Accessing the WordNet database was done using the WordNet.NET [1] project. This
is an open-source project indented to provide access to the WordNet thesaurus from a
.NET project.
WordNet.NET needs the path to the installation of WordNet. This is set using a
static variable and is used for all future calls to the library. It exposes method for
finding the part of speech for a single word and for retrieving a list of synonyms for
that word. This way, the complexity of the WordNet format is hidden and the
developers can concentrate on the actual application.

3 Enhancements

Currently, the Serializer only supports primitive types and collections. But there is a
way to enhance it to support serialization of any object. For this we need to define an
interface, ISerializable (note that this interface was removed in the Silverlight
runtime, so we can reuse this name) with two methods, Serialize(BinaryWriter) and
Deserialize(BinaryWriter). Any object that wants to be serialized has to implement
these two methods so that the Serializer class can call them when an object of type
ISerializable is passed. The class that will be serialized needs to be on both client and
server side.

References
[1] WordNet.NET, http://opensource.ebswift.com/WordNet.Net/
[2] Bing Maps Silverlight Control SDK, http://www.microsoft.com/downloads/details.aspx?
&FamilyID=beb29d27-6f0c-494f-b028-1e0e3187e830
[3] Terence Tsang, http://wwdraw.comw.shine
[4] YouTube video in Silverlight 3, http://www.85turns.com/2009/06/28/youtube-video-in-
silverlight-3/
[5] Silverlight Bing Maps, www.bing.com/maps/explore

Twet

Recomendados

Recomendados

Más contenido relacionado

Similar a Twet

Similar a Twet (20)

Último

Último (20)

Twet