As a search consultant I need to understand how a search application is used with the end goal of providing a better search experience for the end user. That story can come from many places and part of that story can be found in the query logs. Blog post about the same topic: http://nhhagen.wordpress.com/2013/11/28/query-log-analysis-using-logstash-elasticsearch-and-kibana/

1. Query log analytics
28.11.2013
Niels Henrik Hagen
OSLO STOCKHOLM
LONDON
BOSTON

2. Who am I?
Niels Henrik Hagen
@nhhagen
Search enthusiast

3. Analyzing query logs
brings insight into how a
search application is used,

4. use that insight to
improve the search
application

5. What is a query log?

6. {"message":{"time":3016,"hitCounts":{"directmail":27,"refiners":0,"products":2529},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:07:02.011+00:00","@version":1,"source_host":"search3.kaw.
{"message":{"time":492,"hitCounts":{"directmail":0,"refiners":0,"products":5},"searchQuery":{"attributes":{"querystring":"fisk","sortby":"kawdefault","sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:07:19.714+00:00","@version":1,"source_host":"search3.kaw.no
{"message":{"time":459,"hitCounts":{"directmail":27,"refiners":0,"products":2529},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:11:37.852+00:00","@version":1,"source_host":"sea
{"message":{"time":157,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"112530","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:11:43.858+00:00","@version":1,"source_ho
{"message":{"time":155,"hitCounts":{"products":2524},"searchQuery":{"attributes":{"querystring":"skjenk hyller og kommoder ","sortby":"kawdefault","hits":9,"sortorder":"","similarto":"112530","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:11:44.810+00:00","@version":1,"s
{"message":{"time":252,"hitCounts":{"directmail":0,"refiners":0,"products":5},"searchQuery":{"attributes":{"querystring":"fisk","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:12:03.340+00:00","@version":1,"source_host":"sear
{"message":{"time":460,"hitCounts":{"directmail":27,"refiners":0,"products":2529},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:12:47.508+00:00","@version":1,"source_host":"sea
{"message":{"time":303,"hitCounts":{"directmail":5,"refiners":0,"products":122},"searchQuery":{"attributes":{"querystring":"leker","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:12:49.918+00:00","@version":1,"source_host":"s
{"message":{"time":386,"hitCounts":{"directmail":1,"refiners":0,"products":91},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","navigators":[{"key":"shop","displayName":"Butikk","interval":false,"upperValue":null,"lowerValue":null,"modifiers":[{"value":"E
{"message":{"time":22,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"112173","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:13:19.196+00:00","@version":1,"source_hos
{"message":{"time":152,"hitCounts":{"products":58},"searchQuery":{"attributes":{"querystring":"peisovn jøtul ","sortby":"kawdefault","hits":9,"sortorder":"","similarto":"112173","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:13:21.289+00:00","@version":1,"source_host":"s
{"message":{"time":46,"hitCounts":{"directmail":0},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","offset":0,"dmid":"1201","filters":[]}, "metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:13:54.993+00:00","@version":1,"source_host":"sea
{"message":{"time":292,"hitCounts":{"directmail":27,"refiners":0,"products":2529},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:14:08.393+00:00","@version":1,"source_host":"sea
{"message":{"time":335,"hitCounts":{"directmail":0,"refiners":0,"products":8},"searchQuery":{"attributes":{"querystring":"dyner","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:14:20.384+00:00","@version":1,"source_host":"sea
{"message":{"time":326,"hitCounts":{"directmail":27,"refiners":0,"products":2529},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:14:36.072+00:00","@version":1,"source_host":"sea
{"message":{"time":39,"hitCounts":{"directmail":0},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","offset":0,"dmid":"1201","filters":[]}, "metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:14:42.194+00:00","@version":1,"source_host":"sea
{"message":{"time":289,"hitCounts":{"directmail":2,"refiners":0,"products":222},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","navigators":[{"key":"shop","displayName":"Butikk","interval":false,"upperValue":null,"lowerValue":null,"modifiers":[{"value":"
{"message":{"time":333,"hitCounts":{"directmail":27,"refiners":0,"products":2529},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:15:22.236+00:00","@version":1,"source_host":"sea
{"message":{"time":316,"hitCounts":{"directmail":2,"refiners":0,"products":47},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","navigators":[{"key":"dmcategory","displayName":"Kategorier","interval":false,"upperValue":null,"lowerValue":null,"modifiers":[{
{"message":{"time":24,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","offset":0,"dmid":"1447","filters":[]}, "metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:15:55.959+00:00","@version":1,"source_host":"sea
{"message":{"time":294,"hitCounts":{"directmail":0,"refiners":0,"products":12},"searchQuery":{"attributes":{"querystring":"dusjkabinett","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:16:13.633+00:00","@version":1,"source_ho
{"message":{"time":22,"hitCounts":{"directmail":0},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","offset":0,"dmid":"1023","filters":[]}, "metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:16:15.842+00:00","@version":1,"source_host":"sea
{"message":{"time":260,"hitCounts":{"directmail":0,"refiners":0,"products":0,"expiredproducts":0},"searchQuery":{"attributes":{"querystring":"cd591700627no","sortby":"kawdefault","hits":30,"sortorder":"","offset":0}," metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:16:30.752+00:00","@v
{"message":{"time":20,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","offset":0,"dmid":"1447","filters":[]}, "metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:16:31.534+00:00","@version":1,"source_host":"sea
{"message":{"time":26,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"112597","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:16:34.619+00:00","@version":1,"source_hos
{"message":{"time":140,"hitCounts":{"products":132},"searchQuery":{"attributes":{"querystring":"dusjkabinett baderomsinnredning ","sortby":"kawdefault","hits":9,"sortorder":"","similarto":"112597","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:16:37.566+00:00","@version"
{"message":{"time":298,"hitCounts":{"directmail":0,"refiners":0,"products":30},"searchQuery":{"attributes":{"querystring":"kjøkkenutstyr","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:16:42.008+00:00","@version":1,"source_h
{"message":{"time":20,"hitCounts":{"directmail":0},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","offset":0,"dmid":"1062","filters":[]}, "metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:16:46.663+00:00","@version":1,"source_host":"sea
{"message":{"time":22,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","offset":0,"dmid":"1447","filters":[]}, "metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:16:52.785+00:00","@version":1,"source_host":"sea
{"message":{"time":20,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"111811","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:17:02.328+00:00","@version":1,"source_hos
{"message":{"time":146,"hitCounts":{"products":135},"searchQuery":{"attributes":{"querystring":"dusjkabinett element baderomsinnredning ","sortby":"kawdefault","hits":9,"sortorder":"","similarto":"111811","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:17:04.688+00:00","@
{"message":{"time":22,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","offset":0,"dmid":"1447","filters":[]}, "metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:17:04.729+00:00","@version":1,"source_host":"sea
{"message":{"time":22,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","offset":0,"dmid":"1447","filters":[]}, "metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:17:10.554+00:00","@version":1,"source_host":"sea
{"message":{"time":209,"hitCounts":{"products":135},"searchQuery":{"attributes":{"querystring":"dusjkabinett element baderomsinnredning ","sortby":"kawdefault","hits":9,"sortorder":"","similarto":"111811","offset":9},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:17:26.784+00:00","@
{"message":{"time":324,"hitCounts":{"directmail":2,"refiners":0,"products":47},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","navigators":[{"key":"dmcategory","displayName":"Kategorier","interval":false,"upperValue":null,"lowerValue":null,"modifiers":[{
{"message":{"time":21,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"112601","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:18:08.067+00:00","@version":1,"source_hos
{"message":{"time":145,"hitCounts":{"products":132},"searchQuery":{"attributes":{"querystring":"dusjkabinett baderomsinnredning ","sortby":"kawdefault","hits":9,"sortorder":"","similarto":"112601","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:18:10.400+00:00","@version"
{"message":{"time":21,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"111675","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:18:29.017+00:00","@version":1,"source_hos
{"message":{"time":349,"hitCounts":{"directmail":27,"refiners":0,"products":2529},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:18:31.404+00:00","@version":1,"source_host":"sea
{"message":{"time":338,"hitCounts":{"directmail":1,"refiners":0,"products":59},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","navigators":[{"key":"shop","displayName":"Butikk","interval":false,"upperValue":null,"lowerValue":null,"modifiers":[{"value":"
{"message":{"time":22,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"106264","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:19:26.681+00:00","@version":1,"source_hos
{"message":{"time":561,"hitCounts":{"directmail":0,"refiners":0,"products":174},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","navigators":[{"key":"dmcategory","displayName":"Kategorier","interval":false,"upperValue":null,"lowerValue":null,"modifiers":[
{"message":{"time":371,"hitCounts":{"directmail":27,"refiners":0,"products":2529},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:20:58.128+00:00","@version":1,"source_host":"sea
{"message":{"time":415,"hitCounts":{"directmail":0,"refiners":0,"products":11,"expiredproducts":11},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","navigators":[{"key":"dmcategory","displayName":"Kategorier","interval":fals e,"upperValue":null,"lowerValue
{"message":{"time":161,"hitCounts":{"directmail":0,"refiners":0,"products":7},"searchQuery":{"attributes":{"querystring":"snøfreser","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:22:29.636+00:00","@version":1,"source_host":
{"message":{"time":21,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"82298","offset":0,"filters ":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:22:31.727+00:00","@version":1,"source_host
{"message":{"time":318,"hitCounts":{"directmail":27,"refiners":0,"products":2529},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:22:44.942+00:00","@version":1,"source_host":"sea
{"message":{"time":431,"hitCounts":{"directmail":8,"refiners":0,"products":474,"expiredproducts":474},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","navigators":[{"key":"shop","displayName":"Butikk","interval":false,"upperValue":null,"lowerValue":null,"
{"message":{"time":584,"hitCounts":{"directmail":27,"refiners":0,"products":2529},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:23:24.424+00:00","@version":1,"source_host":"sea
{"message":{"time":18,"hitCounts":{"directmail":0},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","offset":0,"dmid":"1201","filters":[]}, "metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:23:41.415+00:00","@version":1,"source_host":"sea
{"message":{"time":19,"hitCounts":{"directmail":0},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","offset":0,"dmid":"1201","filters":[]}, "metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:23:41.696+00:00","@version":1,"source_host":"sea
{"message":{"time":375,"hitCounts":{"directmail":1,"refiners":0,"products":113,"expiredproducts":113},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","navigators":[{"key":"shop","displayName":"Butikk","interval":false,"upperValue":null,"lowerValue":null,"
{"message":{"time":298,"hitCounts":{"directmail":27,"refiners":0,"products":2529},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:24:05.682+00:00","@version":1,"source_host":"sea
{"message":{"time":265,"hitCounts":{"directmail":0,"refiners":0,"products":3},"searchQuery":{"attributes":{"querystring":"nettbrett samsung","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:24:05.936+00:00","@version":1,"sourc
{"message":{"time":331,"hitCounts":{"directmail":27,"refiners":0,"products":2529},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:24:08.566+00:00","@version":1,"source_host":"sea
{"message":{"time":333,"hitCounts":{"directmail":27,"refiners":0,"products":2529},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:25:18.267+00:00","@version":1,"source_host":"sea
{"message":{"time":245,"hitCounts":{"directmail":2,"refiners":0,"products":222},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","navigators":[{"key":"shop","displayName":"Butikk","interval":false,"upperValue":null,"lowerValue":null,"modifiers":[{"value":"
{"message":{"time":286,"hitCounts":{"directmail":0,"refiners":0,"products":100},"searchQuery":{"attributes":{"querystring":"sofa","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:26:36.636+00:00","@version":1,"source_host":"se
{"message":{"time":303,"hitCounts":{"directmail":2,"refiners":0,"products":222},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","navigators":[{"key":"shop","displayName":"Butikk","interval":false,"upperValue":null,"lowerValue":null,"modifiers":[{"value":"
{"message":{"time":305,"hitCounts":{"directmail":0,"refiners":0,"products":100},"searchQuery":{"attributes":{"querystring":"sofa","sortby":"kawdefault","hits":30,"sortorder":"","offset":30},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:26:57.286+00:00","@version":1,"source_host":"s
{"message":{"time":30,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"112781","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:27:07.063+00:00","@version":1,"source_hos
{"message":{"time":22,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","offset":0,"dmid":"1423","filters":[]}, "metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:27:10.700+00:00","@version":1,"source_host":"sea
{"message":{"time":24,"hitCounts":{"directmail":0},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"100610","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:27:10.901+00:00","@version":1,"source_hos
{"message":{"time":23,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"108910","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:27:11.605+00:00","@version":1,"source_hos
{"message":{"time":554,"hitCounts":{"directmail":3,"refiners":0,"products":180,"expiredproducts":180},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","navigators":[{"key":"shop","displayName":"Butikk","interval":false,"upperValue":null,"lowerValue":null,"
{"message":{"time":25,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"113365","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:27:15.965+00:00","@version":1,"source_hos
{"message":{"time":220,"hitCounts":{"products":99},"searchQuery":{"attributes":{"querystring":"buesofa sofa ","sortby":"kawdefault","hits":9,"sortorder":"","similarto":"113365","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:27:17.322+00:00","@version":1,"source_host":"se
{"message":{"time":21,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"113325","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:27:33.139+00:00","@version":1,"source_hos
{"message":{"time":51,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"113419","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:27:39.825+00:00","@version":1,"source_hos
{"message":{"time":20,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"113425","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:27:46.738+00:00","@version":1,"source_hos
{"message":{"time":22,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"113319","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:27:53.654+00:00","@version":1,"source_hos
{"message":{"time":41,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"112136","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:28:00.294+00:00","@version":1,"source_hos
{"message":{"time":20,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"112473","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:28:07.574+00:00","@version":1,"source_hos
{"message":{"time":23,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"112434","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:28:14.495+00:00","@version":1,"source_hos
{"message":{"time":125,"hitCounts":{"directmail":0,"refiners":0,"products":7},"searchQuery":{"attributes":{"querystring":"snøfreser","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:28:47.808+00:00","@version":1,"source_host":
{"message":{"time":22,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"113135","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:28:56.820+00:00","@version":1,"source_hos
{"message":{"time":224,"hitCounts":{"directmail":2,"refiners":0},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","navigators":[{"key":"shop","displayName":"Butikk","interval":false,"upperValue":null,"lowerValue":null,"modifiers":[{"value":""Jula Norge AS
{"message":{"time":27,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"109528","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:30:29.552+00:00","@version":1,"source_hos
{"message":{"time":181,"hitCounts":{"products":59},"searchQuery":{"attributes":{"querystring":"panelovn adax husholdningsapparater ","sortby":"kawdefault","hits":9,"sortorder":"","similarto":"109528","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:30:31.191+00:00","@versi
{"message":{"time":31,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","offset":0,"dmid":"1397","filters":[]}, "metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:30:37.338+00:00","@version":1,"source_host":"sea
{"message":{"time":79,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"112411","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:30:46.252+00:00","@version":1,"source_hos
{"message":{"time":159,"hitCounts":{"products":2524},"searchQuery":{"attributes":{"querystring":"panelovn adax oppvarming og ventilasjon ovn ","sortby":"kawdefault","hits":9,"sortorder":"","similarto":"112411","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:30:47.651+00:0
{"message":{"time":25,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","offset":0,"dmid":"1452","filters":[]}, "metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:31:00.325+00:00","@version":1,"source_host":"sea
{"message":{"time":25,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","offset":0,"dmid":"1452","filters":[]}, "metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:31:05.232+00:00","@version":1,"source_host":"sea
{"message":{"time":34,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","offset":0,"dmid":"1452","filters":[]}, "metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:31:08.092+00:00","@version":1,"source_host":"sea
{"message":{"time":28,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","offset":0,"dmid":"1452","filters":[]}, "metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:31:14.208+00:00","@version":1,"source_host":"sea
{"message":{"time":24,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","offset":0,"dmid":"1452","filters":[]}, "metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:31:18.324+00:00","@version":1,"source_host":"sea
{"message":{"time":25,"hitCounts":{"directmail":0},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"99828","offset":0,"filters ":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:31:43.469+00:00","@version":1,"source_host
{"message":{"time":316,"hitCounts":{"directmail":0,"refiners":0,"products":89},"searchQuery":{"attributes":{"querystring":"verktøy","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:31:45.228+00:00","@version":1,"source_host":"
{"message":{"time":28,"hitCounts":{"directmail":0},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"84401","offset":0,"filters ":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:31:46.311+00:00","@version":1,"source_host
{"message":{"time":248,"hitCounts":{"directmail":2,"refiners":0,"products":222},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","navigators":[{"key":"shop","displayName":"Butikk","interval":false,"upperValue":null,"lowerValue":null,"modifiers":[{"value":"
{"message":{"time":305,"hitCounts":{"directmail":0,"refiners":0,"products":89},"searchQuery":{"attributes":{"querystring":"verktøy","sortby":"kawdefault","hits":30,"sortorder":"","offset":30},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:32:05.858+00:00","@version":1,"source_host":
{"message":{"time":228,"hitCounts":{"directmail":2,"refiners":0,"products":511},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","navigators":[{"key":"shop","displayName":"Butikk","interval":false,"upperValue":null,"lowerValue":null,"modifiers":[{"value":"
{"message":{"time":279,"hitCounts":{"directmail":2,"refiners":0,"products":222},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","navigators":[{"key":"shop","displayName":"Butikk","interval":false,"upperValue":null,"lowerValue":null,"modifiers":[{"value":"
{"message":{"time":252,"hitCounts":{"directmail":0,"refiners":0,"products":35},"searchQuery":{"attributes":{"querystring":"data","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:32:38.465+00:00","@version":1,"source_host":"sea
{"message":{"time":280,"hitCounts":{"directmail":2,"refiners":0,"products":511},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","navigators":[{"key":"shop","displayName":"Butikk","interval":false,"upperValue":null,"lowerValue":null,"modifiers":[{"value":"
{"message":{"time":20,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"112424","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:33:13.364+00:00","@version":1,"source_hos
{"message":{"time":333,"hitCounts":{"products":2524},"searchQuery":{"attributes":{"querystring":"takpanel s�dra gulv, tak og vegg ","sortby":"kawdefault","hits":9,"sortorder":"","similarto":"112424","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:33:20.508+00:00","@versi
{"message":{"time":26,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"82112","offset":0,"filters ":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:33:28.893+00:00","@version":1,"source_host
{"message":{"time":229,"hitCounts":{"directmail":0,"refiners":0,"products":100},"searchQuery":{"attributes":{"querystring":"sofa","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:33:48.443+00:00","@version":1,"source_host":"se
{"message":{"time":211,"hitCounts":{"directmail":0,"refiners":0,"products":7},"searchQuery":{"attributes":{"querystring":"snøfreser","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:33:51.851+00:00","@version":1,"source_host":
{"message":{"time":418,"hitCounts":{"directmail":27,"refiners":0,"products":2529},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:34:08.625+00:00","@version":1,"source_host":"sea
{"message":{"time":25,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"82183","offset":0,"filters ":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:34:16.094+00:00","@version":1,"source_host
{"message":{"time":314,"hitCounts":{"directmail":27,"refiners":0,"products":2529},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:34:20.807+00:00","@version":1,"source_host":"sea
{"message":{"time":21,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"112432","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:34:35.963+00:00","@version":1,"source_hos
{"message":{"time":157,"hitCounts":{"products":64},"searchQuery":{"attributes":{"querystring":"snøfreser peisovn ","sortby":"kawdefault","hits":9,"sortorder":"","similarto":"112432","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:34:37.153+00:00","@version":1,"source_host
{"message":{"time":365,"hitCounts":{"directmail":1,"refiners":0,"products":99},"searchQuery":{"attributes":{"querystring":"ǂ","sortby":"kawdefault","hits":30,"sortorder":"","navigators":[{"key":"shop","displayName":"Butikk","interval":false,"upperValue":null,"lowerValue":null,"modifiers":[{"value":"
{"message":{"time":19,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","offset":0,"dmid":"1452","filters":[]}, "metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:35:12.913+00:00","@version":1,"source_host":"sea
{"message":{"time":89,"hitCounts":{"directmail":1},"searchQuery":{"attributes":{"hits":1,"querystring":"","sortby":"","navigators":[],"sortorder":"","productid":"109114","offset":0,"filter s":[]},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:35:21.318+00:00","@version":1,"source_hos
{"message":{"time":179,"hitCounts":{"products":59},"searchQuery":{"attributes":{"querystring":"panelovn nobø husholdningsapparater ","sortby":"kawdefault","hits":9,"sortorder":"","similarto":"109114","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:35:23.062+00:00","@versi
{"message":{"time":139,"hitCounts":{"directmail":0,"refiners":0,"products":7},"searchQuery":{"attributes":{"querystring":"snøfreser","sortby":"kawdefault","hits":30,"sortorder":"","offset":0},"metaAttributes":{"queryModel":{}}}},"@timestamp":"2013-10-22T11:35:28.482+00:00","@version":1,"source_host":

7. What is a query log made up of ?
•
•
•
•
•
•
Timestamp
Query text
Activated facets
Pagination
Number of hits
…

8. How to understand
the content of a log
file?

9. Visualization!

11. Where to find the log data?

15. How to make text into a graph?

16. Logstash
log crawling, parsing and processing

17. ElasticSearch
distributed restful search and analytics

18. Kibana
real-time analysis of streaming data

19. Simplified architecture

20. The result

23. What could be next?
Pieces missing from the puzzle

24. Track clicks in the result set
– What was clicked and what was not clicked
– Try to calculate how relevant returned results
are

25. Track multiple queries across a “session”
– Data could be used to expand queries

26. Thank you
Niels Henrik Hagen
@nhhagen