Log all the searches going through Elasticsearch
You are looking for a way to retrieve the full Query DSL sent by an application to Elasticsearch in order to debug or simply see what’s going on. This article got you covered.
Sometimes we cannot inspect the HTTP query before it gets to Elasticsearch – maybe because we don’t control the application, or because there is no logger, or because we cannot edit production running code, etc.
Thankfully there is a simple way to log all searches and indexation requests sent to Elasticsearch in the engine log file.
Section intitulée enable-logging-of-all-searchesEnable logging of all searches
We are going to leverage the slowlog functionality. It’s already enabled by default but with a 5 seconds threshold on the info
level.
We lower the threshold of the trace
level and we switch to it:
PUT foobar/_settings
{
"index.search.slowlog.threshold.query.trace": "0s",
"index.search.slowlog.level": "trace"
}
Section intitulée inspecting-the-logsInspecting the logs
Now every search request sent to Elasticsearch will appear in the logs like this:
{"type": "index_search_slowlog", "timestamp": "2021-03-17T22:25:45,839Z", "level": "DEBUG", "component": "i.s.s.query", "cluster.name": "docker-cluster", "node.name": "es01", "message": "[foobar][0]", "took": "95.8micros", "took_millis": "0", "total_hits": "0 hits", "types": "[]", "stats": "[]", "search_type": "QUERY_THEN_FETCH", "total_shards": "1", "source": "{\"query\":{\"multi_match\":{\"query\":\"Covfefe\",\"fields\":[\"description^1.0\",\"title^10.0\"],\"type\":\"best_fields\",\"operator\":\"OR\",\"slop\":0,\"prefix_length\":0,\"max_expansions\":50,\"zero_terms_query\":\"NONE\",\"auto_generate_synonyms_phrase_query\":true,\"fuzzy_transpositions\":true,\"boost\":1.0}}}", "cluster.uuid": "-YsHSUgLTbG20EmG0GRAbg", "node.id": "DwoE3nCWQvGGPk-M-7rKhQ" }
The part we want is inside the source
field. As it is JSON inside a JSON it’s escaped, we just need to replace \"
by "
and we are good to go.
{"query":{"multi_match":{"query":"Covfefe","fields":["description^1.0","title^10.0"],"type":"best_fields","operator":"OR","slop":0,"prefix_length":0,"max_expansions":50,"zero_terms_query":"NONE","auto_generate_synonyms_phrase_query":true,"fuzzy_transpositions":true,"boost":1}}}
We can run this request in Kibana or any tool we like.
Note that the query we see here also contains the default options of search queries, so it’s not exactly the same Query DSL produced by the application.
Section intitulée disable-logging-of-all-searchesDisable logging of all searches
To go back to the default configuration:
PUT foobar/_settings
{
"index.search.slowlog.threshold.query.trace": "500ms",
"index.search.slowlog.level": "info"
}
Section intitulée it-also-works-for-indexing-requestsIt also works for indexing requests
This is less common but we can do the same with the indexing
options.
Enabling slowlog with full source (warning this can be heavy):
PUT foobar/_settings
{
"index.indexing.slowlog.threshold.index.trace": "0s",
"index.indexing.slowlog.level": "trace",
"index.indexing.slowlog.source": true
}
Getting back to the defaults:
PUT foobar/_settings
{
"index.indexing.slowlog.threshold.index.trace": "500ms",
"index.indexing.slowlog.level": "info",
"index.indexing.slowlog.source": "1000"
}
These tips and a lot of other nice “copy and paste” snippets are part of our Elasticsearch Cheatsheet.
Happy debug!
Commentaires et discussions
Elasticsearch the right way in Symfony
You are building an application with Symfony – good choice 😜 – but now you need some full-text search capabilities? This article is for you. Multiple options are available: going full RDMS and using FULLTEXT indexes – yes it works; using a third party SaaS like Algolia or Elastic…
Lire la suite de l’article Elasticsearch the right way in Symfony
Nos articles sur le même sujet
Nos formations sur ce sujet
Notre expertise est aussi disponible sous forme de formations professionnelles !

Elasticsearch
Indexation et recherche avancée, scalable et rapide avec Elasticsearch
Ces clients ont profité de notre expertise

Dans le cadre d’une refonte complète de son architecture Web, Expertissim a sollicité l’expertise de JoliCode afin de tenir les délais et le niveau de qualité attendus. Le domaine métier d’Expertissim n’est pas trivial : les spécificités du marché de l’art apportent une logique métier bien particulière et un processus complexe. La plateforme propose…

JoliCode a été sollicité pour accompagner le développement de la nouvelle version du site. Conçue avec le framework Symfony2, cette nouvelle version bénéficie de la performance et la fiabilité du framework français. Reposant sur des technologies comme Elasticsearch, cette nouvelle version tend à offrir une expérience optimale à l’internaute. Le développement…

Ouibus a pour ambition de devenir la référence du transport en bus longue distance. Dans cette optique, les enjeux à venir de la compagnie sont nombreux (vente multi-produit, agrandissement du réseau, diminution du time-to-market, amélioration de l’expérience et de la satisfaction client) et ont des conséquences sur la structuration de la nouvelle…