Elasticsearch#

Full-text search, relevance ordering, highlighted snippets and facet counts.

Core: app/Core/Integrations/Elasticsearch/. Document sources: app/UI/Api/{module}/Models/Search/*DocumentSource.php. Endpoint: app/Core/Traits/Api/Presenters/SearchTrait.php. Configuration: config/Shared/elasticsearch.neon.

The founding rule#

Elasticsearch is a source of identifiers and ordering, not of data

Document bodies are never read from ES (_source: false). It returns a list of ids, optionally their score order and highlighted snippets. The data is then loaded from the database. The index therefore need not carry every field and cannot return stale content.

The database remains the arbiter of visibility

The caller's filters (active, published, paid, category…) are applied always and on both paths. A stale document can therefore never reveal an entity the user must not see — at worst it shortens a page. The pre-narrowing described below is an optimisation, not a security measure.

1. Two paths, one endpoint#

Every indexed module has an Api action api/{module}/{entity}/search. Its shape is module-neutral — SearchTrait supplies it, the module fills in three hooks.

Situation	Who orders	Who paginates	`totalCount`	`fromEs`	Snippets
search without `orderBys`	ES (score)	ES (`from`/`size`)	ES (`track_total_hits`)	`true`	yes
search with `orderBys` (manual choice)	DB	DB	DB	`false`	no
ES off / outage / breaker / query shorter than 2 characters	DB	DB	DB	`false`	no

A manual ordering choice overrides relevance — by design

If ES paginated even for "Newest", the user would get the newest only within the page of the most relevant. On that branch only the id set is taken from ES (phase one) and the database orders, paginates and counts — exactly as before ES.

The query is always multi_match, never query_string

Both search methods of EsSearcher build a multi_match over the source fields. query_string would turn user input into query syntax: special characters (", (, :, /) would turn a search into a 400 error. Whoever extends the query adds clauses around multi_match, not instead of it.

Shape of the response#

Identical to internalActionGetAll() plus two extra keys:

[
    'items'       => [...],   // already reordered by score
    'totalCount'  => 42,
    '_references' => [...],
    'fromEs'      => true,    // ordering and totalCount come from ES
    'highlights'  => [        // entity id => logical field => safe HTML
        17 => ['name' => 'Selling a <em>drill</em>', 'text' => '…'],
    ],
]

fromEs is the only reliable signal for the front end

Configuration says nothing about ES availability — the breaker may be open, the index may not exist. So the front end asks the payload, not the config. fromEs decides whether the listing is relevance-ordered and whether any sort-box option is highlighted.

The highlights key is ALWAYS present

Even when empty. Callers therefore need not guard it by fromEs. A template that forgets it (highlights: [] when including a nested listing) fails on undefined variables.

Why no query cache here#

The cache would serve stale ordering

SearchIdFilter is a NonCacheableFilter — the key would have to contain the whole id set. On top of that the ES response changes without any database write (reindex, score changes), so the cache would have nothing to invalidate on.

2. Configuration#

parameters:
    elasticsearch:
        enabled: false                    # master switch — false = every service is a no-op
        hosts: ['http://127.0.0.1:9200']
        username: null
        password: null
        indexPrefix: cms
        enabledSources: []                # allowlist of source keys

Hosts and credentials belong in the gitignored config/Shared/local.neon.

Several installations on one cluster: each its own indexPrefix

With the same prefix the installations write into the same indexes and also share the es:<prefix>:… keys in Redis. See Several installations on one server.

Switch layer	Effect
`enabled: false`	no connection is opened, everything behaves as before ES
`enabledSources` without a source key	that one source runs on MySQL, the others on ES
circuit breaker open	ES is not called at all for 60 seconds

Roll out module by module

That allowlist is the whole point — you can have "ES on for the blog, classifieds still on MySQL". A new module is switched on only after a verified reindex.

Two clients#

Service	Timeout	Used for
`esClient`	short	search, monitoring — an outage must fail fast
`esIndexingClient`	60 s	indexing and bulk reindex — hundreds of documents must not die on 2.5 s

Constructing a client opens no connection

That is why it is safe to have the services in the container even with enabled: false.

Timeouts live only in the own Guzzle instance

EsClientFactory hands the client a ready GuzzleHttp\Client through setHttpClient(), with the timeouts in its constructor. If someone adds setHttpClientOptions() or SSL options, the vendor adapter builds a new Guzzle client from those options only and the timeouts are lost: during an outage search then waits without limit (the Guzzle default timeout is 0) instead of 2.5 s.

ES credentials: both values or neither

Only username or only password filled in throws LogicException as soon as the client is built. The vendor would silently drop incomplete credentials and the result would look like an outage (401).

3. Indices, aliases and a reindex without downtime#

A physical index is named {prefix}_{base}_v{N}, for example cms_blog_article_cs_v5. The application never touches it directly — it speaks through two aliases:

Alias	Used by	Points to
`cms_{base}` (read)	search, monitoring	the stable version
`cms_{base}_write`	indexing from save hooks	the new version during a reindex

Course of a full reindex (EsIndexManager):

beginReindex()    creates _v(N+1), switches the WRITE alias to the new version
                  └─ concurrent save hooks then write straight into the new version
   ...bulk...     fills the new version in batches of 200
finishReindex()   refresh the new version → atomic READ alias swap → drop old ones
rollbackReindex() on failure: write alias back to the stable version, discard the new one

Without refresh() before the alias swap, search sees nothing

Bulk writes carry no refresh and the default interval is one second. Were the read alias switched immediately, search and monitoring would briefly see an incomplete new version. The explicit refresh is therefore part of finishReindex().

A concrete index named like an alias blocks the whole bootstrap

It typically appears through auto-create from a write that arrived before the indices were set up. ensureIndexes() raises a readable exception with the fix (DELETE /cms_{base}), because otherwise updateAliases would fail forever.

Czech analysis#

Every full-text field carries two analyzers:

Analyzer	Filters	Purpose
`text_czech`	lowercase, Czech stopwords, Czech stemmer, asciifolding	the base field — also finds inflected forms
`text_folded`	lowercase, asciifolding	the `.folded` subfield — catches queries typed without diacritics

Changing the analysis requires a new index

The analysis of an existing index cannot be changed at runtime. A new version is built and switched to — exactly what a full reindex does.

4. The document source#

One class per entity, living in the module's Api layer (so it can reach services and managers); Core depends on it only through an interface. It is wired by hand in config/Shared/elasticsearch.neon in two places — esSearcher and esMonitor.

Method	Returns	Watch out for
`getKey()`	key for the allowlist and the searcher	must match `enabledSources`
`getIndexBases()`	name bases, one per language for translated entities
`getIndexBaseForLanguage()`	base for a language alias, `null` = unknown language	`null` means falling back to the DB path
`getIndexDefinition()`	settings + mappings	the source of truth about analysis
`getSearchFields()`	fields with boosts for `multi_match`	always the `.folded` variants too
`fetchFresh()`	fresh entities after the commit	`useCache: false`; the entity from the hook is not authoritative
`buildDocuments()`	`base => (_id => document)`	an empty array means the entity is deleted from the index
`getAllDocumentIds()`	every possible `_id` across languages	targeted deletes, never `delete_by_query`
`fetchReindexBatch()`	a batch for the full reindex
`getEntityIdField()`	name of the field holding the entity id	ordering and `search_after` during the orphan scan
`fetchIdsChangedSince()`	ids changed since the watermark	input of the incremental sync
`fetchExistingIds()`	which ids really exist in the database	orphan cleanup
`expectedDocumentBounds()`	`{min, max}` of the expected count	must mirror the `buildDocuments()` guards exactly

expectedDocumentBounds() must agree with buildDocuments()

It is the monitoring input. Once they diverge, a false drift is reported on every check — and then nobody looks at the real one.

A source that can only write leaves deleted records in the index

Ids that no longer exist in the database then show up in results. That is why deletion is part of the contract (getAllDocumentIds plus an empty buildDocuments).

The document `_id`#

One shape across all sources: "{entityId}_{languageId}". The searcher reads the entity id from it via strtok('_'), so a different shape breaks reading. For a single-language entity the insertion language is used.

Two optional extensions#

interface HighlightableDocumentSourceInterface extends DocumentSourceInterface
{
    /** logical key => ES fields in order of preference */
    public function getHighlightFields(): array;
}

interface VisibilityConstrainedDocumentSourceInterface extends DocumentSourceInterface
{
    /** clauses for bool.filter — they only narrow, they do not affect the score */
    public function getVisibilityFilters(): array;
}

Both are opt-in and backwards-neutral

A source without the interface sends a byte-identical query to the one it sent before. That is why rolling out to another module cannot show up in the others.

5. Highlighted snippets#

The security core#

EsHighlighter is the single place where an ES response becomes HTML

A raw fragment never leaves the searcher. Whoever wants to print a snippet gets a finished, safe string.

ES does not wrap matches in  but in two Private Use Area characters (U+E000 / U+E001). The conversion then follows a fixed order:

1. htmlspecialchars(THE WHOLE fragment)  → no executable markup is left in the string
2. only then sentinels → <em> / </em>

Reversing those two steps is stored XSS

The content comes from an editor, so <script> or "><img onerror=… can be in it. Inserting the tags first and escaping afterwards would escape them too — and narrowing the escaping to "the rest" is exactly the task on which XSS is made.

The resulting allowlist is therefore structural, not textual: the only tags that can appear in the output are  and  without a single attribute. The output is always balanced as well — unbalanced or nested sentinels are dropped.

Fields for snippets#

getHighlightFields() returns logical key => list of ES fields in order of preference. The logical key (name, perex, text) is the one under which the template finds the snippet.

Rule	Why
list the base field and its `.folded` variant	Czech and folded analysis match differently; a query that hit only one of them would get no snippet from the other
base field first	the stemmer highlights inflected forms too, `.folded` is the fallback for a query without diacritics
only fields the listing actually prints	`number_of_fragments: 0` returns the whole field; a snippet with nowhere to show it only makes the query dearer (an article's `content`, an advert's `keywords` and `city` are searched but not highlighted)
an empty array means no snippets	no `highlight` block goes into the query
a field without a match is missing from the map	ES does not return it in `highlight`, the template prints the original text

Only PHP escapes, ES does not

The highlight block sends encoder: default, so ES encodes nothing in the fragment. Switching to encoder: html would escape the fragment a second time and the snippet would show entities such as &amp;.

Printing it in a template#

The renderer pulls the snippet into a variable up front ($nameHl, $textHl); the template then only decides whether there is something to replace the original with:

{* element body: the snippet through |noescape, otherwise the original value *}
<a class="list-item__title" href="{$link}" title="{$name}">
    {if $nameHl}{$nameHl|noescape}{else}{$name}{/if}
</a>

{* description: NO |truncate — EsHighlightSnippet already shortened it *}
<p class="list-item__desc">
    {if $textHl}{$textHl|noescape}{else}{$perex|stripHtml|truncate:150}{/if}
</p>

A snippet never belongs in an HTML attribute

The escaping is computed for an element body, not for an attribute value. title= and alt= take the plain text from the database — note that title="{$name}" above does not use the snippet.

Never run |truncate over a snippet

The shortening already happened in the right place (around the match). Truncating again in the template would cut off the very word that is highlighted.

Shortening to the listing's length#

EsHighlighter asks for number_of_fragments: 0, that is the whole field. That suits a title and a lead paragraph, but not the description of a classified ad, which easily runs to 1,200 characters. EsHighlightSnippet::around() shortens it.

A naive |truncate over finished HTML is not enough

It would cut in the middle of an  or of an entity (& → &am), it would count markup towards the limit — and above all it would cut from the start, so a match at character 400 would not be visible at all.

around() therefore picks a window around the first match and cuts by tokens: no new tag can appear,  is always closed and an entity is never split.

Narrowing to the items of the listing#

ES highlights documents the database has thrown away too

A stale document of a deactivated entity drops out of items, but without the filter its title and text would stay in highlights — a leak of text the caller must not see. SearchTrait::highlightsForItems() therefore narrows the map to the ids that really are in the listing. The condition is "the item is in the listing", not "the id is in ES".

6. Pre-narrowing by visibility#

An optional extension of a source. It was born for classified ads: by design the index also keeps expired and inactive ones, so sixty-nine documents stood against two visible ones. The relevance path paginates in ES, so pages came back shorter than the limit and totalCount was badly overstated.

public function getVisibilityFilters(): array
{
    return [
        ['term'  => ['active' => true]],
        ['range' => ['expirationDate' => ['gte' => 'now/d']]],
    ];
}

Property	Why
clauses go into `bool.filter`, not `must`	a filter does not affect the score — relevance ordering stays identical
`active` is a denormalised flag	a generated column in the database; the value is read from the DB, never recomputed in PHP
expiry is a query-time `range`	`NOW()` cannot go into a generated column, and in the index it would go stale every midnight

The definition of active DIFFERS between modules

For classifieds it includes the "sold" flag, for the company catalogue it does not. That is why the value is always read from the database through the generated column's getter and never reassembled in PHP.

Only conditions the source can propagate into the index belong here

The opposite direction (invisible in the index, already visible in the database) is the price of pre-narrowing: such an entity drops out of search until a hook or a reindex catches up. Conditions that depend on settings do not belong here at all — the query would behave one way in the CLI and another on the web.

An article pre-narrows by its publication date: published changes without a write of the entity (the article "comes out" at midnight), so it cannot be a buildDocuments() condition, and the index therefore also holds articles dated in the future. The filter mirrors the front end DateBeforeFilter('published', new DateTime()):

['range' => ['published' => ['lt' => 'now/d', 'time_zone' => date_default_timezone_get()]]]

A date in range needs time_zone

Without it ES rounds now/d to midnight UTC, not to the application's midnight. In summer time, between 00:00 and 02:00 Prague time, articles published yesterday vanish from search although the database shows them. A condition that depends on the user's access (a "free" category) does not belong here; only the database evaluates it.

7. The path to the front end#

The front end never talks to ES or to the database directly — it goes through the Api bridge:

Presenter trait ──► Manager::search() ──► Service ──► Mapper ──► Repository
                                                                     │
                                                        ApiBridge::internalCall
                                                                     ▼
                                                       Api presenter · SearchTrait
                                                           ├─ EsSearcher (ES)
                                                           └─ Manager::findAll (DB)

The manager returns a quadruple [entities, totalCount, fromEs, highlights].

Hydration keeps the payload order — it must not reverse it

The Api side reorders via orderItemsByIds(); the front end then hydrates in the order the items arrived in. Any interference with the order in the mapper silently discards relevance — and on a two-item result it need not even be visible.

The presenter builds three things out of it:

Template variable	Meaning
`items`	the listing in score order
`searchHighlights`	`id => field => HTML`; empty = a listing without highlighting
`sortActiveOrder`	`0` while relevance is active (no sort-box option is highlighted), otherwise the chosen ordering

A deliberate deviation: promoted items are not promoted on the relevance path

Without a search, and with a manual ordering choice, promotion works unchanged. When ordering by score, ES decides the order and the database does not interfere.

When the front end drops the search endpoint#

searchViaApi() in the list trait of the presenter (Blog, Adverts, Company catalog, Discussion, e-shop sets) returns null and the list then takes the presenter's own DB path (prepareFilters() with SearchIdFilter, or MySQL MATCH_AGAINST):

Situation	Why
the mapper returned `null`	the response is not a list: deny envelope, bridge outage, empty payload from the repository (fail-soft `[]`)
no manual order choice and `fromEs = false`	ES is down or the query is short; the DB branch of the endpoint does not order without `orderBys`, the own path gives the list's deterministic default order

The mapper must not return an empty list instead of null

The presenter would render an empty array as "nothing found" although the search merely did not complete. null means "fall back to the DB path", empty items means "really nothing".

8. Permissions and routes#

Without the privilege, search is SILENTLY empty

Api RBAC runs in enforcing mode and a missing privilege is not an error — it is an empty result. Every module therefore needs a :Api:{Module}:{Entity}:search record; the scripts live in docs/sql/ (*-search-privilege.sql).

The listing route must be able to express "ordered by relevance". A sentinel in the mask does that:

o-<order=0>      the default value 0 means relevance

Nette drops the default value from the URL

A segment equal to the default is removed from the address and an incoming /o-0 is redirected (301) to the form without it. Relevance therefore lives on the bare URL; /o-0 exists only as canonicalisation. The scripts live in docs/sql/ (*-route-order-default.sql).

9. Operations#

Job	URL	Frequency	What it does
Full reindex	`/cron/{module}/elastic/reindex`	nightly	builds a new version and swaps the alias — the primary consistency guarantee; runs in parts, see below
Incremental sync	`/cron/{module}/elastic/sync`	every 10 min	indexes what changed since the watermark (5 min overlap) plus orphan cleanup; what "changed" means depends on the source, see below
Status	`/cron/system/elastic/status`	on demand	cluster health and document counts
Watch	`/cron/system/elastic/watch`	every 15 min	checks plus a summary email on findings

Writing to the index goes through the completedSave and completedDelete hooks, that is after the commit.

Bulk operations outside the hooks are invisible to the index

Scheduled jobs, payments, updateColumn and raw SQL write outside the hooks. The incremental sync will not catch them — only the nightly full reindex will. Where freshness matters, targeted reindex triggers sit in the code.

Full reindex in parts#

EsEntityIndexer::reindexAll() processes a source in batches by key (id > cursor) and one call takes at most REINDEX_SLICE_SECONDS (20 s, at least one batch of 200 documents). The progress (the cursor, the version of the index in progress) is in Redis. Until it is done, the cron action …/elastic/reindex answers {"status":"skipped","reason":"reindex_continues"} and the scheduler calls it again on the next tick; it is done when it returns status: ok. The read alias is switched only after the last part; until then the old version is searched and the running writes from the hooks go only to the new one. A run in progress is also finished by the sync of the same source: instead of its own work it calls the next part of the reindex (a safety net in case nobody calls the reindex any more). A run that has not advanced for an hour (REINDEX_STALE_SECONDS) is discarded by the sync. Without Redis the reindex runs in one piece. $timeBudget = null runs the whole reindex in one request (tests, a manual script).

Publication and rollback#

Situation	What happens
even a single bulk item fails	the whole run is rolled back: write alias back to the stable version, the unfinished `_vN` deleted
the refresh of the new version reports a failed shard	an exception before the alias swap; the read alias still points to the old version
the read alias swap went through	the version is published; cleanup of old versions is best-effort, whatever remains is removed by the next reindex
rollback after the swap (only cleanup or clearing the Redis state failed)	`rollbackReindex()` decides by the current aliases: for the version the read alias points to it only realigns the write alias and does not delete the index
reindex killed midway (OOM, timeout, kill)	the write alias stayed on the new version nobody searches; sync points it back to the read alias target outside a running reindex

Never delete the index the read alias points to by hand

Deleting a physical index removes the alias pointing to it as well, and search is left without an index. The unfinished version is cleaned up by the rollback or by the next successful reindex (it deletes all old versions by name pattern).

Run the reindex through the scheduler

A hosting cron that calls the job directly has to call it repeatedly until it returns status: ok. Edits made during the reindex show up in search only after the alias is switched.

What the incremental sync catches#

The sync takes ids from the source's fetchIdsChangedSince() (since the watermark in Redis es:<indexPrefix>:watermark:<source> minus 5 minutes, SYNC_OVERLAP_MINUTES), adds the queue of writes that failed while ES was down, and then deletes documents without a DB row. Which changes the watermark recognises depends on the column the entity has:

Source	Looks at	What follows
article (`ArticleDocumentSource`)	`insertTime` or `updateDatetime`	new and edited articles
sale card (`ProductSetDocumentSource`)	`insertTime` or `updateTime`	mostly new cards — `updateTime` is not reliably bumped on save
advert (`Bazaar\…\AdvertDocumentSource`)	`insertDatetime`	only new adverts — the entity has no edit time
company profile (`Comcat\…\AdvertDocumentSource`)	`insertDatetime`	only new profiles
discussion thread (`MessageDocumentSource`)	`insertDatetime` and `level = 1`	only new threads; replies are not indexed

Edits of the sources without an edit time are written by the completedSave hook right after the commit; payments and expiration of adverts and profiles by targeted triggers. For them the main value of the sync is orphan cleanup — for example threads removed by the tree deleteNode(), which the hook does not delete from the index.

A change outside the hook reaches the index only with the nightly reindex

updateColumn() or raw SQL on an advert, profile or thread is invisible to the sync, because it does not change the insert time. Search keeps the old value until the night. Where that matters, call EsEntityIndexer::index($source, $ids) with the affected ids after the write.

Writes from the administration#

Admin managers have no completedSave/completedDelete hooks and nobody moves the update time on the admin path in most cases, so the sync would not see an admin edit. Indexed entities therefore use ElasticReindexTrait in the admin manager (article, product set, advert, company profile, discussion post). The trait overrides save(), updateColumn(), updateColumns() and delete() and after success calls EsEntityIndexer::index() with the affected ids. The indexer removes a deleted or deactivated record from the index itself. An ES outage is only logged (channel elasticsearch.adminReindex); the database write stays.

The optional touchForEsWatermark() moves the update time before the save (setUpdateTime(), setUpdateDatetime()), so the sync sees the change too as a second safety net.

An own save() in the admin manager switches the reindex off

A method of the class wins over the trait method. Whoever overrides save() must call esReindexBestEffort([$id]) after success, otherwise the save reaches the index only with the nightly reindex.

The recount of advert prices (AdvertManager::recountPrices()) indexes only adverts with a changed price; from 200 changes (RECOUNT_FULL_REINDEX_FROM) it starts a full reindex.

What the monitoring watches#

Check	When it is a problem
cluster availability and health	any status other than `green`
drift of the document count against the database	outside the tolerance (5 %, at least 5 documents)
age of the last successful reindex	over 26 hours

Findings are deduplicated before sending (Redis, 6 hours), so the same problem is not mailed out every quarter of an hour.

The checks are done by EsMonitor, which sends no e-mail itself (Core does not know the Api EmailManager). It returns a report; App\UI\Api\System\Models\Managers\ElasticWatchManager combines the check and the optional e-mail. An error in one check does not stop the others and is itself reported as a finding.

Outage#

During an outage, search returns database results, not an error

The circuit breaker (Redis, a key with a 60-second lifetime) stops calling ES after repeated failures, so nothing waits on timeouts. The caller falls through to MySQL full-text. It does mean, though, that "fewer results" can mean "the index is down" — hence the monitoring.

A 4xx response does NOT open the breaker

A missing index before bootstrap or a malformed query is not a cluster outage and must not turn search off for everyone. The breaker opens only on genuine outage errors.

A database error during indexing does not open the breaker either

A write from a hook has three phases: bootstrap (ES), building the documents from the source (DB, ORM) and bulk (ES). Only an error in an ES phase opens the breaker. An exception in fetchFresh() or buildDocuments() is logged and the ids go to the queue of undelivered writes; if it opened the breaker, every source with the same prefix would lose search for 60 s because of an error unrelated to ES.

Queue of undelivered writes#

A write that did not happen (open breaker, outage, failed bulk item) queues its ids in Redis. Sync reads the queue, reindexes the ids and removes them after success. Without it, writes that do not move the update time (updateColumn, raw SQL, a discussion message without an update column) would be lost during an outage until the nightly reindex. A Redis error is only logged.

Key (`es:<indexPrefix>:…`)	Content
`pendingz:<source>`	ZSET of ids, score = generation of the latest undelivered write
`pendingseq:<source>`	generation counter (`INCR`)
`pending:<source>`	old format (a SET without generation); only read out, nothing new goes into it

The acknowledgement removes an id only when it still carries the generation the sync read. A newer undelivered write of the same id therefore survives the success of an older sync.

Delete the queue and the counter only together

A counter deleted on its own restarts generations from 1. A new write can then get the same number as an id in a sync snapshot in progress, and its acknowledgement deletes it, so it never reaches the index. When cleaning up by hand, delete both pendingz:<source> and pendingseq:<source>, or neither.

Without Redis the breaker does not work

Degradation then only happens through the client's short timeouts — more slowly, and again for every request.

Limits#

Limit	Value	What happens when exceeded
the `from + size` window	10,000	a deeper page → fallback to the DB path (which reaches there)
phase-one id set	10,000	the excess is logged as an overflow
facet buckets	10,000	incomplete counts → fallback to the exact SQL aggregation
minimum query length	2 characters	ES is not called at all

Facet counts refuse imprecision, not just unavailability

When the number of matched documents disagrees with the number of visible ids, unavailability is returned and SQL takes over. Incomplete counts would lie silently, and that is worse than a slower query.

10. Adding a new source#

A document source in app/UI/Api/{Module}/Models/Search/ — take the closest existing one as the model (ArticleDocumentSource for an entity with columns, AdvertDocumentSource for one whose texts live in parameters).
Wire it in elasticsearch.neon in both places — esSearcher and esMonitor.
Api presenter: use SearchTrait and fill in the three hooks — getEsSourceKey(), getEsSearcher(), getFulltextFallbackFields().
The SQL privilege :Api:{Module}:{Entity}:search and the route mask with the sentinel.
The front-end stack: search() in the repository, mapper, service and manager; a search branch in the presenter.
Templates: print the snippet through |noescape, plain text into attributes.
The admin manager of the entity: use ElasticReindexTrait; and EsEntityIndexer $esIndexer plus DocumentSource $esDocumentSource in the constructor, see Writes from the administration.
Reindex, and only then add the key to enabledSources.

The fallback fields must match the indexed ones

getFulltextFallbackFields() is the MySQL variant of the same search. Once it diverges from getSearchFields(), users get different results during an ES outage — and nobody notices, because both paths "work".

For an entity whose texts live in parameters, the RENDERED value is indexed

The values are raw in the database and the replacement of links and blacklisted words happens only at render time. The source must therefore push them through the same path as the template — otherwise words the renderer hides leak into the index, and from there into the snippet.

Index HTML from the editor as plain text

An article's perex and content and an advert's description are HTML. The source runs them through strip_tags() and html_entity_decode() (for the advert also with a space before every tag, so that endStart does not merge into one word). Indexed with markup, tag names would match too (a query for "strong" hits) and the snippet would print  as visible text, because EsHighlighter escapes the whole fragment. Every change of the document shape needs a full reindex of the source, otherwise the index keeps the old text.

11. Tests and verification#

Layer	Where
unit tests	`tests/Unit/Integrations/Elasticsearch/`
manual harnesses against a live ES	`tests/Manual/{Module}/f6_4_*.php`
browser click-throughs	`var/ds-baseline/tools/pages-es.json`

An ordering claim over a single item proves nothing

Over a one-item result the reversed order passes too. When the data is not enough for a proof, the harness needs a tripwire — a claim that fails as soon as more data arrives — not a silent omission.

Nor over two items with identical scores

With tied scores the order is decided by an internal tie-break, not by relevance. Print the scores before writing an ordering test and pick a query that splits the tie. For verifying the front end the right query is the opposite one: the one whose order differs from the default database ordering — only then can the ES path be told from the database path.

Every fix deserves a counter-test

Reintroducing the defect must make the harness fail. Without that you do not know whether the test tests anything.

Cookie settings