Join Log in
Score:1
Andrew Newby avatar Server

fts_solr for Dovecot - can't find plugin file?

in flag
Andrew Newby

I'm trying to follow this to get Solr search setup with my Dovecot install:

https://doc.dovecot.org/configuration_manual/fts/solr/#fts-backend-solr

It all seems to work fine - and I can see the core in Solr admin. I'm using Solr 8.9.0, but it seems happy with the config files.

However, one of the commands it says to run to actually index:

doveadm fts rescan -u [email protected]

I get:

Fatal: Plugin 'fts_solr' not found from directory /usr/lib/dovecot/modules

Sure enough, I don't see anything in anything by that name:

root@east:/usr/lib/dovecot/modules# ls -lh *fts*
-rw-r--r-- 1 root root 263K Jul  7 17:17 lib20_fts_plugin.so
-rw-r--r-- 1 root root  71K Jul  7 17:17 lib21_fts_squat_plugin.so

What am I missing? Do I need to manually install the plugin as well?

dovecot --version
2.3.7.2 (3c910f64b)

UPDATE: Looking around, I can see dovecot-fts-solr a plugin (https://alpine.pkgs.org/3.15/alpine-main-aarch64/dovecot-fts-solr-2.3.17.1-r0.apk.html ) . I can't seem to get it to install through. Not too sure how to proceed? (I could justr download the .so file and put it into /usr/lib/dovecot/modules , but I'm pretty sure that won't work :))

UPDATE 2: So I managed to find:

apt-get install dovecot-solr

This creates the lib21_fts_solr_plugin.so file. However, when I try and re-index the messages I don't get anything:

doveadm -D fts rescan -u [email protected]
Debug: Loading modules from directory: /usr/lib/dovecot/modules
Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so
Debug: Module loaded: /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so
Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm
Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message)
Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so
Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message)
Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so
Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message)
doveadm([email protected])<35131><>: Debug: auth-master: userdb lookup([email protected]): Started userdb lookup
doveadm([email protected])<35131><>: Debug: auth-master: conn unix:/var/run/dovecot//auth-userdb: Connecting
doveadm([email protected])<35131><>: Debug: auth-master: conn unix:/var/run/dovecot//auth-userdb: Client connected (fd=10)
doveadm([email protected])<35131><>: Debug: auth-master: userdb lookup([email protected]): auth USER input: [email protected] uid=1011 gid=8 home=/home/hatlamp quota_rule=*:storage=0M
doveadm([email protected])<35131><>: Debug: auth-master: userdb lookup([email protected]): Finished userdb lookup ([email protected] uid=1011 gid=8 home=/home/hatlamp quota_rule=*:storage=0M)
doveadm([email protected])<35131><>: Debug: Added userdb setting: plugin/quota_rule=*:storage=0M
doveadm([email protected]): Debug: Effective uid=1011, gid=8, home=/home/hatlamp
doveadm([email protected]): Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/hatlamp/mail/hatlamp.com/hello
doveadm([email protected]): Debug: maildir++: root=/home/hatlamp/mail/hatlamp.com/hello, index=, indexpvt=, control=, inbox=/home/hatlamp/mail/hatlamp.com/hello, alt=
doveadm([email protected]): Debug: auth-master: conn unix:/var/run/dovecot//auth-userdb: Disconnected: Connection closed (fd=10)

Looking at Solr's interface, there are no articles:

enter image description here

161
1 + 3
anx avatar
fr flag
anx
Have you configured the URL in dovecot, and have you verified that this URL is accessible? Have you actually tried to access anything from the index after initiating the rescan? Its not super clear from the docs, but *rescan* alone just ensures (e.g. by simply [starting from scratch](https://github.com/dovecot/core/blob/f59f6b660224b9ebd1d5b423a0de0da6ea03dcfb/src/plugins/fts-solr/fts-backend-solr.c#L638)) that the next lookup will not see stale or incomplete data, it does *not* ensure that the index is immediately (re-)created.
1
Reply
Andrew Newby avatar
in flag
Andrew Newby
@anx thanks for the reply. I found the issue in the end. I just needed to send an email to the account, and it would scan the folders! So now its indexed and working well :)
0
Reply
anx avatar
fr flag
anx
AFAIK receiving a message is only sufficient for a mailbox that had been used before (meaning Dovecot already knew what to index). Searching via IMAP should always be sufficient, so I put that into my answer.
0
Reply
Score:1
anx avatar Server
fr flag
anx

Use the FTS index, then you will see results in your fts backend.

Synchronising the internal state of full text content submitted to the FTS engine via the doveadm fts reindex command does not necessarily do much by itself. In the case of the solr binding, it simply moves the cursor back to zero.

This removes mails from the index that have already been expunged and makes sure that the next doveadm index will index all the missing mails (if any). Note that currently most FTS backends do not implement this properly, but instead they delete all the FTS indexes. This may change in the future versions. -- from man doveadm-fts, emphasis mine

Only when something (issuing a search, posting new mail, manual doveadm index) triggers updating the index after that command, that is when the data is sent (in your case, to solr).

+ 0
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.